2024-03-29T03:59:22.595 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-29T03:59:22.595 DEBUG:teuthology.run:Teuthology command: teuthology --name mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi --verbose --owner scheduled_mchangir@teuthology --description fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/uninlining} --archive /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629083 -- /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629083/orig.config.yaml 2024-03-29T03:59:22.647 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-29T03:59:22.736 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629083 branch: wip-mchangir-testing1-main-20240328-072412 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/uninlining} email: mchangir@redhat.com first_in_suite: false job_id: '7629083' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-mchangir-testing1-main-20240328-072412 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability log-whitelist: - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 69e2cd73c52284dfd5096c66b60c6e6b26a3c447 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 69e2cd73c52284dfd5096c66b60c6e6b26a3c447 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-testing1-main-20240328-072412 sha1: 69e2cd73c52284dfd5096c66b60c6e6b26a3c447 owner: scheduled_mchangir@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 8207 sha1: 69e2cd73c52284dfd5096c66b60c6e6b26a3c447 sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-mchangir-testing1-main-20240328-072412 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 69e2cd73c52284dfd5096c66b60c6e6b26a3c447 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi148.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE2Ekop2D5/qeW+aNVldgxNC+pD18+a42gfuvyjNTcjUOI88NVUYIADLe947cAob/WzHHAqLMW1lHVreP4Li6xg= smithi157.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBznV4lGC+LsGWKVf03Y7PBNSd75NFHyTgX517Oz732WskLXmpV6VD/C0X9zCxweqK1PwYSM0nz3uz+E2foVWM8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_uninlining teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-29_02:01:30 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-29T03:59:22.736 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa; will attempt to use it 2024-03-29T03:59:22.737 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks 2024-03-29T03:59:22.737 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-29T03:59:22.739 INFO:teuthology.task.internal:Checking packages... 2024-03-29T03:59:22.763 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '69e2cd73c52284dfd5096c66b60c6e6b26a3c447' 2024-03-29T03:59:22.763 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-29T03:59:22.763 INFO:teuthology.packaging:ref: None 2024-03-29T03:59:22.763 INFO:teuthology.packaging:tag: None 2024-03-29T03:59:22.763 INFO:teuthology.packaging:branch: wip-mchangir-testing1-main-20240328-072412 2024-03-29T03:59:22.763 INFO:teuthology.packaging:sha1: 69e2cd73c52284dfd5096c66b60c6e6b26a3c447 2024-03-29T03:59:22.763 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-mchangir-testing1-main-20240328-072412 2024-03-29T03:59:22.983 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2520.g69e2cd73 2024-03-29T03:59:22.985 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-29T03:59:22.994 INFO:teuthology.task.internal:no buildpackages task found 2024-03-29T03:59:22.994 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-29T03:59:23.034 INFO:teuthology.task.internal:Saving configuration 2024-03-29T03:59:23.053 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-29T03:59:23.063 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-29T03:59:23.088 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629083', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 03:53:41.239490', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE2Ekop2D5/qeW+aNVldgxNC+pD18+a42gfuvyjNTcjUOI88NVUYIADLe947cAob/WzHHAqLMW1lHVreP4Li6xg='} 2024-03-29T03:59:23.114 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629083', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 03:53:41.241399', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBznV4lGC+LsGWKVf03Y7PBNSd75NFHyTgX517Oz732WskLXmpV6VD/C0X9zCxweqK1PwYSM0nz3uz+E2foVWM8='} 2024-03-29T03:59:23.114 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-29T03:59:23.123 INFO:teuthology.task.internal:roles: ubuntu@smithi148.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-03-29T03:59:23.123 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-03-29T03:59:23.123 INFO:teuthology.run_tasks:Running task console_log... 2024-03-29T03:59:23.215 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4698227f70>, signals=[15]) 2024-03-29T03:59:23.215 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-29T03:59:23.224 INFO:teuthology.task.internal:Opening connections... 2024-03-29T03:59:23.224 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi148.front.sepia.ceph.com 2024-03-29T03:59:23.226 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T03:59:23.318 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi157.front.sepia.ceph.com 2024-03-29T03:59:23.319 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T03:59:23.405 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-29T03:59:23.414 DEBUG:teuthology.orchestra.run.smithi148:> uname -m 2024-03-29T03:59:23.431 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64 2024-03-29T03:59:23.432 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2024-03-29T03:59:23.488 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="CentOS Stream" 2024-03-29T03:59:23.488 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="9" 2024-03-29T03:59:23.488 INFO:teuthology.orchestra.run.smithi148.stdout:ID="centos" 2024-03-29T03:59:23.488 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="rhel fedora" 2024-03-29T03:59:23.488 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="9" 2024-03-29T03:59:23.488 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el9" 2024-03-29T03:59:23.488 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-29T03:59:23.488 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2024-03-29T03:59:23.489 INFO:teuthology.orchestra.run.smithi148.stdout:LOGO="fedora-logo-icon" 2024-03-29T03:59:23.489 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-29T03:59:23.489 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://centos.org/" 2024-03-29T03:59:23.489 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-29T03:59:23.489 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-29T03:59:23.489 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-29T03:59:23.489 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2024-03-29T03:59:23.516 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2024-03-29T03:59:23.533 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2024-03-29T03:59:23.533 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2024-03-29T03:59:23.591 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="CentOS Stream" 2024-03-29T03:59:23.591 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="9" 2024-03-29T03:59:23.591 INFO:teuthology.orchestra.run.smithi157.stdout:ID="centos" 2024-03-29T03:59:23.591 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="rhel fedora" 2024-03-29T03:59:23.591 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="9" 2024-03-29T03:59:23.591 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el9" 2024-03-29T03:59:23.591 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-29T03:59:23.591 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2024-03-29T03:59:23.591 INFO:teuthology.orchestra.run.smithi157.stdout:LOGO="fedora-logo-icon" 2024-03-29T03:59:23.591 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-29T03:59:23.592 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://centos.org/" 2024-03-29T03:59:23.592 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-29T03:59:23.592 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-29T03:59:23.592 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-29T03:59:23.592 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2024-03-29T03:59:23.616 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-29T03:59:23.629 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-29T03:59:23.653 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-29T03:59:23.653 DEBUG:teuthology.orchestra.run.smithi148:> test '!' -e /home/ubuntu/cephtest 2024-03-29T03:59:23.657 DEBUG:teuthology.orchestra.run.smithi157:> test '!' -e /home/ubuntu/cephtest 2024-03-29T03:59:23.672 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-29T03:59:23.682 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-29T03:59:23.682 DEBUG:teuthology.orchestra.run.smithi148:> test -z $(ls -A /var/lib/ceph) 2024-03-29T03:59:23.713 DEBUG:teuthology.orchestra.run.smithi157:> test -z $(ls -A /var/lib/ceph) 2024-03-29T03:59:23.824 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-29T03:59:23.878 INFO:teuthology.run_tasks:Running task kernel... 2024-03-29T03:59:23.925 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-29T03:59:23.925 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-29T03:59:23.925 DEBUG:teuthology.orchestra.run.smithi148:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-29T03:59:23.926 DEBUG:teuthology.orchestra.run.smithi157:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-29T03:59:23.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T03:59:23.941 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2024-03-29T03:59:23.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T03:59:23.943 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2024-03-29T03:59:23.997 INFO:teuthology.orchestra.run.smithi157.stdout:5.14.0-361.el9.x86_64 2024-03-29T03:59:23.998 INFO:teuthology.task.kernel:Running kernel on smithi157: 5.14.0-361.el9.x86_64 2024-03-29T03:59:23.998 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum install -y kernel 2024-03-29T03:59:23.999 INFO:teuthology.orchestra.run.smithi148.stdout:5.14.0-361.el9.x86_64 2024-03-29T03:59:23.999 INFO:teuthology.task.kernel:Running kernel on smithi148: 5.14.0-361.el9.x86_64 2024-03-29T03:59:23.999 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum install -y kernel 2024-03-29T03:59:26.213 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-29T03:59:26.362 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS-9 - AppStream 9.8 MB/s | 6.7 MB 00:00 2024-03-29T03:59:27.572 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS-9 - BaseOS 7.5 MB/s | 2.0 MB 00:00 2024-03-29T03:59:27.711 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS-9 - BaseOS 7.6 MB/s | 2.0 MB 00:00 2024-03-29T03:59:28.996 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - BaseOS 7.3 MB/s | 8.0 MB 00:01 2024-03-29T03:59:29.535 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - BaseOS 5.4 MB/s | 8.0 MB 00:01 2024-03-29T03:59:31.522 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-03-29T03:59:33.438 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - AppStream 5.6 MB/s | 19 MB 00:03 2024-03-29T03:59:35.862 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - CRB 8.1 MB/s | 6.1 MB 00:00 2024-03-29T03:59:37.329 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - Extras packages 51 kB/s | 16 kB 00:00 2024-03-29T03:59:38.571 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.1 MB 00:01 2024-03-29T03:59:38.847 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-03-29T03:59:40.056 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 16 kB 00:00 2024-03-29T03:59:41.810 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-03-29T03:59:43.797 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-03-29T03:59:46.071 INFO:teuthology.orchestra.run.smithi148.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-29T03:59:46.071 INFO:teuthology.orchestra.run.smithi148.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-29T03:59:46.168 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T03:59:46.177 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T03:59:46.178 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-03-29T03:59:46.178 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T03:59:46.178 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-03-29T03:59:46.178 INFO:teuthology.orchestra.run.smithi148.stdout: kernel x86_64 5.14.0-434.el9 CentOS-BaseOS 192 k 2024-03-29T03:59:46.178 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-03-29T03:59:46.178 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-core x86_64 5.14.0-434.el9 CentOS-BaseOS 15 M 2024-03-29T03:59:46.178 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules x86_64 5.14.0-434.el9 CentOS-BaseOS 34 M 2024-03-29T03:59:46.178 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules-core x86_64 5.14.0-434.el9 CentOS-BaseOS 28 M 2024-03-29T03:59:46.178 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T03:59:46.178 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-03-29T03:59:46.178 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T03:59:46.178 INFO:teuthology.orchestra.run.smithi148.stdout:Install 4 Packages 2024-03-29T03:59:46.178 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T03:59:46.179 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 76 M 2024-03-29T03:59:46.179 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 123 M 2024-03-29T03:59:46.179 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-03-29T03:59:46.340 INFO:teuthology.orchestra.run.smithi148.stdout:(1/4): kernel-5.14.0-434.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-03-29T03:59:46.690 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-03-29T03:59:46.857 INFO:teuthology.orchestra.run.smithi148.stdout:(2/4): kernel-core-5.14.0-434.el9.x86_64.rpm 22 MB/s | 15 MB 00:00 2024-03-29T03:59:47.449 INFO:teuthology.orchestra.run.smithi148.stdout:(3/4): kernel-modules-5.14.0-434.el9.x86_64.rpm 26 MB/s | 34 MB 00:01 2024-03-29T03:59:48.282 INFO:teuthology.orchestra.run.smithi148.stdout:(4/4): kernel-modules-core-5.14.0-434.el9.x86_6 14 MB/s | 28 MB 00:01 2024-03-29T03:59:48.285 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-03-29T03:59:48.285 INFO:teuthology.orchestra.run.smithi148.stdout:Total 36 MB/s | 76 MB 00:02 2024-03-29T03:59:48.286 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-03-29T03:59:48.670 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-03-29T03:59:48.670 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-03-29T03:59:48.955 INFO:teuthology.orchestra.run.smithi157.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-29T03:59:48.955 INFO:teuthology.orchestra.run.smithi157.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-29T03:59:49.066 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-03-29T03:59:49.071 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-03-29T03:59:49.072 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-03-29T03:59:49.072 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-03-29T03:59:49.072 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-03-29T03:59:49.072 INFO:teuthology.orchestra.run.smithi157.stdout: kernel x86_64 5.14.0-434.el9 CentOS-BaseOS 192 k 2024-03-29T03:59:49.072 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-03-29T03:59:49.072 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-core x86_64 5.14.0-434.el9 CentOS-BaseOS 15 M 2024-03-29T03:59:49.072 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-modules x86_64 5.14.0-434.el9 CentOS-BaseOS 34 M 2024-03-29T03:59:49.072 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-modules-core x86_64 5.14.0-434.el9 CentOS-BaseOS 28 M 2024-03-29T03:59:49.072 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T03:59:49.072 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-03-29T03:59:49.072 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-03-29T03:59:49.072 INFO:teuthology.orchestra.run.smithi157.stdout:Install 4 Packages 2024-03-29T03:59:49.072 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T03:59:49.073 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 76 M 2024-03-29T03:59:49.073 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 123 M 2024-03-29T03:59:49.073 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-03-29T03:59:49.358 INFO:teuthology.orchestra.run.smithi157.stdout:(1/4): kernel-5.14.0-434.el9.x86_64.rpm 673 kB/s | 192 kB 00:00 2024-03-29T03:59:49.511 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-03-29T03:59:49.511 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-03-29T03:59:50.375 INFO:teuthology.orchestra.run.smithi157.stdout:(2/4): kernel-core-5.14.0-434.el9.x86_64.rpm 12 MB/s | 15 MB 00:01 2024-03-29T03:59:50.808 INFO:teuthology.orchestra.run.smithi157.stdout:(3/4): kernel-modules-core-5.14.0-434.el9.x86_6 19 MB/s | 28 MB 00:01 2024-03-29T03:59:51.425 INFO:teuthology.orchestra.run.smithi157.stdout:(4/4): kernel-modules-5.14.0-434.el9.x86_64.rpm 14 MB/s | 34 MB 00:02 2024-03-29T03:59:51.429 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-03-29T03:59:51.429 INFO:teuthology.orchestra.run.smithi157.stdout:Total 32 MB/s | 76 MB 00:02 2024-03-29T03:59:51.429 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-03-29T03:59:51.821 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-03-29T03:59:51.822 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-03-29T03:59:51.840 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-03-29T03:59:52.665 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-modules-core-5.14.0-434.el9.x86_64 1/4 2024-03-29T03:59:52.681 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-03-29T03:59:52.681 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-03-29T03:59:52.757 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-29T03:59:55.002 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-03-29T03:59:55.516 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-29T03:59:55.597 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : kernel-modules-core-5.14.0-434.el9.x86_64 1/4 2024-03-29T03:59:55.650 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-29T03:59:55.684 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-29T03:59:58.464 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-29T03:59:58.596 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-29T04:00:00.380 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-29T04:00:00.415 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-5.14.0-434.el9.x86_64 4/4 2024-03-29T04:00:03.295 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-29T04:00:03.420 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : kernel-5.14.0-434.el9.x86_64 4/4 2024-03-29T04:00:05.235 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-03-29T04:00:08.236 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-03-29T04:00:34.727 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 4/4 2024-03-29T04:00:34.727 INFO:teuthology.orchestra.run.smithi148.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-29T04:00:34.727 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:00:34.771 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 4/4 2024-03-29T04:00:35.151 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-5.14.0-434.el9.x86_64 4/4 2024-03-29T04:00:35.152 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-5.14.0-434.el9.x86_64 1/4 2024-03-29T04:00:35.152 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-29T04:00:35.152 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-29T04:00:35.570 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-03-29T04:00:35.570 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:00:35.570 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-03-29T04:00:35.570 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-5.14.0-434.el9.x86_64 2024-03-29T04:00:35.570 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-core-5.14.0-434.el9.x86_64 2024-03-29T04:00:35.570 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules-5.14.0-434.el9.x86_64 2024-03-29T04:00:35.570 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules-core-5.14.0-434.el9.x86_64 2024-03-29T04:00:35.570 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:00:35.571 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T04:00:35.898 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q kernel | sort -rV | head -n 1 2024-03-29T04:00:36.001 INFO:teuthology.orchestra.run.smithi148.stdout:kernel-5.14.0-434.el9.x86_64 2024-03-29T04:00:36.002 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-03-29T04:00:36.002 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-434.el9.x86_64 2024-03-29T04:00:36.002 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-29T04:00:36.002 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-29T04:00:36.003 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-03-29T04:00:36.003 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi148.front.sepia.ceph.com, path=None, version=distro) 2024-03-29T04:00:36.003 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q kernel | sort -rV | head -n 1 2024-03-29T04:00:36.084 INFO:teuthology.orchestra.run.smithi148.stdout:kernel-5.14.0-434.el9.x86_64 2024-03-29T04:00:36.085 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-03-29T04:00:36.085 DEBUG:teuthology.orchestra.run.smithi148:> sudo rpm -qi grub2-tools 2024-03-29T04:00:36.129 INFO:teuthology.orchestra.run.smithi148.stdout:Name : grub2-tools 2024-03-29T04:00:36.129 INFO:teuthology.orchestra.run.smithi148.stdout:Epoch : 1 2024-03-29T04:00:36.129 INFO:teuthology.orchestra.run.smithi148.stdout:Version : 2.06 2024-03-29T04:00:36.129 INFO:teuthology.orchestra.run.smithi148.stdout:Release : 68.el9 2024-03-29T04:00:36.129 INFO:teuthology.orchestra.run.smithi148.stdout:Architecture: x86_64 2024-03-29T04:00:36.129 INFO:teuthology.orchestra.run.smithi148.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-29T04:00:36.129 INFO:teuthology.orchestra.run.smithi148.stdout:Group : Unspecified 2024-03-29T04:00:36.129 INFO:teuthology.orchestra.run.smithi148.stdout:Size : 8264254 2024-03-29T04:00:36.130 INFO:teuthology.orchestra.run.smithi148.stdout:License : GPLv3+ 2024-03-29T04:00:36.130 INFO:teuthology.orchestra.run.smithi148.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-29T04:00:36.130 INFO:teuthology.orchestra.run.smithi148.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-29T04:00:36.130 INFO:teuthology.orchestra.run.smithi148.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-29T04:00:36.130 INFO:teuthology.orchestra.run.smithi148.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-29T04:00:36.130 INFO:teuthology.orchestra.run.smithi148.stdout:Packager : builder@centos.org 2024-03-29T04:00:36.130 INFO:teuthology.orchestra.run.smithi148.stdout:Vendor : CentOS 2024-03-29T04:00:36.130 INFO:teuthology.orchestra.run.smithi148.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-29T04:00:36.130 INFO:teuthology.orchestra.run.smithi148.stdout:Summary : Support tools for GRUB. 2024-03-29T04:00:36.130 INFO:teuthology.orchestra.run.smithi148.stdout:Description : 2024-03-29T04:00:36.130 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:00:36.130 INFO:teuthology.orchestra.run.smithi148.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-29T04:00:36.130 INFO:teuthology.orchestra.run.smithi148.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-29T04:00:36.130 INFO:teuthology.orchestra.run.smithi148.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-29T04:00:36.131 INFO:teuthology.orchestra.run.smithi148.stdout:hardware devices. 2024-03-29T04:00:36.131 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:00:36.131 INFO:teuthology.orchestra.run.smithi148.stdout:This subpackage provides tools for support of all platforms. 2024-03-29T04:00:36.131 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-29T04:00:36.132 INFO:teuthology.task.kernel:Updating grub on smithi148 to boot 5.14.0-434.el9.x86_64 2024-03-29T04:00:36.132 DEBUG:teuthology.orchestra.run.smithi148:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-29T04:00:36.753 INFO:teuthology.orchestra.run.smithi148.stderr:Generating grub configuration file ... 2024-03-29T04:00:37.492 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 4/4 2024-03-29T04:00:37.492 INFO:teuthology.orchestra.run.smithi157.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-29T04:00:37.493 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:00:37.536 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 4/4 2024-03-29T04:00:37.870 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-5.14.0-434.el9.x86_64 4/4 2024-03-29T04:00:37.871 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : kernel-5.14.0-434.el9.x86_64 1/4 2024-03-29T04:00:37.871 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-29T04:00:37.871 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-29T04:00:38.078 INFO:teuthology.orchestra.run.smithi148.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-29T04:00:38.111 INFO:teuthology.orchestra.run.smithi148.stderr:done 2024-03-29T04:00:38.113 DEBUG:teuthology.orchestra.run.smithi148:> mktemp 2024-03-29T04:00:38.128 INFO:teuthology.orchestra.run.smithi148.stdout:/tmp/tmp.hlcwKjibXk 2024-03-29T04:00:38.128 DEBUG:teuthology.orchestra.run.smithi148:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.hlcwKjibXk 2024-03-29T04:00:38.202 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0666 /tmp/tmp.hlcwKjibXk 2024-03-29T04:00:38.279 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-03-29T04:00:38.280 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:00:38.280 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-03-29T04:00:38.280 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-5.14.0-434.el9.x86_64 2024-03-29T04:00:38.280 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-core-5.14.0-434.el9.x86_64 2024-03-29T04:00:38.280 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-modules-5.14.0-434.el9.x86_64 2024-03-29T04:00:38.280 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-modules-core-5.14.0-434.el9.x86_64 2024-03-29T04:00:38.280 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:00:38.280 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-03-29T04:00:38.342 DEBUG:teuthology.orchestra.remote:smithi148:/tmp/tmp.hlcwKjibXk is 6KB 2024-03-29T04:00:38.390 DEBUG:teuthology.orchestra.run.smithi148:> rm -fr /tmp/tmp.hlcwKjibXk 2024-03-29T04:00:38.404 DEBUG:teuthology.orchestra.run.smithi148:> sudo /bin/ls /boot/loader/entries || true 2024-03-29T04:00:38.470 INFO:teuthology.orchestra.run.smithi148.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-29T04:00:38.470 INFO:teuthology.orchestra.run.smithi148.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-29T04:00:38.470 INFO:teuthology.orchestra.run.smithi148.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-29T04:00:38.470 INFO:teuthology.orchestra.run.smithi148.stdout:15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64.conf 2024-03-29T04:00:38.471 DEBUG:teuthology.orchestra.run.smithi148:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64 2024-03-29T04:00:38.607 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q kernel | sort -rV | head -n 1 2024-03-29T04:00:38.623 DEBUG:teuthology.orchestra.run.smithi148:> sudo shutdown -r now 2024-03-29T04:00:38.707 INFO:teuthology.orchestra.run.smithi157.stdout:kernel-5.14.0-434.el9.x86_64 2024-03-29T04:00:38.708 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-03-29T04:00:38.708 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-434.el9.x86_64 2024-03-29T04:00:38.708 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-29T04:00:38.708 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-29T04:00:38.708 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-03-29T04:00:38.708 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi157.front.sepia.ceph.com, path=None, version=distro) 2024-03-29T04:00:38.708 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q kernel | sort -rV | head -n 1 2024-03-29T04:00:38.788 INFO:teuthology.orchestra.run.smithi157.stdout:kernel-5.14.0-434.el9.x86_64 2024-03-29T04:00:38.788 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-03-29T04:00:38.788 DEBUG:teuthology.orchestra.run.smithi157:> sudo rpm -qi grub2-tools 2024-03-29T04:00:38.832 INFO:teuthology.orchestra.run.smithi157.stdout:Name : grub2-tools 2024-03-29T04:00:38.833 INFO:teuthology.orchestra.run.smithi157.stdout:Epoch : 1 2024-03-29T04:00:38.833 INFO:teuthology.orchestra.run.smithi157.stdout:Version : 2.06 2024-03-29T04:00:38.833 INFO:teuthology.orchestra.run.smithi157.stdout:Release : 68.el9 2024-03-29T04:00:38.833 INFO:teuthology.orchestra.run.smithi157.stdout:Architecture: x86_64 2024-03-29T04:00:38.833 INFO:teuthology.orchestra.run.smithi157.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-29T04:00:38.833 INFO:teuthology.orchestra.run.smithi157.stdout:Group : Unspecified 2024-03-29T04:00:38.833 INFO:teuthology.orchestra.run.smithi157.stdout:Size : 8264254 2024-03-29T04:00:38.833 INFO:teuthology.orchestra.run.smithi157.stdout:License : GPLv3+ 2024-03-29T04:00:38.833 INFO:teuthology.orchestra.run.smithi157.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-29T04:00:38.833 INFO:teuthology.orchestra.run.smithi157.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-29T04:00:38.833 INFO:teuthology.orchestra.run.smithi157.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-29T04:00:38.833 INFO:teuthology.orchestra.run.smithi157.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-29T04:00:38.834 INFO:teuthology.orchestra.run.smithi157.stdout:Packager : builder@centos.org 2024-03-29T04:00:38.834 INFO:teuthology.orchestra.run.smithi157.stdout:Vendor : CentOS 2024-03-29T04:00:38.834 INFO:teuthology.orchestra.run.smithi157.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-29T04:00:38.834 INFO:teuthology.orchestra.run.smithi157.stdout:Summary : Support tools for GRUB. 2024-03-29T04:00:38.834 INFO:teuthology.orchestra.run.smithi157.stdout:Description : 2024-03-29T04:00:38.834 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:00:38.834 INFO:teuthology.orchestra.run.smithi157.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-29T04:00:38.834 INFO:teuthology.orchestra.run.smithi157.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-29T04:00:38.834 INFO:teuthology.orchestra.run.smithi157.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-29T04:00:38.834 INFO:teuthology.orchestra.run.smithi157.stdout:hardware devices. 2024-03-29T04:00:38.834 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:00:38.834 INFO:teuthology.orchestra.run.smithi157.stdout:This subpackage provides tools for support of all platforms. 2024-03-29T04:00:38.835 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-29T04:00:38.835 INFO:teuthology.task.kernel:Updating grub on smithi157 to boot 5.14.0-434.el9.x86_64 2024-03-29T04:00:38.835 DEBUG:teuthology.orchestra.run.smithi157:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-29T04:00:39.452 INFO:teuthology.orchestra.run.smithi157.stderr:Generating grub configuration file ... 2024-03-29T04:00:40.816 INFO:teuthology.orchestra.run.smithi157.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-29T04:00:40.848 INFO:teuthology.orchestra.run.smithi157.stderr:done 2024-03-29T04:00:40.851 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2024-03-29T04:00:40.866 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.4t5XCCqP31 2024-03-29T04:00:40.866 DEBUG:teuthology.orchestra.run.smithi157:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.4t5XCCqP31 2024-03-29T04:00:40.935 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.4t5XCCqP31 2024-03-29T04:00:41.086 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.4t5XCCqP31 is 6KB 2024-03-29T04:00:41.136 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.4t5XCCqP31 2024-03-29T04:00:41.150 DEBUG:teuthology.orchestra.run.smithi157:> sudo /bin/ls /boot/loader/entries || true 2024-03-29T04:00:41.223 INFO:teuthology.orchestra.run.smithi157.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-29T04:00:41.223 INFO:teuthology.orchestra.run.smithi157.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-29T04:00:41.223 INFO:teuthology.orchestra.run.smithi157.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-29T04:00:41.223 INFO:teuthology.orchestra.run.smithi157.stdout:15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64.conf 2024-03-29T04:00:41.225 DEBUG:teuthology.orchestra.run.smithi157:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64 2024-03-29T04:00:41.385 DEBUG:teuthology.orchestra.run.smithi157:> sudo shutdown -r now 2024-03-29T04:01:08.651 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-29T04:01:08.652 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-03-29T04:01:08.653 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:01:11.390 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-29T04:01:11.390 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-03-29T04:01:11.391 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:01:43.199 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-03-29T04:01:46.018 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-03-29T04:01:52.202 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-03-29T04:01:52.203 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:01:55.020 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-03-29T04:01:55.020 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:01:58.078 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-03-29T04:02:10.090 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-03-29T04:02:10.091 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:02:10.590 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-03-29T04:02:22.602 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-03-29T04:02:22.603 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:02:23.017 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-03-29T04:02:23.528 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-03-29T04:02:23.528 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-434.el9.x86_64"... 2024-03-29T04:02:23.528 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2024-03-29T04:02:23.542 INFO:teuthology.orchestra.run.smithi148.stdout:5.14.0-434.el9.x86_64 2024-03-29T04:02:23.542 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-434.el9.x86_64 vs 5.14.0-434.el9.x86_64 2024-03-29T04:02:23.542 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-29T04:02:23.543 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-29T04:02:24.543 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-29T04:02:24.543 DEBUG:teuthology.orchestra.run.smithi148:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-29T04:02:24.606 INFO:teuthology.orchestra.run.smithi148.stdout:ttyS1 2024-03-29T04:02:24.630 DEBUG:teuthology.parallel:result is None 2024-03-29T04:02:28.510 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-03-29T04:02:43.511 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-03-29T04:02:43.512 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:02:43.886 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-03-29T04:02:44.236 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-03-29T04:02:44.236 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-434.el9.x86_64"... 2024-03-29T04:02:44.236 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2024-03-29T04:02:44.251 INFO:teuthology.orchestra.run.smithi157.stdout:5.14.0-434.el9.x86_64 2024-03-29T04:02:44.251 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-434.el9.x86_64 vs 5.14.0-434.el9.x86_64 2024-03-29T04:02:44.251 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-29T04:02:44.251 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-29T04:02:45.252 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-29T04:02:45.253 DEBUG:teuthology.orchestra.run.smithi157:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-29T04:02:45.327 INFO:teuthology.orchestra.run.smithi157.stdout:ttyS1 2024-03-29T04:02:45.366 DEBUG:teuthology.parallel:result is None 2024-03-29T04:02:45.366 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-29T04:02:45.376 INFO:teuthology.task.internal:Creating test directory... 2024-03-29T04:02:45.376 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-29T04:02:45.381 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-29T04:02:45.396 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-29T04:02:45.417 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-29T04:02:45.442 INFO:teuthology.task.internal:Creating archive directory... 2024-03-29T04:02:45.442 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-29T04:02:45.445 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-29T04:02:45.487 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-29T04:02:45.494 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-29T04:02:45.494 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-29T04:02:45.524 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-29T04:02:45.567 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T04:02:45.570 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T04:02:45.583 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T04:02:45.584 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T04:02:45.586 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-29T04:02:45.597 INFO:teuthology.task.internal:Configuring sudo... 2024-03-29T04:02:45.597 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-29T04:02:45.600 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-29T04:02:45.628 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-29T04:02:45.646 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-29T04:02:45.647 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-29T04:02:45.670 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-29T04:02:45.684 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-29T04:02:45.765 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-29T04:02:45.820 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-29T04:02:45.904 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-29T04:02:45.947 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:02:45.947 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-29T04:02:46.014 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-29T04:02:46.054 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-29T04:02:46.096 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-29T04:02:46.175 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-29T04:02:46.213 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:02:46.214 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-29T04:02:46.277 DEBUG:teuthology.orchestra.run.smithi148:> sudo service rsyslog restart 2024-03-29T04:02:46.279 DEBUG:teuthology.orchestra.run.smithi157:> sudo service rsyslog restart 2024-03-29T04:02:46.321 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-29T04:02:46.401 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-29T04:02:46.838 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-29T04:02:46.844 INFO:teuthology.task.internal:Starting timer... 2024-03-29T04:02:46.845 INFO:teuthology.run_tasks:Running task pcp... 2024-03-29T04:02:46.879 INFO:teuthology.run_tasks:Running task selinux... 2024-03-29T04:02:46.902 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-03-29T04:02:46.902 DEBUG:teuthology.orchestra.run.smithi148:> sudo service auditd rotate 2024-03-29T04:02:46.954 INFO:teuthology.orchestra.run.smithi148.stdout:Rotating logs: 2024-03-29T04:02:46.955 DEBUG:teuthology.orchestra.run.smithi157:> sudo service auditd rotate 2024-03-29T04:02:47.013 INFO:teuthology.orchestra.run.smithi157.stdout:Rotating logs: 2024-03-29T04:02:47.015 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-29T04:02:47.015 DEBUG:teuthology.orchestra.run.smithi148:> /usr/sbin/getenforce 2024-03-29T04:02:47.036 INFO:teuthology.orchestra.run.smithi148.stdout:Permissive 2024-03-29T04:02:47.036 DEBUG:teuthology.orchestra.run.smithi157:> /usr/sbin/getenforce 2024-03-29T04:02:47.058 INFO:teuthology.orchestra.run.smithi157.stdout:Permissive 2024-03-29T04:02:47.058 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi148.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi157.front.sepia.ceph.com': 'permissive'} 2024-03-29T04:02:47.058 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-29T04:02:47.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:02:47.105 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-29T04:02:47.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:02:47.133 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-29T04:02:47.133 DEBUG:teuthology.orchestra.run.smithi148:> sudo /usr/sbin/setenforce permissive 2024-03-29T04:02:47.180 DEBUG:teuthology.orchestra.run.smithi157:> sudo /usr/sbin/setenforce permissive 2024-03-29T04:02:47.210 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-29T04:02:47.220 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-29T04:02:47.224 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-29T04:02:47.343 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-29T04:02:47.359 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-29T04:02:47.360 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi148.front.sepia.ceph.com,smithi157.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-29T04:07:00.923 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi148.front.sepia.ceph.com'), Remote(name='ubuntu@smithi157.front.sepia.ceph.com')] 2024-03-29T04:07:00.924 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-03-29T04:07:00.925 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:07:01.017 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-03-29T04:07:01.092 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-03-29T04:07:01.093 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-03-29T04:07:01.093 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:07:01.184 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-03-29T04:07:01.265 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-03-29T04:07:01.266 INFO:teuthology.run_tasks:Running task clock... 2024-03-29T04:07:01.278 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-29T04:07:01.278 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-29T04:07:01.278 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T04:07:01.281 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-29T04:07:01.281 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T04:07:01.309 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-29T04:07:01.326 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-29T04:07:01.335 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-29T04:07:01.352 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-29T04:07:01.355 INFO:teuthology.orchestra.run.smithi148.stderr:sudo: ntpd: command not found 2024-03-29T04:07:01.369 INFO:teuthology.orchestra.run.smithi148.stdout:506 Cannot talk to daemon 2024-03-29T04:07:01.381 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: ntpd: command not found 2024-03-29T04:07:01.385 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-29T04:07:01.395 INFO:teuthology.orchestra.run.smithi157.stdout:506 Cannot talk to daemon 2024-03-29T04:07:01.401 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-29T04:07:01.411 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-29T04:07:01.428 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-29T04:07:01.451 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-03-29T04:07:01.454 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-29T04:07:01.455 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-03-29T04:07:01.455 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:07:01.455 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:07:01.455 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:07:01.455 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:07:01.480 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2024-03-29T04:07:01.483 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-29T04:07:01.483 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-03-29T04:07:01.483 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:07:01.484 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:07:01.484 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:07:01.484 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:07:01.485 INFO:teuthology.run_tasks:Running task install... 2024-03-29T04:07:01.495 DEBUG:teuthology.task.install:project ceph 2024-03-29T04:07:01.495 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447'}} 2024-03-29T04:07:01.496 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447'} 2024-03-29T04:07:01.496 INFO:teuthology.task.install:Using flavor: default 2024-03-29T04:07:01.502 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-29T04:07:01.502 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-29T04:07:01.506 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447', 'tag': None, 'wait_for_package': False} 2024-03-29T04:07:01.507 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=69e2cd73c52284dfd5096c66b60c6e6b26a3c447 2024-03-29T04:07:01.510 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447', 'tag': None, 'wait_for_package': False} 2024-03-29T04:07:01.510 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=69e2cd73c52284dfd5096c66b60c6e6b26a3c447 2024-03-29T04:07:01.655 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/centos/9/flavors/default/ 2024-03-29T04:07:01.655 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2520.g69e2cd73 2024-03-29T04:07:01.733 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/centos/9/flavors/default/ 2024-03-29T04:07:01.733 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2520.g69e2cd73 2024-03-29T04:07:02.030 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-29T04:07:02.031 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:07:02.031 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-29T04:07:02.059 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-29T04:07:02.059 DEBUG:teuthology.orchestra.run.smithi157:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-29T04:07:02.066 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-29T04:07:02.066 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:07:02.066 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-29T04:07:02.094 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-29T04:07:02.095 DEBUG:teuthology.orchestra.run.smithi148:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-29T04:07:02.127 DEBUG:teuthology.orchestra.run.smithi157:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-29T04:07:02.163 DEBUG:teuthology.orchestra.run.smithi148:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-29T04:07:02.206 DEBUG:teuthology.orchestra.run.smithi157:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-29T04:07:02.244 DEBUG:teuthology.orchestra.run.smithi148:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-29T04:07:02.273 INFO:teuthology.orchestra.run.smithi157.stdout:check_obsoletes = 1 2024-03-29T04:07:02.275 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum clean all 2024-03-29T04:07:02.310 INFO:teuthology.orchestra.run.smithi148.stdout:check_obsoletes = 1 2024-03-29T04:07:02.312 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum clean all 2024-03-29T04:07:02.525 INFO:teuthology.orchestra.run.smithi157.stdout:74 files removed 2024-03-29T04:07:02.552 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-29T04:07:02.554 INFO:teuthology.orchestra.run.smithi148.stdout:74 files removed 2024-03-29T04:07:02.581 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-29T04:07:03.581 INFO:teuthology.orchestra.run.smithi148.stdout:ceph packages for x86_64 119 kB/s | 83 kB 00:00 2024-03-29T04:07:03.923 INFO:teuthology.orchestra.run.smithi157.stdout:ceph packages for x86_64 77 kB/s | 83 kB 00:01 2024-03-29T04:07:04.257 INFO:teuthology.orchestra.run.smithi148.stdout:ceph noarch packages 18 kB/s | 12 kB 00:00 2024-03-29T04:07:04.690 INFO:teuthology.orchestra.run.smithi157.stdout:ceph noarch packages 16 kB/s | 12 kB 00:00 2024-03-29T04:07:04.882 INFO:teuthology.orchestra.run.smithi148.stdout:ceph source packages 3.2 kB/s | 1.9 kB 00:00 2024-03-29T04:07:05.157 INFO:teuthology.orchestra.run.smithi157.stdout:ceph source packages 4.5 kB/s | 1.9 kB 00:00 2024-03-29T04:07:05.568 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS-9 - AppStream 18 MB/s | 6.7 MB 00:00 2024-03-29T04:07:05.653 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS-9 - AppStream 9.1 MB/s | 6.7 MB 00:00 2024-03-29T04:07:07.085 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS-9 - BaseOS 6.7 MB/s | 2.0 MB 00:00 2024-03-29T04:07:07.110 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS-9 - BaseOS 4.7 MB/s | 2.0 MB 00:00 2024-03-29T04:07:08.162 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-03-29T04:07:10.425 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - BaseOS 2.7 MB/s | 8.0 MB 00:03 2024-03-29T04:07:10.577 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-03-29T04:07:15.877 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.1 MB 00:01 2024-03-29T04:07:18.386 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-03-29T04:07:20.441 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-03-29T04:07:25.463 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-03-29T04:07:28.026 INFO:teuthology.orchestra.run.smithi157.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-29T04:07:28.027 INFO:teuthology.orchestra.run.smithi157.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-29T04:07:28.041 INFO:teuthology.orchestra.run.smithi157.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-29T04:07:28.042 INFO:teuthology.orchestra.run.smithi157.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-29T04:07:28.043 INFO:teuthology.orchestra.run.smithi157.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-29T04:07:28.044 INFO:teuthology.orchestra.run.smithi157.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-29T04:07:28.046 INFO:teuthology.orchestra.run.smithi157.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-29T04:07:28.049 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-29T04:07:28.050 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-29T04:07:28.129 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-03-29T04:07:28.147 INFO:teuthology.orchestra.run.smithi157.stdout:============================================================================================ 2024-03-29T04:07:28.147 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-03-29T04:07:28.147 INFO:teuthology.orchestra.run.smithi157.stdout:============================================================================================ 2024-03-29T04:07:28.147 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-03-29T04:07:28.147 INFO:teuthology.orchestra.run.smithi157.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-29T04:07:28.147 INFO:teuthology.orchestra.run.smithi157.stdout: ceph x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 6.6 k 2024-03-29T04:07:28.147 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 5.4 M 2024-03-29T04:07:28.147 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 890 k 2024-03-29T04:07:28.147 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 145 k 2024-03-29T04:07:28.148 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 1.5 M 2024-03-29T04:07:28.148 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 141 k 2024-03-29T04:07:28.148 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 3.5 M 2024-03-29T04:07:28.148 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 7.4 M 2024-03-29T04:07:28.148 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 49 k 2024-03-29T04:07:28.148 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 9.8 M 2024-03-29T04:07:28.148 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 48 M 2024-03-29T04:07:28.148 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 264 k 2024-03-29T04:07:28.148 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 762 k 2024-03-29T04:07:28.148 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 221 k 2024-03-29T04:07:28.148 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-shell x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 27 k 2024-03-29T04:07:28.148 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 18 k 2024-03-29T04:07:28.148 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-29T04:07:28.148 INFO:teuthology.orchestra.run.smithi157.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-29T04:07:28.149 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-29T04:07:28.149 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 32 k 2024-03-29T04:07:28.149 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 746 k 2024-03-29T04:07:28.149 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 127 k 2024-03-29T04:07:28.149 INFO:teuthology.orchestra.run.smithi157.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-29T04:07:28.149 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-29T04:07:28.149 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-29T04:07:28.149 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-03-29T04:07:28.149 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-03-29T04:07:28.149 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-03-29T04:07:28.149 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 162 k 2024-03-29T04:07:28.149 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 321 k 2024-03-29T04:07:28.149 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 298 k 2024-03-29T04:07:28.150 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 99 k 2024-03-29T04:07:28.150 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 85 k 2024-03-29T04:07:28.150 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 3.1 M 2024-03-29T04:07:28.150 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 170 k 2024-03-29T04:07:28.150 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-29T04:07:28.150 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrading: 2024-03-29T04:07:28.150 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-03-29T04:07:28.150 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-03-29T04:07:28.150 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-03-29T04:07:28.150 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-03-29T04:07:28.150 INFO:teuthology.orchestra.run.smithi157.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-29T04:07:28.150 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-29T04:07:28.150 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-29T04:07:28.151 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-29T04:07:28.151 INFO:teuthology.orchestra.run.smithi157.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-29T04:07:28.151 INFO:teuthology.orchestra.run.smithi157.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-29T04:07:28.151 INFO:teuthology.orchestra.run.smithi157.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-29T04:07:28.151 INFO:teuthology.orchestra.run.smithi157.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-29T04:07:28.151 INFO:teuthology.orchestra.run.smithi157.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-29T04:07:28.151 INFO:teuthology.orchestra.run.smithi157.stdout: librados2 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 3.4 M 2024-03-29T04:07:28.151 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 3.1 M 2024-03-29T04:07:28.151 INFO:teuthology.orchestra.run.smithi157.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-29T04:07:28.151 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-29T04:07:28.151 INFO:teuthology.orchestra.run.smithi157.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-29T04:07:28.151 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-29T04:07:28.152 INFO:teuthology.orchestra.run.smithi157.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-29T04:07:28.152 INFO:teuthology.orchestra.run.smithi157.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-29T04:07:28.152 INFO:teuthology.orchestra.run.smithi157.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-29T04:07:28.152 INFO:teuthology.orchestra.run.smithi157.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-29T04:07:28.152 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-29T04:07:28.152 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-03-29T04:07:28.152 INFO:teuthology.orchestra.run.smithi157.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-29T04:07:28.152 INFO:teuthology.orchestra.run.smithi157.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-29T04:07:28.152 INFO:teuthology.orchestra.run.smithi157.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-29T04:07:28.152 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 21 M 2024-03-29T04:07:28.152 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 34 k 2024-03-29T04:07:28.152 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 2.4 M 2024-03-29T04:07:28.153 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 249 k 2024-03-29T04:07:28.153 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 4.7 M 2024-03-29T04:07:28.153 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 17 M 2024-03-29T04:07:28.153 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 16 k 2024-03-29T04:07:28.153 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 25 k 2024-03-29T04:07:28.153 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-29T04:07:28.153 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-29T04:07:28.153 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-29T04:07:28.153 INFO:teuthology.orchestra.run.smithi157.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-29T04:07:28.153 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-29T04:07:28.153 INFO:teuthology.orchestra.run.smithi157.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-29T04:07:28.153 INFO:teuthology.orchestra.run.smithi157.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-29T04:07:28.153 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-29T04:07:28.153 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-29T04:07:28.154 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 163 k 2024-03-29T04:07:28.154 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-29T04:07:28.154 INFO:teuthology.orchestra.run.smithi157.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-29T04:07:28.154 INFO:teuthology.orchestra.run.smithi157.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-29T04:07:28.154 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-29T04:07:28.154 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-29T04:07:28.154 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 500 k 2024-03-29T04:07:28.154 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-29T04:07:28.154 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 5.1 M 2024-03-29T04:07:28.154 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-29T04:07:28.154 INFO:teuthology.orchestra.run.smithi157.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-29T04:07:28.154 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-29T04:07:28.155 INFO:teuthology.orchestra.run.smithi157.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-29T04:07:28.155 INFO:teuthology.orchestra.run.smithi157.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-29T04:07:28.155 INFO:teuthology.orchestra.run.smithi157.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-29T04:07:28.155 INFO:teuthology.orchestra.run.smithi157.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-29T04:07:28.155 INFO:teuthology.orchestra.run.smithi157.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-29T04:07:28.155 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-29T04:07:28.155 INFO:teuthology.orchestra.run.smithi157.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-29T04:07:28.155 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-03-29T04:07:28.155 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-29T04:07:28.155 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-29T04:07:28.155 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-29T04:07:28.155 INFO:teuthology.orchestra.run.smithi157.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-29T04:07:28.155 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-29T04:07:28.155 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-29T04:07:28.156 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-29T04:07:28.156 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 45 k 2024-03-29T04:07:28.156 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 134 k 2024-03-29T04:07:28.156 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-29T04:07:28.156 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-29T04:07:28.156 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-29T04:07:28.156 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-29T04:07:28.156 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-29T04:07:28.156 INFO:teuthology.orchestra.run.smithi157.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-29T04:07:28.156 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-29T04:07:28.156 INFO:teuthology.orchestra.run.smithi157.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-29T04:07:28.156 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-29T04:07:28.156 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-29T04:07:28.156 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-29T04:07:28.156 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-29T04:07:28.157 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-29T04:07:28.157 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-29T04:07:28.157 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-29T04:07:28.157 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-29T04:07:28.157 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-29T04:07:28.157 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-29T04:07:28.157 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-29T04:07:28.157 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-29T04:07:28.157 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-29T04:07:28.157 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-29T04:07:28.157 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-29T04:07:28.157 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-29T04:07:28.157 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-29T04:07:28.157 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-29T04:07:28.158 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-29T04:07:28.158 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-29T04:07:28.158 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-29T04:07:28.158 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-29T04:07:28.158 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-29T04:07:28.158 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-29T04:07:28.158 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-29T04:07:28.158 INFO:teuthology.orchestra.run.smithi157.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-29T04:07:28.158 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-29T04:07:28.158 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-29T04:07:28.158 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-29T04:07:28.158 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-29T04:07:28.158 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-29T04:07:28.159 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-29T04:07:28.159 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-29T04:07:28.159 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-29T04:07:28.159 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-29T04:07:28.159 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-29T04:07:28.159 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-29T04:07:28.159 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-29T04:07:28.159 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-29T04:07:28.159 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-29T04:07:28.159 INFO:teuthology.orchestra.run.smithi157.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-29T04:07:28.159 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-29T04:07:28.159 INFO:teuthology.orchestra.run.smithi157.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-29T04:07:28.159 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-29T04:07:28.159 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-29T04:07:28.159 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-29T04:07:28.160 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-29T04:07:28.160 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-03-29T04:07:28.160 INFO:teuthology.orchestra.run.smithi157.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-03-29T04:07:28.160 INFO:teuthology.orchestra.run.smithi157.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-29T04:07:28.160 INFO:teuthology.orchestra.run.smithi157.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-29T04:07:28.160 INFO:teuthology.orchestra.run.smithi157.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-29T04:07:28.160 INFO:teuthology.orchestra.run.smithi157.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-29T04:07:28.160 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-29T04:07:28.160 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-29T04:07:28.160 INFO:teuthology.orchestra.run.smithi157.stdout:Installing weak dependencies: 2024-03-29T04:07:28.160 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-03-29T04:07:28.160 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:07:28.160 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-03-29T04:07:28.160 INFO:teuthology.orchestra.run.smithi157.stdout:============================================================================================ 2024-03-29T04:07:28.161 INFO:teuthology.orchestra.run.smithi157.stdout:Install 150 Packages 2024-03-29T04:07:28.161 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrade 24 Packages 2024-03-29T04:07:28.161 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:07:28.161 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 226 M 2024-03-29T04:07:28.161 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-03-29T04:07:28.792 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:17 2024-03-29T04:07:28.955 INFO:teuthology.orchestra.run.smithi157.stdout:(1/174): ceph-19.0.0-2520.g69e2cd73.el9.x86_64. 30 kB/s | 6.6 kB 00:00 2024-03-29T04:07:29.297 INFO:teuthology.orchestra.run.smithi157.stdout:(2/174): ceph-fuse-19.0.0-2520.g69e2cd73.el9.x8 2.5 MB/s | 890 kB 00:00 2024-03-29T04:07:29.372 INFO:teuthology.orchestra.run.smithi157.stdout:(3/174): ceph-immutable-object-cache-19.0.0-252 1.9 MB/s | 145 kB 00:00 2024-03-29T04:07:29.864 INFO:teuthology.orchestra.run.smithi157.stdout:(4/174): ceph-base-19.0.0-2520.g69e2cd73.el9.x8 4.8 MB/s | 5.4 MB 00:01 2024-03-29T04:07:29.906 INFO:teuthology.orchestra.run.smithi157.stdout:(5/174): ceph-mds-19.0.0-2520.g69e2cd73.el9.x86 4.5 MB/s | 2.4 MB 00:00 2024-03-29T04:07:30.131 INFO:teuthology.orchestra.run.smithi157.stdout:(6/174): ceph-mgr-19.0.0-2520.g69e2cd73.el9.x86 5.6 MB/s | 1.5 MB 00:00 2024-03-29T04:07:30.323 INFO:teuthology.orchestra.run.smithi157.stdout:(7/174): ceph-common-19.0.0-2520.g69e2cd73.el9. 13 MB/s | 21 MB 00:01 2024-03-29T04:07:30.724 INFO:teuthology.orchestra.run.smithi157.stdout:(8/174): ceph-mon-19.0.0-2520.g69e2cd73.el9.x86 5.7 MB/s | 4.7 MB 00:00 2024-03-29T04:07:30.816 INFO:teuthology.orchestra.run.smithi157.stdout:(9/174): ceph-selinux-19.0.0-2520.g69e2cd73.el9 275 kB/s | 25 kB 00:00 2024-03-29T04:07:30.991 INFO:teuthology.orchestra.run.smithi157.stdout:(10/174): ceph-radosgw-19.0.0-2520.g69e2cd73.el 15 MB/s | 9.8 MB 00:00 2024-03-29T04:07:31.066 INFO:teuthology.orchestra.run.smithi157.stdout:(11/174): cephfs-mirror-19.0.0-2520.g69e2cd73.e 2.9 MB/s | 221 kB 00:00 2024-03-29T04:07:31.116 INFO:teuthology.orchestra.run.smithi157.stdout:(12/174): cephfs-shell-19.0.0-2520.g69e2cd73.el 541 kB/s | 27 kB 00:00 2024-03-29T04:07:31.166 INFO:teuthology.orchestra.run.smithi157.stdout:(13/174): libcephfs-devel-19.0.0-2520.g69e2cd73 637 kB/s | 32 kB 00:00 2024-03-29T04:07:31.259 INFO:teuthology.orchestra.run.smithi157.stdout:(14/174): libcephfs2-19.0.0-2520.g69e2cd73.el9. 8.0 MB/s | 746 kB 00:00 2024-03-29T04:07:31.309 INFO:teuthology.orchestra.run.smithi157.stdout:(15/174): libcephsqlite-19.0.0-2520.g69e2cd73.e 3.2 MB/s | 163 kB 00:00 2024-03-29T04:07:31.376 INFO:teuthology.orchestra.run.smithi157.stdout:(16/174): librados-devel-19.0.0-2520.g69e2cd73. 1.9 MB/s | 127 kB 00:00 2024-03-29T04:07:31.468 INFO:teuthology.orchestra.run.smithi157.stdout:(17/174): libradosstriper1-19.0.0-2520.g69e2cd7 5.3 MB/s | 500 kB 00:00 2024-03-29T04:07:31.877 INFO:teuthology.orchestra.run.smithi157.stdout:(18/174): librgw2-19.0.0-2520.g69e2cd73.el9.x86 13 MB/s | 5.1 MB 00:00 2024-03-29T04:07:31.927 INFO:teuthology.orchestra.run.smithi157.stdout:(19/174): python3-ceph-argparse-19.0.0-2520.g69 905 kB/s | 45 kB 00:00 2024-03-29T04:07:31.993 INFO:teuthology.orchestra.run.smithi157.stdout:(20/174): python3-ceph-common-19.0.0-2520.g69e2 2.0 MB/s | 134 kB 00:00 2024-03-29T04:07:32.182 INFO:teuthology.orchestra.run.smithi157.stdout:(21/174): ceph-osd-19.0.0-2520.g69e2cd73.el9.x8 8.2 MB/s | 17 MB 00:02 2024-03-29T04:07:32.211 INFO:teuthology.orchestra.run.smithi157.stdout:(22/174): python3-cephfs-19.0.0-2520.g69e2cd73. 744 kB/s | 162 kB 00:00 2024-03-29T04:07:32.253 INFO:teuthology.orchestra.run.smithi157.stdout:(23/174): python3-rados-19.0.0-2520.g69e2cd73.e 4.5 MB/s | 321 kB 00:00 2024-03-29T04:07:32.278 INFO:teuthology.orchestra.run.smithi157.stdout:(24/174): python3-rbd-19.0.0-2520.g69e2cd73.el9 4.4 MB/s | 298 kB 00:00 2024-03-29T04:07:32.320 INFO:teuthology.orchestra.run.smithi157.stdout:(25/174): python3-rgw-19.0.0-2520.g69e2cd73.el9 1.5 MB/s | 99 kB 00:00 2024-03-29T04:07:32.345 INFO:teuthology.orchestra.run.smithi157.stdout:(26/174): rbd-fuse-19.0.0-2520.g69e2cd73.el9.x8 1.2 MB/s | 85 kB 00:00 2024-03-29T04:07:32.404 INFO:teuthology.orchestra.run.smithi157.stdout:(27/174): rbd-nbd-19.0.0-2520.g69e2cd73.el9.x86 2.9 MB/s | 170 kB 00:00 2024-03-29T04:07:32.471 INFO:teuthology.orchestra.run.smithi157.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2520.g 509 kB/s | 34 kB 00:00 2024-03-29T04:07:32.529 INFO:teuthology.orchestra.run.smithi157.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2520.g69e2cd7 2.4 MB/s | 141 kB 00:00 2024-03-29T04:07:32.755 INFO:teuthology.orchestra.run.smithi157.stdout:(30/174): rbd-mirror-19.0.0-2520.g69e2cd73.el9. 7.1 MB/s | 3.1 MB 00:00 2024-03-29T04:07:32.822 INFO:teuthology.orchestra.run.smithi157.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2520.g69e2c 12 MB/s | 3.5 MB 00:00 2024-03-29T04:07:32.897 INFO:teuthology.orchestra.run.smithi157.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2520.g69 3.2 MB/s | 249 kB 00:00 2024-03-29T04:07:32.947 INFO:teuthology.orchestra.run.smithi157.stdout:(33/174): ceph-mgr-rook-19.0.0-2520.g69e2cd73.e 988 kB/s | 49 kB 00:00 2024-03-29T04:07:33.006 INFO:teuthology.orchestra.run.smithi157.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2520.g6 283 kB/s | 16 kB 00:00 2024-03-29T04:07:33.081 INFO:teuthology.orchestra.run.smithi157.stdout:(35/174): ceph-volume-19.0.0-2520.g69e2cd73.el9 3.4 MB/s | 264 kB 00:00 2024-03-29T04:07:33.173 INFO:teuthology.orchestra.run.smithi157.stdout:(36/174): cephadm-19.0.0-2520.g69e2cd73.el9.noa 8.1 MB/s | 762 kB 00:00 2024-03-29T04:07:33.232 INFO:teuthology.orchestra.run.smithi157.stdout:(37/174): cephfs-top-19.0.0-2520.g69e2cd73.el9. 313 kB/s | 18 kB 00:00 2024-03-29T04:07:33.335 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - CRB 5.7 MB/s | 6.1 MB 00:01 2024-03-29T04:07:33.374 INFO:teuthology.orchestra.run.smithi157.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 4.6 MB/s | 662 kB 00:00 2024-03-29T04:07:33.474 INFO:teuthology.orchestra.run.smithi157.stdout:(39/174): ceph-mgr-diskprediction-local-19.0.0- 10 MB/s | 7.4 MB 00:00 2024-03-29T04:07:33.516 INFO:teuthology.orchestra.run.smithi157.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 6.3 MB/s | 920 kB 00:00 2024-03-29T04:07:33.541 INFO:teuthology.orchestra.run.smithi157.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2024-03-29T04:07:33.566 INFO:teuthology.orchestra.run.smithi157.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-29T04:07:33.584 INFO:teuthology.orchestra.run.smithi157.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 762 kB/s | 32 kB 00:00 2024-03-29T04:07:33.684 INFO:teuthology.orchestra.run.smithi157.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-03-29T04:07:33.700 INFO:teuthology.orchestra.run.smithi157.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 148 kB/s | 17 kB 00:00 2024-03-29T04:07:33.718 INFO:teuthology.orchestra.run.smithi157.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 300 kB/s | 10 kB 00:00 2024-03-29T04:07:33.743 INFO:teuthology.orchestra.run.smithi157.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-03-29T04:07:33.762 INFO:teuthology.orchestra.run.smithi157.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-29T04:07:33.785 INFO:teuthology.orchestra.run.smithi157.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 16 MB/s | 661 kB 00:00 2024-03-29T04:07:33.810 INFO:teuthology.orchestra.run.smithi157.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 12 MB/s | 578 kB 00:00 2024-03-29T04:07:33.836 INFO:teuthology.orchestra.run.smithi157.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 932 kB/s | 46 kB 00:00 2024-03-29T04:07:33.861 INFO:teuthology.orchestra.run.smithi157.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-29T04:07:33.886 INFO:teuthology.orchestra.run.smithi157.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-29T04:07:33.912 INFO:teuthology.orchestra.run.smithi157.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-03-29T04:07:33.937 INFO:teuthology.orchestra.run.smithi157.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 684 kB/s | 34 kB 00:00 2024-03-29T04:07:34.396 INFO:teuthology.orchestra.run.smithi157.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2024-03-29T04:07:34.397 INFO:teuthology.orchestra.run.smithi157.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 9.3 MB/s | 1.5 MB 00:00 2024-03-29T04:07:34.397 INFO:teuthology.orchestra.run.smithi157.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 817 kB 00:00 2024-03-29T04:07:34.397 INFO:teuthology.orchestra.run.smithi157.stdout:(59/174): postgresql-private-libs-13.14-1.el9.x 2.3 MB/s | 132 kB 00:00 2024-03-29T04:07:34.397 INFO:teuthology.orchestra.run.smithi157.stdout:(60/174): postgresql-server-13.14-1.el9.x86_64. 36 MB/s | 5.7 MB 00:00 2024-03-29T04:07:34.397 INFO:teuthology.orchestra.run.smithi157.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.8 MB/s | 1.0 MB 00:00 2024-03-29T04:07:34.397 INFO:teuthology.orchestra.run.smithi157.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 78 kB 00:00 2024-03-29T04:07:34.506 INFO:teuthology.orchestra.run.smithi157.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 35 MB/s | 5.8 MB 00:00 2024-03-29T04:07:34.531 INFO:teuthology.orchestra.run.smithi157.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 148 kB/s | 24 kB 00:00 2024-03-29T04:07:34.556 INFO:teuthology.orchestra.run.smithi157.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 360 kB/s | 18 kB 00:00 2024-03-29T04:07:34.582 INFO:teuthology.orchestra.run.smithi157.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-03-29T04:07:34.607 INFO:teuthology.orchestra.run.smithi157.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 630 kB/s | 32 kB 00:00 2024-03-29T04:07:34.716 INFO:teuthology.orchestra.run.smithi157.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 4.8 MB 00:00 2024-03-29T04:07:34.741 INFO:teuthology.orchestra.run.smithi157.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 217 kB 00:00 2024-03-29T04:07:34.766 INFO:teuthology.orchestra.run.smithi157.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-29T04:07:34.791 INFO:teuthology.orchestra.run.smithi157.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-29T04:07:34.817 INFO:teuthology.orchestra.run.smithi157.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 823 kB/s | 41 kB 00:00 2024-03-29T04:07:34.842 INFO:teuthology.orchestra.run.smithi157.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-03-29T04:07:34.859 INFO:teuthology.orchestra.run.smithi157.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 132 kB 00:00 2024-03-29T04:07:34.884 INFO:teuthology.orchestra.run.smithi157.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.9 MB/s | 211 kB 00:00 2024-03-29T04:07:34.901 INFO:teuthology.orchestra.run.smithi157.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-03-29T04:07:34.918 INFO:teuthology.orchestra.run.smithi157.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 1.2 MB/s | 43 kB 00:00 2024-03-29T04:07:35.085 INFO:teuthology.orchestra.run.smithi157.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 233 kB/s | 39 kB 00:00 2024-03-29T04:07:35.278 INFO:teuthology.orchestra.run.smithi157.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2024-03-29T04:07:35.294 INFO:teuthology.orchestra.run.smithi157.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 198 kB/s | 41 kB 00:00 2024-03-29T04:07:35.320 INFO:teuthology.orchestra.run.smithi157.stdout:(81/174): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2024-03-29T04:07:35.337 INFO:teuthology.orchestra.run.smithi157.stdout:(82/174): qatlib-service-24.02.0-1.el9.x86_64.r 833 kB/s | 35 kB 00:00 2024-03-29T04:07:35.353 INFO:teuthology.orchestra.run.smithi157.stdout:(83/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2024-03-29T04:07:35.379 INFO:teuthology.orchestra.run.smithi157.stdout:(84/174): socat-1.7.4.1-5.el9.x86_64.rpm 7.0 MB/s | 300 kB 00:00 2024-03-29T04:07:35.396 INFO:teuthology.orchestra.run.smithi157.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 56 kB 00:00 2024-03-29T04:07:35.429 INFO:teuthology.orchestra.run.smithi157.stdout:(86/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-29T04:07:35.447 INFO:teuthology.orchestra.run.smithi157.stdout:(87/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 876 kB/s | 44 kB 00:00 2024-03-29T04:07:35.472 INFO:teuthology.orchestra.run.smithi157.stdout:(88/174): fuse-2.9.9-15.el9.x86_64.rpm 1.8 MB/s | 79 kB 00:00 2024-03-29T04:07:35.497 INFO:teuthology.orchestra.run.smithi157.stdout:(89/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-29T04:07:35.531 INFO:teuthology.orchestra.run.smithi157.stdout:(90/174): python3-cryptography-36.0.1-4.el9.x86 20 MB/s | 1.1 MB 00:00 2024-03-29T04:07:35.556 INFO:teuthology.orchestra.run.smithi157.stdout:(91/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-03-29T04:07:35.590 INFO:teuthology.orchestra.run.smithi157.stdout:(92/174): python3-pycparser-2.20-6.el9.noarch.r 2.1 MB/s | 124 kB 00:00 2024-03-29T04:07:35.615 INFO:teuthology.orchestra.run.smithi157.stdout:(93/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 187 kB 00:00 2024-03-29T04:07:35.748 INFO:teuthology.orchestra.run.smithi157.stdout:(94/174): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-03-29T04:07:35.799 INFO:teuthology.orchestra.run.smithi157.stdout:(95/174): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-03-29T04:07:35.803 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-03-29T04:07:35.824 INFO:teuthology.orchestra.run.smithi157.stdout:(96/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 803 kB 00:00 2024-03-29T04:07:36.250 INFO:teuthology.orchestra.run.smithi157.stdout:(97/174): ceph-test-19.0.0-2520.g69e2cd73.el9.x 8.8 MB/s | 48 MB 00:05 2024-03-29T04:07:36.275 INFO:teuthology.orchestra.run.smithi157.stdout:(98/174): python3-requests-2.25.1-8.el9.noarch. 262 kB/s | 125 kB 00:00 2024-03-29T04:07:36.333 INFO:teuthology.orchestra.run.smithi157.stdout:(99/174): elfutils-libelf-devel-0.190-2.el9.x86 42 kB/s | 21 kB 00:00 2024-03-29T04:07:36.359 INFO:teuthology.orchestra.run.smithi157.stdout:(100/174): libuuid-devel-2.37.4-18.el9.x86_64.r 235 kB/s | 25 kB 00:00 2024-03-29T04:07:36.434 INFO:teuthology.orchestra.run.smithi157.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 2.4 MB/s | 245 kB 00:00 2024-03-29T04:07:36.459 INFO:teuthology.orchestra.run.smithi157.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 2.4 MB/s | 248 kB 00:00 2024-03-29T04:07:36.484 INFO:teuthology.orchestra.run.smithi157.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-03-29T04:07:36.510 INFO:teuthology.orchestra.run.smithi157.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-03-29T04:07:36.593 INFO:teuthology.orchestra.run.smithi157.stdout:(105/174): openssl-devel-3.0.7-27.el9.x86_64.rp 13 MB/s | 4.1 MB 00:00 2024-03-29T04:07:36.618 INFO:teuthology.orchestra.run.smithi157.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 6.3 MB/s | 863 kB 00:00 2024-03-29T04:07:36.713 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 26 MB/s | 21 MB 00:00 2024-03-29T04:07:36.744 INFO:teuthology.orchestra.run.smithi157.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 155 kB/s | 19 kB 00:00 2024-03-29T04:07:36.836 INFO:teuthology.orchestra.run.smithi157.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.6 MB/s | 548 kB 00:00 2024-03-29T04:07:36.861 INFO:teuthology.orchestra.run.smithi157.stdout:(109/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.1 MB/s | 309 kB 00:00 2024-03-29T04:07:36.887 INFO:teuthology.orchestra.run.smithi157.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 517 kB/s | 26 kB 00:00 2024-03-29T04:07:36.920 INFO:teuthology.orchestra.run.smithi157.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 828 kB/s | 49 kB 00:00 2024-03-29T04:07:36.945 INFO:teuthology.orchestra.run.smithi157.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-03-29T04:07:36.987 INFO:teuthology.orchestra.run.smithi157.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-03-29T04:07:37.046 INFO:teuthology.orchestra.run.smithi157.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.2 MB/s | 838 kB 00:00 2024-03-29T04:07:37.114 INFO:teuthology.orchestra.run.smithi157.stdout:(115/174): libarrow-9.0.0-10.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-03-29T04:07:37.138 INFO:teuthology.orchestra.run.smithi157.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 472 kB/s | 43 kB 00:00 2024-03-29T04:07:37.163 INFO:teuthology.orchestra.run.smithi157.stdout:(117/174): python3-asyncssh-2.13.2-5.el9.noarch 3.0 MB/s | 548 kB 00:00 2024-03-29T04:07:37.180 INFO:teuthology.orchestra.run.smithi157.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 482 kB/s | 32 kB 00:00 2024-03-29T04:07:37.197 INFO:teuthology.orchestra.run.smithi157.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 255 kB/s | 15 kB 00:00 2024-03-29T04:07:37.231 INFO:teuthology.orchestra.run.smithi157.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-03-29T04:07:37.256 INFO:teuthology.orchestra.run.smithi157.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-03-29T04:07:37.281 INFO:teuthology.orchestra.run.smithi157.stdout:(122/174): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-03-29T04:07:37.298 INFO:teuthology.orchestra.run.smithi157.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 526 kB/s | 35 kB 00:00 2024-03-29T04:07:37.324 INFO:teuthology.orchestra.run.smithi157.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 2.9 MB/s | 203 kB 00:00 2024-03-29T04:07:37.357 INFO:teuthology.orchestra.run.smithi157.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-03-29T04:07:37.382 INFO:teuthology.orchestra.run.smithi157.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 183 kB/s | 11 kB 00:00 2024-03-29T04:07:37.424 INFO:teuthology.orchestra.run.smithi157.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-03-29T04:07:37.450 INFO:teuthology.orchestra.run.smithi157.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-03-29T04:07:37.475 INFO:teuthology.orchestra.run.smithi157.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 251 kB/s | 23 kB 00:00 2024-03-29T04:07:37.500 INFO:teuthology.orchestra.run.smithi157.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-03-29T04:07:37.525 INFO:teuthology.orchestra.run.smithi157.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-03-29T04:07:37.559 INFO:teuthology.orchestra.run.smithi157.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-03-29T04:07:37.584 INFO:teuthology.orchestra.run.smithi157.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 552 kB/s | 46 kB 00:00 2024-03-29T04:07:37.609 INFO:teuthology.orchestra.run.smithi157.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 939 kB/s | 79 kB 00:00 2024-03-29T04:07:37.635 INFO:teuthology.orchestra.run.smithi157.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 766 kB/s | 58 kB 00:00 2024-03-29T04:07:37.660 INFO:teuthology.orchestra.run.smithi157.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-03-29T04:07:37.685 INFO:teuthology.orchestra.run.smithi157.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-03-29T04:07:37.710 INFO:teuthology.orchestra.run.smithi157.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-03-29T04:07:37.736 INFO:teuthology.orchestra.run.smithi157.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 373 kB/s | 28 kB 00:00 2024-03-29T04:07:37.761 INFO:teuthology.orchestra.run.smithi157.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 406 kB/s | 31 kB 00:00 2024-03-29T04:07:37.786 INFO:teuthology.orchestra.run.smithi157.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-03-29T04:07:37.912 INFO:teuthology.orchestra.run.smithi157.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 336 kB/s | 59 kB 00:00 2024-03-29T04:07:37.938 INFO:teuthology.orchestra.run.smithi157.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 203 kB/s | 36 kB 00:00 2024-03-29T04:07:37.963 INFO:teuthology.orchestra.run.smithi157.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 290 kB/s | 51 kB 00:00 2024-03-29T04:07:37.988 INFO:teuthology.orchestra.run.smithi157.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-03-29T04:07:38.013 INFO:teuthology.orchestra.run.smithi157.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-03-29T04:07:38.038 INFO:teuthology.orchestra.run.smithi157.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-03-29T04:07:38.064 INFO:teuthology.orchestra.run.smithi157.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 264 kB/s | 20 kB 00:00 2024-03-29T04:07:38.089 INFO:teuthology.orchestra.run.smithi157.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-03-29T04:07:38.134 INFO:teuthology.orchestra.run.smithi157.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-03-29T04:07:38.214 INFO:teuthology.orchestra.run.smithi157.stdout:(151/174): NetworkManager-1.47.2-1.el9.x86_64.r 27 MB/s | 2.3 MB 00:00 2024-03-29T04:07:38.281 INFO:teuthology.orchestra.run.smithi157.stdout:(152/174): NetworkManager-libnm-1.47.2-1.el9.x8 27 MB/s | 1.8 MB 00:00 2024-03-29T04:07:38.315 INFO:teuthology.orchestra.run.smithi157.stdout:(153/174): NetworkManager-team-1.47.2-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-03-29T04:07:38.357 INFO:teuthology.orchestra.run.smithi157.stdout:(154/174): NetworkManager-tui-1.47.2-1.el9.x86_ 5.8 MB/s | 247 kB 00:00 2024-03-29T04:07:38.399 INFO:teuthology.orchestra.run.smithi157.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-29T04:07:38.491 INFO:teuthology.orchestra.run.smithi157.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 779 kB/s | 71 kB 00:00 2024-03-29T04:07:38.541 INFO:teuthology.orchestra.run.smithi157.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 727 kB/s | 36 kB 00:00 2024-03-29T04:07:38.600 INFO:teuthology.orchestra.run.smithi157.stdout:(158/174): librados2-19.0.0-2520.g69e2cd73.el9. 6.3 MB/s | 3.4 MB 00:00 2024-03-29T04:07:38.625 INFO:teuthology.orchestra.run.smithi157.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.2 MB/s | 192 kB 00:00 2024-03-29T04:07:38.675 INFO:teuthology.orchestra.run.smithi157.stdout:(160/174): librbd1-19.0.0-2520.g69e2cd73.el9.x8 5.4 MB/s | 3.1 MB 00:00 2024-03-29T04:07:38.692 INFO:teuthology.orchestra.run.smithi157.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 344 kB/s | 23 kB 00:00 2024-03-29T04:07:38.742 INFO:teuthology.orchestra.run.smithi157.stdout:(162/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.8 MB/s | 256 kB 00:00 2024-03-29T04:07:38.768 INFO:teuthology.orchestra.run.smithi157.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 155 kB 00:00 2024-03-29T04:07:38.793 INFO:teuthology.orchestra.run.smithi157.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 136 kB 00:00 2024-03-29T04:07:38.810 INFO:teuthology.orchestra.run.smithi157.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.5 MB/s | 64 kB 00:00 2024-03-29T04:07:38.835 INFO:teuthology.orchestra.run.smithi157.stdout:(166/174): libblkid-2.37.4-18.el9.x86_64.rpm 679 kB/s | 108 kB 00:00 2024-03-29T04:07:38.852 INFO:teuthology.orchestra.run.smithi157.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 487 kB/s | 29 kB 00:00 2024-03-29T04:07:38.936 INFO:teuthology.orchestra.run.smithi157.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 9.6 MB/s | 1.2 MB 00:00 2024-03-29T04:07:39.011 INFO:teuthology.orchestra.run.smithi157.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 6.1 MB/s | 465 kB 00:00 2024-03-29T04:07:39.078 INFO:teuthology.orchestra.run.smithi157.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 467 kB/s | 31 kB 00:00 2024-03-29T04:07:39.154 INFO:teuthology.orchestra.run.smithi157.stdout:(171/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 14 MB/s | 1.1 MB 00:00 2024-03-29T04:07:39.195 INFO:teuthology.orchestra.run.smithi157.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 6.7 MB/s | 2.3 MB 00:00 2024-03-29T04:07:39.229 INFO:teuthology.orchestra.run.smithi157.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 989 kB/s | 74 kB 00:00 2024-03-29T04:07:39.337 INFO:teuthology.orchestra.run.smithi157.stdout:(174/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 4.3 MB/s | 2.2 MB 00:00 2024-03-29T04:07:39.341 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:07:39.341 INFO:teuthology.orchestra.run.smithi157.stdout:Total 20 MB/s | 226 MB 00:11 2024-03-29T04:07:39.710 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-03-29T04:07:39.985 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-03-29T04:07:39.985 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-03-29T04:07:41.654 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-03-29T04:07:42.008 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-03-29T04:07:42.008 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-03-29T04:07:44.226 INFO:teuthology.orchestra.run.smithi148.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-29T04:07:44.227 INFO:teuthology.orchestra.run.smithi148.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-29T04:07:44.241 INFO:teuthology.orchestra.run.smithi148.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-29T04:07:44.242 INFO:teuthology.orchestra.run.smithi148.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-29T04:07:44.243 INFO:teuthology.orchestra.run.smithi148.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-29T04:07:44.244 INFO:teuthology.orchestra.run.smithi148.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-29T04:07:44.246 INFO:teuthology.orchestra.run.smithi148.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-29T04:07:44.249 INFO:teuthology.orchestra.run.smithi148.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-29T04:07:44.250 INFO:teuthology.orchestra.run.smithi148.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-29T04:07:44.331 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T04:07:44.356 INFO:teuthology.orchestra.run.smithi148.stdout:============================================================================================ 2024-03-29T04:07:44.356 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-03-29T04:07:44.356 INFO:teuthology.orchestra.run.smithi148.stdout:============================================================================================ 2024-03-29T04:07:44.356 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-03-29T04:07:44.356 INFO:teuthology.orchestra.run.smithi148.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-29T04:07:44.357 INFO:teuthology.orchestra.run.smithi148.stdout: ceph x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 6.6 k 2024-03-29T04:07:44.357 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 5.4 M 2024-03-29T04:07:44.357 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 890 k 2024-03-29T04:07:44.357 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 145 k 2024-03-29T04:07:44.357 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 1.5 M 2024-03-29T04:07:44.357 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 141 k 2024-03-29T04:07:44.357 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 3.5 M 2024-03-29T04:07:44.357 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 7.4 M 2024-03-29T04:07:44.357 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 49 k 2024-03-29T04:07:44.357 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 9.8 M 2024-03-29T04:07:44.357 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 48 M 2024-03-29T04:07:44.357 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 264 k 2024-03-29T04:07:44.357 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 762 k 2024-03-29T04:07:44.358 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-mirror x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 221 k 2024-03-29T04:07:44.358 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-shell x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 27 k 2024-03-29T04:07:44.358 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-top noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 18 k 2024-03-29T04:07:44.358 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-29T04:07:44.358 INFO:teuthology.orchestra.run.smithi148.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-29T04:07:44.358 INFO:teuthology.orchestra.run.smithi148.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-29T04:07:44.358 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 32 k 2024-03-29T04:07:44.358 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 746 k 2024-03-29T04:07:44.358 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 127 k 2024-03-29T04:07:44.358 INFO:teuthology.orchestra.run.smithi148.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-29T04:07:44.358 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-29T04:07:44.358 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-29T04:07:44.358 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-03-29T04:07:44.359 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-03-29T04:07:44.359 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-03-29T04:07:44.359 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 162 k 2024-03-29T04:07:44.359 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 321 k 2024-03-29T04:07:44.359 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 298 k 2024-03-29T04:07:44.359 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 99 k 2024-03-29T04:07:44.359 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 85 k 2024-03-29T04:07:44.359 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 3.1 M 2024-03-29T04:07:44.359 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 170 k 2024-03-29T04:07:44.359 INFO:teuthology.orchestra.run.smithi148.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-29T04:07:44.359 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrading: 2024-03-29T04:07:44.359 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-03-29T04:07:44.359 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-03-29T04:07:44.359 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-03-29T04:07:44.360 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-03-29T04:07:44.360 INFO:teuthology.orchestra.run.smithi148.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-29T04:07:44.360 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-29T04:07:44.360 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-29T04:07:44.360 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-29T04:07:44.360 INFO:teuthology.orchestra.run.smithi148.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-29T04:07:44.360 INFO:teuthology.orchestra.run.smithi148.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-29T04:07:44.360 INFO:teuthology.orchestra.run.smithi148.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-29T04:07:44.360 INFO:teuthology.orchestra.run.smithi148.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-29T04:07:44.360 INFO:teuthology.orchestra.run.smithi148.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-29T04:07:44.360 INFO:teuthology.orchestra.run.smithi148.stdout: librados2 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 3.4 M 2024-03-29T04:07:44.360 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 3.1 M 2024-03-29T04:07:44.360 INFO:teuthology.orchestra.run.smithi148.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-29T04:07:44.360 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-29T04:07:44.361 INFO:teuthology.orchestra.run.smithi148.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-29T04:07:44.361 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-29T04:07:44.361 INFO:teuthology.orchestra.run.smithi148.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-29T04:07:44.361 INFO:teuthology.orchestra.run.smithi148.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-29T04:07:44.361 INFO:teuthology.orchestra.run.smithi148.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-29T04:07:44.361 INFO:teuthology.orchestra.run.smithi148.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-29T04:07:44.361 INFO:teuthology.orchestra.run.smithi148.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-29T04:07:44.361 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-03-29T04:07:44.361 INFO:teuthology.orchestra.run.smithi148.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-29T04:07:44.361 INFO:teuthology.orchestra.run.smithi148.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-29T04:07:44.361 INFO:teuthology.orchestra.run.smithi148.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-29T04:07:44.361 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 21 M 2024-03-29T04:07:44.361 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 34 k 2024-03-29T04:07:44.361 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 2.4 M 2024-03-29T04:07:44.362 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 249 k 2024-03-29T04:07:44.362 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 4.7 M 2024-03-29T04:07:44.362 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 17 M 2024-03-29T04:07:44.362 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 16 k 2024-03-29T04:07:44.362 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 25 k 2024-03-29T04:07:44.362 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-29T04:07:44.362 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-29T04:07:44.362 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-29T04:07:44.362 INFO:teuthology.orchestra.run.smithi148.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-29T04:07:44.362 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-29T04:07:44.362 INFO:teuthology.orchestra.run.smithi148.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-29T04:07:44.362 INFO:teuthology.orchestra.run.smithi148.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-29T04:07:44.362 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-29T04:07:44.363 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-29T04:07:44.363 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 163 k 2024-03-29T04:07:44.363 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-29T04:07:44.363 INFO:teuthology.orchestra.run.smithi148.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-29T04:07:44.363 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-29T04:07:44.363 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-29T04:07:44.363 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-29T04:07:44.363 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 500 k 2024-03-29T04:07:44.363 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-29T04:07:44.363 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 5.1 M 2024-03-29T04:07:44.363 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-29T04:07:44.363 INFO:teuthology.orchestra.run.smithi148.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-29T04:07:44.363 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-29T04:07:44.363 INFO:teuthology.orchestra.run.smithi148.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-29T04:07:44.364 INFO:teuthology.orchestra.run.smithi148.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-29T04:07:44.364 INFO:teuthology.orchestra.run.smithi148.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-29T04:07:44.364 INFO:teuthology.orchestra.run.smithi148.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-29T04:07:44.364 INFO:teuthology.orchestra.run.smithi148.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-29T04:07:44.364 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-29T04:07:44.364 INFO:teuthology.orchestra.run.smithi148.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-29T04:07:44.364 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-03-29T04:07:44.364 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-29T04:07:44.364 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-29T04:07:44.364 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-29T04:07:44.364 INFO:teuthology.orchestra.run.smithi148.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-29T04:07:44.364 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-29T04:07:44.364 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-29T04:07:44.364 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-29T04:07:44.365 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 45 k 2024-03-29T04:07:44.365 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 134 k 2024-03-29T04:07:44.365 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-29T04:07:44.365 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-29T04:07:44.365 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-29T04:07:44.365 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-29T04:07:44.365 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-29T04:07:44.365 INFO:teuthology.orchestra.run.smithi148.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-29T04:07:44.365 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-29T04:07:44.366 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-29T04:07:44.366 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-29T04:07:44.366 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-29T04:07:44.366 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-29T04:07:44.366 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-29T04:07:44.366 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-29T04:07:44.366 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-29T04:07:44.366 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-29T04:07:44.366 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-29T04:07:44.366 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-29T04:07:44.366 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-29T04:07:44.366 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-29T04:07:44.366 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-29T04:07:44.366 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-29T04:07:44.367 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-29T04:07:44.367 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-29T04:07:44.367 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-29T04:07:44.367 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-29T04:07:44.367 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-29T04:07:44.367 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-29T04:07:44.367 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-29T04:07:44.367 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-29T04:07:44.367 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-29T04:07:44.367 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-29T04:07:44.367 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-29T04:07:44.367 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-29T04:07:44.367 INFO:teuthology.orchestra.run.smithi148.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-29T04:07:44.367 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-29T04:07:44.367 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-29T04:07:44.368 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-29T04:07:44.368 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-29T04:07:44.368 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-29T04:07:44.368 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-29T04:07:44.368 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-29T04:07:44.368 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-29T04:07:44.368 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-29T04:07:44.368 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-29T04:07:44.368 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-29T04:07:44.368 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-29T04:07:44.368 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-29T04:07:44.368 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-29T04:07:44.368 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-29T04:07:44.369 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-29T04:07:44.369 INFO:teuthology.orchestra.run.smithi148.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-29T04:07:44.369 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-29T04:07:44.369 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-29T04:07:44.369 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-29T04:07:44.369 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-29T04:07:44.369 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-03-29T04:07:44.369 INFO:teuthology.orchestra.run.smithi148.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-03-29T04:07:44.369 INFO:teuthology.orchestra.run.smithi148.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-29T04:07:44.369 INFO:teuthology.orchestra.run.smithi148.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-29T04:07:44.369 INFO:teuthology.orchestra.run.smithi148.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-29T04:07:44.369 INFO:teuthology.orchestra.run.smithi148.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-29T04:07:44.369 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-29T04:07:44.369 INFO:teuthology.orchestra.run.smithi148.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-29T04:07:44.369 INFO:teuthology.orchestra.run.smithi148.stdout:Installing weak dependencies: 2024-03-29T04:07:44.370 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-03-29T04:07:44.370 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:07:44.370 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-03-29T04:07:44.370 INFO:teuthology.orchestra.run.smithi148.stdout:============================================================================================ 2024-03-29T04:07:44.370 INFO:teuthology.orchestra.run.smithi148.stdout:Install 150 Packages 2024-03-29T04:07:44.370 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrade 24 Packages 2024-03-29T04:07:44.370 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:07:44.371 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 226 M 2024-03-29T04:07:44.371 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-03-29T04:07:45.705 INFO:teuthology.orchestra.run.smithi148.stdout:(1/174): ceph-19.0.0-2520.g69e2cd73.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-03-29T04:07:46.089 INFO:teuthology.orchestra.run.smithi148.stdout:(2/174): ceph-fuse-19.0.0-2520.g69e2cd73.el9.x8 2.3 MB/s | 890 kB 00:00 2024-03-29T04:07:46.172 INFO:teuthology.orchestra.run.smithi148.stdout:(3/174): ceph-immutable-object-cache-19.0.0-252 1.7 MB/s | 145 kB 00:00 2024-03-29T04:07:46.748 INFO:teuthology.orchestra.run.smithi148.stdout:(4/174): ceph-mds-19.0.0-2520.g69e2cd73.el9.x86 4.1 MB/s | 2.4 MB 00:00 2024-03-29T04:07:46.973 INFO:teuthology.orchestra.run.smithi148.stdout:(5/174): ceph-common-19.0.0-2520.g69e2cd73.el9. 12 MB/s | 21 MB 00:01 2024-03-29T04:07:47.048 INFO:teuthology.orchestra.run.smithi148.stdout:(6/174): ceph-mgr-19.0.0-2520.g69e2cd73.el9.x86 5.0 MB/s | 1.5 MB 00:00 2024-03-29T04:07:47.323 INFO:teuthology.orchestra.run.smithi148.stdout:(7/174): ceph-mon-19.0.0-2520.g69e2cd73.el9.x86 13 MB/s | 4.7 MB 00:00 2024-03-29T04:07:47.974 INFO:teuthology.orchestra.run.smithi148.stdout:(8/174): ceph-radosgw-19.0.0-2520.g69e2cd73.el9 15 MB/s | 9.8 MB 00:00 2024-03-29T04:07:48.049 INFO:teuthology.orchestra.run.smithi148.stdout:(9/174): ceph-selinux-19.0.0-2520.g69e2cd73.el9 337 kB/s | 25 kB 00:00 2024-03-29T04:07:48.266 INFO:teuthology.orchestra.run.smithi148.stdout:(10/174): ceph-base-19.0.0-2520.g69e2cd73.el9.x 1.8 MB/s | 5.4 MB 00:02 2024-03-29T04:07:48.408 INFO:teuthology.orchestra.run.smithi148.stdout:(11/174): cephfs-mirror-19.0.0-2520.g69e2cd73.e 1.5 MB/s | 221 kB 00:00 2024-03-29T04:07:48.500 INFO:teuthology.orchestra.run.smithi148.stdout:(12/174): cephfs-shell-19.0.0-2520.g69e2cd73.el 294 kB/s | 27 kB 00:00 2024-03-29T04:07:48.575 INFO:teuthology.orchestra.run.smithi148.stdout:(13/174): libcephfs-devel-19.0.0-2520.g69e2cd73 423 kB/s | 32 kB 00:00 2024-03-29T04:07:48.934 INFO:teuthology.orchestra.run.smithi148.stdout:(14/174): libcephfs2-19.0.0-2520.g69e2cd73.el9. 2.0 MB/s | 746 kB 00:00 2024-03-29T04:07:49.034 INFO:teuthology.orchestra.run.smithi148.stdout:(15/174): libcephsqlite-19.0.0-2520.g69e2cd73.e 1.6 MB/s | 163 kB 00:00 2024-03-29T04:07:49.086 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-03-29T04:07:49.134 INFO:teuthology.orchestra.run.smithi148.stdout:(16/174): librados-devel-19.0.0-2520.g69e2cd73. 1.2 MB/s | 127 kB 00:00 2024-03-29T04:07:49.192 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-29T04:07:49.218 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-29T04:07:49.256 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-29T04:07:49.265 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-29T04:07:49.293 INFO:teuthology.orchestra.run.smithi148.stdout:(17/174): ceph-osd-19.0.0-2520.g69e2cd73.el9.x8 7.5 MB/s | 17 MB 00:02 2024-03-29T04:07:49.311 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-29T04:07:49.360 INFO:teuthology.orchestra.run.smithi148.stdout:(18/174): libradosstriper1-19.0.0-2520.g69e2cd7 2.2 MB/s | 500 kB 00:00 2024-03-29T04:07:49.367 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-29T04:07:49.413 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-29T04:07:49.433 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-29T04:07:49.443 INFO:teuthology.orchestra.run.smithi148.stdout:(19/174): python3-ceph-argparse-19.0.0-2520.g69 543 kB/s | 45 kB 00:00 2024-03-29T04:07:49.478 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-29T04:07:49.535 INFO:teuthology.orchestra.run.smithi148.stdout:(20/174): python3-ceph-common-19.0.0-2520.g69e2 1.4 MB/s | 134 kB 00:00 2024-03-29T04:07:49.627 INFO:teuthology.orchestra.run.smithi148.stdout:(21/174): python3-cephfs-19.0.0-2520.g69e2cd73. 1.7 MB/s | 162 kB 00:00 2024-03-29T04:07:49.803 INFO:teuthology.orchestra.run.smithi148.stdout:(22/174): python3-rados-19.0.0-2520.g69e2cd73.e 1.8 MB/s | 321 kB 00:00 2024-03-29T04:07:49.816 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-29T04:07:49.885 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librados2-2:19.0.0-2520.g69e2cd73.el9.x86_64 10/198 2024-03-29T04:07:49.978 INFO:teuthology.orchestra.run.smithi148.stdout:(23/174): librgw2-19.0.0-2520.g69e2cd73.el9.x86 7.5 MB/s | 5.1 MB 00:00 2024-03-29T04:07:50.003 INFO:teuthology.orchestra.run.smithi148.stdout:(24/174): python3-rbd-19.0.0-2520.g69e2cd73.el9 1.5 MB/s | 298 kB 00:00 2024-03-29T04:07:50.015 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:19.0.0-2520.g69e2cd73.el9.x86_64 10/198 2024-03-29T04:07:50.044 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs2-2:19.0.0-2520.g69e2cd73.el9.x86_64 11/198 2024-03-29T04:07:50.053 INFO:teuthology.orchestra.run.smithi148.stdout:(25/174): python3-rgw-19.0.0-2520.g69e2cd73.el9 1.3 MB/s | 99 kB 00:00 2024-03-29T04:07:50.070 INFO:teuthology.orchestra.run.smithi148.stdout:(26/174): rbd-fuse-19.0.0-2520.g69e2cd73.el9.x8 1.2 MB/s | 85 kB 00:00 2024-03-29T04:07:50.136 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephfs2-2:19.0.0-2520.g69e2cd73.el9.x86_64 11/198 2024-03-29T04:07:50.165 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rados-2:19.0.0-2520.g69e2cd73.el9.x86_64 12/198 2024-03-29T04:07:50.170 INFO:teuthology.orchestra.run.smithi148.stdout:(27/174): rbd-nbd-19.0.0-2520.g69e2cd73.el9.x86 1.7 MB/s | 170 kB 00:00 2024-03-29T04:07:50.201 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-29T04:07:50.295 INFO:teuthology.orchestra.run.smithi148.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2520.g 272 kB/s | 34 kB 00:00 2024-03-29T04:07:50.383 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-29T04:07:50.396 INFO:teuthology.orchestra.run.smithi148.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2520.g69e2cd7 1.4 MB/s | 141 kB 00:00 2024-03-29T04:07:50.446 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-29T04:07:50.474 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephsqlite-2:19.0.0-2520.g69e2cd73.el9.x86_64 16/198 2024-03-29T04:07:50.529 INFO:teuthology.orchestra.run.smithi148.stdout:(30/174): rbd-mirror-19.0.0-2520.g69e2cd73.el9. 6.5 MB/s | 3.1 MB 00:00 2024-03-29T04:07:50.561 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2520.g69e2cd73.el9.x86_64 16/198 2024-03-29T04:07:50.584 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libradosstriper1-2:19.0.0-2520.g69e2cd73.el9.x86 17/198 2024-03-29T04:07:50.665 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2520.g69e2cd73.el9.x86 17/198 2024-03-29T04:07:50.704 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-29T04:07:50.727 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-29T04:07:50.742 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-29T04:07:50.943 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-29T04:07:50.978 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-29T04:07:50.997 INFO:teuthology.orchestra.run.smithi148.stdout:(31/174): ceph-test-19.0.0-2520.g69e2cd73.el9.x 16 MB/s | 48 MB 00:02 2024-03-29T04:07:51.009 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-29T04:07:51.088 INFO:teuthology.orchestra.run.smithi148.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2520.g69 2.7 MB/s | 249 kB 00:00 2024-03-29T04:07:51.172 INFO:teuthology.orchestra.run.smithi148.stdout:(33/174): ceph-mgr-rook-19.0.0-2520.g69e2cd73.e 593 kB/s | 49 kB 00:00 2024-03-29T04:07:51.193 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-29T04:07:51.255 INFO:teuthology.orchestra.run.smithi148.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2520.g6 198 kB/s | 16 kB 00:00 2024-03-29T04:07:51.336 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-29T04:07:51.339 INFO:teuthology.orchestra.run.smithi148.stdout:(35/174): ceph-volume-19.0.0-2520.g69e2cd73.el9 3.1 MB/s | 264 kB 00:00 2024-03-29T04:07:51.396 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-29T04:07:51.423 INFO:teuthology.orchestra.run.smithi148.stdout:(36/174): ceph-mgr-diskprediction-local-19.0.0- 8.3 MB/s | 7.4 MB 00:00 2024-03-29T04:07:51.473 INFO:teuthology.orchestra.run.smithi148.stdout:(37/174): cephadm-19.0.0-2520.g69e2cd73.el9.noa 5.6 MB/s | 762 kB 00:00 2024-03-29T04:07:51.492 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-29T04:07:51.506 INFO:teuthology.orchestra.run.smithi148.stdout:(38/174): cephfs-top-19.0.0-2520.g69e2cd73.el9. 219 kB/s | 18 kB 00:00 2024-03-29T04:07:51.547 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-29T04:07:51.649 INFO:teuthology.orchestra.run.smithi148.stdout:(39/174): ceph-mgr-dashboard-19.0.0-2520.g69e2c 2.8 MB/s | 3.5 MB 00:01 2024-03-29T04:07:51.779 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-29T04:07:51.865 INFO:teuthology.orchestra.run.smithi148.stdout:(40/174): automake-1.16.2-8.el9.noarch.rpm 1.6 MB/s | 662 kB 00:00 2024-03-29T04:07:51.899 INFO:teuthology.orchestra.run.smithi148.stdout:(41/174): bison-3.7.4-5.el9.x86_64.rpm 2.5 MB/s | 920 kB 00:00 2024-03-29T04:07:51.900 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-29T04:07:51.916 INFO:teuthology.orchestra.run.smithi148.stdout:(42/174): boost-program-options-1.75.0-8.el9.x8 398 kB/s | 106 kB 00:00 2024-03-29T04:07:51.941 INFO:teuthology.orchestra.run.smithi148.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 4.0 MB/s | 307 kB 00:00 2024-03-29T04:07:51.966 INFO:teuthology.orchestra.run.smithi148.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 636 kB/s | 32 kB 00:00 2024-03-29T04:07:51.992 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-29T04:07:52.021 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-29T04:07:52.048 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-29T04:07:52.078 INFO:teuthology.orchestra.run.smithi148.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-03-29T04:07:52.093 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-29T04:07:52.100 INFO:teuthology.orchestra.run.smithi148.stdout:(46/174): flexiblas-openblas-openmp-3.0.4-8.el9 108 kB/s | 17 kB 00:00 2024-03-29T04:07:52.117 INFO:teuthology.orchestra.run.smithi148.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 67 kB/s | 10 kB 00:00 2024-03-29T04:07:52.142 INFO:teuthology.orchestra.run.smithi148.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 159 kB 00:00 2024-03-29T04:07:52.159 INFO:teuthology.orchestra.run.smithi148.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-29T04:07:52.166 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-29T04:07:52.184 INFO:teuthology.orchestra.run.smithi148.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 661 kB 00:00 2024-03-29T04:07:52.218 INFO:teuthology.orchestra.run.smithi148.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 578 kB 00:00 2024-03-29T04:07:52.219 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-29T04:07:52.235 INFO:teuthology.orchestra.run.smithi148.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 618 kB/s | 46 kB 00:00 2024-03-29T04:07:52.260 INFO:teuthology.orchestra.run.smithi148.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-29T04:07:52.266 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-29T04:07:52.294 INFO:teuthology.orchestra.run.smithi148.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 187 kB 00:00 2024-03-29T04:07:52.319 INFO:teuthology.orchestra.run.smithi148.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.6 MB/s | 217 kB 00:00 2024-03-29T04:07:52.330 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-29T04:07:52.336 INFO:teuthology.orchestra.run.smithi148.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 452 kB/s | 34 kB 00:00 2024-03-29T04:07:52.365 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-29T04:07:52.408 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-29T04:07:52.441 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-29T04:07:52.469 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-29T04:07:52.537 INFO:teuthology.orchestra.run.smithi148.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-03-29T04:07:52.539 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-29T04:07:52.570 INFO:teuthology.orchestra.run.smithi148.stdout:(58/174): postgresql-13.14-1.el9.x86_64.rpm 6.1 MB/s | 1.5 MB 00:00 2024-03-29T04:07:52.582 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-common-2:19.0.0-2520.g69e2cd73.el9. 39/198 2024-03-29T04:07:52.603 INFO:teuthology.orchestra.run.smithi148.stdout:(59/174): postgresql-contrib-13.14-1.el9.x86_64 3.0 MB/s | 817 kB 00:00 2024-03-29T04:07:52.626 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-argparse-2:19.0.0-2520.g69e2cd73.el 40/198 2024-03-29T04:07:52.629 INFO:teuthology.orchestra.run.smithi148.stdout:(60/174): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 132 kB 00:00 2024-03-29T04:07:52.654 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cephfs-2:19.0.0-2520.g69e2cd73.el9.x86_6 41/198 2024-03-29T04:07:52.725 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-29T04:07:52.754 INFO:teuthology.orchestra.run.smithi148.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.7 MB/s | 1.0 MB 00:00 2024-03-29T04:07:52.777 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-29T04:07:52.830 INFO:teuthology.orchestra.run.smithi148.stdout:(62/174): postgresql-server-13.14-1.el9.x86_64. 25 MB/s | 5.7 MB 00:00 2024-03-29T04:07:52.851 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-29T04:07:52.854 INFO:teuthology.orchestra.run.smithi148.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 347 kB/s | 78 kB 00:00 2024-03-29T04:07:52.963 INFO:teuthology.orchestra.run.smithi148.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 28 MB/s | 5.8 MB 00:00 2024-03-29T04:07:52.980 INFO:teuthology.orchestra.run.smithi148.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 199 kB/s | 24 kB 00:00 2024-03-29T04:07:52.983 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-29T04:07:52.997 INFO:teuthology.orchestra.run.smithi148.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 130 kB/s | 18 kB 00:00 2024-03-29T04:07:53.022 INFO:teuthology.orchestra.run.smithi148.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2024-03-29T04:07:53.038 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-29T04:07:53.039 INFO:teuthology.orchestra.run.smithi148.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 762 kB/s | 32 kB 00:00 2024-03-29T04:07:53.147 INFO:teuthology.orchestra.run.smithi148.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 4.8 MB 00:00 2024-03-29T04:07:53.173 INFO:teuthology.orchestra.run.smithi148.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 217 kB 00:00 2024-03-29T04:07:53.189 INFO:teuthology.orchestra.run.smithi148.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 174 kB 00:00 2024-03-29T04:07:53.206 INFO:teuthology.orchestra.run.smithi148.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 1.2 MB/s | 69 kB 00:00 2024-03-29T04:07:53.232 INFO:teuthology.orchestra.run.smithi148.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 990 kB/s | 41 kB 00:00 2024-03-29T04:07:53.257 INFO:teuthology.orchestra.run.smithi148.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-03-29T04:07:53.282 INFO:teuthology.orchestra.run.smithi148.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-29T04:07:53.316 INFO:teuthology.orchestra.run.smithi148.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.5 MB/s | 211 kB 00:00 2024-03-29T04:07:53.341 INFO:teuthology.orchestra.run.smithi148.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 569 kB/s | 47 kB 00:00 2024-03-29T04:07:53.366 INFO:teuthology.orchestra.run.smithi148.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 513 kB/s | 43 kB 00:00 2024-03-29T04:07:53.433 INFO:teuthology.orchestra.run.smithi148.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 420 kB/s | 39 kB 00:00 2024-03-29T04:07:53.550 INFO:teuthology.orchestra.run.smithi148.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 226 kB/s | 41 kB 00:00 2024-03-29T04:07:53.551 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-29T04:07:53.632 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-29T04:07:53.716 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-29T04:07:53.717 INFO:teuthology.orchestra.run.smithi148.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2024-03-29T04:07:53.742 INFO:teuthology.orchestra.run.smithi148.stdout:(82/174): qatlib-24.02.0-1.el9.x86_64.rpm 719 kB/s | 220 kB 00:00 2024-03-29T04:07:53.753 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-29T04:07:53.762 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-29T04:07:53.767 INFO:teuthology.orchestra.run.smithi148.stdout:(83/174): qatlib-service-24.02.0-1.el9.x86_64.r 162 kB/s | 35 kB 00:00 2024-03-29T04:07:53.793 INFO:teuthology.orchestra.run.smithi148.stdout:(84/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 621 kB/s | 46 kB 00:00 2024-03-29T04:07:53.793 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-29T04:07:53.818 INFO:teuthology.orchestra.run.smithi148.stdout:(85/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-03-29T04:07:53.842 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-29T04:07:53.843 INFO:teuthology.orchestra.run.smithi148.stdout:(86/174): uuid-1.6.2-55.el9.x86_64.rpm 750 kB/s | 56 kB 00:00 2024-03-29T04:07:53.868 INFO:teuthology.orchestra.run.smithi148.stdout:(87/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 836 kB/s | 63 kB 00:00 2024-03-29T04:07:53.880 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librados-devel-2:19.0.0-2520.g69e2cd73.el9.x86_6 53/198 2024-03-29T04:07:53.893 INFO:teuthology.orchestra.run.smithi148.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 880 kB/s | 44 kB 00:00 2024-03-29T04:07:53.920 INFO:teuthology.orchestra.run.smithi148.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-29T04:07:53.920 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-29T04:07:53.944 INFO:teuthology.orchestra.run.smithi148.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-03-29T04:07:53.969 INFO:teuthology.orchestra.run.smithi148.stdout:(91/174): python3-ply-3.11-14.el9.noarch.rpm 2.0 MB/s | 103 kB 00:00 2024-03-29T04:07:54.003 INFO:teuthology.orchestra.run.smithi148.stdout:(92/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-29T04:07:54.028 INFO:teuthology.orchestra.run.smithi148.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-03-29T04:07:54.053 INFO:teuthology.orchestra.run.smithi148.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.2 MB/s | 187 kB 00:00 2024-03-29T04:07:54.204 INFO:teuthology.orchestra.run.smithi148.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2024-03-29T04:07:54.211 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-29T04:07:54.229 INFO:teuthology.orchestra.run.smithi148.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-03-29T04:07:54.254 INFO:teuthology.orchestra.run.smithi148.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-03-29T04:07:54.304 INFO:teuthology.orchestra.run.smithi148.stdout:(98/174): elfutils-libelf-devel-0.190-2.el9.x86 287 kB/s | 21 kB 00:00 2024-03-29T04:07:54.330 INFO:teuthology.orchestra.run.smithi148.stdout:(99/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 338 kB/s | 25 kB 00:00 2024-03-29T04:07:54.363 INFO:teuthology.orchestra.run.smithi148.stdout:(100/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 803 kB 00:00 2024-03-29T04:07:54.369 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-29T04:07:54.461 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-29T04:07:54.472 INFO:teuthology.orchestra.run.smithi148.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 1.7 MB/s | 245 kB 00:00 2024-03-29T04:07:54.506 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-29T04:07:54.522 INFO:teuthology.orchestra.run.smithi148.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-03-29T04:07:54.564 INFO:teuthology.orchestra.run.smithi148.stdout:(103/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.2 MB/s | 248 kB 00:00 2024-03-29T04:07:54.645 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-29T04:07:54.714 INFO:teuthology.orchestra.run.smithi148.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 116 kB/s | 22 kB 00:00 2024-03-29T04:07:54.786 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-29T04:07:54.856 INFO:teuthology.orchestra.run.smithi148.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.9 MB/s | 863 kB 00:00 2024-03-29T04:07:54.957 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-29T04:07:55.037 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-29T04:07:55.068 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-29T04:07:55.104 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-29T04:07:55.132 INFO:teuthology.orchestra.run.smithi148.stdout:(106/174): openssl-devel-3.0.7-27.el9.x86_64.rp 5.0 MB/s | 4.1 MB 00:00 2024-03-29T04:07:55.151 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-29T04:07:55.195 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-29T04:07:55.246 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-29T04:07:55.289 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-29T04:07:55.298 INFO:teuthology.orchestra.run.smithi148.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 940 kB/s | 548 kB 00:00 2024-03-29T04:07:55.365 INFO:teuthology.orchestra.run.smithi148.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 607 kB/s | 309 kB 00:00 2024-03-29T04:07:55.382 INFO:teuthology.orchestra.run.smithi148.stdout:(109/174): grpc-data-1.46.7-10.el9.noarch.rpm 78 kB/s | 19 kB 00:00 2024-03-29T04:07:55.421 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-29T04:07:55.457 INFO:teuthology.orchestra.run.smithi148.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 283 kB/s | 26 kB 00:00 2024-03-29T04:07:55.469 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-29T04:07:55.499 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-29T04:07:55.541 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-29T04:07:55.558 INFO:teuthology.orchestra.run.smithi148.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 673 kB/s | 67 kB 00:00 2024-03-29T04:07:55.575 INFO:teuthology.orchestra.run.smithi148.stdout:(112/174): liboath-2.6.7-2.el9.x86_64.rpm 252 kB/s | 49 kB 00:00 2024-03-29T04:07:55.635 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-29T04:07:55.682 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-29T04:07:55.708 INFO:teuthology.orchestra.run.smithi148.stdout:(113/174): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-03-29T04:07:55.722 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-29T04:07:55.734 INFO:teuthology.orchestra.run.smithi148.stdout:(114/174): luarocks-3.9.2-1.el9.noarch.rpm 861 kB/s | 151 kB 00:00 2024-03-29T04:07:55.748 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-29T04:07:55.773 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-29T04:07:55.809 INFO:teuthology.orchestra.run.smithi148.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 5.4 MB/s | 548 kB 00:00 2024-03-29T04:07:55.826 INFO:teuthology.orchestra.run.smithi148.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 473 kB/s | 43 kB 00:00 2024-03-29T04:07:55.851 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-29T04:07:55.893 INFO:teuthology.orchestra.run.smithi148.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 386 kB/s | 32 kB 00:00 2024-03-29T04:07:55.918 INFO:teuthology.orchestra.run.smithi148.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-03-29T04:07:55.972 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-29T04:07:55.993 INFO:teuthology.orchestra.run.smithi148.stdout:(119/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.0 MB/s | 838 kB 00:00 2024-03-29T04:07:56.018 INFO:teuthology.orchestra.run.smithi148.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-03-29T04:07:56.069 INFO:teuthology.orchestra.run.smithi148.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 2.3 MB/s | 358 kB 00:00 2024-03-29T04:07:56.072 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-29T04:07:56.114 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-29T04:07:56.119 INFO:teuthology.orchestra.run.smithi148.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-03-29T04:07:56.144 INFO:teuthology.orchestra.run.smithi148.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 281 kB/s | 35 kB 00:00 2024-03-29T04:07:56.163 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-29T04:07:56.186 INFO:teuthology.orchestra.run.smithi148.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 1.7 MB/s | 203 kB 00:00 2024-03-29T04:07:56.218 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-29T04:07:56.237 INFO:teuthology.orchestra.run.smithi148.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-03-29T04:07:56.278 INFO:teuthology.orchestra.run.smithi148.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-03-29T04:07:56.345 INFO:teuthology.orchestra.run.smithi148.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 9.0 MB/s | 2.0 MB 00:00 2024-03-29T04:07:56.371 INFO:teuthology.orchestra.run.smithi148.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 132 kB/s | 18 kB 00:00 2024-03-29T04:07:56.396 INFO:teuthology.orchestra.run.smithi148.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 197 kB/s | 23 kB 00:00 2024-03-29T04:07:56.417 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-29T04:07:56.438 INFO:teuthology.orchestra.run.smithi148.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-03-29T04:07:56.460 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-29T04:07:56.463 INFO:teuthology.orchestra.run.smithi148.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-03-29T04:07:56.530 INFO:teuthology.orchestra.run.smithi148.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 504 kB/s | 46 kB 00:00 2024-03-29T04:07:56.555 INFO:teuthology.orchestra.run.smithi148.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 858 kB/s | 79 kB 00:00 2024-03-29T04:07:56.587 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-29T04:07:56.622 INFO:teuthology.orchestra.run.smithi148.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 4.5 MB/s | 1.0 MB 00:00 2024-03-29T04:07:56.625 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-29T04:07:56.647 INFO:teuthology.orchestra.run.smithi148.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 492 kB/s | 58 kB 00:00 2024-03-29T04:07:56.657 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-29T04:07:56.673 INFO:teuthology.orchestra.run.smithi148.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-03-29T04:07:56.688 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-29T04:07:56.706 INFO:teuthology.orchestra.run.smithi148.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-03-29T04:07:56.715 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-29T04:07:56.740 INFO:teuthology.orchestra.run.smithi148.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 976 kB/s | 90 kB 00:00 2024-03-29T04:07:56.758 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-29T04:07:56.773 INFO:teuthology.orchestra.run.smithi148.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 281 kB/s | 28 kB 00:00 2024-03-29T04:07:56.797 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-29T04:07:56.799 INFO:teuthology.orchestra.run.smithi148.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 333 kB/s | 31 kB 00:00 2024-03-29T04:07:56.829 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-29T04:07:56.841 INFO:teuthology.orchestra.run.smithi148.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-03-29T04:07:56.858 INFO:teuthology.orchestra.run.smithi148.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 702 kB/s | 59 kB 00:00 2024-03-29T04:07:56.859 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-29T04:07:56.883 INFO:teuthology.orchestra.run.smithi148.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 426 kB/s | 36 kB 00:00 2024-03-29T04:07:56.925 INFO:teuthology.orchestra.run.smithi148.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 606 kB/s | 51 kB 00:00 2024-03-29T04:07:56.934 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-29T04:07:56.959 INFO:teuthology.orchestra.run.smithi148.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-03-29T04:07:56.969 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-29T04:07:56.984 INFO:teuthology.orchestra.run.smithi148.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 892 kB/s | 90 kB 00:00 2024-03-29T04:07:57.002 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-29T04:07:57.026 INFO:teuthology.orchestra.run.smithi148.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-03-29T04:07:57.035 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-29T04:07:57.051 INFO:teuthology.orchestra.run.smithi148.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-03-29T04:07:57.070 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-29T04:07:57.085 INFO:teuthology.orchestra.run.smithi148.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-03-29T04:07:57.159 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-29T04:07:57.185 INFO:teuthology.orchestra.run.smithi148.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-03-29T04:07:57.240 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-29T04:07:57.285 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-29T04:07:57.316 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-29T04:07:57.352 INFO:teuthology.orchestra.run.smithi148.stdout:(151/174): NetworkManager-1.47.2-1.el9.x86_64.r 14 MB/s | 2.3 MB 00:00 2024-03-29T04:07:57.362 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-29T04:07:57.419 INFO:teuthology.orchestra.run.smithi148.stdout:(152/174): NetworkManager-libnm-1.47.2-1.el9.x8 27 MB/s | 1.8 MB 00:00 2024-03-29T04:07:57.444 INFO:teuthology.orchestra.run.smithi148.stdout:(153/174): NetworkManager-team-1.47.2-1.el9.x86 1.6 MB/s | 41 kB 00:00 2024-03-29T04:07:57.459 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-29T04:07:57.486 INFO:teuthology.orchestra.run.smithi148.stdout:(154/174): NetworkManager-tui-1.47.2-1.el9.x86_ 5.8 MB/s | 247 kB 00:00 2024-03-29T04:07:57.517 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-29T04:07:57.528 INFO:teuthology.orchestra.run.smithi148.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.5 MB/s | 321 kB 00:00 2024-03-29T04:07:57.637 INFO:teuthology.orchestra.run.smithi148.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 658 kB/s | 71 kB 00:00 2024-03-29T04:07:57.670 INFO:teuthology.orchestra.run.smithi148.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 1.1 MB/s | 36 kB 00:00 2024-03-29T04:07:57.720 INFO:teuthology.orchestra.run.smithi148.stdout:(158/174): librados2-19.0.0-2520.g69e2cd73.el9. 5.1 MB/s | 3.4 MB 00:00 2024-03-29T04:07:57.770 INFO:teuthology.orchestra.run.smithi148.stdout:(159/174): librbd1-19.0.0-2520.g69e2cd73.el9.x8 4.6 MB/s | 3.1 MB 00:00 2024-03-29T04:07:57.796 INFO:teuthology.orchestra.run.smithi148.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.5 MB/s | 192 kB 00:00 2024-03-29T04:07:57.846 INFO:teuthology.orchestra.run.smithi148.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-03-29T04:07:57.872 INFO:teuthology.orchestra.run.smithi148.stdout:(162/174): libacl-2.3.1-4.el9.x86_64.rpm 229 kB/s | 23 kB 00:00 2024-03-29T04:07:57.897 INFO:teuthology.orchestra.run.smithi148.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-03-29T04:07:57.922 INFO:teuthology.orchestra.run.smithi148.stdout:(164/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.2 MB/s | 256 kB 00:00 2024-03-29T04:07:57.947 INFO:teuthology.orchestra.run.smithi148.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-03-29T04:07:57.972 INFO:teuthology.orchestra.run.smithi148.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 573 kB/s | 29 kB 00:00 2024-03-29T04:07:57.998 INFO:teuthology.orchestra.run.smithi148.stdout:(167/174): libmount-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 136 kB 00:00 2024-03-29T04:07:58.090 INFO:teuthology.orchestra.run.smithi148.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 8.5 MB/s | 1.2 MB 00:00 2024-03-29T04:07:58.157 INFO:teuthology.orchestra.run.smithi148.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 6.8 MB/s | 465 kB 00:00 2024-03-29T04:07:58.223 INFO:teuthology.orchestra.run.smithi148.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 8.6 MB/s | 2.2 MB 00:00 2024-03-29T04:07:58.240 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-29T04:07:58.249 INFO:teuthology.orchestra.run.smithi148.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 339 kB/s | 31 kB 00:00 2024-03-29T04:07:58.281 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-29T04:07:58.290 INFO:teuthology.orchestra.run.smithi148.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 7.8 MB/s | 2.3 MB 00:00 2024-03-29T04:07:58.323 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-29T04:07:58.333 INFO:teuthology.orchestra.run.smithi148.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 890 kB/s | 74 kB 00:00 2024-03-29T04:07:58.370 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-29T04:07:58.408 INFO:teuthology.orchestra.run.smithi148.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.8 MB/s | 1.1 MB 00:00 2024-03-29T04:07:58.412 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:07:58.412 INFO:teuthology.orchestra.run.smithi148.stdout:Total 16 MB/s | 226 MB 00:14 2024-03-29T04:07:58.449 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-29T04:07:58.491 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-29T04:07:58.669 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-29T04:07:58.713 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-29T04:07:58.776 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-03-29T04:07:58.791 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-29T04:07:59.053 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-03-29T04:07:59.053 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-03-29T04:07:59.424 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-29T04:07:59.551 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-29T04:08:00.764 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-29T04:08:00.847 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-29T04:08:00.878 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-29T04:08:00.909 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-29T04:08:00.936 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-29T04:08:01.079 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-03-29T04:08:01.080 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-03-29T04:08:01.251 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-29T04:08:01.283 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librbd1-2:19.0.0-2520.g69e2cd73.el9.x86_64 124/198 2024-03-29T04:08:01.365 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:19.0.0-2520.g69e2cd73.el9.x86_64 124/198 2024-03-29T04:08:01.396 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 125/198 2024-03-29T04:08:01.432 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-29T04:08:01.900 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-29T04:08:02.078 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librgw2-2:19.0.0-2520.g69e2cd73.el9.x86_64 128/198 2024-03-29T04:08:02.366 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librgw2-2:19.0.0-2520.g69e2cd73.el9.x86_64 128/198 2024-03-29T04:08:02.573 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 129/198 2024-03-29T04:08:04.811 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 130/198 2024-03-29T04:08:04.850 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 130/198 2024-03-29T04:08:05.404 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 130/198 2024-03-29T04:08:05.437 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-base-2:19.0.0-2520.g69e2cd73.el9.x86_64 131/198 2024-03-29T04:08:05.511 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-base-2:19.0.0-2520.g69e2cd73.el9.x86_64 131/198 2024-03-29T04:08:05.511 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-29T04:08:05.511 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-29T04:08:05.511 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:08:05.547 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-selinux-2:19.0.0-2520.g69e2cd73.el9.x86_64 132/198 2024-03-29T04:08:07.988 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-03-29T04:08:08.070 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-29T04:08:08.105 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-29T04:08:08.143 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-29T04:08:08.153 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-29T04:08:08.201 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-29T04:08:08.255 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-29T04:08:08.291 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-29T04:08:08.317 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-29T04:08:08.364 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-29T04:08:08.696 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-29T04:08:08.759 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librados2-2:19.0.0-2520.g69e2cd73.el9.x86_64 10/198 2024-03-29T04:08:08.873 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:19.0.0-2520.g69e2cd73.el9.x86_64 10/198 2024-03-29T04:08:08.903 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs2-2:19.0.0-2520.g69e2cd73.el9.x86_64 11/198 2024-03-29T04:08:08.992 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephfs2-2:19.0.0-2520.g69e2cd73.el9.x86_64 11/198 2024-03-29T04:08:09.023 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rados-2:19.0.0-2520.g69e2cd73.el9.x86_64 12/198 2024-03-29T04:08:09.064 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-29T04:08:09.239 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-29T04:08:09.295 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-29T04:08:09.315 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephsqlite-2:19.0.0-2520.g69e2cd73.el9.x86_64 16/198 2024-03-29T04:08:09.401 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2520.g69e2cd73.el9.x86_64 16/198 2024-03-29T04:08:09.425 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libradosstriper1-2:19.0.0-2520.g69e2cd73.el9.x86 17/198 2024-03-29T04:08:09.481 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2520.g69e2cd73.el9.x86 17/198 2024-03-29T04:08:09.520 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-29T04:08:09.543 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-29T04:08:09.558 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-29T04:08:09.752 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-29T04:08:09.783 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-29T04:08:09.831 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-29T04:08:10.022 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-29T04:08:10.158 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-29T04:08:10.201 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-29T04:08:10.297 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-29T04:08:10.345 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-29T04:08:10.562 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-29T04:08:10.667 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-29T04:08:10.762 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-29T04:08:10.791 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-29T04:08:10.818 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-29T04:08:10.862 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-29T04:08:10.935 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-29T04:08:10.989 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-29T04:08:11.028 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-29T04:08:11.091 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-29T04:08:11.127 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-29T04:08:11.170 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-29T04:08:11.204 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-29T04:08:11.231 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-29T04:08:11.317 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-29T04:08:11.353 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-common-2:19.0.0-2520.g69e2cd73.el9. 39/198 2024-03-29T04:08:11.388 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-argparse-2:19.0.0-2520.g69e2cd73.el 40/198 2024-03-29T04:08:11.416 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cephfs-2:19.0.0-2520.g69e2cd73.el9.x86_6 41/198 2024-03-29T04:08:11.484 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-29T04:08:11.531 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-29T04:08:11.604 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-29T04:08:11.736 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-29T04:08:11.791 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-29T04:08:12.310 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-29T04:08:12.386 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-29T04:08:12.485 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-29T04:08:12.531 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-29T04:08:12.540 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-29T04:08:12.580 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-29T04:08:12.629 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-29T04:08:12.658 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librados-devel-2:19.0.0-2520.g69e2cd73.el9.x86_6 53/198 2024-03-29T04:08:12.689 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-29T04:08:13.054 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-29T04:08:13.260 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-29T04:08:13.341 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-29T04:08:13.384 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-29T04:08:13.766 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-29T04:08:13.815 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-29T04:08:13.870 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-29T04:08:13.942 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-29T04:08:13.974 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-29T04:08:14.010 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-29T04:08:14.065 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-29T04:08:14.101 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-29T04:08:14.143 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-29T04:08:14.186 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-29T04:08:14.318 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-29T04:08:14.374 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-29T04:08:14.405 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-29T04:08:14.446 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-29T04:08:14.540 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-29T04:08:14.580 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-29T04:08:14.620 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-29T04:08:14.645 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-29T04:08:14.670 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-29T04:08:14.747 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-29T04:08:14.871 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-29T04:08:14.978 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-29T04:08:15.020 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-29T04:08:15.068 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-29T04:08:15.124 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-29T04:08:15.309 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-29T04:08:15.357 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-29T04:08:15.487 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-29T04:08:15.530 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-29T04:08:15.562 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-29T04:08:15.593 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-29T04:08:15.638 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-29T04:08:15.672 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-29T04:08:15.711 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-29T04:08:15.743 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-29T04:08:15.773 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-29T04:08:15.849 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-29T04:08:15.883 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-29T04:08:15.916 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-29T04:08:15.958 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-29T04:08:15.992 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-29T04:08:16.112 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-29T04:08:16.197 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-29T04:08:16.248 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-29T04:08:16.280 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-29T04:08:16.326 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-29T04:08:16.423 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-29T04:08:16.473 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-29T04:08:17.038 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2520.g69e2cd73.el9.x86_64 132/198 2024-03-29T04:08:17.038 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /sys 2024-03-29T04:08:17.038 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /proc 2024-03-29T04:08:17.039 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /mnt 2024-03-29T04:08:17.039 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /var/tmp 2024-03-29T04:08:17.039 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /home 2024-03-29T04:08:17.039 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /root 2024-03-29T04:08:17.039 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /tmp 2024-03-29T04:08:17.039 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:08:17.196 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-29T04:08:17.201 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-volume-2:19.0.0-2520.g69e2cd73.el9.noarch 133/198 2024-03-29T04:08:17.217 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-volume-2:19.0.0-2520.g69e2cd73.el9.noarch 133/198 2024-03-29T04:08:17.217 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:08:17.217 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-29T04:08:17.217 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:08:17.245 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-29T04:08:17.279 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-29T04:08:17.335 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-29T04:08:17.414 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-29T04:08:17.455 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-29T04:08:17.633 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-29T04:08:17.661 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-29T04:08:17.742 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-29T04:08:18.389 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-29T04:08:18.498 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-29T04:08:18.712 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-osd-2:19.0.0-2520.g69e2cd73.el9.x86_64 134/198 2024-03-29T04:08:18.738 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-osd-2:19.0.0-2520.g69e2cd73.el9.x86_64 134/198 2024-03-29T04:08:18.738 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:08:18.738 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-29T04:08:18.738 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-29T04:08:18.738 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-29T04:08:18.738 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:08:18.963 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mds-2:19.0.0-2520.g69e2cd73.el9.x86_64 135/198 2024-03-29T04:08:18.986 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mds-2:19.0.0-2520.g69e2cd73.el9.x86_64 135/198 2024-03-29T04:08:18.986 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:08:18.986 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-29T04:08:18.986 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-29T04:08:18.986 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-29T04:08:18.986 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:08:19.430 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mon-2:19.0.0-2520.g69e2cd73.el9.x86_64 136/198 2024-03-29T04:08:19.453 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mon-2:19.0.0-2520.g69e2cd73.el9.x86_64 136/198 2024-03-29T04:08:19.453 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:08:19.453 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-29T04:08:19.453 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-29T04:08:19.453 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-29T04:08:19.453 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:08:19.542 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-29T04:08:19.721 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-29T04:08:19.803 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-29T04:08:19.833 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-29T04:08:19.865 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-29T04:08:19.892 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-29T04:08:19.959 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 138/198 2024-03-29T04:08:19.989 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 138/198 2024-03-29T04:08:20.021 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2520.g69e2cd73.e 139/198 2024-03-29T04:08:20.138 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2520.g69e2cd73. 140/198 2024-03-29T04:08:20.138 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2520.g69e2cd73.el9.noa 141/198 2024-03-29T04:08:20.205 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-29T04:08:20.247 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librbd1-2:19.0.0-2520.g69e2cd73.el9.x86_64 124/198 2024-03-29T04:08:20.320 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:19.0.0-2520.g69e2cd73.el9.x86_64 124/198 2024-03-29T04:08:20.343 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 125/198 2024-03-29T04:08:20.379 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-29T04:08:20.840 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-29T04:08:20.867 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librgw2-2:19.0.0-2520.g69e2cd73.el9.x86_64 128/198 2024-03-29T04:08:20.937 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librgw2-2:19.0.0-2520.g69e2cd73.el9.x86_64 128/198 2024-03-29T04:08:20.960 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 129/198 2024-03-29T04:08:21.149 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2520.g69e2cd73.el9.noa 141/198 2024-03-29T04:08:21.197 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2520.g69e2cd73.el9.n 142/198 2024-03-29T04:08:22.163 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2520.g69e2cd73.el9.n 142/198 2024-03-29T04:08:22.197 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2520.g69e 143/198 2024-03-29T04:08:22.298 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2520.g69e 143/198 2024-03-29T04:08:22.461 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2520.g69e2cd73.el 144/198 2024-03-29T04:08:22.491 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-2:19.0.0-2520.g69e2cd73.el9.x86_64 145/198 2024-03-29T04:08:22.514 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2520.g69e2cd73.el9.x86_64 145/198 2024-03-29T04:08:22.514 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:08:22.514 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-29T04:08:22.514 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-29T04:08:22.515 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-29T04:08:22.515 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:08:22.558 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-rook-2:19.0.0-2520.g69e2cd73.el9.noarch 146/198 2024-03-29T04:08:22.569 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2520.g69e2cd73.el9.noarch 146/198 2024-03-29T04:08:22.624 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-2:19.0.0-2520.g69e2cd73.el9.x86_64 147/198 2024-03-29T04:08:23.136 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 130/198 2024-03-29T04:08:23.173 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 130/198 2024-03-29T04:08:23.506 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-29T04:08:23.544 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-radosgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 149/198 2024-03-29T04:08:23.567 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 149/198 2024-03-29T04:08:23.567 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:08:23.567 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-29T04:08:23.567 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-29T04:08:23.567 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-29T04:08:23.567 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:08:23.611 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2520.g69e2c 150/198 2024-03-29T04:08:23.633 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2520.g69e2c 150/198 2024-03-29T04:08:23.633 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:08:23.633 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-29T04:08:23.633 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:08:23.678 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 151/198 2024-03-29T04:08:23.701 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 151/198 2024-03-29T04:08:23.701 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:08:23.701 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-29T04:08:23.701 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-29T04:08:23.701 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-29T04:08:23.701 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:08:23.722 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 130/198 2024-03-29T04:08:23.760 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-base-2:19.0.0-2520.g69e2cd73.el9.x86_64 131/198 2024-03-29T04:08:23.850 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-base-2:19.0.0-2520.g69e2cd73.el9.x86_64 131/198 2024-03-29T04:08:23.850 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-29T04:08:23.850 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-29T04:08:23.850 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:08:23.894 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-selinux-2:19.0.0-2520.g69e2cd73.el9.x86_64 132/198 2024-03-29T04:08:24.013 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 152/198 2024-03-29T04:08:24.035 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 152/198 2024-03-29T04:08:24.035 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:08:24.035 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-29T04:08:24.035 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-29T04:08:24.035 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-29T04:08:24.035 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:08:28.575 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-test-2:19.0.0-2520.g69e2cd73.el9.x86_64 153/198 2024-03-29T04:08:28.616 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 154/198 2024-03-29T04:08:28.657 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-nbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 155/198 2024-03-29T04:08:28.682 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-29T04:08:28.867 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-shell-2:19.0.0-2520.g69e2cd73.el9.x86_64 157/198 2024-03-29T04:08:28.983 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-29T04:08:29.030 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 159/198 2024-03-29T04:08:29.087 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-29T04:08:29.138 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-29T04:08:29.181 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-29T04:08:29.214 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs-devel-2:19.0.0-2520.g69e2cd73.el9.x86_ 163/198 2024-03-29T04:08:29.254 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-29T04:08:29.293 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-29T04:08:29.347 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-29T04:08:29.412 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-29T04:08:29.449 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-29T04:08:30.016 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-29T04:08:30.065 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-29T04:08:30.086 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-29T04:08:30.904 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-top-2:19.0.0-2520.g69e2cd73.el9.noarch 170/198 2024-03-29T04:08:30.977 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-29T04:08:31.022 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-29T04:08:31.109 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-29T04:08:31.139 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-29T04:08:31.165 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-29T04:08:31.190 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-29T04:08:31.216 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-29T04:08:31.216 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-29T04:08:31.304 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-29T04:08:31.334 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-29T04:08:31.334 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-29T04:08:31.418 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-29T04:08:31.444 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-29T04:08:31.486 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-29T04:08:31.511 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-29T04:08:31.536 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-29T04:08:31.572 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-29T04:08:31.618 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-29T04:08:31.649 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-29T04:08:31.716 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-29T04:08:31.741 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-29T04:08:31.767 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-29T04:08:31.792 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-29T04:08:31.799 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-29T04:08:31.799 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-29T04:08:31.978 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-29T04:08:31.978 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-29T04:08:32.053 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-29T04:08:32.078 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-29T04:08:32.110 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-29T04:08:32.137 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-29T04:08:32.162 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-29T04:08:32.228 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-29T04:08:35.605 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2520.g69e2cd73.el9.x86_64 132/198 2024-03-29T04:08:35.606 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /sys 2024-03-29T04:08:35.606 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /proc 2024-03-29T04:08:35.606 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /mnt 2024-03-29T04:08:35.606 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /var/tmp 2024-03-29T04:08:35.606 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /home 2024-03-29T04:08:35.606 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /root 2024-03-29T04:08:35.607 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /tmp 2024-03-29T04:08:35.607 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:08:35.765 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-volume-2:19.0.0-2520.g69e2cd73.el9.noarch 133/198 2024-03-29T04:08:35.780 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-volume-2:19.0.0-2520.g69e2cd73.el9.noarch 133/198 2024-03-29T04:08:35.780 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:08:35.780 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-29T04:08:35.780 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:08:37.277 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-osd-2:19.0.0-2520.g69e2cd73.el9.x86_64 134/198 2024-03-29T04:08:37.302 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-osd-2:19.0.0-2520.g69e2cd73.el9.x86_64 134/198 2024-03-29T04:08:37.303 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:08:37.303 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-29T04:08:37.303 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-29T04:08:37.303 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-29T04:08:37.303 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:08:37.527 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mds-2:19.0.0-2520.g69e2cd73.el9.x86_64 135/198 2024-03-29T04:08:37.551 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mds-2:19.0.0-2520.g69e2cd73.el9.x86_64 135/198 2024-03-29T04:08:37.551 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:08:37.552 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-29T04:08:37.552 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-29T04:08:37.552 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-29T04:08:37.552 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:08:37.556 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-29T04:08:37.556 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-2:19.0.0-2520.g69e2cd73.el9.x86_64 1/198 2024-03-29T04:08:37.556 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-base-2:19.0.0-2520.g69e2cd73.el9.x86_64 2/198 2024-03-29T04:08:37.556 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 3/198 2024-03-29T04:08:37.556 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 4/198 2024-03-29T04:08:37.556 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2520.g69e2c 5/198 2024-03-29T04:08:37.556 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mds-2:19.0.0-2520.g69e2cd73.el9.x86_64 6/198 2024-03-29T04:08:37.556 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-2:19.0.0-2520.g69e2cd73.el9.x86_64 7/198 2024-03-29T04:08:37.556 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mon-2:19.0.0-2520.g69e2cd73.el9.x86_64 8/198 2024-03-29T04:08:37.556 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-osd-2:19.0.0-2520.g69e2cd73.el9.x86_64 9/198 2024-03-29T04:08:37.556 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-radosgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 10/198 2024-03-29T04:08:37.557 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-selinux-2:19.0.0-2520.g69e2cd73.el9.x86_64 11/198 2024-03-29T04:08:37.557 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-test-2:19.0.0-2520.g69e2cd73.el9.x86_64 12/198 2024-03-29T04:08:37.557 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 13/198 2024-03-29T04:08:37.557 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-shell-2:19.0.0-2520.g69e2cd73.el9.x86_64 14/198 2024-03-29T04:08:37.557 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs-devel-2:19.0.0-2520.g69e2cd73.el9.x86_ 15/198 2024-03-29T04:08:37.557 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs2-2:19.0.0-2520.g69e2cd73.el9.x86_64 16/198 2024-03-29T04:08:37.557 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephsqlite-2:19.0.0-2520.g69e2cd73.el9.x86_64 17/198 2024-03-29T04:08:37.557 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados-devel-2:19.0.0-2520.g69e2cd73.el9.x86_6 18/198 2024-03-29T04:08:37.557 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libradosstriper1-2:19.0.0-2520.g69e2cd73.el9.x86 19/198 2024-03-29T04:08:37.557 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librgw2-2:19.0.0-2520.g69e2cd73.el9.x86_64 20/198 2024-03-29T04:08:37.557 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2520.g69e2cd73.el 21/198 2024-03-29T04:08:37.557 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-common-2:19.0.0-2520.g69e2cd73.el9. 22/198 2024-03-29T04:08:37.557 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cephfs-2:19.0.0-2520.g69e2cd73.el9.x86_6 23/198 2024-03-29T04:08:37.557 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rados-2:19.0.0-2520.g69e2cd73.el9.x86_64 24/198 2024-03-29T04:08:37.558 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 25/198 2024-03-29T04:08:37.558 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 26/198 2024-03-29T04:08:37.558 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 27/198 2024-03-29T04:08:37.558 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 28/198 2024-03-29T04:08:37.558 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-nbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 29/198 2024-03-29T04:08:37.558 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2520.g69e2cd73. 30/198 2024-03-29T04:08:37.558 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2520.g69e2cd73.el9.noa 31/198 2024-03-29T04:08:37.558 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2520.g69e2cd73.el9.n 32/198 2024-03-29T04:08:37.558 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2520.g69e 33/198 2024-03-29T04:08:37.558 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2520.g69e2cd73.el 34/198 2024-03-29T04:08:37.558 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2520.g69e2cd73.el9.noarch 35/198 2024-03-29T04:08:37.558 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2520.g69e2cd73.e 36/198 2024-03-29T04:08:37.558 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-volume-2:19.0.0-2520.g69e2cd73.el9.noarch 37/198 2024-03-29T04:08:37.558 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 38/198 2024-03-29T04:08:37.559 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-top-2:19.0.0-2520.g69e2cd73.el9.noarch 39/198 2024-03-29T04:08:37.559 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-29T04:08:37.559 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-29T04:08:37.559 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-29T04:08:37.559 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-29T04:08:37.559 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-29T04:08:37.559 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-29T04:08:37.559 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-29T04:08:37.559 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-29T04:08:37.559 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-29T04:08:37.559 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-29T04:08:37.559 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-29T04:08:37.559 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-29T04:08:37.559 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-29T04:08:37.560 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-29T04:08:37.560 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-29T04:08:37.560 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-29T04:08:37.560 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-29T04:08:37.560 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-29T04:08:37.560 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-03-29T04:08:37.560 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-03-29T04:08:37.560 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-03-29T04:08:37.560 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-03-29T04:08:37.560 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-29T04:08:37.560 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-29T04:08:37.560 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-29T04:08:37.560 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-29T04:08:37.560 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-29T04:08:37.561 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-29T04:08:37.561 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-29T04:08:37.561 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-29T04:08:37.561 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-29T04:08:37.561 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-29T04:08:37.561 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-29T04:08:37.561 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-29T04:08:37.561 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-29T04:08:37.561 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-29T04:08:37.561 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-29T04:08:37.561 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-29T04:08:37.561 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-29T04:08:37.561 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-29T04:08:37.561 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-29T04:08:37.561 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-29T04:08:37.562 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-03-29T04:08:37.562 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-03-29T04:08:37.562 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-03-29T04:08:37.562 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-03-29T04:08:37.562 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-03-29T04:08:37.562 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-03-29T04:08:37.562 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-03-29T04:08:37.562 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-29T04:08:37.562 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-03-29T04:08:37.562 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-03-29T04:08:37.562 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-03-29T04:08:37.562 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-03-29T04:08:37.562 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-03-29T04:08:37.562 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-03-29T04:08:37.563 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-03-29T04:08:37.563 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-03-29T04:08:37.563 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-03-29T04:08:37.563 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-03-29T04:08:37.563 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-03-29T04:08:37.563 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-03-29T04:08:37.563 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-03-29T04:08:37.563 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-03-29T04:08:37.563 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-29T04:08:37.563 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-29T04:08:37.563 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-29T04:08:37.563 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-29T04:08:37.563 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-29T04:08:37.563 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-29T04:08:37.564 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-29T04:08:37.564 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-29T04:08:37.564 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-29T04:08:37.564 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-29T04:08:37.564 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-29T04:08:37.564 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-29T04:08:37.564 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-29T04:08:37.564 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-29T04:08:37.564 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-29T04:08:37.564 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-29T04:08:37.564 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-29T04:08:37.564 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-29T04:08:37.564 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-29T04:08:37.564 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-29T04:08:37.564 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-29T04:08:37.565 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-29T04:08:37.565 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-29T04:08:37.565 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-29T04:08:37.565 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-29T04:08:37.565 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-29T04:08:37.565 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-29T04:08:37.565 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-29T04:08:37.565 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-29T04:08:37.565 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-29T04:08:37.565 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-29T04:08:37.565 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-29T04:08:37.565 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-29T04:08:37.565 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-29T04:08:37.565 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-29T04:08:37.565 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-29T04:08:37.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-29T04:08:37.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-29T04:08:37.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-29T04:08:37.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-29T04:08:37.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-29T04:08:37.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-29T04:08:37.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-29T04:08:37.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-29T04:08:37.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-29T04:08:37.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-29T04:08:37.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-29T04:08:37.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:19.0.0-2520.g69e2cd73.el9.x86_64 151/198 2024-03-29T04:08:37.566 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-29T04:08:37.567 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:19.0.0-2520.g69e2cd73.el9.x86_64 153/198 2024-03-29T04:08:37.567 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-29T04:08:37.567 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-03-29T04:08:37.567 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-03-29T04:08:37.567 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-03-29T04:08:37.567 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-03-29T04:08:37.567 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-03-29T04:08:37.567 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-03-29T04:08:37.567 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-03-29T04:08:37.567 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-03-29T04:08:37.567 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-03-29T04:08:37.567 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-03-29T04:08:37.567 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-29T04:08:37.567 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-29T04:08:37.568 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-29T04:08:37.568 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-29T04:08:37.568 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-29T04:08:37.568 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-29T04:08:37.568 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-29T04:08:37.568 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-29T04:08:37.568 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-29T04:08:37.568 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-29T04:08:37.568 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-29T04:08:37.568 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-29T04:08:37.568 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-29T04:08:37.568 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-29T04:08:37.568 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-29T04:08:37.568 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-29T04:08:37.568 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-29T04:08:37.569 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-29T04:08:37.569 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-29T04:08:37.569 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-29T04:08:37.569 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-29T04:08:37.569 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-29T04:08:37.569 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-29T04:08:37.569 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-29T04:08:37.569 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-29T04:08:37.569 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-29T04:08:37.569 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-29T04:08:37.569 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-29T04:08:37.569 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-29T04:08:37.569 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-29T04:08:37.569 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-29T04:08:37.569 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-29T04:08:37.570 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-29T04:08:37.986 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mon-2:19.0.0-2520.g69e2cd73.el9.x86_64 136/198 2024-03-29T04:08:38.008 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mon-2:19.0.0-2520.g69e2cd73.el9.x86_64 136/198 2024-03-29T04:08:38.008 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:08:38.008 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-29T04:08:38.008 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-29T04:08:38.008 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-29T04:08:38.009 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:08:38.096 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-29T04:08:38.448 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 138/198 2024-03-29T04:08:38.470 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 138/198 2024-03-29T04:08:38.502 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2520.g69e2cd73.e 139/198 2024-03-29T04:08:38.562 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2520.g69e2cd73. 140/198 2024-03-29T04:08:38.593 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2520.g69e2cd73.el9.noa 141/198 2024-03-29T04:08:39.626 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2520.g69e2cd73.el9.noa 141/198 2024-03-29T04:08:39.661 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2520.g69e2cd73.el9.n 142/198 2024-03-29T04:08:40.435 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-29T04:08:40.435 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:08:40.435 INFO:teuthology.orchestra.run.smithi157.stdout:Upgraded: 2024-03-29T04:08:40.435 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-29T04:08:40.435 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-29T04:08:40.435 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-29T04:08:40.436 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-29T04:08:40.436 INFO:teuthology.orchestra.run.smithi157.stdout: acl-2.3.1-4.el9.x86_64 2024-03-29T04:08:40.436 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-29T04:08:40.436 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-29T04:08:40.436 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-29T04:08:40.436 INFO:teuthology.orchestra.run.smithi157.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-29T04:08:40.436 INFO:teuthology.orchestra.run.smithi157.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-29T04:08:40.436 INFO:teuthology.orchestra.run.smithi157.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-29T04:08:40.436 INFO:teuthology.orchestra.run.smithi157.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-29T04:08:40.436 INFO:teuthology.orchestra.run.smithi157.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-29T04:08:40.436 INFO:teuthology.orchestra.run.smithi157.stdout: librados2-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.437 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.437 INFO:teuthology.orchestra.run.smithi157.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-29T04:08:40.437 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-29T04:08:40.437 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-29T04:08:40.437 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-29T04:08:40.437 INFO:teuthology.orchestra.run.smithi157.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-29T04:08:40.437 INFO:teuthology.orchestra.run.smithi157.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-29T04:08:40.437 INFO:teuthology.orchestra.run.smithi157.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-29T04:08:40.437 INFO:teuthology.orchestra.run.smithi157.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-29T04:08:40.438 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-29T04:08:40.438 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-03-29T04:08:40.438 INFO:teuthology.orchestra.run.smithi157.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-29T04:08:40.438 INFO:teuthology.orchestra.run.smithi157.stdout: automake-1.16.2-8.el9.noarch 2024-03-29T04:08:40.438 INFO:teuthology.orchestra.run.smithi157.stdout: bison-3.7.4-5.el9.x86_64 2024-03-29T04:08:40.438 INFO:teuthology.orchestra.run.smithi157.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-29T04:08:40.438 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.438 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.438 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.438 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.438 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:40.438 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.438 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.438 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.439 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:40.439 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:40.439 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:40.439 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:40.439 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:40.439 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.439 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.439 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:40.439 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.439 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.439 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.439 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:40.440 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:40.440 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.440 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-shell-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.440 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:40.440 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-29T04:08:40.440 INFO:teuthology.orchestra.run.smithi157.stdout: flex-2.6.4-9.el9.x86_64 2024-03-29T04:08:40.440 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-29T04:08:40.440 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-29T04:08:40.440 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-29T04:08:40.440 INFO:teuthology.orchestra.run.smithi157.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-29T04:08:40.440 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-29T04:08:40.440 INFO:teuthology.orchestra.run.smithi157.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-29T04:08:40.440 INFO:teuthology.orchestra.run.smithi157.stdout: jq-1.6-16.el9.x86_64 2024-03-29T04:08:40.440 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-29T04:08:40.441 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-29T04:08:40.441 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-29T04:08:40.441 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.441 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.441 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.441 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-29T04:08:40.441 INFO:teuthology.orchestra.run.smithi157.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-29T04:08:40.441 INFO:teuthology.orchestra.run.smithi157.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-29T04:08:40.441 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-29T04:08:40.441 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-29T04:08:40.441 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.441 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.441 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-29T04:08:40.441 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.441 INFO:teuthology.orchestra.run.smithi157.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-29T04:08:40.441 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-29T04:08:40.442 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-29T04:08:40.442 INFO:teuthology.orchestra.run.smithi157.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-29T04:08:40.442 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-29T04:08:40.442 INFO:teuthology.orchestra.run.smithi157.stdout: lua-5.4.4-4.el9.x86_64 2024-03-29T04:08:40.442 INFO:teuthology.orchestra.run.smithi157.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-29T04:08:40.442 INFO:teuthology.orchestra.run.smithi157.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-29T04:08:40.442 INFO:teuthology.orchestra.run.smithi157.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-29T04:08:40.442 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-29T04:08:40.442 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-29T04:08:40.442 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-29T04:08:40.442 INFO:teuthology.orchestra.run.smithi157.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-29T04:08:40.443 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-13.14-1.el9.x86_64 2024-03-29T04:08:40.443 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-29T04:08:40.443 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-29T04:08:40.443 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-29T04:08:40.443 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-29T04:08:40.443 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-29T04:08:40.443 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-29T04:08:40.443 INFO:teuthology.orchestra.run.smithi157.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-29T04:08:40.443 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-29T04:08:40.443 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-29T04:08:40.443 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-29T04:08:40.443 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.443 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.443 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.444 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-29T04:08:40.444 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-29T04:08:40.444 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-29T04:08:40.444 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-29T04:08:40.444 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-29T04:08:40.444 INFO:teuthology.orchestra.run.smithi157.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-29T04:08:40.444 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-29T04:08:40.444 INFO:teuthology.orchestra.run.smithi157.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-29T04:08:40.444 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-29T04:08:40.444 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-29T04:08:40.444 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-29T04:08:40.444 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-29T04:08:40.445 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-29T04:08:40.445 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-29T04:08:40.445 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-29T04:08:40.445 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-29T04:08:40.445 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-29T04:08:40.445 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-29T04:08:40.445 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-29T04:08:40.445 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-29T04:08:40.445 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-29T04:08:40.445 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-29T04:08:40.445 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-29T04:08:40.445 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-29T04:08:40.445 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-29T04:08:40.445 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-29T04:08:40.446 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-29T04:08:40.446 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-29T04:08:40.446 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-29T04:08:40.446 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-29T04:08:40.446 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ply-3.11-14.el9.noarch 2024-03-29T04:08:40.446 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-29T04:08:40.446 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-29T04:08:40.446 INFO:teuthology.orchestra.run.smithi157.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-29T04:08:40.446 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-29T04:08:40.446 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-29T04:08:40.446 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-29T04:08:40.446 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-29T04:08:40.446 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-29T04:08:40.447 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-29T04:08:40.447 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.447 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.447 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-29T04:08:40.447 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-29T04:08:40.447 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-29T04:08:40.447 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.447 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-29T04:08:40.447 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-29T04:08:40.447 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-29T04:08:40.448 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-29T04:08:40.448 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-29T04:08:40.448 INFO:teuthology.orchestra.run.smithi157.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-29T04:08:40.448 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-29T04:08:40.448 INFO:teuthology.orchestra.run.smithi157.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-29T04:08:40.448 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-29T04:08:40.448 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-29T04:08:40.448 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-29T04:08:40.448 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-29T04:08:40.448 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-29T04:08:40.448 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-29T04:08:40.448 INFO:teuthology.orchestra.run.smithi157.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-29T04:08:40.449 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.449 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.449 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:40.449 INFO:teuthology.orchestra.run.smithi157.stdout: re2-1:20211101-3.el9.x86_64 2024-03-29T04:08:40.449 INFO:teuthology.orchestra.run.smithi157.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-29T04:08:40.449 INFO:teuthology.orchestra.run.smithi157.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-29T04:08:40.449 INFO:teuthology.orchestra.run.smithi157.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-29T04:08:40.449 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-29T04:08:40.449 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-29T04:08:40.449 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-29T04:08:40.449 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:08:40.450 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-03-29T04:08:40.621 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2520.g69e2cd73.el9.n 142/198 2024-03-29T04:08:40.645 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2520.g69e 143/198 2024-03-29T04:08:40.744 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2520.g69e 143/198 2024-03-29T04:08:40.898 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2520.g69e2cd73.el 144/198 2024-03-29T04:08:40.922 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-2:19.0.0-2520.g69e2cd73.el9.x86_64 145/198 2024-03-29T04:08:40.944 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2520.g69e2cd73.el9.x86_64 145/198 2024-03-29T04:08:40.945 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:08:40.945 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-29T04:08:40.945 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-29T04:08:40.945 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-29T04:08:40.945 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:08:40.988 DEBUG:teuthology.parallel:result is None 2024-03-29T04:08:40.989 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-rook-2:19.0.0-2520.g69e2cd73.el9.noarch 146/198 2024-03-29T04:08:41.000 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2520.g69e2cd73.el9.noarch 146/198 2024-03-29T04:08:41.041 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-2:19.0.0-2520.g69e2cd73.el9.x86_64 147/198 2024-03-29T04:08:41.915 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-29T04:08:41.941 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-radosgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 149/198 2024-03-29T04:08:41.963 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 149/198 2024-03-29T04:08:41.964 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:08:41.964 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-29T04:08:41.964 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-29T04:08:41.964 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-29T04:08:41.964 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:08:42.008 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2520.g69e2c 150/198 2024-03-29T04:08:42.030 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2520.g69e2c 150/198 2024-03-29T04:08:42.030 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:08:42.030 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-29T04:08:42.031 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:08:42.076 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 151/198 2024-03-29T04:08:42.098 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 151/198 2024-03-29T04:08:42.098 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:08:42.098 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-29T04:08:42.098 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-29T04:08:42.098 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-29T04:08:42.099 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:08:42.410 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 152/198 2024-03-29T04:08:42.432 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 152/198 2024-03-29T04:08:42.432 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:08:42.432 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-29T04:08:42.432 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-29T04:08:42.432 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-29T04:08:42.432 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:08:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-test-2:19.0.0-2520.g69e2cd73.el9.x86_64 153/198 2024-03-29T04:08:46.930 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 154/198 2024-03-29T04:08:46.961 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-nbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 155/198 2024-03-29T04:08:46.979 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-29T04:08:47.164 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-shell-2:19.0.0-2520.g69e2cd73.el9.x86_64 157/198 2024-03-29T04:08:47.279 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-29T04:08:47.320 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 159/198 2024-03-29T04:08:47.366 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-29T04:08:47.427 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-29T04:08:47.470 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-29T04:08:47.495 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs-devel-2:19.0.0-2520.g69e2cd73.el9.x86_ 163/198 2024-03-29T04:08:47.535 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-29T04:08:47.565 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-29T04:08:47.625 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-29T04:08:47.676 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-29T04:08:47.705 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-29T04:08:48.305 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-29T04:08:48.344 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-29T04:08:48.365 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-29T04:08:49.188 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-top-2:19.0.0-2520.g69e2cd73.el9.noarch 170/198 2024-03-29T04:08:49.264 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-29T04:08:49.336 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-29T04:08:49.422 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-29T04:08:49.451 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-29T04:08:49.476 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-29T04:08:49.502 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-29T04:08:49.528 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-29T04:08:49.528 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-29T04:08:49.624 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-29T04:08:49.654 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-29T04:08:49.654 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-29T04:08:49.722 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-29T04:08:49.748 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-29T04:08:49.790 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-29T04:08:49.815 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-29T04:08:49.840 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-29T04:08:49.875 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-29T04:08:49.912 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-29T04:08:49.944 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-29T04:08:49.970 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-29T04:08:49.995 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-29T04:08:50.020 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-29T04:08:50.046 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-29T04:08:50.052 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-29T04:08:50.052 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-29T04:08:50.232 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-29T04:08:50.232 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-29T04:08:50.314 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-29T04:08:50.339 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-29T04:08:50.372 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-29T04:08:50.399 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-29T04:08:50.424 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-29T04:08:50.495 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-29T04:08:55.778 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-29T04:08:55.778 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-2:19.0.0-2520.g69e2cd73.el9.x86_64 1/198 2024-03-29T04:08:55.778 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-base-2:19.0.0-2520.g69e2cd73.el9.x86_64 2/198 2024-03-29T04:08:55.778 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 3/198 2024-03-29T04:08:55.778 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 4/198 2024-03-29T04:08:55.778 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2520.g69e2c 5/198 2024-03-29T04:08:55.778 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mds-2:19.0.0-2520.g69e2cd73.el9.x86_64 6/198 2024-03-29T04:08:55.778 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-2:19.0.0-2520.g69e2cd73.el9.x86_64 7/198 2024-03-29T04:08:55.778 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mon-2:19.0.0-2520.g69e2cd73.el9.x86_64 8/198 2024-03-29T04:08:55.778 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-osd-2:19.0.0-2520.g69e2cd73.el9.x86_64 9/198 2024-03-29T04:08:55.778 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-radosgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 10/198 2024-03-29T04:08:55.779 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-selinux-2:19.0.0-2520.g69e2cd73.el9.x86_64 11/198 2024-03-29T04:08:55.779 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-test-2:19.0.0-2520.g69e2cd73.el9.x86_64 12/198 2024-03-29T04:08:55.779 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 13/198 2024-03-29T04:08:55.779 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-shell-2:19.0.0-2520.g69e2cd73.el9.x86_64 14/198 2024-03-29T04:08:55.779 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs-devel-2:19.0.0-2520.g69e2cd73.el9.x86_ 15/198 2024-03-29T04:08:55.779 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs2-2:19.0.0-2520.g69e2cd73.el9.x86_64 16/198 2024-03-29T04:08:55.779 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephsqlite-2:19.0.0-2520.g69e2cd73.el9.x86_64 17/198 2024-03-29T04:08:55.779 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados-devel-2:19.0.0-2520.g69e2cd73.el9.x86_6 18/198 2024-03-29T04:08:55.779 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libradosstriper1-2:19.0.0-2520.g69e2cd73.el9.x86 19/198 2024-03-29T04:08:55.779 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librgw2-2:19.0.0-2520.g69e2cd73.el9.x86_64 20/198 2024-03-29T04:08:55.779 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2520.g69e2cd73.el 21/198 2024-03-29T04:08:55.779 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-common-2:19.0.0-2520.g69e2cd73.el9. 22/198 2024-03-29T04:08:55.779 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cephfs-2:19.0.0-2520.g69e2cd73.el9.x86_6 23/198 2024-03-29T04:08:55.779 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rados-2:19.0.0-2520.g69e2cd73.el9.x86_64 24/198 2024-03-29T04:08:55.780 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 25/198 2024-03-29T04:08:55.780 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 26/198 2024-03-29T04:08:55.780 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 27/198 2024-03-29T04:08:55.780 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 28/198 2024-03-29T04:08:55.780 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-nbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 29/198 2024-03-29T04:08:55.795 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2520.g69e2cd73. 30/198 2024-03-29T04:08:55.795 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2520.g69e2cd73.el9.noa 31/198 2024-03-29T04:08:55.795 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2520.g69e2cd73.el9.n 32/198 2024-03-29T04:08:55.795 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2520.g69e 33/198 2024-03-29T04:08:55.795 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2520.g69e2cd73.el 34/198 2024-03-29T04:08:55.795 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2520.g69e2cd73.el9.noarch 35/198 2024-03-29T04:08:55.795 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2520.g69e2cd73.e 36/198 2024-03-29T04:08:55.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-volume-2:19.0.0-2520.g69e2cd73.el9.noarch 37/198 2024-03-29T04:08:55.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 38/198 2024-03-29T04:08:55.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-top-2:19.0.0-2520.g69e2cd73.el9.noarch 39/198 2024-03-29T04:08:55.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-29T04:08:55.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-29T04:08:55.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-29T04:08:55.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-29T04:08:55.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-29T04:08:55.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-29T04:08:55.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-29T04:08:55.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-29T04:08:55.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-29T04:08:55.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-29T04:08:55.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-29T04:08:55.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-29T04:08:55.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-29T04:08:55.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-29T04:08:55.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-29T04:08:55.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-29T04:08:55.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-29T04:08:55.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-29T04:08:55.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-03-29T04:08:55.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-03-29T04:08:55.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-03-29T04:08:55.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-03-29T04:08:55.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-29T04:08:55.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-29T04:08:55.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-29T04:08:55.798 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-29T04:08:55.798 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-29T04:08:55.798 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-29T04:08:55.798 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-29T04:08:55.798 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-29T04:08:55.798 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-29T04:08:55.798 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-29T04:08:55.798 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-29T04:08:55.798 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-29T04:08:55.798 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-29T04:08:55.798 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-29T04:08:55.798 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-29T04:08:55.798 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-29T04:08:55.799 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-29T04:08:55.799 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-29T04:08:55.799 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-29T04:08:55.799 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-29T04:08:55.799 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-03-29T04:08:55.799 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-03-29T04:08:55.799 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-03-29T04:08:55.799 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-03-29T04:08:55.799 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-03-29T04:08:55.799 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-03-29T04:08:55.799 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-03-29T04:08:55.799 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-29T04:08:55.799 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-03-29T04:08:55.799 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-03-29T04:08:55.800 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-03-29T04:08:55.800 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-03-29T04:08:55.800 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-03-29T04:08:55.800 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-03-29T04:08:55.800 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-03-29T04:08:55.800 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-03-29T04:08:55.800 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-03-29T04:08:55.800 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-03-29T04:08:55.800 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-03-29T04:08:55.800 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-03-29T04:08:55.800 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-03-29T04:08:55.800 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-03-29T04:08:55.800 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-29T04:08:55.800 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-29T04:08:55.800 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-29T04:08:55.801 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-29T04:08:55.801 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-29T04:08:55.801 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-29T04:08:55.801 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-29T04:08:55.801 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-29T04:08:55.801 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-29T04:08:55.801 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-29T04:08:55.801 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-29T04:08:55.801 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-29T04:08:55.801 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-29T04:08:55.801 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-29T04:08:55.801 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-29T04:08:55.801 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-29T04:08:55.801 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-29T04:08:55.802 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-29T04:08:55.802 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-29T04:08:55.802 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-29T04:08:55.802 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-29T04:08:55.802 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-29T04:08:55.802 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-29T04:08:55.802 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-29T04:08:55.802 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-29T04:08:55.802 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-29T04:08:55.802 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-29T04:08:55.802 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-29T04:08:55.802 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-29T04:08:55.802 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-29T04:08:55.802 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-29T04:08:55.803 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-29T04:08:55.803 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-29T04:08:55.803 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-29T04:08:55.803 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-29T04:08:55.803 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-29T04:08:55.803 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-29T04:08:55.803 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-29T04:08:55.803 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-29T04:08:55.803 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-29T04:08:55.803 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-29T04:08:55.803 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-29T04:08:55.803 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-29T04:08:55.803 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-29T04:08:55.803 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-29T04:08:55.803 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-29T04:08:55.804 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-29T04:08:55.804 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:19.0.0-2520.g69e2cd73.el9.x86_64 151/198 2024-03-29T04:08:55.804 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-29T04:08:55.804 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:19.0.0-2520.g69e2cd73.el9.x86_64 153/198 2024-03-29T04:08:55.804 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-29T04:08:55.804 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-03-29T04:08:55.804 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-03-29T04:08:55.804 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-03-29T04:08:55.804 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-03-29T04:08:55.804 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-03-29T04:08:55.804 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-03-29T04:08:55.804 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-03-29T04:08:55.804 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-03-29T04:08:55.805 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-03-29T04:08:55.805 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-03-29T04:08:55.805 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-29T04:08:55.805 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-29T04:08:55.805 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-29T04:08:55.805 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-29T04:08:55.805 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-29T04:08:55.805 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-29T04:08:55.805 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-29T04:08:55.805 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-29T04:08:55.805 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-29T04:08:55.805 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-29T04:08:55.805 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-29T04:08:55.805 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-29T04:08:55.805 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-29T04:08:55.806 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-29T04:08:55.806 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-29T04:08:55.806 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-29T04:08:55.806 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-29T04:08:55.806 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-29T04:08:55.806 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-29T04:08:55.806 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-29T04:08:55.806 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-29T04:08:55.806 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-29T04:08:55.806 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-29T04:08:55.806 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-29T04:08:55.806 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-29T04:08:55.806 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-29T04:08:55.806 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-29T04:08:55.807 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-29T04:08:55.807 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-29T04:08:55.807 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-29T04:08:55.807 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-29T04:08:55.807 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-29T04:08:55.807 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-29T04:08:58.964 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-29T04:08:58.965 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:08:58.965 INFO:teuthology.orchestra.run.smithi148.stdout:Upgraded: 2024-03-29T04:08:58.965 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-29T04:08:58.965 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-29T04:08:58.965 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-29T04:08:58.965 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-29T04:08:58.965 INFO:teuthology.orchestra.run.smithi148.stdout: acl-2.3.1-4.el9.x86_64 2024-03-29T04:08:58.965 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-29T04:08:58.965 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-29T04:08:58.965 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-29T04:08:58.965 INFO:teuthology.orchestra.run.smithi148.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-29T04:08:58.965 INFO:teuthology.orchestra.run.smithi148.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-29T04:08:58.966 INFO:teuthology.orchestra.run.smithi148.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-29T04:08:58.966 INFO:teuthology.orchestra.run.smithi148.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-29T04:08:58.966 INFO:teuthology.orchestra.run.smithi148.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-29T04:08:58.966 INFO:teuthology.orchestra.run.smithi148.stdout: librados2-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.966 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.966 INFO:teuthology.orchestra.run.smithi148.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-29T04:08:58.966 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-29T04:08:58.966 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-29T04:08:58.966 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-29T04:08:58.966 INFO:teuthology.orchestra.run.smithi148.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-29T04:08:58.966 INFO:teuthology.orchestra.run.smithi148.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-29T04:08:58.966 INFO:teuthology.orchestra.run.smithi148.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-29T04:08:58.966 INFO:teuthology.orchestra.run.smithi148.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-29T04:08:58.967 INFO:teuthology.orchestra.run.smithi148.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-29T04:08:58.967 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-03-29T04:08:58.967 INFO:teuthology.orchestra.run.smithi148.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-29T04:08:58.967 INFO:teuthology.orchestra.run.smithi148.stdout: automake-1.16.2-8.el9.noarch 2024-03-29T04:08:58.967 INFO:teuthology.orchestra.run.smithi148.stdout: bison-3.7.4-5.el9.x86_64 2024-03-29T04:08:58.967 INFO:teuthology.orchestra.run.smithi148.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-29T04:08:58.967 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.967 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.967 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.967 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.967 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:58.967 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.967 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.967 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.967 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:58.968 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:58.968 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:58.968 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:58.968 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:58.968 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.968 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.968 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:58.968 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.968 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.968 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.968 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:58.968 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:58.968 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.968 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-shell-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.968 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-top-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:58.969 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-29T04:08:58.969 INFO:teuthology.orchestra.run.smithi148.stdout: flex-2.6.4-9.el9.x86_64 2024-03-29T04:08:58.969 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-29T04:08:58.969 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-29T04:08:58.969 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-29T04:08:58.969 INFO:teuthology.orchestra.run.smithi148.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-29T04:08:58.969 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-29T04:08:58.969 INFO:teuthology.orchestra.run.smithi148.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-29T04:08:58.969 INFO:teuthology.orchestra.run.smithi148.stdout: jq-1.6-16.el9.x86_64 2024-03-29T04:08:58.969 INFO:teuthology.orchestra.run.smithi148.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-29T04:08:58.969 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-29T04:08:58.969 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-29T04:08:58.969 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.969 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.970 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.970 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-29T04:08:58.970 INFO:teuthology.orchestra.run.smithi148.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-29T04:08:58.970 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-29T04:08:58.970 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-29T04:08:58.970 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-29T04:08:58.970 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.970 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.970 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-29T04:08:58.970 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.970 INFO:teuthology.orchestra.run.smithi148.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-29T04:08:58.970 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-29T04:08:58.970 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-29T04:08:58.970 INFO:teuthology.orchestra.run.smithi148.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-29T04:08:58.971 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-29T04:08:58.971 INFO:teuthology.orchestra.run.smithi148.stdout: lua-5.4.4-4.el9.x86_64 2024-03-29T04:08:58.971 INFO:teuthology.orchestra.run.smithi148.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-29T04:08:58.971 INFO:teuthology.orchestra.run.smithi148.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-29T04:08:58.971 INFO:teuthology.orchestra.run.smithi148.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-29T04:08:58.971 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-29T04:08:58.971 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-29T04:08:58.971 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-29T04:08:58.971 INFO:teuthology.orchestra.run.smithi148.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-29T04:08:58.971 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-13.14-1.el9.x86_64 2024-03-29T04:08:58.971 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-29T04:08:58.971 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-29T04:08:58.971 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-29T04:08:58.971 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-29T04:08:58.971 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-29T04:08:58.972 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-29T04:08:58.972 INFO:teuthology.orchestra.run.smithi148.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-29T04:08:58.972 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-29T04:08:58.972 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-29T04:08:58.972 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-29T04:08:58.972 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.972 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.972 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.972 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-29T04:08:58.972 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-29T04:08:58.972 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-29T04:08:58.972 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-29T04:08:58.972 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-29T04:08:58.972 INFO:teuthology.orchestra.run.smithi148.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-29T04:08:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-29T04:08:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-29T04:08:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-29T04:08:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-29T04:08:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-29T04:08:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-29T04:08:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-29T04:08:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-29T04:08:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-29T04:08:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-29T04:08:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-29T04:08:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-29T04:08:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-29T04:08:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-29T04:08:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-29T04:08:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-29T04:08:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-29T04:08:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-29T04:08:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-29T04:08:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-29T04:08:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-29T04:08:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-29T04:08:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-29T04:08:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-29T04:08:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply-3.11-14.el9.noarch 2024-03-29T04:08:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-29T04:08:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-29T04:08:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-29T04:08:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-29T04:08:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-29T04:08:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-29T04:08:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-29T04:08:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-29T04:08:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-29T04:08:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-29T04:08:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-29T04:08:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-29T04:08:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-29T04:08:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-29T04:08:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-29T04:08:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-29T04:08:58.976 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-29T04:08:58.976 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-29T04:08:58.976 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-29T04:08:58.976 INFO:teuthology.orchestra.run.smithi148.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-29T04:08:58.976 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-29T04:08:58.976 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-29T04:08:58.976 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-29T04:08:58.976 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-29T04:08:58.976 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-29T04:08:58.976 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-29T04:08:58.976 INFO:teuthology.orchestra.run.smithi148.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-29T04:08:58.976 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.976 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.976 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:58.976 INFO:teuthology.orchestra.run.smithi148.stdout: re2-1:20211101-3.el9.x86_64 2024-03-29T04:08:58.977 INFO:teuthology.orchestra.run.smithi148.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-29T04:08:58.977 INFO:teuthology.orchestra.run.smithi148.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-29T04:08:58.977 INFO:teuthology.orchestra.run.smithi148.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-29T04:08:58.977 INFO:teuthology.orchestra.run.smithi148.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-29T04:08:58.977 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-29T04:08:58.977 INFO:teuthology.orchestra.run.smithi148.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-29T04:08:58.977 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:08:58.977 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T04:08:59.457 DEBUG:teuthology.parallel:result is None 2024-03-29T04:08:59.458 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=69e2cd73c52284dfd5096c66b60c6e6b26a3c447 2024-03-29T04:08:59.682 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-29T04:08:59.704 INFO:teuthology.orchestra.run.smithi148.stdout:19.0.0-2520.g69e2cd73.el9 2024-03-29T04:08:59.705 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2520.g69e2cd73.el9 2024-03-29T04:08:59.705 INFO:teuthology.task.install:The correct ceph version 19.0.0-2520.g69e2cd73 is installed. 2024-03-29T04:08:59.709 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=69e2cd73c52284dfd5096c66b60c6e6b26a3c447 2024-03-29T04:08:59.928 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-29T04:08:59.954 INFO:teuthology.orchestra.run.smithi157.stdout:19.0.0-2520.g69e2cd73.el9 2024-03-29T04:08:59.955 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2520.g69e2cd73.el9 2024-03-29T04:08:59.955 INFO:teuthology.task.install:The correct ceph version 19.0.0-2520.g69e2cd73 is installed. 2024-03-29T04:08:59.956 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-29T04:08:59.957 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:08:59.957 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-29T04:08:59.988 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:08:59.988 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-29T04:09:00.028 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-29T04:09:00.028 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:09:00.028 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/daemon-helper 2024-03-29T04:09:00.055 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-29T04:09:00.122 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:09:00.122 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/daemon-helper 2024-03-29T04:09:00.152 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-29T04:09:00.222 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-29T04:09:00.222 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:09:00.222 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-29T04:09:00.249 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-29T04:09:00.316 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:09:00.316 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-29T04:09:00.343 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-29T04:09:00.410 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-29T04:09:00.410 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:09:00.410 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/stdin-killer 2024-03-29T04:09:00.437 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-29T04:09:00.503 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:09:00.503 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/stdin-killer 2024-03-29T04:09:00.531 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-29T04:09:00.598 INFO:teuthology.run_tasks:Running task ceph... 2024-03-29T04:09:00.690 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-29T04:09:00.690 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 777 /var/log/ceph 2024-03-29T04:09:00.693 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 777 /var/log/ceph 2024-03-29T04:09:00.720 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-29T04:09:00.720 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-29T04:09:00.760 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-29T04:09:00.787 INFO:tasks.ceph:Creating extra log directories... 2024-03-29T04:09:00.788 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-29T04:09:00.828 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-29T04:09:00.862 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-29T04:09:00.863 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-29T04:09:00.863 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-29T04:09:00.863 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:09:00.863 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-29T04:09:00.908 DEBUG:teuthology.orchestra.run.smithi148:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-29T04:09:01.028 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-29T04:09:01.055 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:09:01.056 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-29T04:09:01.071 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-29T04:09:01.188 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-29T04:09:01.221 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-29T04:09:01.221 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-29T04:09:01.222 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629083', 'branch': 'wip-mchangir-testing1-main-20240328-072412', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/uninlining}', 'email': 'mchangir@redhat.com', 'first_in_suite': False, 'job_id': '7629083', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-mchangir-testing1-main-20240328-072412'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-mchangir-testing1-main-20240328-072412', 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447'}}, 'owner': 'scheduled_mchangir@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 8207, 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-mchangir-testing1-main-20240328-072412', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi148.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE2Ekop2D5/qeW+aNVldgxNC+pD18+a42gfuvyjNTcjUOI88NVUYIADLe947cAob/WzHHAqLMW1lHVreP4Li6xg=', 'smithi157.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBznV4lGC+LsGWKVf03Y7PBNSd75NFHyTgX517Oz732WskLXmpV6VD/C0X9zCxweqK1PwYSM0nz3uz+E2foVWM8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_uninlining']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-29_02:01:30', 'tube': 'smithi', 'user': 'mchangir', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-29T04:09:01.222 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-29T04:09:01.224 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-29T04:09:01.272 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m0777 -- /var/run/ceph 2024-03-29T04:09:01.279 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m0777 -- /var/run/ceph 2024-03-29T04:09:01.340 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:09:01.340 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/scratch_devs of=/dev/stdout 2024-03-29T04:09:01.359 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-29T04:09:01.359 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_1 2024-03-29T04:09:01.413 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-29T04:09:01.414 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:09:01.414 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-03-29T04:09:01.414 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:09:01.414 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:09:01.414 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-03-29 04:08:50.835430392 +0000 2024-03-29T04:09:01.414 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-03-29 04:06:37.929006362 +0000 2024-03-29T04:09:01.414 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-03-29 04:06:37.929006362 +0000 2024-03-29T04:09:01.414 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-03-29 04:06:37.929006362 +0000 2024-03-29T04:09:01.414 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-29T04:09:01.479 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-03-29T04:09:01.480 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-03-29T04:09:01.480 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000264097 s, 1.9 MB/s 2024-03-29T04:09:01.481 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-29T04:09:01.536 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_2 2024-03-29T04:09:01.592 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-29T04:09:01.592 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:09:01.592 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-03-29T04:09:01.592 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:09:01.592 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:09:01.592 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-03-29 04:08:50.835430392 +0000 2024-03-29T04:09:01.592 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-03-29 04:06:38.274997883 +0000 2024-03-29T04:09:01.592 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-03-29 04:06:38.274997883 +0000 2024-03-29T04:09:01.592 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-03-29 04:06:38.274997883 +0000 2024-03-29T04:09:01.592 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-29T04:09:01.658 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-03-29T04:09:01.658 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-03-29T04:09:01.658 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000188125 s, 2.7 MB/s 2024-03-29T04:09:01.659 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-29T04:09:01.714 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_3 2024-03-29T04:09:01.768 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-29T04:09:01.768 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:09:01.768 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-03-29T04:09:01.768 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:09:01.768 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:09:01.769 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-03-29 04:08:50.836430368 +0000 2024-03-29T04:09:01.769 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-03-29 04:06:38.616989502 +0000 2024-03-29T04:09:01.769 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-03-29 04:06:38.616989502 +0000 2024-03-29T04:09:01.769 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-03-29 04:06:38.616989502 +0000 2024-03-29T04:09:01.769 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-29T04:09:01.833 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-03-29T04:09:01.833 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-03-29T04:09:01.833 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000208531 s, 2.5 MB/s 2024-03-29T04:09:01.834 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-29T04:09:01.890 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_4 2024-03-29T04:09:01.946 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-29T04:09:01.946 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:09:01.946 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-03-29T04:09:01.946 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:09:01.946 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:09:01.946 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-03-29 04:08:50.836430368 +0000 2024-03-29T04:09:01.946 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-03-29 04:06:38.962981024 +0000 2024-03-29T04:09:01.946 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-03-29 04:06:38.962981024 +0000 2024-03-29T04:09:01.946 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-03-29 04:06:38.962981024 +0000 2024-03-29T04:09:01.947 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-29T04:09:02.011 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-03-29T04:09:02.011 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-03-29T04:09:02.011 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000162074 s, 3.2 MB/s 2024-03-29T04:09:02.012 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-29T04:09:02.067 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T04:09:02.067 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:09:02.067 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/scratch_devs of=/dev/stdout 2024-03-29T04:09:02.085 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-29T04:09:02.085 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_1 2024-03-29T04:09:02.141 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-29T04:09:02.141 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:09:02.141 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-03-29T04:09:02.141 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:09:02.141 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:09:02.141 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-03-29 04:08:32.582313510 +0000 2024-03-29T04:09:02.142 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-03-29 04:06:37.350437788 +0000 2024-03-29T04:09:02.142 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-03-29 04:06:37.350437788 +0000 2024-03-29T04:09:02.142 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-03-29 04:06:37.350437788 +0000 2024-03-29T04:09:02.142 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-29T04:09:02.210 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-03-29T04:09:02.210 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-03-29T04:09:02.210 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000107135 s, 4.8 MB/s 2024-03-29T04:09:02.211 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-29T04:09:02.270 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_2 2024-03-29T04:09:02.326 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-29T04:09:02.326 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:09:02.326 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-03-29T04:09:02.326 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:09:02.326 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:09:02.326 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-03-29 04:08:32.582313510 +0000 2024-03-29T04:09:02.326 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-03-29 04:06:37.669432555 +0000 2024-03-29T04:09:02.326 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-03-29 04:06:37.669432555 +0000 2024-03-29T04:09:02.327 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-03-29 04:06:37.669432555 +0000 2024-03-29T04:09:02.327 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-29T04:09:02.392 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-03-29T04:09:02.393 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-03-29T04:09:02.393 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 8.9177e-05 s, 5.7 MB/s 2024-03-29T04:09:02.394 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-29T04:09:02.450 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_3 2024-03-29T04:09:02.505 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-29T04:09:02.505 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:09:02.505 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-03-29T04:09:02.505 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:09:02.505 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:09:02.505 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-03-29 04:08:32.582313510 +0000 2024-03-29T04:09:02.505 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-03-29 04:06:37.985427370 +0000 2024-03-29T04:09:02.506 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-03-29 04:06:37.985427370 +0000 2024-03-29T04:09:02.506 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-03-29 04:06:37.985427370 +0000 2024-03-29T04:09:02.506 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-29T04:09:02.571 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-03-29T04:09:02.572 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-03-29T04:09:02.572 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 9.9977e-05 s, 5.1 MB/s 2024-03-29T04:09:02.573 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-29T04:09:02.629 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_4 2024-03-29T04:09:02.683 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-29T04:09:02.683 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:09:02.684 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-03-29T04:09:02.684 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:09:02.684 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:09:02.684 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-03-29 04:08:32.583313493 +0000 2024-03-29T04:09:02.684 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-03-29 04:06:38.312422005 +0000 2024-03-29T04:09:02.684 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-03-29 04:06:38.312422005 +0000 2024-03-29T04:09:02.684 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-03-29 04:06:38.312422005 +0000 2024-03-29T04:09:02.684 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-29T04:09:02.749 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-03-29T04:09:02.749 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-03-29T04:09:02.749 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000101165 s, 5.1 MB/s 2024-03-29T04:09:02.750 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-29T04:09:02.806 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T04:09:02.806 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi148.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi157.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-29T04:09:02.806 INFO:tasks.ceph:Generating config... 2024-03-29T04:09:02.807 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-29T04:09:02.807 INFO:tasks.ceph:[client] debug client = 20 2024-03-29T04:09:02.807 INFO:tasks.ceph:[client] debug ms = 1 2024-03-29T04:09:02.808 INFO:tasks.ceph:[client] pid file = 2024-03-29T04:09:02.808 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-29T04:09:02.808 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-29T04:09:02.808 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-29T04:09:02.808 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-29T04:09:02.808 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-29T04:09:02.808 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-29T04:09:02.808 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-29T04:09:02.808 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-29T04:09:02.808 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-29T04:09:02.808 INFO:tasks.ceph:[mds] mds_bal_replicate_threshold = 1 2024-03-29T04:09:02.809 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-29T04:09:02.809 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-29T04:09:02.809 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-29T04:09:02.809 INFO:tasks.ceph:[mgr] debug client = 20 2024-03-29T04:09:02.809 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-03-29T04:09:02.809 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-29T04:09:02.809 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-29T04:09:02.809 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-29T04:09:02.809 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-29T04:09:02.809 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-29T04:09:02.809 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-03-29T04:09:02.809 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-29T04:09:02.809 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-29T04:09:02.809 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-29T04:09:02.809 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-03-29T04:09:02.810 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-03-29T04:09:02.810 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-29T04:09:02.810 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-29T04:09:02.810 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-03-29T04:09:02.810 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-29T04:09:02.810 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-29T04:09:02.810 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-29T04:09:02.810 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-29T04:09:02.810 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-29T04:09:02.810 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-29T04:09:02.810 INFO:tasks.ceph:Setting up mon.a... 2024-03-29T04:09:02.811 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-29T04:09:02.861 INFO:teuthology.orchestra.run.smithi148.stdout:creating /etc/ceph/ceph.keyring 2024-03-29T04:09:02.864 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-29T04:09:02.910 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-29T04:09:02.938 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.148'), ('mon.b', '172.21.15.157'), ('mon.c', '[v2:172.21.15.157:3301,v1:172.21.15.157:6790]')] 2024-03-29T04:09:02.938 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.148,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-29T04:09:02.939 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:09:02.939 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-29T04:09:02.993 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.148 --add b 172.21.15.157 --addv c '[v2:172.21.15.157:3301,v1:172.21.15.157:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:09:03.079 INFO:teuthology.orchestra.run.smithi148.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:09:03.079 INFO:teuthology.orchestra.run.smithi148.stdout:monmaptool: generated fsid 56b659bb-6db8-45e8-b612-135349c3fd6b 2024-03-29T04:09:03.079 INFO:teuthology.orchestra.run.smithi148.stdout:setting min_mon_release = quincy 2024-03-29T04:09:03.079 INFO:teuthology.orchestra.run.smithi148.stdout:epoch 0 2024-03-29T04:09:03.079 INFO:teuthology.orchestra.run.smithi148.stdout:fsid 56b659bb-6db8-45e8-b612-135349c3fd6b 2024-03-29T04:09:03.080 INFO:teuthology.orchestra.run.smithi148.stdout:last_changed 2024-03-29T04:09:03.100697+0000 2024-03-29T04:09:03.080 INFO:teuthology.orchestra.run.smithi148.stdout:created 2024-03-29T04:09:03.100697+0000 2024-03-29T04:09:03.080 INFO:teuthology.orchestra.run.smithi148.stdout:min_mon_release 17 (quincy) 2024-03-29T04:09:03.080 INFO:teuthology.orchestra.run.smithi148.stdout:election_strategy: 1 2024-03-29T04:09:03.080 INFO:teuthology.orchestra.run.smithi148.stdout:0: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.a 2024-03-29T04:09:03.080 INFO:teuthology.orchestra.run.smithi148.stdout:1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.b 2024-03-29T04:09:03.080 INFO:teuthology.orchestra.run.smithi148.stdout:2: [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] mon.c 2024-03-29T04:09:03.080 INFO:teuthology.orchestra.run.smithi148.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-29T04:09:03.081 DEBUG:teuthology.orchestra.run.smithi148:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-29T04:09:03.096 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 56b659bb-6db8-45e8-b612-135349c3fd6b... 2024-03-29T04:09:03.097 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-29T04:09:03.138 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-29T04:09:03.186 INFO:teuthology.orchestra.run.smithi148.stdout:[global] 2024-03-29T04:09:03.187 INFO:teuthology.orchestra.run.smithi148.stdout: chdir = "" 2024-03-29T04:09:03.187 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-29T04:09:03.187 INFO:teuthology.orchestra.run.smithi148.stdout: auth supported = cephx 2024-03-29T04:09:03.187 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.187 INFO:teuthology.orchestra.run.smithi148.stdout: filestore xattr use omap = true 2024-03-29T04:09:03.187 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.187 INFO:teuthology.orchestra.run.smithi148.stdout: mon clock drift allowed = 1.000 2024-03-29T04:09:03.187 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.187 INFO:teuthology.orchestra.run.smithi148.stdout: osd crush chooseleaf type = 0 2024-03-29T04:09:03.187 INFO:teuthology.orchestra.run.smithi148.stdout: auth debug = true 2024-03-29T04:09:03.187 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.187 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on old message = true 2024-03-29T04:09:03.188 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on bug = true 2024-03-29T04:09:03.188 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.188 INFO:teuthology.orchestra.run.smithi148.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-29T04:09:03.188 INFO:teuthology.orchestra.run.smithi148.stdout: mon pg warn max object skew = 0 2024-03-29T04:09:03.188 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.188 INFO:teuthology.orchestra.run.smithi148.stdout: # disable pg_autoscaler by default for new pools 2024-03-29T04:09:03.188 INFO:teuthology.orchestra.run.smithi148.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-29T04:09:03.188 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.188 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default size = 2 2024-03-29T04:09:03.188 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.188 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow primary affinity = true 2024-03-29T04:09:03.188 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow pg remap = true 2024-03-29T04:09:03.189 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on legacy crush tunables = false 2024-03-29T04:09:03.189 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on crush straw calc version zero = false 2024-03-29T04:09:03.189 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on no sortbitwise = false 2024-03-29T04:09:03.189 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on osd down out interval zero = false 2024-03-29T04:09:03.189 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on too few osds = false 2024-03-29T04:09:03.189 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-29T04:09:03.189 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-29T04:09:03.189 INFO:teuthology.orchestra.run.smithi148.stdout: mon_allow_pool_size_one = true 2024-03-29T04:09:03.189 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.189 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-29T04:09:03.189 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.190 INFO:teuthology.orchestra.run.smithi148.stdout: osd default data pool replay window = 5 2024-03-29T04:09:03.190 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.190 INFO:teuthology.orchestra.run.smithi148.stdout: mon allow pool delete = true 2024-03-29T04:09:03.190 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.190 INFO:teuthology.orchestra.run.smithi148.stdout: mon cluster log file level = debug 2024-03-29T04:09:03.190 INFO:teuthology.orchestra.run.smithi148.stdout: debug asserts on shutdown = true 2024-03-29T04:09:03.190 INFO:teuthology.orchestra.run.smithi148.stdout: mon health detail to clog = false 2024-03-29T04:09:03.190 INFO:teuthology.orchestra.run.smithi148.stdout: mon host = "172.21.15.148,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]" 2024-03-29T04:09:03.190 INFO:teuthology.orchestra.run.smithi148.stdout: fsid = 56b659bb-6db8-45e8-b612-135349c3fd6b 2024-03-29T04:09:03.190 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.190 INFO:teuthology.orchestra.run.smithi148.stdout:[osd] 2024-03-29T04:09:03.191 INFO:teuthology.orchestra.run.smithi148.stdout: osd journal size = 100 2024-03-29T04:09:03.191 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.191 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub load threshold = 5.0 2024-03-29T04:09:03.191 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub max interval = 600 2024-03-29T04:09:03.191 INFO:teuthology.orchestra.run.smithi148.stdout: osd mclock profile = high_recovery_ops 2024-03-29T04:09:03.191 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.191 INFO:teuthology.orchestra.run.smithi148.stdout: osd recover clone overlap = true 2024-03-29T04:09:03.191 INFO:teuthology.orchestra.run.smithi148.stdout: osd recovery max chunk = 1048576 2024-03-29T04:09:03.191 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.191 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug shutdown = true 2024-03-29T04:09:03.191 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug op order = true 2024-03-29T04:09:03.191 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug verify stray on activate = true 2024-03-29T04:09:03.191 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug trim objects = true 2024-03-29T04:09:03.191 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.191 INFO:teuthology.orchestra.run.smithi148.stdout: osd open classes on start = true 2024-03-29T04:09:03.191 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug pg log writeout = true 2024-03-29T04:09:03.192 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.192 INFO:teuthology.orchestra.run.smithi148.stdout: osd deep scrub update digest min age = 30 2024-03-29T04:09:03.192 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.192 INFO:teuthology.orchestra.run.smithi148.stdout: osd map max advance = 10 2024-03-29T04:09:03.192 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.192 INFO:teuthology.orchestra.run.smithi148.stdout: journal zero on create = true 2024-03-29T04:09:03.192 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.193 INFO:teuthology.orchestra.run.smithi148.stdout: filestore ondisk finisher threads = 3 2024-03-29T04:09:03.193 INFO:teuthology.orchestra.run.smithi148.stdout: filestore apply finisher threads = 3 2024-03-29T04:09:03.193 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.193 INFO:teuthology.orchestra.run.smithi148.stdout: bdev debug aio = true 2024-03-29T04:09:03.193 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug misdirected ops = true 2024-03-29T04:09:03.193 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore block size = 96636764160 2024-03-29T04:09:03.194 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore fsck on mount = True 2024-03-29T04:09:03.194 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluefs = 20 2024-03-29T04:09:03.194 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluestore = 20 2024-03-29T04:09:03.194 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-03-29T04:09:03.194 INFO:teuthology.orchestra.run.smithi148.stdout: debug osd = 20 2024-03-29T04:09:03.194 INFO:teuthology.orchestra.run.smithi148.stdout: debug rocksdb = 10 2024-03-29T04:09:03.194 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd backfillfull_ratio = 0.85 2024-03-29T04:09:03.194 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd full ratio = 0.9 2024-03-29T04:09:03.194 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd nearfull ratio = 0.8 2024-03-29T04:09:03.194 INFO:teuthology.orchestra.run.smithi148.stdout: osd failsafe full ratio = 0.95 2024-03-29T04:09:03.195 INFO:teuthology.orchestra.run.smithi148.stdout: osd objectstore = bluestore 2024-03-29T04:09:03.195 INFO:teuthology.orchestra.run.smithi148.stdout: osd op complaint time = 180 2024-03-29T04:09:03.195 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.195 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr] 2024-03-29T04:09:03.195 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-03-29T04:09:03.195 INFO:teuthology.orchestra.run.smithi148.stdout: debug mgr = 20 2024-03-29T04:09:03.196 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-03-29T04:09:03.196 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-03-29T04:09:03.196 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-03-29T04:09:03.196 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-03-29T04:09:03.196 INFO:teuthology.orchestra.run.smithi148.stdout: mgr/telemetry/nag = false 2024-03-29T04:09:03.196 INFO:teuthology.orchestra.run.smithi148.stdout: debug client = 20 2024-03-29T04:09:03.196 INFO:teuthology.orchestra.run.smithi148.stdout: debug finisher = 20 2024-03-29T04:09:03.197 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.197 INFO:teuthology.orchestra.run.smithi148.stdout:[mon] 2024-03-29T04:09:03.197 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-03-29T04:09:03.197 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-03-29T04:09:03.197 INFO:teuthology.orchestra.run.smithi148.stdout: debug paxos = 20 2024-03-29T04:09:03.197 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-03-29T04:09:03.197 INFO:teuthology.orchestra.run.smithi148.stdout: mon data avail warn = 5 2024-03-29T04:09:03.197 INFO:teuthology.orchestra.run.smithi148.stdout: mon mgr mkfs grace = 240 2024-03-29T04:09:03.197 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-03-29T04:09:03.197 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd reporter subtree level = osd 2024-03-29T04:09:03.197 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd prime pg temp = true 2024-03-29T04:09:03.197 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-03-29T04:09:03.197 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.198 INFO:teuthology.orchestra.run.smithi148.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-29T04:09:03.198 INFO:teuthology.orchestra.run.smithi148.stdout: auth mon ticket ttl = 660 # 11m 2024-03-29T04:09:03.198 INFO:teuthology.orchestra.run.smithi148.stdout: auth service ticket ttl = 240 # 4m 2024-03-29T04:09:03.198 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.198 INFO:teuthology.orchestra.run.smithi148.stdout: # don't complain about insecure global_id in the test suite 2024-03-29T04:09:03.198 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-29T04:09:03.198 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-29T04:09:03.198 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.198 INFO:teuthology.orchestra.run.smithi148.stdout: # 1m isn't quite enough 2024-03-29T04:09:03.198 INFO:teuthology.orchestra.run.smithi148.stdout: mon_down_mkfs_grace = 2m 2024-03-29T04:09:03.199 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.199 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_filestore_osds = false 2024-03-29T04:09:03.199 INFO:teuthology.orchestra.run.smithi148.stdout: mon down mkfs grace = 300 2024-03-29T04:09:03.199 INFO:teuthology.orchestra.run.smithi148.stdout: mon op complaint time = 120 2024-03-29T04:09:03.199 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:03.199 INFO:teuthology.orchestra.run.smithi148.stdout:[client] 2024-03-29T04:09:03.199 INFO:teuthology.orchestra.run.smithi148.stdout: rgw cache enabled = true 2024-03-29T04:09:03.199 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable ops log = true 2024-03-29T04:09:03.199 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable usage log = true 2024-03-29T04:09:03.199 INFO:teuthology.orchestra.run.smithi148.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-29T04:09:03.199 INFO:teuthology.orchestra.run.smithi148.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-29T04:09:03.199 INFO:teuthology.orchestra.run.smithi148.stdout: client mount timeout = 600 2024-03-29T04:09:03.199 INFO:teuthology.orchestra.run.smithi148.stdout: debug client = 20 2024-03-29T04:09:03.199 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-03-29T04:09:03.199 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = "" 2024-03-29T04:09:03.200 INFO:teuthology.orchestra.run.smithi148.stdout: rados mon op timeout = 900 2024-03-29T04:09:03.200 INFO:teuthology.orchestra.run.smithi148.stdout: rados osd op timeout = 900 2024-03-29T04:09:03.200 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.a] 2024-03-29T04:09:03.200 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.b] 2024-03-29T04:09:03.200 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.c] 2024-03-29T04:09:03.200 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.a] 2024-03-29T04:09:03.200 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.c] 2024-03-29T04:09:03.200 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.b] 2024-03-29T04:09:03.200 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.d] 2024-03-29T04:09:03.200 INFO:teuthology.orchestra.run.smithi148.stdout:[mds] 2024-03-29T04:09:03.200 INFO:teuthology.orchestra.run.smithi148.stdout: debug mds = 20 2024-03-29T04:09:03.200 INFO:teuthology.orchestra.run.smithi148.stdout: debug mds balancer = 20 2024-03-29T04:09:03.200 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-03-29T04:09:03.200 INFO:teuthology.orchestra.run.smithi148.stdout: mds debug frag = True 2024-03-29T04:09:03.200 INFO:teuthology.orchestra.run.smithi148.stdout: mds debug scatterstat = True 2024-03-29T04:09:03.201 INFO:teuthology.orchestra.run.smithi148.stdout: mds op complaint time = 180 2024-03-29T04:09:03.201 INFO:teuthology.orchestra.run.smithi148.stdout: mds verify scatter = True 2024-03-29T04:09:03.201 INFO:teuthology.orchestra.run.smithi148.stdout: mds_bal_replicate_threshold = 1 2024-03-29T04:09:03.201 INFO:teuthology.orchestra.run.smithi148.stdout: osd op complaint time = 180 2024-03-29T04:09:03.201 INFO:teuthology.orchestra.run.smithi148.stdout: rados mon op timeout = 900 2024-03-29T04:09:03.201 INFO:teuthology.orchestra.run.smithi148.stdout: rados osd op timeout = 900 2024-03-29T04:09:03.201 INFO:teuthology.orchestra.run.smithi157.stdout:[global] 2024-03-29T04:09:03.202 INFO:teuthology.orchestra.run.smithi157.stdout: chdir = "" 2024-03-29T04:09:03.202 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-29T04:09:03.202 INFO:teuthology.orchestra.run.smithi157.stdout: auth supported = cephx 2024-03-29T04:09:03.202 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.202 INFO:teuthology.orchestra.run.smithi157.stdout: filestore xattr use omap = true 2024-03-29T04:09:03.202 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.202 INFO:teuthology.orchestra.run.smithi157.stdout: mon clock drift allowed = 1.000 2024-03-29T04:09:03.202 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.202 INFO:teuthology.orchestra.run.smithi157.stdout: osd crush chooseleaf type = 0 2024-03-29T04:09:03.202 INFO:teuthology.orchestra.run.smithi157.stdout: auth debug = true 2024-03-29T04:09:03.202 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.203 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on old message = true 2024-03-29T04:09:03.203 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on bug = true 2024-03-29T04:09:03.203 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.203 INFO:teuthology.orchestra.run.smithi157.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-29T04:09:03.203 INFO:teuthology.orchestra.run.smithi157.stdout: mon pg warn max object skew = 0 2024-03-29T04:09:03.203 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.203 INFO:teuthology.orchestra.run.smithi157.stdout: # disable pg_autoscaler by default for new pools 2024-03-29T04:09:03.203 INFO:teuthology.orchestra.run.smithi157.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-29T04:09:03.203 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.203 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default size = 2 2024-03-29T04:09:03.203 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.203 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow primary affinity = true 2024-03-29T04:09:03.203 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow pg remap = true 2024-03-29T04:09:03.203 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on legacy crush tunables = false 2024-03-29T04:09:03.204 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on crush straw calc version zero = false 2024-03-29T04:09:03.204 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on no sortbitwise = false 2024-03-29T04:09:03.204 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on osd down out interval zero = false 2024-03-29T04:09:03.204 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on too few osds = false 2024-03-29T04:09:03.204 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-29T04:09:03.204 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-29T04:09:03.204 INFO:teuthology.orchestra.run.smithi157.stdout: mon_allow_pool_size_one = true 2024-03-29T04:09:03.204 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.204 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-29T04:09:03.204 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.204 INFO:teuthology.orchestra.run.smithi157.stdout: osd default data pool replay window = 5 2024-03-29T04:09:03.204 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.204 INFO:teuthology.orchestra.run.smithi157.stdout: mon allow pool delete = true 2024-03-29T04:09:03.204 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.204 INFO:teuthology.orchestra.run.smithi157.stdout: mon cluster log file level = debug 2024-03-29T04:09:03.205 INFO:teuthology.orchestra.run.smithi157.stdout: debug asserts on shutdown = true 2024-03-29T04:09:03.205 INFO:teuthology.orchestra.run.smithi157.stdout: mon health detail to clog = false 2024-03-29T04:09:03.205 INFO:teuthology.orchestra.run.smithi157.stdout: mon host = "172.21.15.148,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]" 2024-03-29T04:09:03.205 INFO:teuthology.orchestra.run.smithi157.stdout: fsid = 56b659bb-6db8-45e8-b612-135349c3fd6b 2024-03-29T04:09:03.205 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.205 INFO:teuthology.orchestra.run.smithi157.stdout:[osd] 2024-03-29T04:09:03.205 INFO:teuthology.orchestra.run.smithi157.stdout: osd journal size = 100 2024-03-29T04:09:03.205 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.205 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub load threshold = 5.0 2024-03-29T04:09:03.205 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub max interval = 600 2024-03-29T04:09:03.205 INFO:teuthology.orchestra.run.smithi157.stdout: osd mclock profile = high_recovery_ops 2024-03-29T04:09:03.205 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.205 INFO:teuthology.orchestra.run.smithi157.stdout: osd recover clone overlap = true 2024-03-29T04:09:03.205 INFO:teuthology.orchestra.run.smithi157.stdout: osd recovery max chunk = 1048576 2024-03-29T04:09:03.205 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.206 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug shutdown = true 2024-03-29T04:09:03.206 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug op order = true 2024-03-29T04:09:03.206 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug verify stray on activate = true 2024-03-29T04:09:03.206 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug trim objects = true 2024-03-29T04:09:03.206 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.206 INFO:teuthology.orchestra.run.smithi157.stdout: osd open classes on start = true 2024-03-29T04:09:03.206 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug pg log writeout = true 2024-03-29T04:09:03.206 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.206 INFO:teuthology.orchestra.run.smithi157.stdout: osd deep scrub update digest min age = 30 2024-03-29T04:09:03.206 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.206 INFO:teuthology.orchestra.run.smithi157.stdout: osd map max advance = 10 2024-03-29T04:09:03.206 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.206 INFO:teuthology.orchestra.run.smithi157.stdout: journal zero on create = true 2024-03-29T04:09:03.207 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.207 INFO:teuthology.orchestra.run.smithi157.stdout: filestore ondisk finisher threads = 3 2024-03-29T04:09:03.207 INFO:teuthology.orchestra.run.smithi157.stdout: filestore apply finisher threads = 3 2024-03-29T04:09:03.207 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.207 INFO:teuthology.orchestra.run.smithi157.stdout: bdev debug aio = true 2024-03-29T04:09:03.207 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug misdirected ops = true 2024-03-29T04:09:03.207 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore block size = 96636764160 2024-03-29T04:09:03.207 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore fsck on mount = True 2024-03-29T04:09:03.207 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluefs = 20 2024-03-29T04:09:03.207 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluestore = 20 2024-03-29T04:09:03.207 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-03-29T04:09:03.207 INFO:teuthology.orchestra.run.smithi157.stdout: debug osd = 20 2024-03-29T04:09:03.207 INFO:teuthology.orchestra.run.smithi157.stdout: debug rocksdb = 10 2024-03-29T04:09:03.207 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd backfillfull_ratio = 0.85 2024-03-29T04:09:03.207 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd full ratio = 0.9 2024-03-29T04:09:03.208 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd nearfull ratio = 0.8 2024-03-29T04:09:03.208 INFO:teuthology.orchestra.run.smithi157.stdout: osd failsafe full ratio = 0.95 2024-03-29T04:09:03.208 INFO:teuthology.orchestra.run.smithi157.stdout: osd objectstore = bluestore 2024-03-29T04:09:03.208 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2024-03-29T04:09:03.208 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.208 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr] 2024-03-29T04:09:03.208 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-03-29T04:09:03.208 INFO:teuthology.orchestra.run.smithi157.stdout: debug mgr = 20 2024-03-29T04:09:03.208 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2024-03-29T04:09:03.208 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2024-03-29T04:09:03.208 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2024-03-29T04:09:03.208 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2024-03-29T04:09:03.208 INFO:teuthology.orchestra.run.smithi157.stdout: mgr/telemetry/nag = false 2024-03-29T04:09:03.208 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2024-03-29T04:09:03.208 INFO:teuthology.orchestra.run.smithi157.stdout: debug finisher = 20 2024-03-29T04:09:03.208 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.209 INFO:teuthology.orchestra.run.smithi157.stdout:[mon] 2024-03-29T04:09:03.209 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-03-29T04:09:03.209 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2024-03-29T04:09:03.209 INFO:teuthology.orchestra.run.smithi157.stdout: debug paxos = 20 2024-03-29T04:09:03.209 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2024-03-29T04:09:03.209 INFO:teuthology.orchestra.run.smithi157.stdout: mon data avail warn = 5 2024-03-29T04:09:03.209 INFO:teuthology.orchestra.run.smithi157.stdout: mon mgr mkfs grace = 240 2024-03-29T04:09:03.209 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2024-03-29T04:09:03.209 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd reporter subtree level = osd 2024-03-29T04:09:03.209 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd prime pg temp = true 2024-03-29T04:09:03.209 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2024-03-29T04:09:03.209 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.209 INFO:teuthology.orchestra.run.smithi157.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-29T04:09:03.209 INFO:teuthology.orchestra.run.smithi157.stdout: auth mon ticket ttl = 660 # 11m 2024-03-29T04:09:03.209 INFO:teuthology.orchestra.run.smithi157.stdout: auth service ticket ttl = 240 # 4m 2024-03-29T04:09:03.210 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.210 INFO:teuthology.orchestra.run.smithi157.stdout: # don't complain about insecure global_id in the test suite 2024-03-29T04:09:03.210 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-29T04:09:03.210 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-29T04:09:03.210 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.210 INFO:teuthology.orchestra.run.smithi157.stdout: # 1m isn't quite enough 2024-03-29T04:09:03.210 INFO:teuthology.orchestra.run.smithi157.stdout: mon_down_mkfs_grace = 2m 2024-03-29T04:09:03.210 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.210 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_filestore_osds = false 2024-03-29T04:09:03.210 INFO:teuthology.orchestra.run.smithi157.stdout: mon down mkfs grace = 300 2024-03-29T04:09:03.210 INFO:teuthology.orchestra.run.smithi157.stdout: mon op complaint time = 120 2024-03-29T04:09:03.210 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-29T04:09:03.210 INFO:teuthology.orchestra.run.smithi157.stdout:[client] 2024-03-29T04:09:03.210 INFO:teuthology.orchestra.run.smithi157.stdout: rgw cache enabled = true 2024-03-29T04:09:03.211 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable ops log = true 2024-03-29T04:09:03.211 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable usage log = true 2024-03-29T04:09:03.211 INFO:teuthology.orchestra.run.smithi157.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-29T04:09:03.211 INFO:teuthology.orchestra.run.smithi157.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-29T04:09:03.211 INFO:teuthology.orchestra.run.smithi157.stdout: client mount timeout = 600 2024-03-29T04:09:03.211 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2024-03-29T04:09:03.211 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-03-29T04:09:03.211 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = "" 2024-03-29T04:09:03.211 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2024-03-29T04:09:03.211 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2024-03-29T04:09:03.211 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.a] 2024-03-29T04:09:03.211 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.b] 2024-03-29T04:09:03.211 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.c] 2024-03-29T04:09:03.211 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2024-03-29T04:09:03.211 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2024-03-29T04:09:03.212 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2024-03-29T04:09:03.212 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2024-03-29T04:09:03.212 INFO:teuthology.orchestra.run.smithi157.stdout:[mds] 2024-03-29T04:09:03.212 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds = 20 2024-03-29T04:09:03.212 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds balancer = 20 2024-03-29T04:09:03.212 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-03-29T04:09:03.212 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug frag = True 2024-03-29T04:09:03.212 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug scatterstat = True 2024-03-29T04:09:03.212 INFO:teuthology.orchestra.run.smithi157.stdout: mds op complaint time = 180 2024-03-29T04:09:03.212 INFO:teuthology.orchestra.run.smithi157.stdout: mds verify scatter = True 2024-03-29T04:09:03.212 INFO:teuthology.orchestra.run.smithi157.stdout: mds_bal_replicate_threshold = 1 2024-03-29T04:09:03.212 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2024-03-29T04:09:03.212 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2024-03-29T04:09:03.212 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2024-03-29T04:09:03.214 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-29T04:09:03.214 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-29T04:09:03.309 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-29T04:09:03.309 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:09:03.309 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-29T04:09:03.324 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:09:03.324 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-29T04:09:03.379 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi148.front.sepia.ceph.com 2024-03-29T04:09:03.379 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:09:03.379 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-29T04:09:03.379 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-29T04:09:03.459 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:09:03.459 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:09:03.513 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi157.front.sepia.ceph.com 2024-03-29T04:09:03.514 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:09:03.514 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-29T04:09:03.514 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-29T04:09:03.555 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:09:03.555 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:09:03.611 INFO:tasks.ceph:Setting up mon nodes... 2024-03-29T04:09:03.611 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-29T04:09:03.611 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-29T04:09:03.666 INFO:teuthology.orchestra.run.smithi148.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-29T04:09:03.669 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-29T04:09:03.733 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-29T04:09:03.735 INFO:tasks.ceph:Setting up mds nodes... 2024-03-29T04:09:03.735 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-29T04:09:03.791 INFO:teuthology.orchestra.run.smithi148.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-29T04:09:03.793 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-29T04:09:03.820 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-29T04:09:03.915 INFO:teuthology.orchestra.run.smithi148.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-29T04:09:03.918 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-29T04:09:03.944 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-29T04:09:03.999 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-29T04:09:04.002 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-29T04:09:04.029 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-29T04:09:04.123 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-29T04:09:04.125 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-29T04:09:04.152 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-29T04:09:04.153 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-29T04:09:04.195 INFO:teuthology.orchestra.run.smithi148.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-29T04:09:04.210 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-03-29T04:09:04.292 INFO:teuthology.orchestra.run.smithi148.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-03-29T04:09:04.307 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-29T04:09:04.350 INFO:teuthology.orchestra.run.smithi157.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-29T04:09:04.365 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-03-29T04:09:04.447 INFO:teuthology.orchestra.run.smithi157.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-03-29T04:09:04.463 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-29T04:09:04.463 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi148.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi157.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-29T04:09:04.463 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-29T04:09:04.490 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T04:09:04.490 INFO:tasks.ceph:role: osd.0 2024-03-29T04:09:04.490 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi148.front.sepia.ceph.com 2024-03-29T04:09:04.491 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-29T04:09:04.565 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:09:04.565 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:09:04.565 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:09:04.565 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-29T04:09:04.565 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:09:04.566 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:09:04.566 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:09:04.566 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-29T04:09:04.566 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:09:04.566 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:09:05.071 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2024-03-29T04:09:05.073 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-03-29T04:09:05.073 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-29T04:09:05.229 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-29T04:09:05.264 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-29T04:09:05.332 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T04:09:05.332 INFO:tasks.ceph:role: osd.1 2024-03-29T04:09:05.333 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi148.front.sepia.ceph.com 2024-03-29T04:09:05.333 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-29T04:09:05.407 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:09:05.407 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:09:05.407 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:09:05.407 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-29T04:09:05.407 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:09:05.408 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:09:05.408 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:09:05.408 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-29T04:09:05.408 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:09:05.408 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:09:05.843 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2024-03-29T04:09:05.845 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-03-29T04:09:05.845 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-29T04:09:05.929 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-29T04:09:06.001 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-29T04:09:06.068 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T04:09:06.068 INFO:tasks.ceph:role: osd.2 2024-03-29T04:09:06.068 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi148.front.sepia.ceph.com 2024-03-29T04:09:06.068 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-29T04:09:06.142 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:09:06.142 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:09:06.142 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:09:06.143 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-29T04:09:06.143 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:09:06.143 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:09:06.143 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:09:06.143 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-29T04:09:06.143 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:09:06.143 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:09:06.650 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2024-03-29T04:09:06.652 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-03-29T04:09:06.653 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-29T04:09:06.785 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-29T04:09:06.818 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-29T04:09:06.886 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T04:09:06.886 INFO:tasks.ceph:role: osd.3 2024-03-29T04:09:06.886 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi148.front.sepia.ceph.com 2024-03-29T04:09:06.887 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-29T04:09:06.961 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:09:06.961 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:09:06.961 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:09:06.961 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-29T04:09:06.961 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:09:06.961 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:09:06.961 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:09:06.961 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-29T04:09:06.962 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:09:06.962 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:09:07.453 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2024-03-29T04:09:07.455 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-03-29T04:09:07.455 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-29T04:09:07.571 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-29T04:09:07.605 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:09:07.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:07.726+0000 7f366409d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-29T04:09:07.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:07.727+0000 7f366409d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-29T04:09:07.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:07.727+0000 7f366409d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-29T04:09:07.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:07.727+0000 7f366409d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-29T04:09:07.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:07.727+0000 7f366409d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-29T04:09:07.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:07.727+0000 7f366409d640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-29T04:09:10.552 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-29T04:09:10.582 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:09:10.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:10.691+0000 7fc482a8b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-29T04:09:10.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:10.691+0000 7fc482a8b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-29T04:09:10.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:10.691+0000 7fc482a8b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-29T04:09:10.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:10.691+0000 7fc482a8b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-29T04:09:10.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:10.691+0000 7fc482a8b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-29T04:09:10.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:10.692+0000 7fc482a8b640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-29T04:09:13.525 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-29T04:09:13.557 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:09:13.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:13.668+0000 7fb945b85640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-29T04:09:13.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:13.669+0000 7fb945b85640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-29T04:09:13.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:13.669+0000 7fb945b85640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-29T04:09:13.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:13.669+0000 7fb945b85640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-29T04:09:13.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:13.669+0000 7fb945b85640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-29T04:09:13.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:13.669+0000 7fb945b85640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-29T04:09:16.498 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-29T04:09:16.528 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:09:16.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:16.637+0000 7ff073c0f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-29T04:09:16.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:16.637+0000 7ff073c0f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-29T04:09:16.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:16.637+0000 7ff073c0f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-29T04:09:16.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:16.638+0000 7ff073c0f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-29T04:09:16.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:16.638+0000 7ff073c0f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-29T04:09:16.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:16.638+0000 7ff073c0f640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-29T04:09:19.475 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-29T04:09:19.505 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-29T04:09:19.538 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T04:09:19.538 INFO:tasks.ceph:role: osd.4 2024-03-29T04:09:19.538 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi157.front.sepia.ceph.com 2024-03-29T04:09:19.538 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-29T04:09:19.608 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:09:19.609 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:09:19.609 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:09:19.609 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-29T04:09:19.609 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:09:19.609 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:09:19.609 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:09:19.609 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-29T04:09:19.609 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:09:19.609 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:09:19.641 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2024-03-29T04:09:19.643 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2024-03-29T04:09:19.643 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-29T04:09:19.722 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-29T04:09:19.794 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-29T04:09:19.862 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T04:09:19.863 INFO:tasks.ceph:role: osd.5 2024-03-29T04:09:19.863 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi157.front.sepia.ceph.com 2024-03-29T04:09:19.863 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-29T04:09:19.930 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:09:19.930 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:09:19.930 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:09:19.930 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-29T04:09:19.930 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:09:19.930 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:09:19.931 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:09:19.931 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-29T04:09:19.931 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:09:19.931 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:09:19.962 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2024-03-29T04:09:19.964 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2024-03-29T04:09:19.964 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-29T04:09:20.036 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-29T04:09:20.108 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-29T04:09:20.176 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T04:09:20.177 INFO:tasks.ceph:role: osd.6 2024-03-29T04:09:20.177 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi157.front.sepia.ceph.com 2024-03-29T04:09:20.177 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-29T04:09:20.243 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:09:20.243 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:09:20.244 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:09:20.244 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-29T04:09:20.244 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:09:20.244 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:09:20.244 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:09:20.244 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-29T04:09:20.244 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:09:20.244 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:09:20.275 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2024-03-29T04:09:20.278 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2024-03-29T04:09:20.278 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-29T04:09:20.358 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-29T04:09:20.430 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-29T04:09:20.498 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T04:09:20.498 INFO:tasks.ceph:role: osd.7 2024-03-29T04:09:20.498 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi157.front.sepia.ceph.com 2024-03-29T04:09:20.498 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-29T04:09:20.564 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:09:20.564 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:09:20.564 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:09:20.564 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-29T04:09:20.564 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:09:20.564 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:09:20.564 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:09:20.564 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-29T04:09:20.565 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:09:20.565 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:09:20.598 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2024-03-29T04:09:20.599 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2024-03-29T04:09:20.599 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-29T04:09:20.670 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-29T04:09:20.740 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:09:20.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:20.858+0000 7fdba3224640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-29T04:09:20.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:20.858+0000 7fdba3224640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-29T04:09:20.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:20.858+0000 7fdba3224640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-29T04:09:20.839 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:20.858+0000 7fdba3224640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-29T04:09:20.839 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:20.858+0000 7fdba3224640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-29T04:09:20.839 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:20.858+0000 7fdba3224640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-29T04:09:23.690 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-29T04:09:23.721 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:09:23.812 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:23.831+0000 7f3905749640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-29T04:09:23.812 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:23.831+0000 7f3905749640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-29T04:09:23.812 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:23.831+0000 7f3905749640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-29T04:09:23.812 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:23.831+0000 7f3905749640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-29T04:09:23.812 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:23.831+0000 7f3905749640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-29T04:09:23.812 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:23.831+0000 7f3905749640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-29T04:09:26.653 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-29T04:09:26.685 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:09:26.778 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:26.798+0000 7f1cc1a10640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-29T04:09:26.779 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:26.798+0000 7f1cc1a10640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-29T04:09:26.779 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:26.798+0000 7f1cc1a10640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-29T04:09:26.779 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:26.798+0000 7f1cc1a10640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-29T04:09:26.779 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:26.798+0000 7f1cc1a10640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-29T04:09:26.779 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:26.799+0000 7f1cc1a10640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-29T04:09:29.626 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-29T04:09:29.658 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:09:29.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:29.770+0000 7f686be48640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-29T04:09:29.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:29.770+0000 7f686be48640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-29T04:09:29.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:29.770+0000 7f686be48640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-29T04:09:29.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:29.770+0000 7f686be48640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-29T04:09:29.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:29.770+0000 7f686be48640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-29T04:09:29.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:29.771+0000 7f686be48640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-29T04:09:31.223 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:09:31.226 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:09:32.597 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-29T04:09:32.627 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-29T04:09:32.628 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:09:32.628 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-29T04:09:32.660 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:09:32.660 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-29T04:09:32.731 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:09:32.731 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-29T04:09:32.800 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:09:32.800 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-29T04:09:32.869 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:09:32.869 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-29T04:09:32.938 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:09:32.938 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-29T04:09:33.006 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:09:33.006 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-29T04:09:33.073 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:09:33.073 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-29T04:09:33.105 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:09:33.105 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-29T04:09:33.176 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:09:33.176 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-29T04:09:33.246 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:09:33.246 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-29T04:09:33.316 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:09:33.316 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-29T04:09:33.386 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:09:33.386 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-29T04:09:33.454 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:09:33.454 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-29T04:09:33.521 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:09:33.521 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-29T04:09:33.537 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:09:33.537 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-03-29T04:09:33.593 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:09:33.593 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-29T04:09:33.608 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:09:33.608 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-03-29T04:09:33.663 INFO:tasks.ceph:Adding keys to all mons... 2024-03-29T04:09:33.664 DEBUG:teuthology.orchestra.run.smithi148:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-29T04:09:33.666 DEBUG:teuthology.orchestra.run.smithi157:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-29T04:09:33.707 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr.y] 2024-03-29T04:09:33.707 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDfPgZmyAMBKRAA46y2Bz584MMFjMBsrsdVHg== 2024-03-29T04:09:33.707 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.a] 2024-03-29T04:09:33.707 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDfPgZmTrdyMBAAvPgJ04SmD7mCUz2pcAdZ3g== 2024-03-29T04:09:33.707 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.c] 2024-03-29T04:09:33.707 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDfPgZmXvzaNxAAOuU3MIAJe5jS9MAoxcgP9Q== 2024-03-29T04:09:33.708 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.0] 2024-03-29T04:09:33.708 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDjPgZmXItfKxAA2Jikd2Er5Kn/zTRZj3aeHg== 2024-03-29T04:09:33.708 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.1] 2024-03-29T04:09:33.708 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDmPgZmTiJPKRAAECwIVwapx3UnsEQG6IzVlg== 2024-03-29T04:09:33.708 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.2] 2024-03-29T04:09:33.708 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDpPgZmeNf2JxAANLSoLBF1pm2tRiYVQezq0A== 2024-03-29T04:09:33.708 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.3] 2024-03-29T04:09:33.708 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDsPgZmfKgbJhAAczXpuXVH2PniXy4WufH+lg== 2024-03-29T04:09:33.708 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr.x] 2024-03-29T04:09:33.708 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDfPgZma5P5LBAA0aGdlks9ozVfEYCSPc/Kcg== 2024-03-29T04:09:33.709 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.b] 2024-03-29T04:09:33.709 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDgPgZmvrdIARAA4vZcLEbAmmcLdaOZbgls8A== 2024-03-29T04:09:33.709 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.d] 2024-03-29T04:09:33.709 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDgPgZmK5OnCBAA/MqxtSGpybkRCfz71RftJw== 2024-03-29T04:09:33.709 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.4] 2024-03-29T04:09:33.709 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDwPgZmwT4/MxAAuCDZmNYrb+bk5XwG3epZ+Q== 2024-03-29T04:09:33.709 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.5] 2024-03-29T04:09:33.710 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDzPgZmSoOiMRAAQefLkDGw3zw2jWszEcpFnw== 2024-03-29T04:09:33.710 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.6] 2024-03-29T04:09:33.710 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQD2PgZm3uivLxAA1EfZ+gq3r0mgUbab0XcAAQ== 2024-03-29T04:09:33.710 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.7] 2024-03-29T04:09:33.710 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQD5PgZmbFv0LRAAWX++H2hdVRnjF+vFhXS5mA== 2024-03-29T04:09:33.710 INFO:teuthology.orchestra.run.smithi148.stdout:[client.0] 2024-03-29T04:09:33.711 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDgPgZmzmjtDBAAZ0mAEI3GHeThXHriL59lXg== 2024-03-29T04:09:33.711 INFO:teuthology.orchestra.run.smithi148.stdout:[client.2] 2024-03-29T04:09:33.711 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDgPgZmeXm6EhAAcTw1856TQ/rI+Y/9YWGuMQ== 2024-03-29T04:09:33.711 INFO:teuthology.orchestra.run.smithi148.stdout:[client.1] 2024-03-29T04:09:33.711 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDgPgZmOhImFhAAyeFmJdddMuNJNn529wAv7A== 2024-03-29T04:09:33.711 INFO:teuthology.orchestra.run.smithi148.stdout:[client.3] 2024-03-29T04:09:33.711 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDgPgZmpOn5GxAA1qplaVeRlhA6pGPbpzdrrQ== 2024-03-29T04:09:33.729 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr.y] 2024-03-29T04:09:33.730 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDfPgZmyAMBKRAA46y2Bz584MMFjMBsrsdVHg== 2024-03-29T04:09:33.730 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2024-03-29T04:09:33.730 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDfPgZmTrdyMBAAvPgJ04SmD7mCUz2pcAdZ3g== 2024-03-29T04:09:33.730 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2024-03-29T04:09:33.730 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDfPgZmXvzaNxAAOuU3MIAJe5jS9MAoxcgP9Q== 2024-03-29T04:09:33.730 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.0] 2024-03-29T04:09:33.730 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDjPgZmXItfKxAA2Jikd2Er5Kn/zTRZj3aeHg== 2024-03-29T04:09:33.730 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.1] 2024-03-29T04:09:33.730 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDmPgZmTiJPKRAAECwIVwapx3UnsEQG6IzVlg== 2024-03-29T04:09:33.730 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.2] 2024-03-29T04:09:33.731 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDpPgZmeNf2JxAANLSoLBF1pm2tRiYVQezq0A== 2024-03-29T04:09:33.731 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.3] 2024-03-29T04:09:33.731 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDsPgZmfKgbJhAAczXpuXVH2PniXy4WufH+lg== 2024-03-29T04:09:33.731 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr.x] 2024-03-29T04:09:33.731 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDfPgZma5P5LBAA0aGdlks9ozVfEYCSPc/Kcg== 2024-03-29T04:09:33.731 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2024-03-29T04:09:33.731 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDgPgZmvrdIARAA4vZcLEbAmmcLdaOZbgls8A== 2024-03-29T04:09:33.731 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2024-03-29T04:09:33.731 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDgPgZmK5OnCBAA/MqxtSGpybkRCfz71RftJw== 2024-03-29T04:09:33.731 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.4] 2024-03-29T04:09:33.731 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDwPgZmwT4/MxAAuCDZmNYrb+bk5XwG3epZ+Q== 2024-03-29T04:09:33.731 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.5] 2024-03-29T04:09:33.731 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDzPgZmSoOiMRAAQefLkDGw3zw2jWszEcpFnw== 2024-03-29T04:09:33.731 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.6] 2024-03-29T04:09:33.732 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQD2PgZm3uivLxAA1EfZ+gq3r0mgUbab0XcAAQ== 2024-03-29T04:09:33.732 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.7] 2024-03-29T04:09:33.732 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQD5PgZmbFv0LRAAWX++H2hdVRnjF+vFhXS5mA== 2024-03-29T04:09:33.732 INFO:teuthology.orchestra.run.smithi157.stdout:[client.0] 2024-03-29T04:09:33.732 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDgPgZmzmjtDBAAZ0mAEI3GHeThXHriL59lXg== 2024-03-29T04:09:33.732 INFO:teuthology.orchestra.run.smithi157.stdout:[client.2] 2024-03-29T04:09:33.733 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDgPgZmeXm6EhAAcTw1856TQ/rI+Y/9YWGuMQ== 2024-03-29T04:09:33.733 INFO:teuthology.orchestra.run.smithi157.stdout:[client.1] 2024-03-29T04:09:33.733 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDgPgZmOhImFhAAyeFmJdddMuNJNn529wAv7A== 2024-03-29T04:09:33.733 INFO:teuthology.orchestra.run.smithi157.stdout:[client.3] 2024-03-29T04:09:33.733 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDgPgZmpOn5GxAA1qplaVeRlhA6pGPbpzdrrQ== 2024-03-29T04:09:33.735 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-29T04:09:33.756 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-29T04:09:33.827 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:09:33.829 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:09:33.885 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:09:33.887 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:09:33.943 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:09:33.945 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:09:33.997 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:09:33.999 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:09:34.051 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:09:34.054 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:09:34.106 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:09:34.108 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:09:34.160 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-29T04:09:34.162 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-29T04:09:34.214 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:09:34.216 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:09:34.271 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:09:34.274 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:09:34.330 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:09:34.336 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:09:34.388 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:09:34.390 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:09:34.442 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:09:34.444 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:09:34.497 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:09:34.500 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:09:34.552 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:09:34.554 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:09:34.609 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:09:34.611 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:09:34.666 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:09:34.669 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:09:34.725 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:09:34.727 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:09:34.782 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-29T04:09:34.783 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-29T04:09:34.809 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-29T04:09:34.906 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-29T04:09:34.933 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-29T04:09:34.960 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-29T04:09:35.058 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-29T04:09:35.084 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-29T04:09:35.154 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-29T04:09:35.253 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-29T04:09:35.280 DEBUG:teuthology.orchestra.run.smithi148:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:09:35.283 DEBUG:teuthology.orchestra.run.smithi157:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:09:35.335 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-29T04:09:35.335 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-29T04:09:35.336 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-29T04:09:35.338 INFO:tasks.ceph.mon.a:Started 2024-03-29T04:09:35.338 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-29T04:09:35.338 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-29T04:09:35.378 INFO:tasks.ceph.mon.b:Started 2024-03-29T04:09:35.378 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-29T04:09:35.378 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-29T04:09:35.380 INFO:tasks.ceph.mon.c:Started 2024-03-29T04:09:35.380 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-29T04:09:35.380 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-29T04:09:35.381 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-29T04:09:35.383 INFO:tasks.ceph.mgr.y:Started 2024-03-29T04:09:35.383 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-29T04:09:35.383 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-29T04:09:35.385 INFO:tasks.ceph.mgr.x:Started 2024-03-29T04:09:35.385 INFO:tasks.ceph:Setting crush tunables to default 2024-03-29T04:09:35.385 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd crush tunables default 2024-03-29T04:09:35.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.506+0000 7f3f01a4a640 1 Processor -- start 2024-03-29T04:09:35.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.506+0000 7f3f01a4a640 1 -- start start 2024-03-29T04:09:35.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.507+0000 7f3f01a4a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3efc06efe0 0x7f3efc0700d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.507+0000 7f3f01a4a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3efc0ce310 0x7f3efc0b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.507+0000 7f3f01a4a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3efc15d460 con 0x7f3efc15a120 2024-03-29T04:09:35.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.507+0000 7f3f01a4a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f3efc0d1b20 con 0x7f3efc0ce310 2024-03-29T04:09:35.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.507+0000 7f3f01a4a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3efc0d0f60 con 0x7f3efc06efe0 2024-03-29T04:09:35.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.507+0000 7f3efb7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3efc0ce310 0x7f3efc0b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.507+0000 7f3efb7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3efc0ce310 0x7f3efc0b6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34710/0 (socket says 172.21.15.148:34710) 2024-03-29T04:09:35.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.507+0000 7f3efb7fe640 1 -- 172.21.15.148:0/4232108335 learned_addr learned my addr 172.21.15.148:0/4232108335 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:09:35.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.507+0000 7f3efb7fe640 1 -- 172.21.15.148:0/4232108335 >> v2:172.21.15.148:3300/0 conn(0x7f3efc0ce310 msgr2=0x7f3efc0b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-29T04:09:35.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.507+0000 7f3efb7fe640 1 -- 172.21.15.148:0/4232108335 >> v2:172.21.15.148:3300/0 conn(0x7f3efc0ce310 msgr2=0x7f3efc0b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T04:09:35.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.507+0000 7f3efb7fe640 1 --2- 172.21.15.148:0/4232108335 >> v2:172.21.15.148:3300/0 conn(0x7f3efc0ce310 0x7f3efc0b6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:09:35.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.507+0000 7f3efb7fe640 1 --2- 172.21.15.148:0/4232108335 >> v2:172.21.15.148:3300/0 conn(0x7f3efc0ce310 0x7f3efc0b6f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T04:09:35.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.507+0000 7f3efa7fc640 1 --2- 172.21.15.148:0/4232108335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3efc06efe0 0x7f3efc0700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.507+0000 7f3efa7fc640 1 -- 172.21.15.148:0/4232108335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3efc06efe0 msgr2=0x7f3efc0700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-29T04:09:35.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.507+0000 7f3efa7fc640 1 -- 172.21.15.148:0/4232108335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3efc06efe0 msgr2=0x7f3efc0700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T04:09:35.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.507+0000 7f3efa7fc640 1 --2- 172.21.15.148:0/4232108335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3efc06efe0 0x7f3efc0700d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:09:35.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.507+0000 7f3efa7fc640 1 --2- 172.21.15.148:0/4232108335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3efc06efe0 0x7f3efc0700d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T04:09:35.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.663+0000 7f3ef9ffb640 1 -- 172.21.15.148:0/4232108335 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4104830216 0 0) 0x7f3efc15d460 con 0x7f3efc15a120 2024-03-29T04:09:35.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.663+0000 7f3ef9ffb640 1 -- 172.21.15.148:0/4232108335 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ee8003520 con 0x7f3efc15a120 2024-03-29T04:09:35.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.664+0000 7f3ef9ffb640 1 -- 172.21.15.148:0/4232108335 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1216177880 0 0) 0x7f3ee8003520 con 0x7f3efc15a120 2024-03-29T04:09:35.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.664+0000 7f3ef9ffb640 1 -- 172.21.15.148:0/4232108335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3efc06efe0 msgr2=0x7f3efc0700d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:09:35.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.664+0000 7f3ef9ffb640 1 --2- 172.21.15.148:0/4232108335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3efc06efe0 0x7f3efc0700d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.664+0000 7f3ef9ffb640 1 -- 172.21.15.148:0/4232108335 >> v2:172.21.15.148:3300/0 conn(0x7f3efc0ce310 msgr2=0x7f3efc0b6f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:09:35.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.664+0000 7f3ef9ffb640 1 --2- 172.21.15.148:0/4232108335 >> v2:172.21.15.148:3300/0 conn(0x7f3efc0ce310 0x7f3efc0b6f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.664+0000 7f3ef9ffb640 1 -- 172.21.15.148:0/4232108335 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3efc15d700 con 0x7f3efc15a120 2024-03-29T04:09:35.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.664+0000 7f3ef9ffb640 1 -- 172.21.15.148:0/4232108335 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f3eec001670 con 0x7f3efc15a120 2024-03-29T04:09:35.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.664+0000 7f3ef9ffb640 1 -- 172.21.15.148:0/4232108335 >> v1:172.21.15.148:6789/0 conn(0x7f3efc15a120 legacy=0x7f3efc06e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.664+0000 7f3ef9ffb640 1 --2- 172.21.15.148:0/4232108335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3efc06efe0 0x7f3ee80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.664+0000 7f3ef9ffb640 1 --2- 172.21.15.148:0/4232108335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ee8004030 0x7f3ee8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.664+0000 7f3ef9ffb640 1 --2- 172.21.15.148:0/4232108335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ee8004d40 0x7f3ee80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.664+0000 7f3ef9ffb640 1 -- 172.21.15.148:0/4232108335 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc0d0f60 con 0x7f3ee8004d40 2024-03-29T04:09:35.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.664+0000 7f3ef9ffb640 1 -- 172.21.15.148:0/4232108335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee800baa0 con 0x7f3ee8004030 2024-03-29T04:09:35.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.664+0000 7f3ef9ffb640 1 -- 172.21.15.148:0/4232108335 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3ee800bca0 con 0x7f3efc06efe0 2024-03-29T04:09:35.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.664+0000 7f3efb7fe640 1 --2- 172.21.15.148:0/4232108335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ee8004d40 0x7f3ee80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.664+0000 7f3efaffd640 1 --2- 172.21.15.148:0/4232108335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3efc06efe0 0x7f3ee80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.664+0000 7f3efa7fc640 1 --2- 172.21.15.148:0/4232108335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ee8004030 0x7f3ee8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.664+0000 7f3efb7fe640 1 -- 172.21.15.148:0/4232108335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3efc06efe0 msgr2=0x7f3ee80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:35.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.664+0000 7f3efb7fe640 1 --2- 172.21.15.148:0/4232108335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3efc06efe0 0x7f3ee80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.664+0000 7f3efb7fe640 1 -- 172.21.15.148:0/4232108335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ee8004030 msgr2=0x7f3ee8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:35.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.664+0000 7f3efb7fe640 1 --2- 172.21.15.148:0/4232108335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ee8004030 0x7f3ee8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.664+0000 7f3efb7fe640 1 -- 172.21.15.148:0/4232108335 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee8003520 con 0x7f3ee8004d40 2024-03-29T04:09:35.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.665+0000 7f3efb7fe640 1 --2- 172.21.15.148:0/4232108335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ee8004d40 0x7f3ee80051c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3efc160ff0 tx=0x7f3ee8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a7b3d8d5820e8df server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.665+0000 7f3ef9ffb640 1 -- 172.21.15.148:0/4232108335 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4032070 con 0x7f3ee8004d40 2024-03-29T04:09:35.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.665+0000 7f3ef9ffb640 1 -- 172.21.15.148:0/4232108335 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef402cab0 con 0x7f3ee8004d40 2024-03-29T04:09:35.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.665+0000 7f3ef9ffb640 1 -- 172.21.15.148:0/4232108335 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef40098c0 con 0x7f3ee8004d40 2024-03-29T04:09:35.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.666+0000 7f3f01a4a640 1 -- 172.21.15.148:0/4232108335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ee8004d40 msgr2=0x7f3ee80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.666+0000 7f3f01a4a640 1 --2- 172.21.15.148:0/4232108335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ee8004d40 0x7f3ee80051c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3efc160ff0 tx=0x7f3ee8005830 comp rx=0 tx=0).stop 2024-03-29T04:09:35.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.666+0000 7f3efb7fe640 1 -- 172.21.15.148:0/4232108335 reap_dead start 2024-03-29T04:09:35.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.666+0000 7f3f01a4a640 1 -- 172.21.15.148:0/4232108335 shutdown_connections 2024-03-29T04:09:35.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.666+0000 7f3f01a4a640 1 -- 172.21.15.148:0/4232108335 >> 172.21.15.148:0/4232108335 conn(0x7f3efc0cead0 msgr2=0x7f3efc0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:35.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.666+0000 7f3f01a4a640 1 -- 172.21.15.148:0/4232108335 shutdown_connections 2024-03-29T04:09:35.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.666+0000 7f3f01a4a640 1 -- 172.21.15.148:0/4232108335 wait complete. 2024-03-29T04:09:35.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.667+0000 7f3f01a4a640 1 Processor -- start 2024-03-29T04:09:35.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.668+0000 7f3f01a4a640 1 -- start start 2024-03-29T04:09:35.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.668+0000 7f3f01a4a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ee8004d40 0x7f3efc0cac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.668+0000 7f3f01a4a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3efc0b78f0 0x7f3efc06e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.668+0000 7f3f01a4a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3efc1f8040 0x7f3efc1f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.668+0000 7f3f01a4a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc06f5c0 con 0x7f3efc0b78f0 2024-03-29T04:09:35.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.668+0000 7f3f01a4a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc0ce7d0 con 0x7f3efc1f8040 2024-03-29T04:09:35.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.668+0000 7f3f01a4a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3efc06f740 con 0x7f3ee8004d40 2024-03-29T04:09:35.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.668+0000 7f3efa7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3efc0b78f0 0x7f3efc06e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.668+0000 7f3efa7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3efc0b78f0 0x7f3efc06e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34730/0 (socket says 172.21.15.148:34730) 2024-03-29T04:09:35.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.668+0000 7f3efa7fc640 1 -- 172.21.15.148:0/2167969199 learned_addr learned my addr 172.21.15.148:0/2167969199 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:09:35.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.668+0000 7f3efb7fe640 1 --2- 172.21.15.148:0/2167969199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3efc1f8040 0x7f3efc1f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.669+0000 7f3efaffd640 1 --2- 172.21.15.148:0/2167969199 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ee8004d40 0x7f3efc0cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.669+0000 7f3efa7fc640 1 -- 172.21.15.148:0/2167969199 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ee8004d40 msgr2=0x7f3efc0cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.669+0000 7f3efa7fc640 1 --2- 172.21.15.148:0/2167969199 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ee8004d40 0x7f3efc0cac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.669+0000 7f3efa7fc640 1 -- 172.21.15.148:0/2167969199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3efc1f8040 msgr2=0x7f3efc1f84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.669+0000 7f3efa7fc640 1 --2- 172.21.15.148:0/2167969199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3efc1f8040 0x7f3efc1f84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.669+0000 7f3efa7fc640 1 -- 172.21.15.148:0/2167969199 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3efc06e680 con 0x7f3efc0b78f0 2024-03-29T04:09:35.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.669+0000 7f3efaffd640 1 --2- 172.21.15.148:0/2167969199 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ee8004d40 0x7f3efc0cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:35.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.669+0000 7f3efa7fc640 1 --2- 172.21.15.148:0/2167969199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3efc0b78f0 0x7f3efc06e140 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4010920 tx=0x7f3ee4010de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.669+0000 7f3ef3fff640 1 -- 172.21.15.148:0/2167969199 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee401a020 con 0x7f3efc0b78f0 2024-03-29T04:09:35.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.669+0000 7f3ef3fff640 1 -- 172.21.15.148:0/2167969199 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ee400e960 con 0x7f3efc0b78f0 2024-03-29T04:09:35.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.669+0000 7f3f01a4a640 1 -- 172.21.15.148:0/2167969199 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3efc0cb240 con 0x7f3efc0b78f0 2024-03-29T04:09:35.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.669+0000 7f3ef3fff640 1 -- 172.21.15.148:0/2167969199 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee400ed50 con 0x7f3efc0b78f0 2024-03-29T04:09:35.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.669+0000 7f3f01a4a640 1 -- 172.21.15.148:0/2167969199 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3efc0cb800 con 0x7f3efc0b78f0 2024-03-29T04:09:35.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.669+0000 7f3ef3fff640 1 -- 172.21.15.148:0/2167969199 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3ee401d020 con 0x7f3efc0b78f0 2024-03-29T04:09:35.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.669+0000 7f3ef3fff640 1 -- 172.21.15.148:0/2167969199 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3ee4014910 con 0x7f3efc0b78f0 2024-03-29T04:09:35.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.670+0000 7f3f01a4a640 1 -- 172.21.15.148:0/2167969199 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3efc068850 con 0x7f3efc0b78f0 2024-03-29T04:09:35.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.672+0000 7f3ef3fff640 1 -- 172.21.15.148:0/2167969199 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3ee4024020 con 0x7f3efc0b78f0 2024-03-29T04:09:35.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.738+0000 7f3f01a4a640 1 -- 172.21.15.148:0/2167969199 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f3efc0d1cb0 con 0x7f3efc0b78f0 2024-03-29T04:09:35.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.739+0000 7f3ef3fff640 1 -- 172.21.15.148:0/2167969199 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f3ee4014430 con 0x7f3efc0b78f0 2024-03-29T04:09:35.718 INFO:teuthology.orchestra.run.smithi148.stderr:adjusted tunables profile to default 2024-03-29T04:09:35.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.740+0000 7f3f01a4a640 1 -- 172.21.15.148:0/2167969199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3efc0b78f0 msgr2=0x7f3efc06e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.740+0000 7f3f01a4a640 1 --2- 172.21.15.148:0/2167969199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3efc0b78f0 0x7f3efc06e140 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4010920 tx=0x7f3ee4010de0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.740+0000 7f3f01a4a640 1 -- 172.21.15.148:0/2167969199 shutdown_connections 2024-03-29T04:09:35.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.740+0000 7f3f01a4a640 1 --2- 172.21.15.148:0/2167969199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3efc1f8040 0x7f3efc1f84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.740+0000 7f3f01a4a640 1 --2- 172.21.15.148:0/2167969199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3efc0b78f0 0x7f3efc06e140 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.740+0000 7f3f01a4a640 1 --2- 172.21.15.148:0/2167969199 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ee8004d40 0x7f3efc0cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.740+0000 7f3f01a4a640 1 -- 172.21.15.148:0/2167969199 >> 172.21.15.148:0/2167969199 conn(0x7f3efc0cead0 msgr2=0x7f3efc06e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:35.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.741+0000 7f3f01a4a640 1 -- 172.21.15.148:0/2167969199 shutdown_connections 2024-03-29T04:09:35.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:35.741+0000 7f3f01a4a640 1 -- 172.21.15.148:0/2167969199 wait complete. 2024-03-29T04:09:35.731 INFO:tasks.ceph:check_enable_crimson: False 2024-03-29T04:09:35.731 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-29T04:09:35.731 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:09:35.731 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-29T04:09:35.758 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:09:35.758 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-29T04:09:35.826 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:09:35.826 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-29T04:09:35.892 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:09:35.893 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-29T04:09:35.960 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:09:35.960 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-29T04:09:35.987 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:09:35.987 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-29T04:09:36.054 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:09:36.054 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-29T04:09:36.119 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-03-29T04:09:36.140+0000 7fc059617040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-29T04:09:36.121 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:09:36.121 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-29T04:09:36.166 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-29T04:09:36.187+0000 7faf20847040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-29T04:09:36.175 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-03-29T04:09:36.197+0000 7fc059617040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-29T04:09:36.188 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 443b9c39-6bbf-4ed3-92a1-914d9f9cc53a 0 2024-03-29T04:09:36.219 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-29T04:09:36.240+0000 7faf20847040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-29T04:09:36.330 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.351+0000 7fe3bb374640 1 Processor -- start 2024-03-29T04:09:36.330 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.351+0000 7fe3bb374640 1 -- start start 2024-03-29T04:09:36.331 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.352+0000 7fe3bb374640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe3b40d1cd0 0x7fe3b406e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.331 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.352+0000 7fe3bb374640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3b415cd40 con 0x7fe3b40707e0 2024-03-29T04:09:36.331 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.352+0000 7fe3bb374640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3b415d020 con 0x7fe3b4070400 2024-03-29T04:09:36.332 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.352+0000 7fe3bb374640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fe3b415d300 con 0x7fe3b40d1cd0 2024-03-29T04:09:36.332 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.352+0000 7fe3b9110640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe3b4070400 0x7fe3b40ce340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41464/0 (socket says 172.21.15.157:41464) 2024-03-29T04:09:36.332 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.352+0000 7fe3b9110640 1 -- 172.21.15.157:0/1898425160 learned_addr learned my addr 172.21.15.157:0/1898425160 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-03-29T04:09:36.332 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.352+0000 7fe3b890f640 1 --2- 172.21.15.157:0/1898425160 >> v2:172.21.15.148:3300/0 conn(0x7fe3b40d1cd0 0x7fe3b406e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.332 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.352+0000 7fe3abfff640 1 -- 172.21.15.157:0/1898425160 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3009953943 0 0) 0x7fe3b415d020 con 0x7fe3b4070400 2024-03-29T04:09:36.332 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.352+0000 7fe3abfff640 1 -- 172.21.15.157:0/1898425160 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe3a0003520 con 0x7fe3b4070400 2024-03-29T04:09:36.332 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.352+0000 7fe3abfff640 1 -- 172.21.15.157:0/1898425160 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 448217078 0 0) 0x7fe3b415cd40 con 0x7fe3b40707e0 2024-03-29T04:09:36.332 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.352+0000 7fe3abfff640 1 -- 172.21.15.157:0/1898425160 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe3b415d020 con 0x7fe3b40707e0 2024-03-29T04:09:36.332 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.352+0000 7fe3b890f640 1 -- 172.21.15.157:0/1898425160 >> v1:172.21.15.157:6789/0 conn(0x7fe3b4070400 legacy=0x7fe3b40ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.332 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.352+0000 7fe3abfff640 1 -- 172.21.15.157:0/1898425160 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2958994955 0 0) 0x7fe3a0003520 con 0x7fe3b4070400 2024-03-29T04:09:36.333 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.352+0000 7fe3b890f640 1 -- 172.21.15.157:0/1898425160 >> v1:172.21.15.148:6789/0 conn(0x7fe3b40707e0 legacy=0x7fe3b4160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.333 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.352+0000 7fe3b890f640 1 -- 172.21.15.157:0/1898425160 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b415d5e0 con 0x7fe3b40d1cd0 2024-03-29T04:09:36.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.353+0000 7fe3b890f640 1 --2- 172.21.15.157:0/1898425160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3b40d1cd0 0x7fe3b406e8d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4009720 tx=0x7fe3a402b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a594a33eec9c45d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.353+0000 7fe3abfff640 1 -- 172.21.15.157:0/1898425160 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4037070 con 0x7fe3b40d1cd0 2024-03-29T04:09:36.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.353+0000 7fe3abfff640 1 -- 172.21.15.157:0/1898425160 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3a4030510 con 0x7fe3b40d1cd0 2024-03-29T04:09:36.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.353+0000 7fe3abfff640 1 -- 172.21.15.157:0/1898425160 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0001670 con 0x7fe3b40d1cd0 2024-03-29T04:09:36.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.354+0000 7fe3bb374640 1 -- 172.21.15.157:0/1898425160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3b40d1cd0 msgr2=0x7fe3b406e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.334 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.354+0000 7fe3bb374640 1 --2- 172.21.15.157:0/1898425160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3b40d1cd0 0x7fe3b406e8d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4009720 tx=0x7fe3a402b730 comp rx=0 tx=0).stop 2024-03-29T04:09:36.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.354+0000 7fe3bb374640 1 -- 172.21.15.157:0/1898425160 shutdown_connections 2024-03-29T04:09:36.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.354+0000 7fe3bb374640 1 --2- 172.21.15.157:0/1898425160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3b40d1cd0 0x7fe3b406e8d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.354+0000 7fe3bb374640 1 -- 172.21.15.157:0/1898425160 >> 172.21.15.157:0/1898425160 conn(0x7fe3b40b3ce0 msgr2=0x7fe3b40b40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:36.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.354+0000 7fe3bb374640 1 -- 172.21.15.157:0/1898425160 shutdown_connections 2024-03-29T04:09:36.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.354+0000 7fe3bb374640 1 -- 172.21.15.157:0/1898425160 wait complete. 2024-03-29T04:09:36.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.355+0000 7fe3bb374640 1 Processor -- start 2024-03-29T04:09:36.335 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.355+0000 7fe3bb374640 1 -- start start 2024-03-29T04:09:36.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.356+0000 7fe3bb374640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3b4070400 0x7fe3b4157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.356+0000 7fe3bb374640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3b40707e0 0x7fe3b40cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.356+0000 7fe3bb374640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b40d1cd0 0x7fe3b40cae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.356+0000 7fe3bb374640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b40cc740 con 0x7fe3b4070400 2024-03-29T04:09:36.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.356+0000 7fe3bb374640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b415cab0 con 0x7fe3b40d1cd0 2024-03-29T04:09:36.337 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.356+0000 7fe3bb374640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe3b40cc8c0 con 0x7fe3b40707e0 2024-03-29T04:09:36.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.356+0000 7fe3b9911640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b40d1cd0 0x7fe3b40cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.356+0000 7fe3b890f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3b40707e0 0x7fe3b40cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.356+0000 7fe3b890f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3b40707e0 0x7fe3b40cbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:47848/0 (socket says 172.21.15.157:47848) 2024-03-29T04:09:36.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.356+0000 7fe3b9911640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b40d1cd0 0x7fe3b40cae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46762/0 (socket says 172.21.15.157:46762) 2024-03-29T04:09:36.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.356+0000 7fe3b890f640 1 -- 172.21.15.157:0/2083134700 learned_addr learned my addr 172.21.15.157:0/2083134700 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-03-29T04:09:36.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.356+0000 7fe3b9110640 1 --2- 172.21.15.157:0/2083134700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3b4070400 0x7fe3b4157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.357+0000 7fe3b890f640 1 -- 172.21.15.157:0/2083134700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b40d1cd0 msgr2=0x7fe3b40cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.357+0000 7fe3b890f640 1 --2- 172.21.15.157:0/2083134700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b40d1cd0 0x7fe3b40cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.357+0000 7fe3b890f640 1 -- 172.21.15.157:0/2083134700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3b4070400 msgr2=0x7fe3b4157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.357+0000 7fe3b890f640 1 --2- 172.21.15.157:0/2083134700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3b4070400 0x7fe3b4157c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.357+0000 7fe3b890f640 1 -- 172.21.15.157:0/2083134700 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b4156b60 con 0x7fe3b40707e0 2024-03-29T04:09:36.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.357+0000 7fe3b9911640 1 --2- 172.21.15.157:0/2083134700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b40d1cd0 0x7fe3b40cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:36.338 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.357+0000 7fe3b9110640 1 --2- 172.21.15.157:0/2083134700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3b4070400 0x7fe3b4157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:36.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.357+0000 7fe3b890f640 1 --2- 172.21.15.157:0/2083134700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3b40707e0 0x7fe3b40cbc40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4035040 tx=0x7fe3a40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.357+0000 7fe3a9ffb640 1 -- 172.21.15.157:0/2083134700 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4042020 con 0x7fe3b40707e0 2024-03-29T04:09:36.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.357+0000 7fe3a9ffb640 1 -- 172.21.15.157:0/2083134700 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3a403d730 con 0x7fe3b40707e0 2024-03-29T04:09:36.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.357+0000 7fe3bb374640 1 -- 172.21.15.157:0/2083134700 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3b4156e40 con 0x7fe3b40707e0 2024-03-29T04:09:36.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.357+0000 7fe3a9ffb640 1 -- 172.21.15.157:0/2083134700 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4037040 con 0x7fe3b40707e0 2024-03-29T04:09:36.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.357+0000 7fe3bb374640 1 -- 172.21.15.157:0/2083134700 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3b4157400 con 0x7fe3b40707e0 2024-03-29T04:09:36.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.357+0000 7fe3a9ffb640 1 -- 172.21.15.157:0/2083134700 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe3a4045020 con 0x7fe3b40707e0 2024-03-29T04:09:36.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.357+0000 7fe3a9ffb640 1 -- 172.21.15.157:0/2083134700 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fe3a403ec80 con 0x7fe3b40707e0 2024-03-29T04:09:36.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.358+0000 7fe3bb374640 1 -- 172.21.15.157:0/2083134700 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3b4068850 con 0x7fe3b40707e0 2024-03-29T04:09:36.339 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.361+0000 7fe3a9ffb640 1 -- 172.21.15.157:0/2083134700 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe3a404c030 con 0x7fe3b40707e0 2024-03-29T04:09:36.412 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.433+0000 7fe3bb374640 1 -- 172.21.15.157:0/2083134700 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd new", "uuid": "443b9c39-6bbf-4ed3-92a1-914d9f9cc53a", "id": 0} v 0) -- 0x7fe3b40d17a0 con 0x7fe3b40707e0 2024-03-29T04:09:36.415 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.436+0000 7fe3a9ffb640 1 -- 172.21.15.157:0/2083134700 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "443b9c39-6bbf-4ed3-92a1-914d9f9cc53a", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fe3a403e420 con 0x7fe3b40707e0 2024-03-29T04:09:36.415 INFO:teuthology.orchestra.run.smithi157.stdout:0 2024-03-29T04:09:36.416 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.437+0000 7fe3bb374640 1 -- 172.21.15.157:0/2083134700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3b40707e0 msgr2=0x7fe3b40cbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.416 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.437+0000 7fe3bb374640 1 --2- 172.21.15.157:0/2083134700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3b40707e0 0x7fe3b40cbc40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4035040 tx=0x7fe3a40046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.416 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.437+0000 7fe3bb374640 1 -- 172.21.15.157:0/2083134700 shutdown_connections 2024-03-29T04:09:36.416 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.437+0000 7fe3bb374640 1 --2- 172.21.15.157:0/2083134700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b40d1cd0 0x7fe3b40cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.417 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.437+0000 7fe3bb374640 1 --2- 172.21.15.157:0/2083134700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3b40707e0 0x7fe3b40cbc40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.417 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.437+0000 7fe3bb374640 1 --2- 172.21.15.157:0/2083134700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3b4070400 0x7fe3b4157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.417 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.437+0000 7fe3bb374640 1 -- 172.21.15.157:0/2083134700 >> 172.21.15.157:0/2083134700 conn(0x7fe3b40b3ce0 msgr2=0x7fe3b40b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:36.417 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.437+0000 7fe3bb374640 1 -- 172.21.15.157:0/2083134700 shutdown_connections 2024-03-29T04:09:36.417 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.437+0000 7fe3bb374640 1 -- 172.21.15.157:0/2083134700 wait complete. 2024-03-29T04:09:36.426 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42 1 2024-03-29T04:09:36.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.548+0000 7f8505475640 1 Processor -- start 2024-03-29T04:09:36.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.548+0000 7f8505475640 1 -- start start 2024-03-29T04:09:36.528 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.549+0000 7f8505475640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8500147f20 0x7f85000ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.528 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.549+0000 7f8505475640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f85000b3fd0 0x7f850006c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.528 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.549+0000 7f8505475640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85000b4520 con 0x7f850006f930 2024-03-29T04:09:36.528 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.549+0000 7f8505475640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f85000b4800 con 0x7f8500147f20 2024-03-29T04:09:36.528 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.549+0000 7f8505475640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f85000b4ae0 con 0x7f85000b3fd0 2024-03-29T04:09:36.529 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.549+0000 7f84f7fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f85000b3fd0 0x7f850006c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.529 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.549+0000 7f84f7fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f85000b3fd0 0x7f850006c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46768/0 (socket says 172.21.15.157:46768) 2024-03-29T04:09:36.529 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.549+0000 7f84f7fff640 1 -- 172.21.15.157:0/2858409672 learned_addr learned my addr 172.21.15.157:0/2858409672 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-03-29T04:09:36.529 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.549+0000 7f84feffd640 1 --2- 172.21.15.157:0/2858409672 >> v2:172.21.15.148:3300/0 conn(0x7f8500147f20 0x7f85000ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.529 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.549+0000 7f84f7fff640 1 -- 172.21.15.157:0/2858409672 >> v2:172.21.15.148:3300/0 conn(0x7f8500147f20 msgr2=0x7f85000ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:36.529 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.549+0000 7f84f7fff640 1 --2- 172.21.15.157:0/2858409672 >> v2:172.21.15.148:3300/0 conn(0x7f8500147f20 0x7f85000ad8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.529 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.549+0000 7f84f7fff640 1 -- 172.21.15.157:0/2858409672 >> v1:172.21.15.148:6789/0 conn(0x7f850006f930 legacy=0x7f85000b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.529 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.549+0000 7f84f7fff640 1 -- 172.21.15.157:0/2858409672 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85000b4dc0 con 0x7f85000b3fd0 2024-03-29T04:09:36.530 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.550+0000 7f84f7fff640 1 --2- 172.21.15.157:0/2858409672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85000b3fd0 0x7f850006c470 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f85001483f0 tx=0x7f84e802b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fea89f23e63143c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.530 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.550+0000 7f84fe7fc640 1 -- 172.21.15.157:0/2858409672 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8034070 con 0x7f85000b3fd0 2024-03-29T04:09:36.530 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.550+0000 7f84fe7fc640 1 -- 172.21.15.157:0/2858409672 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84e8030860 con 0x7f85000b3fd0 2024-03-29T04:09:36.530 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.550+0000 7f84fe7fc640 1 -- 172.21.15.157:0/2858409672 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8030c30 con 0x7f85000b3fd0 2024-03-29T04:09:36.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.551+0000 7f8505475640 1 -- 172.21.15.157:0/2858409672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85000b3fd0 msgr2=0x7f850006c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.551+0000 7f8505475640 1 --2- 172.21.15.157:0/2858409672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85000b3fd0 0x7f850006c470 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f85001483f0 tx=0x7f84e802b9a0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.551+0000 7f8505475640 1 -- 172.21.15.157:0/2858409672 shutdown_connections 2024-03-29T04:09:36.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.551+0000 7f8505475640 1 --2- 172.21.15.157:0/2858409672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85000b3fd0 0x7f850006c470 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.551+0000 7f8505475640 1 --2- 172.21.15.157:0/2858409672 >> v2:172.21.15.148:3300/0 conn(0x7f8500147f20 0x7f85000ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.551+0000 7f8505475640 1 -- 172.21.15.157:0/2858409672 >> 172.21.15.157:0/2858409672 conn(0x7f85000d1060 msgr2=0x7f85000d1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:36.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.551+0000 7f8505475640 1 -- 172.21.15.157:0/2858409672 shutdown_connections 2024-03-29T04:09:36.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.551+0000 7f8505475640 1 -- 172.21.15.157:0/2858409672 wait complete. 2024-03-29T04:09:36.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.552+0000 7f8505475640 1 Processor -- start 2024-03-29T04:09:36.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.552+0000 7f8505475640 1 -- start start 2024-03-29T04:09:36.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.553+0000 7f8505475640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f850006f930 0x7f85000ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.553+0000 7f8505475640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85000b3fd0 0x7f85000c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.553+0000 7f8505475640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8500147f20 0x7f85000cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.553+0000 7f8505475640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f850006ff40 con 0x7f85000b3fd0 2024-03-29T04:09:36.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.553+0000 7f8505475640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f850006fdc0 con 0x7f8500147f20 2024-03-29T04:09:36.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.553+0000 7f8505475640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f85000700c0 con 0x7f850006f930 2024-03-29T04:09:36.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.553+0000 7f84feffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f850006f930 0x7f85000ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.553+0000 7f84f7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85000b3fd0 0x7f85000c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.553+0000 7f84feffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f850006f930 0x7f85000ce1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:47858/0 (socket says 172.21.15.157:47858) 2024-03-29T04:09:36.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.553+0000 7f84ff7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8500147f20 0x7f85000cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.553+0000 7f84feffd640 1 -- 172.21.15.157:0/2334903335 learned_addr learned my addr 172.21.15.157:0/2334903335 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-03-29T04:09:36.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.553+0000 7f84ff7fe640 1 -- 172.21.15.157:0/2334903335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f850006f930 msgr2=0x7f85000ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.553+0000 7f84ff7fe640 1 --2- 172.21.15.157:0/2334903335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f850006f930 0x7f85000ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.553+0000 7f84ff7fe640 1 -- 172.21.15.157:0/2334903335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85000b3fd0 msgr2=0x7f85000c68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.553+0000 7f84ff7fe640 1 --2- 172.21.15.157:0/2334903335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85000b3fd0 0x7f85000c68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.553+0000 7f84ff7fe640 1 -- 172.21.15.157:0/2334903335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85000cea00 con 0x7f8500147f20 2024-03-29T04:09:36.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.553+0000 7f84feffd640 1 --2- 172.21.15.157:0/2334903335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f850006f930 0x7f85000ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:36.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.553+0000 7f84ff7fe640 1 --2- 172.21.15.157:0/2334903335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8500147f20 0x7f85000cfaa0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f84f800c690 tx=0x7f84f800cb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.554+0000 7f84f77fe640 1 -- 172.21.15.157:0/2334903335 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f8003b70 con 0x7f8500147f20 2024-03-29T04:09:36.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.554+0000 7f84f77fe640 1 -- 172.21.15.157:0/2334903335 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84f8003d10 con 0x7f8500147f20 2024-03-29T04:09:36.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.554+0000 7f8505475640 1 -- 172.21.15.157:0/2334903335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85000cece0 con 0x7f8500147f20 2024-03-29T04:09:36.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.554+0000 7f84f77fe640 1 -- 172.21.15.157:0/2334903335 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f8005310 con 0x7f8500147f20 2024-03-29T04:09:36.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.554+0000 7f8505475640 1 -- 172.21.15.157:0/2334903335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85000cf2a0 con 0x7f8500147f20 2024-03-29T04:09:36.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.554+0000 7f84f77fe640 1 -- 172.21.15.157:0/2334903335 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f84f80054b0 con 0x7f8500147f20 2024-03-29T04:09:36.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.554+0000 7f84f77fe640 1 -- 172.21.15.157:0/2334903335 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f84f80125d0 con 0x7f8500147f20 2024-03-29T04:09:36.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.555+0000 7f8505475640 1 -- 172.21.15.157:0/2334903335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8500068850 con 0x7f8500147f20 2024-03-29T04:09:36.536 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.557+0000 7f84f77fe640 1 -- 172.21.15.157:0/2334903335 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f84f800a640 con 0x7f8500147f20 2024-03-29T04:09:36.609 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.630+0000 7f8505475640 1 -- 172.21.15.157:0/2334903335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42", "id": 1} v 0) -- 0x7f85000705a0 con 0x7f8500147f20 2024-03-29T04:09:36.612 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.633+0000 7f84f77fe640 1 -- 172.21.15.157:0/2334903335 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f84f800a820 con 0x7f8500147f20 2024-03-29T04:09:36.612 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-03-29T04:09:36.613 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.634+0000 7f8505475640 1 -- 172.21.15.157:0/2334903335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8500147f20 msgr2=0x7f85000cfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.613 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.634+0000 7f8505475640 1 --2- 172.21.15.157:0/2334903335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8500147f20 0x7f85000cfaa0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f84f800c690 tx=0x7f84f800cb50 comp rx=0 tx=0).stop 2024-03-29T04:09:36.613 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.634+0000 7f8505475640 1 -- 172.21.15.157:0/2334903335 shutdown_connections 2024-03-29T04:09:36.613 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.634+0000 7f8505475640 1 --2- 172.21.15.157:0/2334903335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8500147f20 0x7f85000cfaa0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.613 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.634+0000 7f8505475640 1 --2- 172.21.15.157:0/2334903335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85000b3fd0 0x7f85000c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.613 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.634+0000 7f8505475640 1 --2- 172.21.15.157:0/2334903335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f850006f930 0x7f85000ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.614 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.634+0000 7f8505475640 1 -- 172.21.15.157:0/2334903335 >> 172.21.15.157:0/2334903335 conn(0x7f85000d1060 msgr2=0x7f8500072470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:36.614 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.634+0000 7f8505475640 1 -- 172.21.15.157:0/2334903335 shutdown_connections 2024-03-29T04:09:36.614 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.634+0000 7f8505475640 1 -- 172.21.15.157:0/2334903335 wait complete. 2024-03-29T04:09:36.624 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 63302fb8-9a90-4da0-a37a-a6470547e394 2 2024-03-29T04:09:36.726 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.747+0000 7f6b6fd1c640 1 Processor -- start 2024-03-29T04:09:36.726 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.747+0000 7f6b6fd1c640 1 -- start start 2024-03-29T04:09:36.727 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.748+0000 7f6b6fd1c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6b680cc6a0 0x7f6b680cff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.727 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.748+0000 7f6b6fd1c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6b6815f560 0x7f6b680b7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.727 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.748+0000 7f6b6fd1c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b680b5030 con 0x7f6b680ce110 2024-03-29T04:09:36.727 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.748+0000 7f6b6fd1c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f6b680ce5a0 con 0x7f6b6815f560 2024-03-29T04:09:36.727 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.748+0000 7f6b6fd1c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f6b680ce7d0 con 0x7f6b680cc6a0 2024-03-29T04:09:36.728 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.748+0000 7f6b6dab8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6b680cc6a0 0x7f6b680cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.728 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.748+0000 7f6b6dab8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6b680cc6a0 0x7f6b680cff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46784/0 (socket says 172.21.15.157:46784) 2024-03-29T04:09:36.728 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.748+0000 7f6b6dab8640 1 -- 172.21.15.157:0/963369402 learned_addr learned my addr 172.21.15.157:0/963369402 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-03-29T04:09:36.728 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.748+0000 7f6b6e2b9640 1 --2- 172.21.15.157:0/963369402 >> v2:172.21.15.148:3300/0 conn(0x7f6b6815f560 0x7f6b680b7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.728 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.748+0000 7f6b6dab8640 1 -- 172.21.15.157:0/963369402 >> v2:172.21.15.148:3300/0 conn(0x7f6b6815f560 msgr2=0x7f6b680b7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:36.728 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.748+0000 7f6b6dab8640 1 --2- 172.21.15.157:0/963369402 >> v2:172.21.15.148:3300/0 conn(0x7f6b6815f560 0x7f6b680b7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.728 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.748+0000 7f6b6dab8640 1 -- 172.21.15.157:0/963369402 >> v1:172.21.15.148:6789/0 conn(0x7f6b680ce110 legacy=0x7f6b680cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.728 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.748+0000 7f6b6dab8640 1 -- 172.21.15.157:0/963369402 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b680b5310 con 0x7f6b680cc6a0 2024-03-29T04:09:36.729 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.749+0000 7f6b6dab8640 1 --2- 172.21.15.157:0/963369402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b680cc6a0 0x7f6b680cff50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6b680b4ca0 tx=0x7f6b6402b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5104f59ef55690e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.729 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.749+0000 7f6b6cab6640 1 -- 172.21.15.157:0/963369402 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b64035020 con 0x7f6b680cc6a0 2024-03-29T04:09:36.729 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.749+0000 7f6b6cab6640 1 -- 172.21.15.157:0/963369402 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b64031870 con 0x7f6b680cc6a0 2024-03-29T04:09:36.729 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.749+0000 7f6b6cab6640 1 -- 172.21.15.157:0/963369402 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b64031c40 con 0x7f6b680cc6a0 2024-03-29T04:09:36.729 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.750+0000 7f6b6fd1c640 1 -- 172.21.15.157:0/963369402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b680cc6a0 msgr2=0x7f6b680cff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.730 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.750+0000 7f6b6fd1c640 1 --2- 172.21.15.157:0/963369402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b680cc6a0 0x7f6b680cff50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6b680b4ca0 tx=0x7f6b6402b910 comp rx=0 tx=0).stop 2024-03-29T04:09:36.730 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.750+0000 7f6b6fd1c640 1 -- 172.21.15.157:0/963369402 shutdown_connections 2024-03-29T04:09:36.730 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.750+0000 7f6b6fd1c640 1 --2- 172.21.15.157:0/963369402 >> v2:172.21.15.148:3300/0 conn(0x7f6b6815f560 0x7f6b680b7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.730 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.750+0000 7f6b6fd1c640 1 --2- 172.21.15.157:0/963369402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b680cc6a0 0x7f6b680cff50 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.730 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.750+0000 7f6b6fd1c640 1 -- 172.21.15.157:0/963369402 >> 172.21.15.157:0/963369402 conn(0x7f6b680cead0 msgr2=0x7f6b680ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:36.730 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.750+0000 7f6b6fd1c640 1 -- 172.21.15.157:0/963369402 shutdown_connections 2024-03-29T04:09:36.730 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.750+0000 7f6b6fd1c640 1 -- 172.21.15.157:0/963369402 wait complete. 2024-03-29T04:09:36.733 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.751+0000 7f6b6fd1c640 1 Processor -- start 2024-03-29T04:09:36.733 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.751+0000 7f6b6fd1c640 1 -- start start 2024-03-29T04:09:36.733 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.752+0000 7f6b6fd1c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b680cc6a0 0x7f6b681540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.733 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.752+0000 7f6b6fd1c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b680ce110 0x7f6b680ae510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.733 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.752+0000 7f6b6fd1c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b6815f560 0x7f6b68161f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.733 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.752+0000 7f6b6fd1c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6b68158ee0 con 0x7f6b680ce110 2024-03-29T04:09:36.733 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.752+0000 7f6b6fd1c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6b680b6d90 con 0x7f6b680cc6a0 2024-03-29T04:09:36.733 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.752+0000 7f6b6fd1c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6b68159060 con 0x7f6b6815f560 2024-03-29T04:09:36.733 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.752+0000 7f6b6dab8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b680cc6a0 0x7f6b681540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.733 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.752+0000 7f6b6dab8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b680cc6a0 0x7f6b681540c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46798/0 (socket says 172.21.15.157:46798) 2024-03-29T04:09:36.733 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.752+0000 7f6b6dab8640 1 -- 172.21.15.157:0/2317967963 learned_addr learned my addr 172.21.15.157:0/2317967963 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-03-29T04:09:36.733 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.752+0000 7f6b6e2b9640 1 --2- 172.21.15.157:0/2317967963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b6815f560 0x7f6b68161f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.734 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.752+0000 7f6b6d2b7640 1 --2- 172.21.15.157:0/2317967963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b680ce110 0x7f6b680ae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.734 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.752+0000 7f6b6dab8640 1 -- 172.21.15.157:0/2317967963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b6815f560 msgr2=0x7f6b68161f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.734 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.752+0000 7f6b6dab8640 1 --2- 172.21.15.157:0/2317967963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b6815f560 0x7f6b68161f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.734 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.752+0000 7f6b6dab8640 1 -- 172.21.15.157:0/2317967963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b680ce110 msgr2=0x7f6b680ae510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.734 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.752+0000 7f6b6dab8640 1 --2- 172.21.15.157:0/2317967963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b680ce110 0x7f6b680ae510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.734 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.752+0000 7f6b6dab8640 1 -- 172.21.15.157:0/2317967963 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b681628e0 con 0x7f6b680cc6a0 2024-03-29T04:09:36.734 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.752+0000 7f6b6dab8640 1 --2- 172.21.15.157:0/2317967963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b680cc6a0 0x7f6b681540c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6b6402b950 tx=0x7f6b640046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.734 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.752+0000 7f6b5e7fc640 1 -- 172.21.15.157:0/2317967963 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b64040070 con 0x7f6b680cc6a0 2024-03-29T04:09:36.734 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.752+0000 7f6b5e7fc640 1 -- 172.21.15.157:0/2317967963 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b64030d00 con 0x7f6b680cc6a0 2024-03-29T04:09:36.734 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.752+0000 7f6b5e7fc640 1 -- 172.21.15.157:0/2317967963 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b64035040 con 0x7f6b680cc6a0 2024-03-29T04:09:36.734 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.752+0000 7f6b6fd1c640 1 -- 172.21.15.157:0/2317967963 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b680c9370 con 0x7f6b680cc6a0 2024-03-29T04:09:36.734 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.753+0000 7f6b6fd1c640 1 -- 172.21.15.157:0/2317967963 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b680c9930 con 0x7f6b680cc6a0 2024-03-29T04:09:36.734 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.753+0000 7f6b5e7fc640 1 -- 172.21.15.157:0/2317967963 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6b6404b060 con 0x7f6b680cc6a0 2024-03-29T04:09:36.734 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.753+0000 7f6b5e7fc640 1 -- 172.21.15.157:0/2317967963 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f6b64044820 con 0x7f6b680cc6a0 2024-03-29T04:09:36.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.754+0000 7f6b6fd1c640 1 -- 172.21.15.157:0/2317967963 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b68068850 con 0x7f6b680cc6a0 2024-03-29T04:09:36.738 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.756+0000 7f6b5e7fc640 1 -- 172.21.15.157:0/2317967963 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f6b64044400 con 0x7f6b680cc6a0 2024-03-29T04:09:36.752 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-03-29T04:09:36.774+0000 7fc059617040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-29T04:09:36.759 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-29T04:09:36.780+0000 7faf20847040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-29T04:09:36.814 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-29T04:09:36.834+0000 7faf20847040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-29T04:09:36.844 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-03-29T04:09:36.865+0000 7fc059617040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-29T04:09:36.858 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-29T04:09:36.879+0000 7faf20847040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-29T04:09:36.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.895+0000 7f6b6fd1c640 1 -- 172.21.15.157:0/2317967963 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "63302fb8-9a90-4da0-a37a-a6470547e394", "id": 2} v 0) -- 0x7f6b680cf8a0 con 0x7f6b680cc6a0 2024-03-29T04:09:36.879 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.900+0000 7f6b5e7fc640 1 -- 172.21.15.157:0/2317967963 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "63302fb8-9a90-4da0-a37a-a6470547e394", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f6b6403ab10 con 0x7f6b680cc6a0 2024-03-29T04:09:36.879 INFO:teuthology.orchestra.run.smithi157.stdout:2 2024-03-29T04:09:36.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.901+0000 7f6b6fd1c640 1 -- 172.21.15.157:0/2317967963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b680cc6a0 msgr2=0x7f6b681540c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.901+0000 7f6b6fd1c640 1 --2- 172.21.15.157:0/2317967963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b680cc6a0 0x7f6b681540c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6b6402b950 tx=0x7f6b640046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.901+0000 7f6b6fd1c640 1 -- 172.21.15.157:0/2317967963 shutdown_connections 2024-03-29T04:09:36.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.901+0000 7f6b6fd1c640 1 --2- 172.21.15.157:0/2317967963 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b6815f560 0x7f6b68161f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.901+0000 7f6b6fd1c640 1 --2- 172.21.15.157:0/2317967963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b680ce110 0x7f6b680ae510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.880 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.901+0000 7f6b6fd1c640 1 --2- 172.21.15.157:0/2317967963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b680cc6a0 0x7f6b681540c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.881 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.901+0000 7f6b6fd1c640 1 -- 172.21.15.157:0/2317967963 >> 172.21.15.157:0/2317967963 conn(0x7f6b680cead0 msgr2=0x7f6b680cff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:36.881 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.901+0000 7f6b6fd1c640 1 -- 172.21.15.157:0/2317967963 shutdown_connections 2024-03-29T04:09:36.881 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:36.901+0000 7f6b6fd1c640 1 -- 172.21.15.157:0/2317967963 wait complete. 2024-03-29T04:09:36.891 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb 3 2024-03-29T04:09:36.899 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-03-29T04:09:36.920+0000 7fc059617040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-29T04:09:36.913 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-29T04:09:36.934+0000 7faf20847040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-29T04:09:36.956 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-29T04:09:36.977+0000 7faf20847040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-29T04:09:36.956 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-03-29T04:09:36.977+0000 7fc059617040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-29T04:09:36.992 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.013+0000 7f3936bcd640 1 Processor -- start 2024-03-29T04:09:36.992 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.013+0000 7f3936bcd640 1 -- start start 2024-03-29T04:09:36.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.014+0000 7f3936bcd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f39300b4fe0 0x7f3930162c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.014+0000 7f3936bcd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f393015e670 0x7f39300b73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.014+0000 7f3936bcd640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39300cf0f0 con 0x7f393015e290 2024-03-29T04:09:36.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.014+0000 7f3936bcd640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f3930162180 con 0x7f39300b4fe0 2024-03-29T04:09:36.994 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.014+0000 7f3936bcd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3930162460 con 0x7f393015e670 2024-03-29T04:09:36.994 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.014+0000 7f393516a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f393015e670 0x7f39300b73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.994 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.014+0000 7f3934969640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f393015e290 0x7f39300b6cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41476/0 (socket says 172.21.15.157:41476) 2024-03-29T04:09:36.994 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.014+0000 7f3934969640 1 -- 172.21.15.157:0/4130381454 learned_addr learned my addr 172.21.15.157:0/4130381454 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-03-29T04:09:36.994 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.014+0000 7f392ffff640 1 --2- 172.21.15.157:0/4130381454 >> v2:172.21.15.148:3300/0 conn(0x7f39300b4fe0 0x7f3930162c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.994 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.014+0000 7f393516a640 1 -- 172.21.15.157:0/4130381454 >> v2:172.21.15.148:3300/0 conn(0x7f39300b4fe0 msgr2=0x7f3930162c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:36.994 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.014+0000 7f392f7fe640 1 -- 172.21.15.157:0/4130381454 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2217706103 0 0) 0x7f39300cf0f0 con 0x7f393015e290 2024-03-29T04:09:36.994 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.014+0000 7f393516a640 1 --2- 172.21.15.157:0/4130381454 >> v2:172.21.15.148:3300/0 conn(0x7f39300b4fe0 0x7f3930162c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.994 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.014+0000 7f393516a640 1 -- 172.21.15.157:0/4130381454 >> v1:172.21.15.157:6789/0 conn(0x7f393015e290 legacy=0x7f39300b6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.994 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.014+0000 7f393516a640 1 -- 172.21.15.157:0/4130381454 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3930162740 con 0x7f393015e670 2024-03-29T04:09:36.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.015+0000 7f393516a640 1 --2- 172.21.15.157:0/4130381454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f393015e670 0x7f39300b73d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3928009d90 tx=0x7f392802bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e33e5b7298e62b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.015+0000 7f392f7fe640 1 -- 172.21.15.157:0/4130381454 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928039070 con 0x7f393015e670 2024-03-29T04:09:36.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.015+0000 7f392f7fe640 1 -- 172.21.15.157:0/4130381454 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f392802cb30 con 0x7f393015e670 2024-03-29T04:09:36.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.015+0000 7f392f7fe640 1 -- 172.21.15.157:0/4130381454 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928034440 con 0x7f393015e670 2024-03-29T04:09:36.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.016+0000 7f3936bcd640 1 -- 172.21.15.157:0/4130381454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f393015e670 msgr2=0x7f39300b73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.016+0000 7f3936bcd640 1 --2- 172.21.15.157:0/4130381454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f393015e670 0x7f39300b73d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3928009d90 tx=0x7f392802bd80 comp rx=0 tx=0).stop 2024-03-29T04:09:36.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.016+0000 7f3936bcd640 1 -- 172.21.15.157:0/4130381454 shutdown_connections 2024-03-29T04:09:36.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.016+0000 7f3936bcd640 1 --2- 172.21.15.157:0/4130381454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f393015e670 0x7f39300b73d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.016+0000 7f3936bcd640 1 --2- 172.21.15.157:0/4130381454 >> v2:172.21.15.148:3300/0 conn(0x7f39300b4fe0 0x7f3930162c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.016+0000 7f3936bcd640 1 -- 172.21.15.157:0/4130381454 >> 172.21.15.157:0/4130381454 conn(0x7f39300b3ce0 msgr2=0x7f39300b40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:36.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.016+0000 7f3936bcd640 1 -- 172.21.15.157:0/4130381454 shutdown_connections 2024-03-29T04:09:36.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.016+0000 7f3936bcd640 1 -- 172.21.15.157:0/4130381454 wait complete. 2024-03-29T04:09:36.997 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.017+0000 7f3936bcd640 1 Processor -- start 2024-03-29T04:09:36.997 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.017+0000 7f3936bcd640 1 -- start start 2024-03-29T04:09:36.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.018+0000 7f3936bcd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39300b4fe0 0x7f39300ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.018+0000 7f3936bcd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f393015e290 0x7f3930149e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.018+0000 7f3936bcd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f393015e670 0x7f39300cc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.018+0000 7f3936bcd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f39300b7910 con 0x7f393015e670 2024-03-29T04:09:36.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.018+0000 7f3936bcd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f39300cf3d0 con 0x7f39300b4fe0 2024-03-29T04:09:36.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.018+0000 7f3936bcd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f39300b7a90 con 0x7f393015e290 2024-03-29T04:09:36.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.018+0000 7f392ffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f393015e290 0x7f3930149e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.018+0000 7f3934969640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39300b4fe0 0x7f39300ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.018+0000 7f392ffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f393015e290 0x7f3930149e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:47878/0 (socket says 172.21.15.157:47878) 2024-03-29T04:09:36.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.018+0000 7f392ffff640 1 -- 172.21.15.157:0/2300478644 learned_addr learned my addr 172.21.15.157:0/2300478644 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-03-29T04:09:36.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.018+0000 7f393516a640 1 --2- 172.21.15.157:0/2300478644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f393015e670 0x7f39300cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.018+0000 7f392ffff640 1 -- 172.21.15.157:0/2300478644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39300b4fe0 msgr2=0x7f39300ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.018+0000 7f392ffff640 1 --2- 172.21.15.157:0/2300478644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39300b4fe0 0x7f39300ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.018+0000 7f392ffff640 1 -- 172.21.15.157:0/2300478644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f393015e670 msgr2=0x7f39300cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.018+0000 7f392ffff640 1 --2- 172.21.15.157:0/2300478644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f393015e670 0x7f39300cc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.018+0000 7f392ffff640 1 -- 172.21.15.157:0/2300478644 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39300ade20 con 0x7f393015e290 2024-03-29T04:09:37.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.018+0000 7f392ffff640 1 --2- 172.21.15.157:0/2300478644 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f393015e290 0x7f3930149e90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f391c00e9b0 tx=0x7f391c00ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:37.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.019+0000 7f392d7fa640 1 -- 172.21.15.157:0/2300478644 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391c017070 con 0x7f393015e290 2024-03-29T04:09:37.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.019+0000 7f392d7fa640 1 -- 172.21.15.157:0/2300478644 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f391c00c9a0 con 0x7f393015e290 2024-03-29T04:09:37.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.019+0000 7f392d7fa640 1 -- 172.21.15.157:0/2300478644 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391c00cd90 con 0x7f393015e290 2024-03-29T04:09:37.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.019+0000 7f3936bcd640 1 -- 172.21.15.157:0/2300478644 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f393014d4d0 con 0x7f393015e290 2024-03-29T04:09:37.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.019+0000 7f3936bcd640 1 -- 172.21.15.157:0/2300478644 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f393006cb00 con 0x7f393015e290 2024-03-29T04:09:37.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.019+0000 7f392d7fa640 1 -- 172.21.15.157:0/2300478644 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f391c01a070 con 0x7f393015e290 2024-03-29T04:09:37.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.019+0000 7f392d7fa640 1 -- 172.21.15.157:0/2300478644 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f391c006660 con 0x7f393015e290 2024-03-29T04:09:37.000 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.020+0000 7f3936bcd640 1 -- 172.21.15.157:0/2300478644 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3930068850 con 0x7f393015e290 2024-03-29T04:09:37.001 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.022+0000 7f392d7fa640 1 -- 172.21.15.157:0/2300478644 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f391c006020 con 0x7f393015e290 2024-03-29T04:09:37.002 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-03-29T04:09:37.023+0000 7fc059617040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-29T04:09:37.072 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.094+0000 7f3936bcd640 1 -- 172.21.15.157:0/2300478644 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb", "id": 3} v 0) -- 0x7f393015f270 con 0x7f393015e290 2024-03-29T04:09:37.073 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-29T04:09:37.093+0000 7faf20847040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-29T04:09:37.078 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.099+0000 7f392d7fa640 1 -- 172.21.15.157:0/2300478644 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f391c005990 con 0x7f393015e290 2024-03-29T04:09:37.078 INFO:teuthology.orchestra.run.smithi157.stdout:3 2024-03-29T04:09:37.079 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.100+0000 7f3936bcd640 1 -- 172.21.15.157:0/2300478644 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f393015e290 msgr2=0x7f3930149e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.079 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.100+0000 7f3936bcd640 1 --2- 172.21.15.157:0/2300478644 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f393015e290 0x7f3930149e90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f391c00e9b0 tx=0x7f391c00ee70 comp rx=0 tx=0).stop 2024-03-29T04:09:37.079 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.100+0000 7f3936bcd640 1 -- 172.21.15.157:0/2300478644 shutdown_connections 2024-03-29T04:09:37.079 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.100+0000 7f3936bcd640 1 --2- 172.21.15.157:0/2300478644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f393015e670 0x7f39300cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.079 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.100+0000 7f3936bcd640 1 --2- 172.21.15.157:0/2300478644 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f393015e290 0x7f3930149e90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.100+0000 7f3936bcd640 1 --2- 172.21.15.157:0/2300478644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39300b4fe0 0x7f39300ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.100+0000 7f3936bcd640 1 -- 172.21.15.157:0/2300478644 >> 172.21.15.157:0/2300478644 conn(0x7f39300b3ce0 msgr2=0x7f393015eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:37.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.100+0000 7f3936bcd640 1 -- 172.21.15.157:0/2300478644 shutdown_connections 2024-03-29T04:09:37.080 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.100+0000 7f3936bcd640 1 -- 172.21.15.157:0/2300478644 wait complete. 2024-03-29T04:09:37.089 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 96c44823-5bd3-44a9-a145-f59514696617 4 2024-03-29T04:09:37.124 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-03-29T04:09:37.145+0000 7fc059617040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-29T04:09:37.142 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-29T04:09:37.163+0000 7faf20847040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-29T04:09:37.188 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-29T04:09:37.209+0000 7faf20847040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-29T04:09:37.198 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-03-29T04:09:37.219+0000 7fc059617040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-29T04:09:37.225 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.246+0000 7f98cc322640 1 Processor -- start 2024-03-29T04:09:37.225 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.246+0000 7f98cc322640 1 -- start start 2024-03-29T04:09:37.227 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.247+0000 7f98cc322640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f98c4145770 0x7f98c406f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.227 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.247+0000 7f98cc322640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98c406fe70 0x7f98c40d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.228 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.247+0000 7f98cc322640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98c4146140 con 0x7f98c40d0930 2024-03-29T04:09:37.228 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.247+0000 7f98cc322640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f98c40702d0 con 0x7f98c4145770 2024-03-29T04:09:37.228 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.247+0000 7f98cc322640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f98c406b5b0 con 0x7f98c406fe70 2024-03-29T04:09:37.228 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.247+0000 7f98ca0be640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f98c4145770 0x7f98c406f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.228 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.247+0000 7f98c98bd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98c406fe70 0x7f98c40d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.228 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.247+0000 7f98ca8bf640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f98c40d0930 0x7f98c415fe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41480/0 (socket says 172.21.15.157:41480) 2024-03-29T04:09:37.228 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.247+0000 7f98ca8bf640 1 -- 172.21.15.157:0/2648284640 learned_addr learned my addr 172.21.15.157:0/2648284640 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-03-29T04:09:37.228 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.247+0000 7f98c98bd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98c406fe70 0x7f98c40d03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:47888/0 (socket says 172.21.15.157:47888) 2024-03-29T04:09:37.229 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.248+0000 7f98c90bc640 1 -- 172.21.15.157:0/2648284640 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3601194182 0 0) 0x7f98bc001490 con 0x7f98c40d0930 2024-03-29T04:09:37.229 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.248+0000 7f98c90bc640 1 -- 172.21.15.157:0/2648284640 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98b0003520 con 0x7f98c40d0930 2024-03-29T04:09:37.229 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.248+0000 7f98ca0be640 1 -- 172.21.15.157:0/2648284640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98c406fe70 msgr2=0x7f98c40d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:37.229 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.248+0000 7f98ca0be640 1 --2- 172.21.15.157:0/2648284640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98c406fe70 0x7f98c40d03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.229 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.248+0000 7f98ca0be640 1 -- 172.21.15.157:0/2648284640 >> v1:172.21.15.157:6789/0 conn(0x7f98c40d0930 legacy=0x7f98c415fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.229 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.248+0000 7f98ca0be640 1 -- 172.21.15.157:0/2648284640 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c406b800 con 0x7f98c4145770 2024-03-29T04:09:37.229 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.248+0000 7f98c98bd640 1 --2- 172.21.15.157:0/2648284640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98c406fe70 0x7f98c40d03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:37.229 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.248+0000 7f98ca0be640 1 --2- 172.21.15.157:0/2648284640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98c4145770 0x7f98c406f930 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f98b400dbc0 tx=0x7f98b402fc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f41f7e922ee318bf server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:37.229 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.249+0000 7f98c90bc640 1 -- 172.21.15.157:0/2648284640 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b40307d0 con 0x7f98c4145770 2024-03-29T04:09:37.229 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.249+0000 7f98c90bc640 1 -- 172.21.15.157:0/2648284640 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98b4030970 con 0x7f98c4145770 2024-03-29T04:09:37.229 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.249+0000 7f98c90bc640 1 -- 172.21.15.157:0/2648284640 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b4030d40 con 0x7f98c4145770 2024-03-29T04:09:37.233 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.249+0000 7f98cc322640 1 -- 172.21.15.157:0/2648284640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98c4145770 msgr2=0x7f98c406f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.233 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.249+0000 7f98cc322640 1 --2- 172.21.15.157:0/2648284640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98c4145770 0x7f98c406f930 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f98b400dbc0 tx=0x7f98b402fc30 comp rx=0 tx=0).stop 2024-03-29T04:09:37.233 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.249+0000 7f98cc322640 1 -- 172.21.15.157:0/2648284640 shutdown_connections 2024-03-29T04:09:37.233 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.249+0000 7f98cc322640 1 --2- 172.21.15.157:0/2648284640 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98c406fe70 0x7f98c40d03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.233 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.249+0000 7f98cc322640 1 --2- 172.21.15.157:0/2648284640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98c4145770 0x7f98c406f930 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.233 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.249+0000 7f98cc322640 1 -- 172.21.15.157:0/2648284640 >> 172.21.15.157:0/2648284640 conn(0x7f98c40d1060 msgr2=0x7f98c40d1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:37.233 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.249+0000 7f98cc322640 1 -- 172.21.15.157:0/2648284640 shutdown_connections 2024-03-29T04:09:37.233 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.249+0000 7f98cc322640 1 -- 172.21.15.157:0/2648284640 wait complete. 2024-03-29T04:09:37.233 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.250+0000 7f98cc322640 1 Processor -- start 2024-03-29T04:09:37.234 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.251+0000 7f98cc322640 1 -- start start 2024-03-29T04:09:37.234 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.251+0000 7f98cc322640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98c406fe70 0x7f98c415c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.234 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.251+0000 7f98cc322640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98c40d0930 0x7f98c415b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.234 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.251+0000 7f98cc322640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98c4145770 0x7f98c415e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.234 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.251+0000 7f98cc322640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f98c406c0a0 con 0x7f98c40d0930 2024-03-29T04:09:37.234 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.251+0000 7f98cc322640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f98c406bdc0 con 0x7f98c4145770 2024-03-29T04:09:37.234 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.251+0000 7f98cc322640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f98c4145b50 con 0x7f98c406fe70 2024-03-29T04:09:37.234 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.251+0000 7f98ca0be640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98c406fe70 0x7f98c415c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.234 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.251+0000 7f98ca8bf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98c4145770 0x7f98c415e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.234 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.251+0000 7f98ca0be640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98c406fe70 0x7f98c415c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:47894/0 (socket says 172.21.15.157:47894) 2024-03-29T04:09:37.234 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.251+0000 7f98ca0be640 1 -- 172.21.15.157:0/2968080810 learned_addr learned my addr 172.21.15.157:0/2968080810 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-03-29T04:09:37.234 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.251+0000 7f98c98bd640 1 --2- 172.21.15.157:0/2968080810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98c40d0930 0x7f98c415b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.234 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.252+0000 7f98ca8bf640 1 -- 172.21.15.157:0/2968080810 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98c406fe70 msgr2=0x7f98c415c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.252+0000 7f98ca8bf640 1 --2- 172.21.15.157:0/2968080810 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98c406fe70 0x7f98c415c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.252+0000 7f98ca8bf640 1 -- 172.21.15.157:0/2968080810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98c40d0930 msgr2=0x7f98c415b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.252+0000 7f98ca8bf640 1 --2- 172.21.15.157:0/2968080810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98c40d0930 0x7f98c415b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.252+0000 7f98ca8bf640 1 -- 172.21.15.157:0/2968080810 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c415ceb0 con 0x7f98c4145770 2024-03-29T04:09:37.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.252+0000 7f98c98bd640 1 --2- 172.21.15.157:0/2968080810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98c40d0930 0x7f98c415b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:37.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.252+0000 7f98ca8bf640 1 --2- 172.21.15.157:0/2968080810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98c4145770 0x7f98c415e340 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00caa0 tx=0x7f98bc00cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:37.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.252+0000 7f98baffd640 1 -- 172.21.15.157:0/2968080810 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc017070 con 0x7f98c4145770 2024-03-29T04:09:37.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.252+0000 7f98baffd640 1 -- 172.21.15.157:0/2968080810 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98bc00ac00 con 0x7f98c4145770 2024-03-29T04:09:37.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.252+0000 7f98cc322640 1 -- 172.21.15.157:0/2968080810 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c413b080 con 0x7f98c4145770 2024-03-29T04:09:37.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.252+0000 7f98baffd640 1 -- 172.21.15.157:0/2968080810 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc012400 con 0x7f98c4145770 2024-03-29T04:09:37.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.252+0000 7f98cc322640 1 -- 172.21.15.157:0/2968080810 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c413ce50 con 0x7f98c4145770 2024-03-29T04:09:37.236 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.252+0000 7f98baffd640 1 -- 172.21.15.157:0/2968080810 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f98bc01a070 con 0x7f98c4145770 2024-03-29T04:09:37.236 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.252+0000 7f98baffd640 1 -- 172.21.15.157:0/2968080810 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f98bc007580 con 0x7f98c4145770 2024-03-29T04:09:37.236 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.253+0000 7f98cc322640 1 -- 172.21.15.157:0/2968080810 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98c4068850 con 0x7f98c4145770 2024-03-29T04:09:37.236 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.255+0000 7f98baffd640 1 -- 172.21.15.157:0/2968080810 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f98bc021070 con 0x7f98c4145770 2024-03-29T04:09:37.241 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-29T04:09:37.262+0000 7faf20847040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-29T04:09:37.248 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-03-29T04:09:37.269+0000 7fc059617040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-29T04:09:37.287 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-29T04:09:37.309+0000 7faf20847040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-29T04:09:37.302 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-03-29T04:09:37.323+0000 7fc059617040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-29T04:09:37.306 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.328+0000 7f98cc322640 1 -- 172.21.15.157:0/2968080810 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "96c44823-5bd3-44a9-a145-f59514696617", "id": 4} v 0) -- 0x7f98c4070460 con 0x7f98c4145770 2024-03-29T04:09:37.312 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.333+0000 7f98baffd640 1 -- 172.21.15.157:0/2968080810 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "96c44823-5bd3-44a9-a145-f59514696617", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f98c4146140 con 0x7f98c4145770 2024-03-29T04:09:37.312 INFO:teuthology.orchestra.run.smithi157.stdout:4 2024-03-29T04:09:37.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.334+0000 7f98cc322640 1 -- 172.21.15.157:0/2968080810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98c4145770 msgr2=0x7f98c415e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.334+0000 7f98cc322640 1 --2- 172.21.15.157:0/2968080810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98c4145770 0x7f98c415e340 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00caa0 tx=0x7f98bc00cf60 comp rx=0 tx=0).stop 2024-03-29T04:09:37.314 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.334+0000 7f98cc322640 1 -- 172.21.15.157:0/2968080810 shutdown_connections 2024-03-29T04:09:37.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.334+0000 7f98cc322640 1 --2- 172.21.15.157:0/2968080810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98c4145770 0x7f98c415e340 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.334+0000 7f98cc322640 1 --2- 172.21.15.157:0/2968080810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98c40d0930 0x7f98c415b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.334+0000 7f98cc322640 1 --2- 172.21.15.157:0/2968080810 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98c406fe70 0x7f98c415c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.334+0000 7f98cc322640 1 -- 172.21.15.157:0/2968080810 >> 172.21.15.157:0/2968080810 conn(0x7f98c40d1060 msgr2=0x7f98c406cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:37.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.334+0000 7f98cc322640 1 -- 172.21.15.157:0/2968080810 shutdown_connections 2024-03-29T04:09:37.315 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.334+0000 7f98cc322640 1 -- 172.21.15.157:0/2968080810 wait complete. 2024-03-29T04:09:37.323 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 00de4401-55e6-4463-8a34-f9dadcc8aff3 5 2024-03-29T04:09:37.353 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-03-29T04:09:37.374+0000 7fc059617040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-29T04:09:37.425 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.445+0000 7f1cb0b59640 1 Processor -- start 2024-03-29T04:09:37.425 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.446+0000 7f1cb0b59640 1 -- start start 2024-03-29T04:09:37.427 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.446+0000 7f1cb0b59640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1cac070400 0x7f1cac0ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.427 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.446+0000 7f1cb0b59640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1cac0d1cd0 0x7f1cac06e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.428 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.446+0000 7f1cb0b59640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cac15cd40 con 0x7f1cac0707e0 2024-03-29T04:09:37.428 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.446+0000 7f1cb0b59640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f1cac15d020 con 0x7f1cac070400 2024-03-29T04:09:37.428 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.446+0000 7f1cb0b59640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f1cac15d300 con 0x7f1cac0d1cd0 2024-03-29T04:09:37.428 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.447+0000 7f1ca9d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1cac0d1cd0 0x7f1cac06e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.428 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.447+0000 7f1ca9d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1cac0d1cd0 0x7f1cac06e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46824/0 (socket says 172.21.15.157:46824) 2024-03-29T04:09:37.428 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.447+0000 7f1ca9d9b640 1 -- 172.21.15.157:0/1902309492 learned_addr learned my addr 172.21.15.157:0/1902309492 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-03-29T04:09:37.428 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.447+0000 7f1caa59c640 1 --2- 172.21.15.157:0/1902309492 >> v2:172.21.15.148:3300/0 conn(0x7f1cac070400 0x7f1cac0ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.428 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.447+0000 7f1ca9d9b640 1 -- 172.21.15.157:0/1902309492 >> v2:172.21.15.148:3300/0 conn(0x7f1cac070400 msgr2=0x7f1cac0ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:37.428 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.447+0000 7f1ca9d9b640 1 --2- 172.21.15.157:0/1902309492 >> v2:172.21.15.148:3300/0 conn(0x7f1cac070400 0x7f1cac0ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.428 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.447+0000 7f1ca9d9b640 1 -- 172.21.15.157:0/1902309492 >> v1:172.21.15.148:6789/0 conn(0x7f1cac0707e0 legacy=0x7f1cac160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.428 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.447+0000 7f1ca9d9b640 1 -- 172.21.15.157:0/1902309492 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cac15d5e0 con 0x7f1cac0d1cd0 2024-03-29T04:09:37.428 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.448+0000 7f1ca9d9b640 1 --2- 172.21.15.157:0/1902309492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cac0d1cd0 0x7f1cac06e8d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0cd020 tx=0x7f1c9c02b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65a9d33f4a31fdbc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:37.429 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.448+0000 7f1ca959a640 1 -- 172.21.15.157:0/1902309492 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c035020 con 0x7f1cac0d1cd0 2024-03-29T04:09:37.429 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.448+0000 7f1ca959a640 1 -- 172.21.15.157:0/1902309492 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c9c031870 con 0x7f1cac0d1cd0 2024-03-29T04:09:37.429 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.448+0000 7f1ca959a640 1 -- 172.21.15.157:0/1902309492 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c031c40 con 0x7f1cac0d1cd0 2024-03-29T04:09:37.429 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.448+0000 7f1cb0b59640 1 -- 172.21.15.157:0/1902309492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cac0d1cd0 msgr2=0x7f1cac06e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.429 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.448+0000 7f1cb0b59640 1 --2- 172.21.15.157:0/1902309492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cac0d1cd0 0x7f1cac06e8d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0cd020 tx=0x7f1c9c02b910 comp rx=0 tx=0).stop 2024-03-29T04:09:37.429 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.449+0000 7f1cb0b59640 1 -- 172.21.15.157:0/1902309492 shutdown_connections 2024-03-29T04:09:37.429 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.449+0000 7f1cb0b59640 1 --2- 172.21.15.157:0/1902309492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cac0d1cd0 0x7f1cac06e8d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.429 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.449+0000 7f1cb0b59640 1 --2- 172.21.15.157:0/1902309492 >> v2:172.21.15.148:3300/0 conn(0x7f1cac070400 0x7f1cac0ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.429 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.449+0000 7f1cb0b59640 1 -- 172.21.15.157:0/1902309492 >> 172.21.15.157:0/1902309492 conn(0x7f1cac0b3ce0 msgr2=0x7f1cac0b40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:37.429 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.449+0000 7f1cb0b59640 1 -- 172.21.15.157:0/1902309492 shutdown_connections 2024-03-29T04:09:37.429 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.449+0000 7f1cb0b59640 1 -- 172.21.15.157:0/1902309492 wait complete. 2024-03-29T04:09:37.432 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.450+0000 7f1cb0b59640 1 Processor -- start 2024-03-29T04:09:37.432 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.450+0000 7f1cb0b59640 1 -- start start 2024-03-29T04:09:37.432 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.451+0000 7f1cb0b59640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cac070400 0x7f1cac152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.433 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.451+0000 7f1cb0b59640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cac0707e0 0x7f1cac155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.433 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.451+0000 7f1cb0b59640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cac0d1cd0 0x7f1cac1557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.433 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.451+0000 7f1cb0b59640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1cac15c9e0 con 0x7f1cac0707e0 2024-03-29T04:09:37.433 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.451+0000 7f1cb0b59640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1cac15c860 con 0x7f1cac070400 2024-03-29T04:09:37.433 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.451+0000 7f1cb0b59640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1cac0cc740 con 0x7f1cac0d1cd0 2024-03-29T04:09:37.433 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.451+0000 7f1caa59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cac070400 0x7f1cac152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.433 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.451+0000 7f1caa59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cac070400 0x7f1cac152a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46836/0 (socket says 172.21.15.157:46836) 2024-03-29T04:09:37.433 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.451+0000 7f1caa59c640 1 -- 172.21.15.157:0/704573782 learned_addr learned my addr 172.21.15.157:0/704573782 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-03-29T04:09:37.433 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.451+0000 7f1caad9d640 1 --2- 172.21.15.157:0/704573782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cac0d1cd0 0x7f1cac1557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.433 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.451+0000 7f1ca9d9b640 1 --2- 172.21.15.157:0/704573782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cac0707e0 0x7f1cac155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.433 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.451+0000 7f1caa59c640 1 -- 172.21.15.157:0/704573782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cac0d1cd0 msgr2=0x7f1cac1557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.433 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.451+0000 7f1caa59c640 1 --2- 172.21.15.157:0/704573782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cac0d1cd0 0x7f1cac1557b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.434 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.451+0000 7f1caa59c640 1 -- 172.21.15.157:0/704573782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cac0707e0 msgr2=0x7f1cac155270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.434 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.451+0000 7f1caa59c640 1 --2- 172.21.15.157:0/704573782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cac0707e0 0x7f1cac155270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.434 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.451+0000 7f1caa59c640 1 -- 172.21.15.157:0/704573782 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cac154410 con 0x7f1cac070400 2024-03-29T04:09:37.434 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.451+0000 7f1ca9d9b640 1 --2- 172.21.15.157:0/704573782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cac0707e0 0x7f1cac155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:37.434 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.451+0000 7f1caa59c640 1 --2- 172.21.15.157:0/704573782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cac070400 0x7f1cac152a90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400e9b0 tx=0x7f1c9400ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:37.434 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.451+0000 7f1ca2ffd640 1 -- 172.21.15.157:0/704573782 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c94017070 con 0x7f1cac070400 2024-03-29T04:09:37.434 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.451+0000 7f1ca2ffd640 1 -- 172.21.15.157:0/704573782 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c9400c9a0 con 0x7f1cac070400 2024-03-29T04:09:37.435 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.451+0000 7f1cb0b59640 1 -- 172.21.15.157:0/704573782 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cac1546f0 con 0x7f1cac070400 2024-03-29T04:09:37.435 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.451+0000 7f1ca2ffd640 1 -- 172.21.15.157:0/704573782 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9400cd90 con 0x7f1cac070400 2024-03-29T04:09:37.435 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.451+0000 7f1cb0b59640 1 -- 172.21.15.157:0/704573782 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cac14ff10 con 0x7f1cac070400 2024-03-29T04:09:37.435 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.452+0000 7f1ca2ffd640 1 -- 172.21.15.157:0/704573782 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1c9401a070 con 0x7f1cac070400 2024-03-29T04:09:37.435 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.452+0000 7f1ca2ffd640 1 -- 172.21.15.157:0/704573782 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f1c9401cac0 con 0x7f1cac070400 2024-03-29T04:09:37.435 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.452+0000 7f1cb0b59640 1 -- 172.21.15.157:0/704573782 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cac068850 con 0x7f1cac070400 2024-03-29T04:09:37.435 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.454+0000 7f1ca2ffd640 1 -- 172.21.15.157:0/704573782 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f1c9401c600 con 0x7f1cac070400 2024-03-29T04:09:37.505 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.527+0000 7f1cb0b59640 1 -- 172.21.15.157:0/704573782 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00de4401-55e6-4463-8a34-f9dadcc8aff3", "id": 5} v 0) -- 0x7f1cac15f7c0 con 0x7f1cac070400 2024-03-29T04:09:37.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.532+0000 7f1ca2ffd640 1 -- 172.21.15.157:0/704573782 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00de4401-55e6-4463-8a34-f9dadcc8aff3", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f1c94005990 con 0x7f1cac070400 2024-03-29T04:09:37.511 INFO:teuthology.orchestra.run.smithi157.stdout:5 2024-03-29T04:09:37.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.533+0000 7f1cb0b59640 1 -- 172.21.15.157:0/704573782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cac070400 msgr2=0x7f1cac152a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.533+0000 7f1cb0b59640 1 --2- 172.21.15.157:0/704573782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cac070400 0x7f1cac152a90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400e9b0 tx=0x7f1c9400ee70 comp rx=0 tx=0).stop 2024-03-29T04:09:37.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.533+0000 7f1cb0b59640 1 -- 172.21.15.157:0/704573782 shutdown_connections 2024-03-29T04:09:37.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.533+0000 7f1cb0b59640 1 --2- 172.21.15.157:0/704573782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cac0d1cd0 0x7f1cac1557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.533+0000 7f1cb0b59640 1 --2- 172.21.15.157:0/704573782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cac0707e0 0x7f1cac155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.513 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.533+0000 7f1cb0b59640 1 --2- 172.21.15.157:0/704573782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cac070400 0x7f1cac152a90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.513 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.533+0000 7f1cb0b59640 1 -- 172.21.15.157:0/704573782 >> 172.21.15.157:0/704573782 conn(0x7f1cac0b3ce0 msgr2=0x7f1cac0b2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:37.513 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.533+0000 7f1cb0b59640 1 -- 172.21.15.157:0/704573782 shutdown_connections 2024-03-29T04:09:37.513 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.533+0000 7f1cb0b59640 1 -- 172.21.15.157:0/704573782 wait complete. 2024-03-29T04:09:37.523 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a 6 2024-03-29T04:09:37.668 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.687+0000 7fe989ec8640 1 Processor -- start 2024-03-29T04:09:37.668 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.688+0000 7fe989ec8640 1 -- start start 2024-03-29T04:09:37.668 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.688+0000 7fe989ec8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe9840b4fe0 0x7fe984162c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.668 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.688+0000 7fe989ec8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe98415e670 0x7fe9840b73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.668 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.688+0000 7fe989ec8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9840cf0f0 con 0x7fe98415e290 2024-03-29T04:09:37.668 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.688+0000 7fe989ec8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fe984162180 con 0x7fe9840b4fe0 2024-03-29T04:09:37.668 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.688+0000 7fe989ec8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe984162460 con 0x7fe98415e670 2024-03-29T04:09:37.668 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.688+0000 7fe983fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe98415e670 0x7fe9840b73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.668 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.688+0000 7fe9837fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe98415e290 0x7fe9840b6cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41492/0 (socket says 172.21.15.157:41492) 2024-03-29T04:09:37.668 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.688+0000 7fe9837fe640 1 -- 172.21.15.157:0/2483199626 learned_addr learned my addr 172.21.15.157:0/2483199626 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-03-29T04:09:37.668 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.689+0000 7fe982ffd640 1 --2- 172.21.15.157:0/2483199626 >> v2:172.21.15.148:3300/0 conn(0x7fe9840b4fe0 0x7fe984162c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.669 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.689+0000 7fe9827fc640 1 -- 172.21.15.157:0/2483199626 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3059030507 0 0) 0x7fe9840cf0f0 con 0x7fe98415e290 2024-03-29T04:09:37.669 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.689+0000 7fe9827fc640 1 -- 172.21.15.157:0/2483199626 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe96c003520 con 0x7fe98415e290 2024-03-29T04:09:37.669 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.689+0000 7fe983fff640 1 -- 172.21.15.157:0/2483199626 >> v2:172.21.15.148:3300/0 conn(0x7fe9840b4fe0 msgr2=0x7fe984162c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:37.669 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.689+0000 7fe983fff640 1 --2- 172.21.15.157:0/2483199626 >> v2:172.21.15.148:3300/0 conn(0x7fe9840b4fe0 0x7fe984162c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.669 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.689+0000 7fe983fff640 1 -- 172.21.15.157:0/2483199626 >> v1:172.21.15.157:6789/0 conn(0x7fe98415e290 legacy=0x7fe9840b6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.669 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.689+0000 7fe983fff640 1 -- 172.21.15.157:0/2483199626 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe984162740 con 0x7fe98415e670 2024-03-29T04:09:37.670 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-29T04:09:37.690+0000 7faf20847040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-29T04:09:37.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.689+0000 7fe983fff640 1 --2- 172.21.15.157:0/2483199626 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe98415e670 0x7fe9840b73d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe97c009d90 tx=0x7fe97c02bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eca80831e4c7b957 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:37.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.690+0000 7fe9827fc640 1 -- 172.21.15.157:0/2483199626 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c039070 con 0x7fe98415e670 2024-03-29T04:09:37.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.690+0000 7fe9827fc640 1 -- 172.21.15.157:0/2483199626 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe97c02cb30 con 0x7fe98415e670 2024-03-29T04:09:37.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.690+0000 7fe9827fc640 1 -- 172.21.15.157:0/2483199626 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c034440 con 0x7fe98415e670 2024-03-29T04:09:37.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.690+0000 7fe989ec8640 1 -- 172.21.15.157:0/2483199626 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe98415e670 msgr2=0x7fe9840b73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.673 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.690+0000 7fe989ec8640 1 --2- 172.21.15.157:0/2483199626 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe98415e670 0x7fe9840b73d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe97c009d90 tx=0x7fe97c02bd80 comp rx=0 tx=0).stop 2024-03-29T04:09:37.674 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.690+0000 7fe989ec8640 1 -- 172.21.15.157:0/2483199626 shutdown_connections 2024-03-29T04:09:37.674 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.690+0000 7fe989ec8640 1 --2- 172.21.15.157:0/2483199626 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe98415e670 0x7fe9840b73d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.674 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.690+0000 7fe989ec8640 1 --2- 172.21.15.157:0/2483199626 >> v2:172.21.15.148:3300/0 conn(0x7fe9840b4fe0 0x7fe984162c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.674 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.690+0000 7fe989ec8640 1 -- 172.21.15.157:0/2483199626 >> 172.21.15.157:0/2483199626 conn(0x7fe9840b3ce0 msgr2=0x7fe9840b40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:37.674 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.690+0000 7fe989ec8640 1 -- 172.21.15.157:0/2483199626 shutdown_connections 2024-03-29T04:09:37.674 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.690+0000 7fe989ec8640 1 -- 172.21.15.157:0/2483199626 wait complete. 2024-03-29T04:09:37.674 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.691+0000 7fe989ec8640 1 Processor -- start 2024-03-29T04:09:37.674 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.692+0000 7fe989ec8640 1 -- start start 2024-03-29T04:09:37.674 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.692+0000 7fe989ec8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9840b4fe0 0x7fe9841516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.674 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.692+0000 7fe989ec8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe98415e290 0x7fe984151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.674 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.692+0000 7fe989ec8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe98415e670 0x7fe984152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.674 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.692+0000 7fe989ec8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe9840b7910 con 0x7fe9840b4fe0 2024-03-29T04:09:37.674 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.692+0000 7fe989ec8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe9840cf3d0 con 0x7fe98415e670 2024-03-29T04:09:37.674 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.692+0000 7fe989ec8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe9840b7a90 con 0x7fe98415e290 2024-03-29T04:09:37.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.692+0000 7fe982ffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe98415e290 0x7fe984151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.692+0000 7fe983fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe98415e670 0x7fe984152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.692+0000 7fe982ffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe98415e290 0x7fe984151be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:47926/0 (socket says 172.21.15.157:47926) 2024-03-29T04:09:37.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.692+0000 7fe982ffd640 1 -- 172.21.15.157:0/2505377833 learned_addr learned my addr 172.21.15.157:0/2505377833 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-03-29T04:09:37.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.692+0000 7fe983fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe98415e670 0x7fe984152510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46846/0 (socket says 172.21.15.157:46846) 2024-03-29T04:09:37.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.692+0000 7fe9837fe640 1 --2- 172.21.15.157:0/2505377833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9840b4fe0 0x7fe9841516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.693+0000 7fe983fff640 1 -- 172.21.15.157:0/2505377833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe98415e290 msgr2=0x7fe984151be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.693+0000 7fe983fff640 1 --2- 172.21.15.157:0/2505377833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe98415e290 0x7fe984151be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.693+0000 7fe983fff640 1 -- 172.21.15.157:0/2505377833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9840b4fe0 msgr2=0x7fe9841516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.693+0000 7fe983fff640 1 --2- 172.21.15.157:0/2505377833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9840b4fe0 0x7fe9841516a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.693+0000 7fe983fff640 1 -- 172.21.15.157:0/2505377833 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9841565d0 con 0x7fe98415e670 2024-03-29T04:09:37.675 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.693+0000 7fe983fff640 1 --2- 172.21.15.157:0/2505377833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe98415e670 0x7fe984152510 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe97c033000 tx=0x7fe97c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:37.676 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.693+0000 7fe96bfff640 1 -- 172.21.15.157:0/2505377833 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c039050 con 0x7fe98415e670 2024-03-29T04:09:37.676 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.693+0000 7fe96bfff640 1 -- 172.21.15.157:0/2505377833 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe97c045070 con 0x7fe98415e670 2024-03-29T04:09:37.676 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.693+0000 7fe989ec8640 1 -- 172.21.15.157:0/2505377833 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe984156870 con 0x7fe98415e670 2024-03-29T04:09:37.676 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.693+0000 7fe96bfff640 1 -- 172.21.15.157:0/2505377833 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c009450 con 0x7fe98415e670 2024-03-29T04:09:37.676 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.693+0000 7fe989ec8640 1 -- 172.21.15.157:0/2505377833 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe984156e30 con 0x7fe98415e670 2024-03-29T04:09:37.676 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.693+0000 7fe96bfff640 1 -- 172.21.15.157:0/2505377833 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe97c03e850 con 0x7fe98415e670 2024-03-29T04:09:37.676 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.693+0000 7fe96bfff640 1 -- 172.21.15.157:0/2505377833 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fe97c040450 con 0x7fe98415e670 2024-03-29T04:09:37.676 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.694+0000 7fe989ec8640 1 -- 172.21.15.157:0/2505377833 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe984068850 con 0x7fe98415e670 2024-03-29T04:09:37.677 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.698+0000 7fe96bfff640 1 -- 172.21.15.157:0/2505377833 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe97c04a460 con 0x7fe98415e670 2024-03-29T04:09:37.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.770+0000 7fe989ec8640 1 -- 172.21.15.157:0/2505377833 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a", "id": 6} v 0) -- 0x7fe9840d17a0 con 0x7fe98415e670 2024-03-29T04:09:37.756 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.776+0000 7fe96bfff640 1 -- 172.21.15.157:0/2505377833 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fe97c03ec80 con 0x7fe98415e670 2024-03-29T04:09:37.756 INFO:teuthology.orchestra.run.smithi157.stdout:6 2024-03-29T04:09:37.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.776+0000 7fe989ec8640 1 -- 172.21.15.157:0/2505377833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe98415e670 msgr2=0x7fe984152510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.776+0000 7fe989ec8640 1 --2- 172.21.15.157:0/2505377833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe98415e670 0x7fe984152510 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe97c033000 tx=0x7fe97c004a40 comp rx=0 tx=0).stop 2024-03-29T04:09:37.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.776+0000 7fe989ec8640 1 -- 172.21.15.157:0/2505377833 shutdown_connections 2024-03-29T04:09:37.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.776+0000 7fe989ec8640 1 --2- 172.21.15.157:0/2505377833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe98415e670 0x7fe984152510 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.777+0000 7fe989ec8640 1 --2- 172.21.15.157:0/2505377833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe98415e290 0x7fe984151be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.757 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.777+0000 7fe989ec8640 1 --2- 172.21.15.157:0/2505377833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9840b4fe0 0x7fe9841516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.758 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.777+0000 7fe989ec8640 1 -- 172.21.15.157:0/2505377833 >> 172.21.15.157:0/2505377833 conn(0x7fe9840b3ce0 msgr2=0x7fe98415d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:37.758 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.777+0000 7fe989ec8640 1 -- 172.21.15.157:0/2505377833 shutdown_connections 2024-03-29T04:09:37.758 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.777+0000 7fe989ec8640 1 -- 172.21.15.157:0/2505377833 wait complete. 2024-03-29T04:09:37.758 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-03-29T04:09:37.780+0000 7fc059617040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-29T04:09:37.766 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 95766452-2e6e-4091-8373-a3a7a0f22315 7 2024-03-29T04:09:37.806 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-29T04:09:37.827+0000 7faf20847040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-29T04:09:37.867 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.888+0000 7f93391d4640 1 Processor -- start 2024-03-29T04:09:37.867 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.888+0000 7f93391d4640 1 -- start start 2024-03-29T04:09:37.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.889+0000 7f93391d4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f933415c230 0x7f933406f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.889+0000 7f93391d4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f933406fe70 0x7f93340b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.889+0000 7f93391d4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9334070280 con 0x7f9334158be0 2024-03-29T04:09:37.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.889+0000 7f93391d4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f933406cf10 con 0x7f933406fe70 2024-03-29T04:09:37.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.889+0000 7f93391d4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f933406d1f0 con 0x7f933415c230 2024-03-29T04:09:37.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.889+0000 7f933259c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f933415c230 0x7f933406f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.889+0000 7f933259c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f933415c230 0x7f933406f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46848/0 (socket says 172.21.15.157:46848) 2024-03-29T04:09:37.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.889+0000 7f933259c640 1 -- 172.21.15.157:0/1350869598 learned_addr learned my addr 172.21.15.157:0/1350869598 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-03-29T04:09:37.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.889+0000 7f933359e640 1 --2- 172.21.15.157:0/1350869598 >> v2:172.21.15.148:3300/0 conn(0x7f933406fe70 0x7f93340b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.889+0000 7f933259c640 1 -- 172.21.15.157:0/1350869598 >> v2:172.21.15.148:3300/0 conn(0x7f933406fe70 msgr2=0x7f93340b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:37.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.889+0000 7f933259c640 1 --2- 172.21.15.157:0/1350869598 >> v2:172.21.15.148:3300/0 conn(0x7f933406fe70 0x7f93340b4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.889+0000 7f933259c640 1 -- 172.21.15.157:0/1350869598 >> v1:172.21.15.148:6789/0 conn(0x7f9334158be0 legacy=0x7f933406c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.889+0000 7f933259c640 1 -- 172.21.15.157:0/1350869598 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f933406d4d0 con 0x7f933415c230 2024-03-29T04:09:37.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.890+0000 7f933259c640 1 --2- 172.21.15.157:0/1350869598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f933415c230 0x7f933406f930 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9334073da0 tx=0x7f932402b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=906bea60f823c2b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:37.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.890+0000 7f9331d9b640 1 -- 172.21.15.157:0/1350869598 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324034070 con 0x7f933415c230 2024-03-29T04:09:37.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.890+0000 7f9331d9b640 1 -- 172.21.15.157:0/1350869598 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9324030860 con 0x7f933415c230 2024-03-29T04:09:37.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.890+0000 7f9331d9b640 1 -- 172.21.15.157:0/1350869598 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324030c30 con 0x7f933415c230 2024-03-29T04:09:37.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.891+0000 7f93391d4640 1 -- 172.21.15.157:0/1350869598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f933415c230 msgr2=0x7f933406f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.891+0000 7f93391d4640 1 --2- 172.21.15.157:0/1350869598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f933415c230 0x7f933406f930 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9334073da0 tx=0x7f932402b9a0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.891+0000 7f93391d4640 1 -- 172.21.15.157:0/1350869598 shutdown_connections 2024-03-29T04:09:37.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.891+0000 7f93391d4640 1 --2- 172.21.15.157:0/1350869598 >> v2:172.21.15.148:3300/0 conn(0x7f933406fe70 0x7f93340b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.891+0000 7f93391d4640 1 --2- 172.21.15.157:0/1350869598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f933415c230 0x7f933406f930 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.871 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.891+0000 7f93391d4640 1 -- 172.21.15.157:0/1350869598 >> 172.21.15.157:0/1350869598 conn(0x7f93340d1060 msgr2=0x7f93340d1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:37.872 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.891+0000 7f93391d4640 1 -- 172.21.15.157:0/1350869598 shutdown_connections 2024-03-29T04:09:37.872 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.891+0000 7f93391d4640 1 -- 172.21.15.157:0/1350869598 wait complete. 2024-03-29T04:09:37.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.892+0000 7f93391d4640 1 Processor -- start 2024-03-29T04:09:37.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.892+0000 7f93391d4640 1 -- start start 2024-03-29T04:09:37.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.893+0000 7f93391d4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f933406fe70 0x7f9334147dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.893+0000 7f93391d4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9334158be0 0x7f9334152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.893+0000 7f93391d4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f933415c230 0x7f93341528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.893+0000 7f93391d4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f93340b4bd0 con 0x7f933406fe70 2024-03-29T04:09:37.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.893+0000 7f93391d4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9334158fc0 con 0x7f933415c230 2024-03-29T04:09:37.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.893+0000 7f93391d4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f93340b4d50 con 0x7f9334158be0 2024-03-29T04:09:37.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.893+0000 7f933359e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f933415c230 0x7f93341528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.893+0000 7f933259c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9334158be0 0x7f9334152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.893+0000 7f933359e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f933415c230 0x7f93341528b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46862/0 (socket says 172.21.15.157:46862) 2024-03-29T04:09:37.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.893+0000 7f933359e640 1 -- 172.21.15.157:0/1231472453 learned_addr learned my addr 172.21.15.157:0/1231472453 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-03-29T04:09:37.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.893+0000 7f9332d9d640 1 --2- 172.21.15.157:0/1231472453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f933406fe70 0x7f9334147dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.893+0000 7f933259c640 1 -- 172.21.15.157:0/1231472453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f933415c230 msgr2=0x7f93341528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.893+0000 7f933259c640 1 --2- 172.21.15.157:0/1231472453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f933415c230 0x7f93341528b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.875 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.893+0000 7f933259c640 1 -- 172.21.15.157:0/1231472453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f933406fe70 msgr2=0x7f9334147dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.893+0000 7f933259c640 1 --2- 172.21.15.157:0/1231472453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f933406fe70 0x7f9334147dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.893+0000 7f933259c640 1 -- 172.21.15.157:0/1231472453 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f933414fee0 con 0x7f9334158be0 2024-03-29T04:09:37.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.893+0000 7f933359e640 1 --2- 172.21.15.157:0/1231472453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f933415c230 0x7f93341528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:37.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.894+0000 7f933259c640 1 --2- 172.21.15.157:0/1231472453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9334158be0 0x7f9334152370 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9324033000 tx=0x7f93240046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:37.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.894+0000 7f931b7fe640 1 -- 172.21.15.157:0/1231472453 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324034040 con 0x7f9334158be0 2024-03-29T04:09:37.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.894+0000 7f931b7fe640 1 -- 172.21.15.157:0/1231472453 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9324039d90 con 0x7f9334158be0 2024-03-29T04:09:37.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.894+0000 7f93391d4640 1 -- 172.21.15.157:0/1231472453 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93340cf9f0 con 0x7f9334158be0 2024-03-29T04:09:37.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.894+0000 7f931b7fe640 1 -- 172.21.15.157:0/1231472453 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324043630 con 0x7f9334158be0 2024-03-29T04:09:37.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.894+0000 7f93391d4640 1 -- 172.21.15.157:0/1231472453 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f93340cffb0 con 0x7f9334158be0 2024-03-29T04:09:37.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.894+0000 7f931b7fe640 1 -- 172.21.15.157:0/1231472453 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f932404a060 con 0x7f9334158be0 2024-03-29T04:09:37.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.894+0000 7f931b7fe640 1 -- 172.21.15.157:0/1231472453 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f932404cab0 con 0x7f9334158be0 2024-03-29T04:09:37.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.895+0000 7f93391d4640 1 -- 172.21.15.157:0/1231472453 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9334068850 con 0x7f9334158be0 2024-03-29T04:09:37.877 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.897+0000 7f931b7fe640 1 -- 172.21.15.157:0/1231472453 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f932403e070 con 0x7f9334158be0 2024-03-29T04:09:37.902 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-03-29T04:09:37.923+0000 7fc059617040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-29T04:09:37.914 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-29T04:09:37.935+0000 7faf20847040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-29T04:09:37.948 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.970+0000 7f93391d4640 1 -- 172.21.15.157:0/1231472453 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd new", "uuid": "95766452-2e6e-4091-8373-a3a7a0f22315", "id": 7} v 0) -- 0x7f9334070ed0 con 0x7f9334158be0 2024-03-29T04:09:37.954 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.975+0000 7f931b7fe640 1 -- 172.21.15.157:0/1231472453 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "95766452-2e6e-4091-8373-a3a7a0f22315", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f932404cd50 con 0x7f9334158be0 2024-03-29T04:09:37.954 INFO:teuthology.orchestra.run.smithi157.stdout:7 2024-03-29T04:09:37.955 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.976+0000 7f93391d4640 1 -- 172.21.15.157:0/1231472453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9334158be0 msgr2=0x7f9334152370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.955 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.976+0000 7f93391d4640 1 --2- 172.21.15.157:0/1231472453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9334158be0 0x7f9334152370 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9324033000 tx=0x7f93240046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.955 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.976+0000 7f93391d4640 1 -- 172.21.15.157:0/1231472453 shutdown_connections 2024-03-29T04:09:37.956 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.976+0000 7f93391d4640 1 --2- 172.21.15.157:0/1231472453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f933415c230 0x7f93341528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.956 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.976+0000 7f93391d4640 1 --2- 172.21.15.157:0/1231472453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9334158be0 0x7f9334152370 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.956 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.976+0000 7f93391d4640 1 --2- 172.21.15.157:0/1231472453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f933406fe70 0x7f9334147dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.956 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.976+0000 7f93391d4640 1 -- 172.21.15.157:0/1231472453 >> 172.21.15.157:0/1231472453 conn(0x7f93340d1060 msgr2=0x7f933415a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:37.956 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.976+0000 7f93391d4640 1 -- 172.21.15.157:0/1231472453 shutdown_connections 2024-03-29T04:09:37.956 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:09:37.976+0000 7f93391d4640 1 -- 172.21.15.157:0/1231472453 wait complete. 2024-03-29T04:09:37.960 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-29T04:09:37.981+0000 7faf20847040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-29T04:09:37.965 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-29T04:09:37.965 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-29T04:09:37.968 INFO:tasks.ceph.osd.0:Started 2024-03-29T04:09:37.968 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-29T04:09:37.968 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-29T04:09:37.972 INFO:tasks.ceph.osd.1:Started 2024-03-29T04:09:37.972 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-29T04:09:37.972 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-29T04:09:37.975 INFO:tasks.ceph.osd.2:Started 2024-03-29T04:09:37.975 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-29T04:09:37.975 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-29T04:09:37.977 INFO:tasks.ceph.osd.3:Started 2024-03-29T04:09:37.977 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-29T04:09:37.978 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-29T04:09:37.980 INFO:tasks.ceph.osd.4:Started 2024-03-29T04:09:37.980 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-29T04:09:37.980 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-29T04:09:37.982 INFO:tasks.ceph.osd.5:Started 2024-03-29T04:09:37.982 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-29T04:09:37.982 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-29T04:09:37.984 INFO:tasks.ceph.osd.6:Started 2024-03-29T04:09:37.985 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-29T04:09:37.985 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-29T04:09:37.989 INFO:tasks.ceph.osd.7:Started 2024-03-29T04:09:37.989 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:38.035 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-03-29T04:09:38.056+0000 7fc059617040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-29T04:09:38.105 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-03-29T04:09:38.126+0000 7fc059617040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-29T04:09:38.114 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-29T04:09:38.135+0000 7faf20847040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-29T04:09:38.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.138+0000 7fc3f2c14640 1 Processor -- start 2024-03-29T04:09:38.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.138+0000 7fc3f2c14640 1 -- start start 2024-03-29T04:09:38.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.138+0000 7fc3f2c14640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3ec0ba030 0x7fc3ec0d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.138+0000 7fc3f2c14640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc3ec0d10d0 0x7fc3ec0d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.138+0000 7fc3f2c14640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3ec15e7c0 con 0x7fc3ec15e3e0 2024-03-29T04:09:38.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.138+0000 7fc3f2c14640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc3ec0d3840 con 0x7fc3ec0d10d0 2024-03-29T04:09:38.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.138+0000 7fc3f2c14640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc3ec0d2c30 con 0x7fc3ec0ba030 2024-03-29T04:09:38.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.139+0000 7fc3ebfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3ec0ba030 0x7fc3ec0d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.139+0000 7fc3ebfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3ec0ba030 0x7fc3ec0d3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.148:49400/0 (socket says 172.21.15.148:49400) 2024-03-29T04:09:38.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.139+0000 7fc3ebfff640 1 -- 172.21.15.148:0/3624064399 learned_addr learned my addr 172.21.15.148:0/3624064399 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:09:38.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.139+0000 7fc3f11b1640 1 --2- 172.21.15.148:0/3624064399 >> v2:172.21.15.157:3300/0 conn(0x7fc3ec0d10d0 0x7fc3ec0d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.139+0000 7fc3ebfff640 1 -- 172.21.15.148:0/3624064399 >> v2:172.21.15.157:3300/0 conn(0x7fc3ec0d10d0 msgr2=0x7fc3ec0d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:38.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.139+0000 7fc3ebfff640 1 --2- 172.21.15.148:0/3624064399 >> v2:172.21.15.157:3300/0 conn(0x7fc3ec0d10d0 0x7fc3ec0d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.139+0000 7fc3ebfff640 1 -- 172.21.15.148:0/3624064399 >> v1:172.21.15.157:6789/0 conn(0x7fc3ec15e3e0 legacy=0x7fc3ec0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.139+0000 7fc3ebfff640 1 -- 172.21.15.148:0/3624064399 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ec0d2f10 con 0x7fc3ec0ba030 2024-03-29T04:09:38.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.139+0000 7fc3f11b1640 1 --2- 172.21.15.148:0/3624064399 >> v2:172.21.15.157:3300/0 conn(0x7fc3ec0d10d0 0x7fc3ec0d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:38.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.140+0000 7fc3ebfff640 1 --2- 172.21.15.148:0/3624064399 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3ec0ba030 0x7fc3ec0d3cd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009870 tx=0x7fc3d802ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2607dbd983c683ae server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:38.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.140+0000 7fc3eb7fe640 1 -- 172.21.15.148:0/3624064399 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d802f840 con 0x7fc3ec0ba030 2024-03-29T04:09:38.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.140+0000 7fc3eb7fe640 1 -- 172.21.15.148:0/3624064399 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d802f9e0 con 0x7fc3ec0ba030 2024-03-29T04:09:38.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.140+0000 7fc3eb7fe640 1 -- 172.21.15.148:0/3624064399 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d802fdb0 con 0x7fc3ec0ba030 2024-03-29T04:09:38.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.140+0000 7fc3f2c14640 1 -- 172.21.15.148:0/3624064399 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3ec0ba030 msgr2=0x7fc3ec0d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.140+0000 7fc3f2c14640 1 --2- 172.21.15.148:0/3624064399 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3ec0ba030 0x7fc3ec0d3cd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009870 tx=0x7fc3d802ba20 comp rx=0 tx=0).stop 2024-03-29T04:09:38.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.140+0000 7fc3f2c14640 1 -- 172.21.15.148:0/3624064399 shutdown_connections 2024-03-29T04:09:38.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.140+0000 7fc3f2c14640 1 --2- 172.21.15.148:0/3624064399 >> v2:172.21.15.157:3300/0 conn(0x7fc3ec0d10d0 0x7fc3ec0d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.140+0000 7fc3f2c14640 1 --2- 172.21.15.148:0/3624064399 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3ec0ba030 0x7fc3ec0d3cd0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.141+0000 7fc3f2c14640 1 -- 172.21.15.148:0/3624064399 >> 172.21.15.148:0/3624064399 conn(0x7fc3ec0b6210 msgr2=0x7fc3ec0b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:38.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.141+0000 7fc3f2c14640 1 -- 172.21.15.148:0/3624064399 shutdown_connections 2024-03-29T04:09:38.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.141+0000 7fc3f2c14640 1 -- 172.21.15.148:0/3624064399 wait complete. 2024-03-29T04:09:38.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.142+0000 7fc3f2c14640 1 Processor -- start 2024-03-29T04:09:38.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.142+0000 7fc3f2c14640 1 -- start start 2024-03-29T04:09:38.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.143+0000 7fc3f2c14640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3ec0ba030 0x7fc3ec1f5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.143+0000 7fc3f2c14640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3ec0d10d0 0x7fc3ec14f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.143+0000 7fc3f2c14640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3ec15e3e0 0x7fc3ec153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.143+0000 7fc3f2c14640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec167f50 con 0x7fc3ec0ba030 2024-03-29T04:09:38.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.143+0000 7fc3f2c14640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec0d34d0 con 0x7fc3ec0d10d0 2024-03-29T04:09:38.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.143+0000 7fc3f2c14640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc3ec1680d0 con 0x7fc3ec15e3e0 2024-03-29T04:09:38.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.143+0000 7fc3f09b0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3ec0ba030 0x7fc3ec1f5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.143+0000 7fc3f09b0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3ec0ba030 0x7fc3ec1f5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34792/0 (socket says 172.21.15.148:34792) 2024-03-29T04:09:38.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.143+0000 7fc3f09b0640 1 -- 172.21.15.148:0/2133501141 learned_addr learned my addr 172.21.15.148:0/2133501141 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:09:38.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.143+0000 7fc3f11b1640 1 --2- 172.21.15.148:0/2133501141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3ec15e3e0 0x7fc3ec153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.143+0000 7fc3ebfff640 1 --2- 172.21.15.148:0/2133501141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3ec0d10d0 0x7fc3ec14f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.143+0000 7fc3f09b0640 1 -- 172.21.15.148:0/2133501141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3ec15e3e0 msgr2=0x7fc3ec153a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.143+0000 7fc3f09b0640 1 --2- 172.21.15.148:0/2133501141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3ec15e3e0 0x7fc3ec153a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.143+0000 7fc3f09b0640 1 -- 172.21.15.148:0/2133501141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3ec0d10d0 msgr2=0x7fc3ec14f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.143+0000 7fc3f09b0640 1 --2- 172.21.15.148:0/2133501141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3ec0d10d0 0x7fc3ec14f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.143+0000 7fc3f09b0640 1 -- 172.21.15.148:0/2133501141 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ec1542e0 con 0x7fc3ec0ba030 2024-03-29T04:09:38.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.143+0000 7fc3f09b0640 1 --2- 172.21.15.148:0/2133501141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3ec0ba030 0x7fc3ec1f5120 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000c3c0 tx=0x7fc3e000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:38.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.143+0000 7fc3e97fa640 1 -- 172.21.15.148:0/2133501141 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e00052d0 con 0x7fc3ec0ba030 2024-03-29T04:09:38.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.143+0000 7fc3e97fa640 1 -- 172.21.15.148:0/2133501141 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3e0005490 con 0x7fc3ec0ba030 2024-03-29T04:09:38.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.143+0000 7fc3f2c14640 1 -- 172.21.15.148:0/2133501141 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3ec1545c0 con 0x7fc3ec0ba030 2024-03-29T04:09:38.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.144+0000 7fc3e97fa640 1 -- 172.21.15.148:0/2133501141 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0007260 con 0x7fc3ec0ba030 2024-03-29T04:09:38.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.144+0000 7fc3f2c14640 1 -- 172.21.15.148:0/2133501141 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3ec153fd0 con 0x7fc3ec0ba030 2024-03-29T04:09:38.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.144+0000 7fc3e97fa640 1 -- 172.21.15.148:0/2133501141 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc3e0007400 con 0x7fc3ec0ba030 2024-03-29T04:09:38.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.144+0000 7fc3e97fa640 1 -- 172.21.15.148:0/2133501141 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fc3e0003a00 con 0x7fc3ec0ba030 2024-03-29T04:09:38.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.145+0000 7fc3f2c14640 1 -- 172.21.15.148:0/2133501141 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3ec0687c0 con 0x7fc3ec0ba030 2024-03-29T04:09:38.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.147+0000 7fc3e97fa640 1 -- 172.21.15.148:0/2133501141 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fc3e00035f0 con 0x7fc3ec0ba030 2024-03-29T04:09:38.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.208+0000 7fc3f2c14640 1 -- 172.21.15.148:0/2133501141 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3ec0d21c0 con 0x7fc3ec0ba030 2024-03-29T04:09:38.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.208+0000 7fc3e97fa640 1 -- 172.21.15.148:0/2133501141 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fc3e00035f0 con 0x7fc3ec0ba030 2024-03-29T04:09:38.187 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:38.188 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":10,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:09:37.971465+0000","last_up_change":"0.000000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:38.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.209+0000 7fc3f2c14640 1 -- 172.21.15.148:0/2133501141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3ec0ba030 msgr2=0x7fc3ec1f5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.209+0000 7fc3f2c14640 1 --2- 172.21.15.148:0/2133501141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3ec0ba030 0x7fc3ec1f5120 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000c3c0 tx=0x7fc3e000c7c0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.209+0000 7fc3f2c14640 1 -- 172.21.15.148:0/2133501141 shutdown_connections 2024-03-29T04:09:38.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.209+0000 7fc3f2c14640 1 --2- 172.21.15.148:0/2133501141 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3ec15e3e0 0x7fc3ec153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.209+0000 7fc3f2c14640 1 --2- 172.21.15.148:0/2133501141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3ec0d10d0 0x7fc3ec14f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.209+0000 7fc3f2c14640 1 --2- 172.21.15.148:0/2133501141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3ec0ba030 0x7fc3ec1f5120 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.209+0000 7fc3f2c14640 1 -- 172.21.15.148:0/2133501141 >> 172.21.15.148:0/2133501141 conn(0x7fc3ec0b6210 msgr2=0x7fc3ec0b77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:38.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.209+0000 7fc3f2c14640 1 -- 172.21.15.148:0/2133501141 shutdown_connections 2024-03-29T04:09:38.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:38.209+0000 7fc3f2c14640 1 -- 172.21.15.148:0/2133501141 wait complete. 2024-03-29T04:09:38.199 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-29T04:09:38.199 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-29T04:09:38.215 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-29T04:09:38.236+0000 7faf20847040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-29T04:09:38.219 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-03-29T04:09:38.241+0000 7fc059617040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-29T04:09:38.315 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-03-29T04:09:38.336+0000 7fc059617040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-29T04:09:38.374 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-29T04:09:38.395+0000 7faf20847040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-29T04:09:38.417 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-29T04:09:38.438+0000 7faf20847040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-29T04:09:38.471 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-03-29T04:09:38.492+0000 7fc059617040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-29T04:09:38.513 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-03-29T04:09:38.534+0000 7fc059617040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-29T04:09:38.576 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-29T04:09:38.596+0000 7faf20847040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-29T04:09:38.579 INFO:tasks.ceph.osd.4.smithi157.stderr:2024-03-29T04:09:38.600+0000 7fb8ad61c640 -1 Falling back to public interface 2024-03-29T04:09:38.582 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-29T04:09:38.603+0000 7f2015618640 -1 Falling back to public interface 2024-03-29T04:09:38.585 INFO:tasks.ceph.osd.6.smithi157.stderr:2024-03-29T04:09:38.606+0000 7f37c3126640 -1 Falling back to public interface 2024-03-29T04:09:38.589 INFO:tasks.ceph.osd.5.smithi157.stderr:2024-03-29T04:09:38.610+0000 7f6b0108b640 -1 Falling back to public interface 2024-03-29T04:09:38.593 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-03-29T04:09:38.614+0000 7fe6732b3640 -1 Falling back to public interface 2024-03-29T04:09:38.595 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-03-29T04:09:38.616+0000 7fc35660c640 -1 Falling back to public interface 2024-03-29T04:09:38.599 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-29T04:09:38.620+0000 7feb24a24640 -1 Falling back to public interface 2024-03-29T04:09:38.618 INFO:tasks.ceph.osd.7.smithi157.stderr:2024-03-29T04:09:38.640+0000 7f16a4cae640 -1 Falling back to public interface 2024-03-29T04:09:38.676 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-03-29T04:09:38.698+0000 7fc059617040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-29T04:09:39.004 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:39.095 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.115+0000 7f58b46d9640 1 Processor -- start 2024-03-29T04:09:39.095 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.115+0000 7f58b46d9640 1 -- start start 2024-03-29T04:09:39.096 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.116+0000 7f58b46d9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f58ac0d02e0 0x7f58ac167130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.096 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.116+0000 7f58b46d9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58ac0701a0 con 0x7f58ac160940 2024-03-29T04:09:39.096 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.116+0000 7f58b46d9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58ac070480 con 0x7f58ac15c1e0 2024-03-29T04:09:39.096 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.116+0000 7f58b46d9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f58ac070760 con 0x7f58ac0d02e0 2024-03-29T04:09:39.096 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.116+0000 7f58b2c76640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f58ac0d02e0 0x7f58ac167130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.096 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.117+0000 7f58b2c76640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f58ac0d02e0 0x7f58ac167130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34796/0 (socket says 172.21.15.148:34796) 2024-03-29T04:09:39.096 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.117+0000 7f58b1c74640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f58ac160940 0x7f58ac0cfbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:53904/0 (socket says 172.21.15.148:53904) 2024-03-29T04:09:39.096 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.117+0000 7f58b2c76640 1 -- 172.21.15.148:0/372718802 learned_addr learned my addr 172.21.15.148:0/372718802 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:09:39.097 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.117+0000 7f58b2c76640 1 -- 172.21.15.148:0/372718802 >> v1:172.21.15.157:6789/0 conn(0x7f58ac15c1e0 legacy=0x7f58ac06f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.097 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.117+0000 7f58b2c76640 1 -- 172.21.15.148:0/372718802 >> v1:172.21.15.148:6789/0 conn(0x7f58ac160940 legacy=0x7f58ac0cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.097 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.117+0000 7f58b2c76640 1 -- 172.21.15.148:0/372718802 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58ac070a40 con 0x7f58ac0d02e0 2024-03-29T04:09:39.097 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.117+0000 7f58b1473640 1 -- 172.21.15.148:0/372718802 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3791450038 0 0) 0x7f58ac0701a0 con 0x7f58ac160940 2024-03-29T04:09:39.098 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.117+0000 7f58b2c76640 1 --2- 172.21.15.148:0/372718802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58ac0d02e0 0x7f58ac167130 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009870 tx=0x7f58a802bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a86d36dc932034 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:39.098 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.118+0000 7f58b1473640 1 -- 172.21.15.148:0/372718802 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a802d870 con 0x7f58ac0d02e0 2024-03-29T04:09:39.099 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.118+0000 7f58b1473640 1 -- 172.21.15.148:0/372718802 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58a802da10 con 0x7f58ac0d02e0 2024-03-29T04:09:39.099 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.118+0000 7f58b1473640 1 -- 172.21.15.148:0/372718802 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a802dde0 con 0x7f58ac0d02e0 2024-03-29T04:09:39.099 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.118+0000 7f58b46d9640 1 -- 172.21.15.148:0/372718802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58ac0d02e0 msgr2=0x7f58ac167130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.099 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.118+0000 7f58b46d9640 1 --2- 172.21.15.148:0/372718802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58ac0d02e0 0x7f58ac167130 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009870 tx=0x7f58a802bab0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.099 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.118+0000 7f58b46d9640 1 -- 172.21.15.148:0/372718802 shutdown_connections 2024-03-29T04:09:39.099 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.118+0000 7f58b46d9640 1 --2- 172.21.15.148:0/372718802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58ac0d02e0 0x7f58ac167130 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.099 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.118+0000 7f58b46d9640 1 -- 172.21.15.148:0/372718802 >> 172.21.15.148:0/372718802 conn(0x7f58ac072490 msgr2=0x7f58ac0728a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:39.099 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.118+0000 7f58b46d9640 1 -- 172.21.15.148:0/372718802 shutdown_connections 2024-03-29T04:09:39.099 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.118+0000 7f58b46d9640 1 -- 172.21.15.148:0/372718802 wait complete. 2024-03-29T04:09:39.099 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.119+0000 7f58b46d9640 1 Processor -- start 2024-03-29T04:09:39.099 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.119+0000 7f58b46d9640 1 -- start start 2024-03-29T04:09:39.101 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.120+0000 7f58b46d9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58ac0d02e0 0x7f58ac0cb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.101 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.120+0000 7f58b46d9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58ac15c1e0 0x7f58ac0cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.102 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.120+0000 7f58b46d9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58ac160940 0x7f58ac1f4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.102 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.120+0000 7f58b46d9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac06d050 con 0x7f58ac160940 2024-03-29T04:09:39.102 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.120+0000 7f58b46d9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac15c5c0 con 0x7f58ac15c1e0 2024-03-29T04:09:39.102 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.120+0000 7f58b46d9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f58ac06d1d0 con 0x7f58ac0d02e0 2024-03-29T04:09:39.102 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.120+0000 7f58b2c76640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58ac160940 0x7f58ac1f4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.102 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.120+0000 7f58b2c76640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58ac160940 0x7f58ac1f4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34802/0 (socket says 172.21.15.148:34802) 2024-03-29T04:09:39.102 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.120+0000 7f58b2c76640 1 -- 172.21.15.148:0/2591026701 learned_addr learned my addr 172.21.15.148:0/2591026701 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:09:39.102 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.120+0000 7f58b1c74640 1 --2- 172.21.15.148:0/2591026701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58ac15c1e0 0x7f58ac0cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.102 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.120+0000 7f58b2475640 1 --2- 172.21.15.148:0/2591026701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58ac0d02e0 0x7f58ac0cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.102 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.120+0000 7f58b2c76640 1 -- 172.21.15.148:0/2591026701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58ac0d02e0 msgr2=0x7f58ac0cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.102 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.120+0000 7f58b2c76640 1 --2- 172.21.15.148:0/2591026701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58ac0d02e0 0x7f58ac0cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.102 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.120+0000 7f58b2c76640 1 -- 172.21.15.148:0/2591026701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58ac15c1e0 msgr2=0x7f58ac0cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.102 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.120+0000 7f58b2c76640 1 --2- 172.21.15.148:0/2591026701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58ac15c1e0 0x7f58ac0cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.103 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.120+0000 7f58b2c76640 1 -- 172.21.15.148:0/2591026701 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58ac0ca560 con 0x7f58ac160940 2024-03-29T04:09:39.103 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.120+0000 7f58b1c74640 1 --2- 172.21.15.148:0/2591026701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58ac15c1e0 0x7f58ac0cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:39.103 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.121+0000 7f58b2c76640 1 --2- 172.21.15.148:0/2591026701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58ac160940 0x7f58ac1f4b70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009990 tx=0x7f58a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:39.103 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.121+0000 7f58a2ffd640 1 -- 172.21.15.148:0/2591026701 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a8047070 con 0x7f58ac160940 2024-03-29T04:09:39.103 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.121+0000 7f58a2ffd640 1 -- 172.21.15.148:0/2591026701 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58a8037950 con 0x7f58ac160940 2024-03-29T04:09:39.103 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.121+0000 7f58a2ffd640 1 -- 172.21.15.148:0/2591026701 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a8037d00 con 0x7f58ac160940 2024-03-29T04:09:39.103 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.121+0000 7f58b46d9640 1 -- 172.21.15.148:0/2591026701 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58ac0ca840 con 0x7f58ac160940 2024-03-29T04:09:39.103 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.121+0000 7f58b46d9640 1 -- 172.21.15.148:0/2591026701 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58ac0cae00 con 0x7f58ac160940 2024-03-29T04:09:39.103 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.121+0000 7f58a2ffd640 1 -- 172.21.15.148:0/2591026701 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f58a802e440 con 0x7f58ac160940 2024-03-29T04:09:39.103 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.121+0000 7f58a2ffd640 1 -- 172.21.15.148:0/2591026701 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f58a8042ac0 con 0x7f58ac160940 2024-03-29T04:09:39.103 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.122+0000 7f58b46d9640 1 -- 172.21.15.148:0/2591026701 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58ac068ef0 con 0x7f58ac160940 2024-03-29T04:09:39.103 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.124+0000 7f58a2ffd640 1 -- 172.21.15.148:0/2591026701 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f58a8050070 con 0x7f58ac160940 2024-03-29T04:09:39.165 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.186+0000 7f58b46d9640 1 -- 172.21.15.148:0/2591026701 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58ac0b6f80 con 0x7f58ac160940 2024-03-29T04:09:39.166 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.187+0000 7f58a2ffd640 1 -- 172.21.15.148:0/2591026701 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f58a802e910 con 0x7f58ac160940 2024-03-29T04:09:39.166 INFO:teuthology.misc.health.smithi148.stdout: 2024-03-29T04:09:39.166 INFO:teuthology.misc.health.smithi148.stdout:{"epoch":10,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:09:37.971465+0000","last_up_change":"0.000000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:39.167 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.188+0000 7f58b46d9640 1 -- 172.21.15.148:0/2591026701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58ac160940 msgr2=0x7f58ac1f4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.167 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.188+0000 7f58b46d9640 1 --2- 172.21.15.148:0/2591026701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58ac160940 0x7f58ac1f4b70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009990 tx=0x7f58a8004730 comp rx=0 tx=0).stop 2024-03-29T04:09:39.168 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.188+0000 7f58b46d9640 1 -- 172.21.15.148:0/2591026701 shutdown_connections 2024-03-29T04:09:39.168 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.188+0000 7f58b46d9640 1 --2- 172.21.15.148:0/2591026701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58ac160940 0x7f58ac1f4b70 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.168 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.188+0000 7f58b46d9640 1 --2- 172.21.15.148:0/2591026701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58ac15c1e0 0x7f58ac0cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.168 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.188+0000 7f58b46d9640 1 --2- 172.21.15.148:0/2591026701 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f58ac0d02e0 0x7f58ac0cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.168 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.188+0000 7f58b46d9640 1 -- 172.21.15.148:0/2591026701 >> 172.21.15.148:0/2591026701 conn(0x7f58ac072490 msgr2=0x7f58ac15fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:39.168 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.188+0000 7f58b46d9640 1 -- 172.21.15.148:0/2591026701 shutdown_connections 2024-03-29T04:09:39.168 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:39.188+0000 7f58b46d9640 1 -- 172.21.15.148:0/2591026701 wait complete. 2024-03-29T04:09:39.176 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-29T04:09:39.404 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-29T04:09:39.425+0000 7faf20847040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-29T04:09:39.496 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-03-29T04:09:39.518+0000 7fc059617040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-29T04:09:39.683 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-29T04:09:39.704+0000 7faf20847040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-29T04:09:39.736 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-29T04:09:39.757+0000 7faf20847040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-29T04:09:39.777 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-03-29T04:09:39.798+0000 7fc059617040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-29T04:09:39.828 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-03-29T04:09:39.850+0000 7fc059617040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-29T04:09:39.852 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-29T04:09:39.872+0000 7faf20847040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-29T04:09:39.952 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-03-29T04:09:39.973+0000 7fc059617040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-29T04:09:40.865 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-29T04:09:40.886+0000 7faec13d5640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-29T04:09:40.865 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-03-29T04:09:40.886+0000 7faec13d5640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-29T04:09:42.074 INFO:tasks.ceph.osd.4.smithi157.stderr:2024-03-29T04:09:42.096+0000 7fb8ad61c640 -1 osd.4 0 log_to_monitors true 2024-03-29T04:09:42.101 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-03-29T04:09:42.100+0000 7fe6732b3640 -1 osd.2 0 log_to_monitors true 2024-03-29T04:09:42.101 INFO:tasks.ceph.osd.5.smithi157.stderr:2024-03-29T04:09:42.102+0000 7f6b0108b640 -1 osd.5 0 log_to_monitors true 2024-03-29T04:09:42.112 INFO:tasks.ceph.osd.6.smithi157.stderr:2024-03-29T04:09:42.133+0000 7f37c3126640 -1 osd.6 0 log_to_monitors true 2024-03-29T04:09:42.117 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-29T04:09:42.138+0000 7f2015618640 -1 osd.1 0 log_to_monitors true 2024-03-29T04:09:42.119 INFO:tasks.ceph.osd.7.smithi157.stderr:2024-03-29T04:09:42.140+0000 7f16a4cae640 -1 osd.7 0 log_to_monitors true 2024-03-29T04:09:42.121 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-29T04:09:42.142+0000 7feb24a24640 -1 osd.0 0 log_to_monitors true 2024-03-29T04:09:42.153 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-03-29T04:09:42.173+0000 7fc35660c640 -1 osd.3 0 log_to_monitors true 2024-03-29T04:09:45.257 INFO:tasks.ceph.osd.4.smithi157.stderr:2024-03-29T04:09:45.278+0000 7fb8a900c640 -1 osd.4 0 waiting for initial osdmap 2024-03-29T04:09:45.272 INFO:tasks.ceph.osd.4.smithi157.stderr:2024-03-29T04:09:45.293+0000 7fb8a0e2c640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:09:45.289 INFO:tasks.ceph.osd.5.smithi157.stderr:2024-03-29T04:09:45.310+0000 7f6afd021640 -1 osd.5 0 waiting for initial osdmap 2024-03-29T04:09:45.301 INFO:tasks.ceph.osd.5.smithi157.stderr:2024-03-29T04:09:45.323+0000 7f6af4e41640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:09:45.325 INFO:tasks.ceph.osd.7.smithi157.stderr:2024-03-29T04:09:45.346+0000 7f16a0c44640 -1 osd.7 0 waiting for initial osdmap 2024-03-29T04:09:45.333 INFO:tasks.ceph.osd.6.smithi157.stderr:2024-03-29T04:09:45.354+0000 7f37bf0bc640 -1 osd.6 0 waiting for initial osdmap 2024-03-29T04:09:45.336 INFO:tasks.ceph.osd.7.smithi157.stderr:2024-03-29T04:09:45.358+0000 7f1698a64640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:09:45.344 INFO:tasks.ceph.osd.6.smithi157.stderr:2024-03-29T04:09:45.366+0000 7f37b6edc640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:09:45.967 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-03-29T04:09:45.988+0000 7fe66f249640 -1 osd.2 0 waiting for initial osdmap 2024-03-29T04:09:45.978 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-03-29T04:09:45.999+0000 7fc351f48640 -1 osd.3 0 waiting for initial osdmap 2024-03-29T04:09:45.983 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:45.984 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-03-29T04:09:46.005+0000 7fe667069640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:09:45.991 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-03-29T04:09:46.012+0000 7fc349d68640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:09:46.005 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-29T04:09:46.026+0000 7f2010f85640 -1 osd.1 0 waiting for initial osdmap 2024-03-29T04:09:46.011 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-29T04:09:46.031+0000 7feb209ba640 -1 osd.0 0 waiting for initial osdmap 2024-03-29T04:09:46.020 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-03-29T04:09:46.041+0000 7f2008da5640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:09:46.026 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-03-29T04:09:46.047+0000 7feb187da640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:09:46.077 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.098+0000 7f931df1c640 1 Processor -- start 2024-03-29T04:09:46.077 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.098+0000 7f931df1c640 1 -- start start 2024-03-29T04:09:46.079 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.098+0000 7f931df1c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f93180d38e0 0x7f93180b7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:46.079 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.098+0000 7f931df1c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f931806cce0 0x7f93180b81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:46.079 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.098+0000 7f931df1c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9318162170 con 0x7f93180b34b0 2024-03-29T04:09:46.079 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.098+0000 7f931df1c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f93180b77d0 con 0x7f93180d38e0 2024-03-29T04:09:46.079 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.098+0000 7f931df1c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f93180d01d0 con 0x7f931806cce0 2024-03-29T04:09:46.079 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.099+0000 7f9316ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f93180d38e0 0x7f93180b7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:46.079 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.099+0000 7f93177fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f93180b34b0 0x7f93180d31d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42718/0 (socket says 172.21.15.148:42718) 2024-03-29T04:09:46.079 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.099+0000 7f93177fe640 1 -- 172.21.15.148:0/1530989194 learned_addr learned my addr 172.21.15.148:0/1530989194 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:09:46.079 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.099+0000 7f9317fff640 1 --2- 172.21.15.148:0/1530989194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f931806cce0 0x7f93180b81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:46.079 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.099+0000 7f9316ffd640 1 -- 172.21.15.148:0/1530989194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f931806cce0 msgr2=0x7f93180b81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:46.079 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.099+0000 7f9316ffd640 1 --2- 172.21.15.148:0/1530989194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f931806cce0 0x7f93180b81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:46.079 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.099+0000 7f9316ffd640 1 -- 172.21.15.148:0/1530989194 >> v1:172.21.15.148:6789/0 conn(0x7f93180b34b0 legacy=0x7f93180d31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:46.080 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.099+0000 7f9316ffd640 1 -- 172.21.15.148:0/1530989194 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93180d04b0 con 0x7f93180d38e0 2024-03-29T04:09:46.080 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.099+0000 7f93167fc640 1 -- 172.21.15.148:0/1530989194 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2164360007 0 0) 0x7f9318162170 con 0x7f93180b34b0 2024-03-29T04:09:46.080 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.099+0000 7f9317fff640 1 --2- 172.21.15.148:0/1530989194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f931806cce0 0x7f93180b81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:46.080 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.100+0000 7f9316ffd640 1 --2- 172.21.15.148:0/1530989194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93180d38e0 0x7f93180b7c60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f930c0097a0 tx=0x7f930c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2180050c31973f21 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:46.080 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.100+0000 7f93167fc640 1 -- 172.21.15.148:0/1530989194 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c034070 con 0x7f93180d38e0 2024-03-29T04:09:46.080 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.100+0000 7f93167fc640 1 -- 172.21.15.148:0/1530989194 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f930c030610 con 0x7f93180d38e0 2024-03-29T04:09:46.080 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.100+0000 7f93167fc640 1 -- 172.21.15.148:0/1530989194 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c0309e0 con 0x7f93180d38e0 2024-03-29T04:09:46.083 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.100+0000 7f931df1c640 1 -- 172.21.15.148:0/1530989194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93180d38e0 msgr2=0x7f93180b7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:46.083 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.100+0000 7f931df1c640 1 --2- 172.21.15.148:0/1530989194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93180d38e0 0x7f93180b7c60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f930c0097a0 tx=0x7f930c02b960 comp rx=0 tx=0).stop 2024-03-29T04:09:46.083 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.100+0000 7f931df1c640 1 -- 172.21.15.148:0/1530989194 shutdown_connections 2024-03-29T04:09:46.083 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.100+0000 7f931df1c640 1 --2- 172.21.15.148:0/1530989194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f931806cce0 0x7f93180b81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:46.083 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.100+0000 7f931df1c640 1 --2- 172.21.15.148:0/1530989194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93180d38e0 0x7f93180b7c60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:46.083 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.100+0000 7f931df1c640 1 -- 172.21.15.148:0/1530989194 >> 172.21.15.148:0/1530989194 conn(0x7f93180b5db0 msgr2=0x7f93180b61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:46.083 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.100+0000 7f931df1c640 1 -- 172.21.15.148:0/1530989194 shutdown_connections 2024-03-29T04:09:46.084 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.100+0000 7f931df1c640 1 -- 172.21.15.148:0/1530989194 wait complete. 2024-03-29T04:09:46.084 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.101+0000 7f931df1c640 1 Processor -- start 2024-03-29T04:09:46.084 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.102+0000 7f931df1c640 1 -- start start 2024-03-29T04:09:46.084 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.102+0000 7f931df1c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f931806cce0 0x7f93181605b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:46.084 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.102+0000 7f931df1c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93180b34b0 0x7f9318158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:46.084 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.102+0000 7f931df1c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93180d38e0 0x7f9318158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:46.084 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.102+0000 7f931df1c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f93180b3890 con 0x7f93180d38e0 2024-03-29T04:09:46.084 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.102+0000 7f931df1c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f93180d0a70 con 0x7f93180b34b0 2024-03-29T04:09:46.084 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.102+0000 7f931df1c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f93180b3a10 con 0x7f931806cce0 2024-03-29T04:09:46.084 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.102+0000 7f9317fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93180d38e0 0x7f9318158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:46.084 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.102+0000 7f9317fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93180d38e0 0x7f9318158780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38512/0 (socket says 172.21.15.148:38512) 2024-03-29T04:09:46.084 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.102+0000 7f9317fff640 1 -- 172.21.15.148:0/1355083071 learned_addr learned my addr 172.21.15.148:0/1355083071 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:09:46.084 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.102+0000 7f93177fe640 1 --2- 172.21.15.148:0/1355083071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f931806cce0 0x7f93181605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:46.084 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.102+0000 7f9316ffd640 1 --2- 172.21.15.148:0/1355083071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93180b34b0 0x7f9318158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:46.085 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.103+0000 7f9317fff640 1 -- 172.21.15.148:0/1355083071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f931806cce0 msgr2=0x7f93181605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:46.085 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.103+0000 7f9317fff640 1 --2- 172.21.15.148:0/1355083071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f931806cce0 0x7f93181605b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:46.085 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.103+0000 7f9317fff640 1 -- 172.21.15.148:0/1355083071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93180b34b0 msgr2=0x7f9318158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:46.085 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.103+0000 7f9317fff640 1 --2- 172.21.15.148:0/1355083071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93180b34b0 0x7f9318158240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:46.085 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.103+0000 7f9317fff640 1 -- 172.21.15.148:0/1355083071 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f931815f550 con 0x7f93180d38e0 2024-03-29T04:09:46.085 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.103+0000 7f9317fff640 1 --2- 172.21.15.148:0/1355083071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93180d38e0 0x7f9318158780 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f931000cef0 tx=0x7f931000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:46.085 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.103+0000 7f92fffff640 1 -- 172.21.15.148:0/1355083071 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931001a070 con 0x7f93180d38e0 2024-03-29T04:09:46.085 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.103+0000 7f92fffff640 1 -- 172.21.15.148:0/1355083071 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f931000d860 con 0x7f93180d38e0 2024-03-29T04:09:46.085 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.103+0000 7f92fffff640 1 -- 172.21.15.148:0/1355083071 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931000dc50 con 0x7f93180d38e0 2024-03-29T04:09:46.085 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.103+0000 7f931df1c640 1 -- 172.21.15.148:0/1355083071 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f931815f7f0 con 0x7f93180d38e0 2024-03-29T04:09:46.085 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.103+0000 7f931df1c640 1 -- 172.21.15.148:0/1355083071 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f931815fdb0 con 0x7f93180d38e0 2024-03-29T04:09:46.085 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.104+0000 7f931df1c640 1 -- 172.21.15.148:0/1355083071 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9318074cf0 con 0x7f93180d38e0 2024-03-29T04:09:46.085 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.104+0000 7f92fffff640 1 -- 172.21.15.148:0/1355083071 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 4) ==== 90800+0+0 (secure 0 0 0) 0x7f931001d070 con 0x7f93180d38e0 2024-03-29T04:09:46.086 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.105+0000 7f92fffff640 1 --2- 172.21.15.148:0/1355083071 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f930406c4a0 0x7f930406e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:46.086 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.105+0000 7f92fffff640 1 -- 172.21.15.148:0/1355083071 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f931008b040 con 0x7f93180d38e0 2024-03-29T04:09:46.086 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.105+0000 7f93177fe640 1 --2- 172.21.15.148:0/1355083071 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f930406c4a0 0x7f930406e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:46.086 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.105+0000 7f93177fe640 1 --2- 172.21.15.148:0/1355083071 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f930406c4a0 0x7f930406e960 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9308000f60 tx=0x7f9308007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:46.088 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.109+0000 7f92fffff640 1 -- 172.21.15.148:0/1355083071 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9318074cf0 con 0x7f93180d38e0 2024-03-29T04:09:46.262 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.283+0000 7f931df1c640 1 -- 172.21.15.148:0/1355083071 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9318068ef0 con 0x7f93180d38e0 2024-03-29T04:09:46.263 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.284+0000 7f92fffff640 1 -- 172.21.15.148:0/1355083071 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f9310059f40 con 0x7f93180d38e0 2024-03-29T04:09:46.263 INFO:teuthology.misc.health.smithi148.stdout: 2024-03-29T04:09:46.263 INFO:teuthology.misc.health.smithi148.stdout:{"epoch":12,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:09:43.886882+0000","last_up_change":"0.000000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:46.265 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.286+0000 7f931df1c640 1 -- 172.21.15.148:0/1355083071 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f930406c4a0 msgr2=0x7f930406e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:46.265 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.286+0000 7f931df1c640 1 --2- 172.21.15.148:0/1355083071 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f930406c4a0 0x7f930406e960 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9308000f60 tx=0x7f9308007d50 comp rx=0 tx=0).stop 2024-03-29T04:09:46.266 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.286+0000 7f931df1c640 1 -- 172.21.15.148:0/1355083071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93180d38e0 msgr2=0x7f9318158780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:46.266 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.286+0000 7f931df1c640 1 --2- 172.21.15.148:0/1355083071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93180d38e0 0x7f9318158780 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f931000cef0 tx=0x7f931000a710 comp rx=0 tx=0).stop 2024-03-29T04:09:46.266 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.286+0000 7f931df1c640 1 -- 172.21.15.148:0/1355083071 shutdown_connections 2024-03-29T04:09:46.266 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.286+0000 7f931df1c640 1 --2- 172.21.15.148:0/1355083071 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f930406c4a0 0x7f930406e960 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:46.266 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.286+0000 7f931df1c640 1 --2- 172.21.15.148:0/1355083071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93180d38e0 0x7f9318158780 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:46.266 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.286+0000 7f931df1c640 1 --2- 172.21.15.148:0/1355083071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93180b34b0 0x7f9318158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:46.266 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.286+0000 7f931df1c640 1 --2- 172.21.15.148:0/1355083071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f931806cce0 0x7f93181605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:46.266 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.286+0000 7f931df1c640 1 -- 172.21.15.148:0/1355083071 >> 172.21.15.148:0/1355083071 conn(0x7f93180b5db0 msgr2=0x7f93180b7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:46.266 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.286+0000 7f931df1c640 1 -- 172.21.15.148:0/1355083071 shutdown_connections 2024-03-29T04:09:46.266 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:46.286+0000 7f931df1c640 1 -- 172.21.15.148:0/1355083071 wait complete. 2024-03-29T04:09:46.275 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-29T04:09:53.081 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:53.182 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.201+0000 7fd028b15640 1 Processor -- start 2024-03-29T04:09:53.182 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.201+0000 7fd028b15640 1 -- start start 2024-03-29T04:09:53.183 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.202+0000 7fd028b15640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0240b34b0 0x7fd0240d31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.183 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.202+0000 7fd028b15640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd02406cce0 0x7fd0240d3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.183 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.202+0000 7fd028b15640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd024162170 con 0x7fd02406ba70 2024-03-29T04:09:53.183 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.202+0000 7fd028b15640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd0240b77d0 con 0x7fd02406cce0 2024-03-29T04:09:53.184 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.202+0000 7fd028b15640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd0240d01d0 con 0x7fd0240b34b0 2024-03-29T04:09:53.184 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.203+0000 7fd021d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd02406cce0 0x7fd0240d3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.184 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.203+0000 7fd021d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd02406cce0 0x7fd0240d3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:57496/0 (socket says 172.21.15.148:57496) 2024-03-29T04:09:53.184 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.203+0000 7fd021d9b640 1 -- 172.21.15.148:0/2077998709 learned_addr learned my addr 172.21.15.148:0/2077998709 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:09:53.184 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.203+0000 7fd02259c640 1 --2- 172.21.15.148:0/2077998709 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0240b34b0 0x7fd0240d31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.184 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.203+0000 7fd02259c640 1 -- 172.21.15.148:0/2077998709 >> v2:172.21.15.157:3300/0 conn(0x7fd02406cce0 msgr2=0x7fd0240d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:53.184 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.203+0000 7fd02259c640 1 --2- 172.21.15.148:0/2077998709 >> v2:172.21.15.157:3300/0 conn(0x7fd02406cce0 0x7fd0240d3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.184 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.203+0000 7fd02259c640 1 -- 172.21.15.148:0/2077998709 >> v1:172.21.15.157:6789/0 conn(0x7fd02406ba70 legacy=0x7fd0240ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.184 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.203+0000 7fd02259c640 1 -- 172.21.15.148:0/2077998709 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0240d04b0 con 0x7fd0240b34b0 2024-03-29T04:09:53.184 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.203+0000 7fd021d9b640 1 --2- 172.21.15.148:0/2077998709 >> v2:172.21.15.157:3300/0 conn(0x7fd02406cce0 0x7fd0240d3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:53.185 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.204+0000 7fd02259c640 1 --2- 172.21.15.148:0/2077998709 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0240b34b0 0x7fd0240d31d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd014009d90 tx=0x7fd01402bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32d627d9c84cca76 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:53.185 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.204+0000 7fd02159a640 1 -- 172.21.15.148:0/2077998709 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd014039070 con 0x7fd0240b34b0 2024-03-29T04:09:53.185 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.204+0000 7fd02159a640 1 -- 172.21.15.148:0/2077998709 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd01402cce0 con 0x7fd0240b34b0 2024-03-29T04:09:53.186 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.204+0000 7fd02159a640 1 -- 172.21.15.148:0/2077998709 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd014034660 con 0x7fd0240b34b0 2024-03-29T04:09:53.186 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.205+0000 7fd028b15640 1 -- 172.21.15.148:0/2077998709 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0240b34b0 msgr2=0x7fd0240d31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.186 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.205+0000 7fd028b15640 1 --2- 172.21.15.148:0/2077998709 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0240b34b0 0x7fd0240d31d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd014009d90 tx=0x7fd01402bf30 comp rx=0 tx=0).stop 2024-03-29T04:09:53.186 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.205+0000 7fd028b15640 1 -- 172.21.15.148:0/2077998709 shutdown_connections 2024-03-29T04:09:53.186 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.205+0000 7fd028b15640 1 --2- 172.21.15.148:0/2077998709 >> v2:172.21.15.157:3300/0 conn(0x7fd02406cce0 0x7fd0240d3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.186 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.205+0000 7fd028b15640 1 --2- 172.21.15.148:0/2077998709 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0240b34b0 0x7fd0240d31d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.186 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.205+0000 7fd028b15640 1 -- 172.21.15.148:0/2077998709 >> 172.21.15.148:0/2077998709 conn(0x7fd0240b5db0 msgr2=0x7fd0240b61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:53.186 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.205+0000 7fd028b15640 1 -- 172.21.15.148:0/2077998709 shutdown_connections 2024-03-29T04:09:53.186 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.205+0000 7fd028b15640 1 -- 172.21.15.148:0/2077998709 wait complete. 2024-03-29T04:09:53.187 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.206+0000 7fd028b15640 1 Processor -- start 2024-03-29T04:09:53.187 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.206+0000 7fd028b15640 1 -- start start 2024-03-29T04:09:53.188 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.207+0000 7fd028b15640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd02406ba70 0x7fd024158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.189 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.207+0000 7fd028b15640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd02406cce0 0x7fd02415e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.189 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.207+0000 7fd028b15640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0240b34b0 0x7fd02415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.189 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.207+0000 7fd028b15640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd0240b3890 con 0x7fd0240b34b0 2024-03-29T04:09:53.189 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.207+0000 7fd028b15640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd0240d0a70 con 0x7fd02406ba70 2024-03-29T04:09:53.189 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.207+0000 7fd028b15640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd0240b3a10 con 0x7fd02406cce0 2024-03-29T04:09:53.189 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.207+0000 7fd022d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0240b34b0 0x7fd02415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.189 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.207+0000 7fd022d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0240b34b0 0x7fd02415b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38538/0 (socket says 172.21.15.148:38538) 2024-03-29T04:09:53.189 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.207+0000 7fd022d9d640 1 -- 172.21.15.148:0/3796734194 learned_addr learned my addr 172.21.15.148:0/3796734194 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:09:53.189 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.207+0000 7fd021d9b640 1 --2- 172.21.15.148:0/3796734194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd02406cce0 0x7fd02415e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.189 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.207+0000 7fd02259c640 1 --2- 172.21.15.148:0/3796734194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd02406ba70 0x7fd024158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.189 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.208+0000 7fd022d9d640 1 -- 172.21.15.148:0/3796734194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd02406cce0 msgr2=0x7fd02415e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.189 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.208+0000 7fd022d9d640 1 --2- 172.21.15.148:0/3796734194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd02406cce0 0x7fd02415e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.189 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.208+0000 7fd022d9d640 1 -- 172.21.15.148:0/3796734194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd02406ba70 msgr2=0x7fd024158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.190 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.208+0000 7fd022d9d640 1 --2- 172.21.15.148:0/3796734194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd02406ba70 0x7fd024158c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.190 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.208+0000 7fd022d9d640 1 -- 172.21.15.148:0/3796734194 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd024159690 con 0x7fd0240b34b0 2024-03-29T04:09:53.190 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.208+0000 7fd021d9b640 1 --2- 172.21.15.148:0/3796734194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd02406cce0 0x7fd02415e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:53.190 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.208+0000 7fd022d9d640 1 --2- 172.21.15.148:0/3796734194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0240b34b0 0x7fd02415b6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd01c001eb0 tx=0x7fd01c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:53.190 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.208+0000 7fd012ffd640 1 -- 172.21.15.148:0/3796734194 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c017070 con 0x7fd0240b34b0 2024-03-29T04:09:53.190 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.208+0000 7fd012ffd640 1 -- 172.21.15.148:0/3796734194 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd01c0052d0 con 0x7fd0240b34b0 2024-03-29T04:09:53.190 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.208+0000 7fd028b15640 1 -- 172.21.15.148:0/3796734194 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd024159970 con 0x7fd0240b34b0 2024-03-29T04:09:53.190 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.208+0000 7fd012ffd640 1 -- 172.21.15.148:0/3796734194 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c003810 con 0x7fd0240b34b0 2024-03-29T04:09:53.190 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.208+0000 7fd028b15640 1 -- 172.21.15.148:0/3796734194 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd024159f30 con 0x7fd0240b34b0 2024-03-29T04:09:53.191 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.209+0000 7fd028b15640 1 -- 172.21.15.148:0/3796734194 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd024068ef0 con 0x7fd0240b34b0 2024-03-29T04:09:53.191 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.209+0000 7fd012ffd640 1 -- 172.21.15.148:0/3796734194 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd01c01a070 con 0x7fd0240b34b0 2024-03-29T04:09:53.191 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.210+0000 7fd012ffd640 1 --2- 172.21.15.148:0/3796734194 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd00406c530 0x7fd00406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.191 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.210+0000 7fd012ffd640 1 -- 172.21.15.148:0/3796734194 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fd01c08a650 con 0x7fd0240b34b0 2024-03-29T04:09:53.191 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.210+0000 7fd02259c640 1 --2- 172.21.15.148:0/3796734194 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd00406c530 0x7fd00406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.191 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.210+0000 7fd02259c640 1 --2- 172.21.15.148:0/3796734194 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd00406c530 0x7fd00406e9f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd014036000 tx=0x7fd014030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:53.194 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.215+0000 7fd012ffd640 1 -- 172.21.15.148:0/3796734194 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd01c058970 con 0x7fd0240b34b0 2024-03-29T04:09:53.369 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.389+0000 7fd028b15640 1 -- 172.21.15.148:0/3796734194 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd02415f8c0 con 0x7fd0240b34b0 2024-03-29T04:09:53.370 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.390+0000 7fd012ffd640 1 -- 172.21.15.148:0/3796734194 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12896 (secure 0 0 0) 0x7fd01c05cc70 con 0x7fd0240b34b0 2024-03-29T04:09:53.370 INFO:teuthology.misc.health.smithi148.stdout: 2024-03-29T04:09:53.370 INFO:teuthology.misc.health.smithi148.stdout:{"epoch":16,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:09:49.904034+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:53.372 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.392+0000 7fd028b15640 1 -- 172.21.15.148:0/3796734194 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd00406c530 msgr2=0x7fd00406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.372 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.392+0000 7fd028b15640 1 --2- 172.21.15.148:0/3796734194 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd00406c530 0x7fd00406e9f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd014036000 tx=0x7fd014030000 comp rx=0 tx=0).stop 2024-03-29T04:09:53.372 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.392+0000 7fd028b15640 1 -- 172.21.15.148:0/3796734194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0240b34b0 msgr2=0x7fd02415b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.372 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.392+0000 7fd028b15640 1 --2- 172.21.15.148:0/3796734194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0240b34b0 0x7fd02415b6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd01c001eb0 tx=0x7fd01c00c890 comp rx=0 tx=0).stop 2024-03-29T04:09:53.372 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.392+0000 7fd028b15640 1 -- 172.21.15.148:0/3796734194 shutdown_connections 2024-03-29T04:09:53.372 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.392+0000 7fd028b15640 1 --2- 172.21.15.148:0/3796734194 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd00406c530 0x7fd00406e9f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.373 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.392+0000 7fd028b15640 1 --2- 172.21.15.148:0/3796734194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0240b34b0 0x7fd02415b6c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.373 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.392+0000 7fd028b15640 1 --2- 172.21.15.148:0/3796734194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd02406cce0 0x7fd02415e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.373 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.392+0000 7fd028b15640 1 --2- 172.21.15.148:0/3796734194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd02406ba70 0x7fd024158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.373 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.392+0000 7fd028b15640 1 -- 172.21.15.148:0/3796734194 >> 172.21.15.148:0/3796734194 conn(0x7fd0240b5db0 msgr2=0x7fd0240b4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:53.373 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.392+0000 7fd028b15640 1 -- 172.21.15.148:0/3796734194 shutdown_connections 2024-03-29T04:09:53.373 INFO:teuthology.misc.health.smithi148.stderr:2024-03-29T04:09:53.392+0000 7fd028b15640 1 -- 172.21.15.148:0/3796734194 wait complete. 2024-03-29T04:09:53.382 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-29T04:09:53.382 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-29T04:09:53.383 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-29T04:09:53.383 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-29T04:09:53.385 INFO:tasks.ceph.mds.a:Started 2024-03-29T04:09:53.385 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-29T04:09:53.385 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-29T04:09:53.388 INFO:tasks.ceph.mds.c:Started 2024-03-29T04:09:53.388 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-29T04:09:53.388 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-29T04:09:53.390 INFO:tasks.ceph.mds.b:Started 2024-03-29T04:09:53.390 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-29T04:09:53.391 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-29T04:09:53.393 INFO:tasks.ceph.mds.d:Started 2024-03-29T04:09:53.393 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-29T04:09:53.393 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:53.462 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2024-03-29T04:09:53.463 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2024-03-29T04:09:53.463 INFO:tasks.ceph.mds.a.smithi148.stderr:2024-03-29T04:09:53.483+0000 7fded860d640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-03-29T04:09:53.477 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-03-29T04:09:53.478 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-03-29T04:09:53.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.519+0000 7f974afc8640 1 Processor -- start 2024-03-29T04:09:53.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.519+0000 7f974afc8640 1 -- start start 2024-03-29T04:09:53.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.520+0000 7f974afc8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f974415e3e0 0x7f97440d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.520+0000 7f974afc8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f97440ba030 0x7f97440d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.520+0000 7f974afc8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97440d10d0 0x7f97440d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.520+0000 7f974afc8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f97440d3840 con 0x7f97440ba030 2024-03-29T04:09:53.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.520+0000 7f974afc8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f974415e7c0 con 0x7f974415e3e0 2024-03-29T04:09:53.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.520+0000 7f974afc8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f97440d2c30 con 0x7f97440d10d0 2024-03-29T04:09:53.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.520+0000 7f9743fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f97440ba030 0x7f97440d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.520+0000 7f9743fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f97440ba030 0x7f97440d2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38576/0 (socket says 172.21.15.148:38576) 2024-03-29T04:09:53.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.520+0000 7f9743fff640 1 -- 172.21.15.148:0/1682433240 learned_addr learned my addr 172.21.15.148:0/1682433240 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:09:53.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.520+0000 7f9749565640 1 --2- 172.21.15.148:0/1682433240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97440d10d0 0x7f97440d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.520+0000 7f9748d64640 1 --2- 172.21.15.148:0/1682433240 >> v2:172.21.15.157:3300/0 conn(0x7f974415e3e0 0x7f97440d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.520+0000 7f9743fff640 1 -- 172.21.15.148:0/1682433240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97440d10d0 msgr2=0x7f97440d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:53.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.520+0000 7f9743fff640 1 --2- 172.21.15.148:0/1682433240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97440d10d0 0x7f97440d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.520+0000 7f9743fff640 1 -- 172.21.15.148:0/1682433240 >> v2:172.21.15.157:3300/0 conn(0x7f974415e3e0 msgr2=0x7f97440d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:53.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.520+0000 7f9743fff640 1 --2- 172.21.15.148:0/1682433240 >> v2:172.21.15.157:3300/0 conn(0x7f974415e3e0 0x7f97440d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.520+0000 7f9743fff640 1 -- 172.21.15.148:0/1682433240 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97440d2f10 con 0x7f97440ba030 2024-03-29T04:09:53.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.521+0000 7f9743fff640 1 --2- 172.21.15.148:0/1682433240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97440ba030 0x7f97440d2330 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f97440b4eb0 tx=0x7f973002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d4fa24c5913c7fc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:53.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.521+0000 7f97437fe640 1 -- 172.21.15.148:0/1682433240 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9730034070 con 0x7f97440ba030 2024-03-29T04:09:53.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.521+0000 7f97437fe640 1 -- 172.21.15.148:0/1682433240 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9730030860 con 0x7f97440ba030 2024-03-29T04:09:53.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.521+0000 7f97437fe640 1 -- 172.21.15.148:0/1682433240 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9730030c30 con 0x7f97440ba030 2024-03-29T04:09:53.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.521+0000 7f974afc8640 1 -- 172.21.15.148:0/1682433240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97440ba030 msgr2=0x7f97440d2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.521+0000 7f974afc8640 1 --2- 172.21.15.148:0/1682433240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97440ba030 0x7f97440d2330 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f97440b4eb0 tx=0x7f973002b9a0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.522+0000 7f974afc8640 1 -- 172.21.15.148:0/1682433240 shutdown_connections 2024-03-29T04:09:53.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.522+0000 7f974afc8640 1 --2- 172.21.15.148:0/1682433240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97440ba030 0x7f97440d2330 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.522+0000 7f974afc8640 1 --2- 172.21.15.148:0/1682433240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97440d10d0 0x7f97440d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.522+0000 7f974afc8640 1 --2- 172.21.15.148:0/1682433240 >> v2:172.21.15.157:3300/0 conn(0x7f974415e3e0 0x7f97440d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.522+0000 7f974afc8640 1 -- 172.21.15.148:0/1682433240 >> 172.21.15.148:0/1682433240 conn(0x7f97440b6210 msgr2=0x7f97440b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:53.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.522+0000 7f974afc8640 1 -- 172.21.15.148:0/1682433240 shutdown_connections 2024-03-29T04:09:53.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.522+0000 7f974afc8640 1 -- 172.21.15.148:0/1682433240 wait complete. 2024-03-29T04:09:53.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.523+0000 7f974afc8640 1 Processor -- start 2024-03-29T04:09:53.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.523+0000 7f974afc8640 1 -- start start 2024-03-29T04:09:53.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.524+0000 7f974afc8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97440ba030 0x7f974415d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.524+0000 7f974afc8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97440d10d0 0x7f97441f7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.524+0000 7f974afc8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f974415e3e0 0x7f974414b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.524+0000 7f974afc8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f97440d2870 con 0x7f97440ba030 2024-03-29T04:09:53.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.524+0000 7f974afc8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f97440d34d0 con 0x7f97440d10d0 2024-03-29T04:09:53.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.524+0000 7f974afc8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f97440d29f0 con 0x7f974415e3e0 2024-03-29T04:09:53.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.524+0000 7f9748d64640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97440ba030 0x7f974415d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.524+0000 7f9748d64640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97440ba030 0x7f974415d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38586/0 (socket says 172.21.15.148:38586) 2024-03-29T04:09:53.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.524+0000 7f9748d64640 1 -- 172.21.15.148:0/102550582 learned_addr learned my addr 172.21.15.148:0/102550582 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:09:53.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.524+0000 7f9749565640 1 --2- 172.21.15.148:0/102550582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f974415e3e0 0x7f974414b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.524+0000 7f9743fff640 1 --2- 172.21.15.148:0/102550582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97440d10d0 0x7f97441f7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.524+0000 7f9748d64640 1 -- 172.21.15.148:0/102550582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f974415e3e0 msgr2=0x7f974414b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.524+0000 7f9748d64640 1 --2- 172.21.15.148:0/102550582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f974415e3e0 0x7f974414b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.524+0000 7f9748d64640 1 -- 172.21.15.148:0/102550582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97440d10d0 msgr2=0x7f97441f7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.524+0000 7f9748d64640 1 --2- 172.21.15.148:0/102550582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97440d10d0 0x7f97441f7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.524+0000 7f9748d64640 1 -- 172.21.15.148:0/102550582 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f974415ad60 con 0x7f97440ba030 2024-03-29T04:09:53.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.524+0000 7f9748d64640 1 --2- 172.21.15.148:0/102550582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97440ba030 0x7f974415d490 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f973800d620 tx=0x7f973800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:53.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.524+0000 7f97417fa640 1 -- 172.21.15.148:0/102550582 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973800b620 con 0x7f97440ba030 2024-03-29T04:09:53.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.524+0000 7f97417fa640 1 -- 172.21.15.148:0/102550582 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9738019070 con 0x7f97440ba030 2024-03-29T04:09:53.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.524+0000 7f97417fa640 1 -- 172.21.15.148:0/102550582 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973800ba00 con 0x7f97440ba030 2024-03-29T04:09:53.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.524+0000 7f974afc8640 1 -- 172.21.15.148:0/102550582 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f974415b040 con 0x7f97440ba030 2024-03-29T04:09:53.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.524+0000 7f974afc8640 1 -- 172.21.15.148:0/102550582 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f974415d9d0 con 0x7f97440ba030 2024-03-29T04:09:53.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.525+0000 7f974afc8640 1 -- 172.21.15.148:0/102550582 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97440687e0 con 0x7f97440ba030 2024-03-29T04:09:53.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.526+0000 7f97417fa640 1 -- 172.21.15.148:0/102550582 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f973800bba0 con 0x7f97440ba030 2024-03-29T04:09:53.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.526+0000 7f97417fa640 1 --2- 172.21.15.148:0/102550582 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f972806c530 0x7f972806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.526+0000 7f97417fa640 1 -- 172.21.15.148:0/102550582 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f973808b6f0 con 0x7f97440ba030 2024-03-29T04:09:53.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.527+0000 7f9743fff640 1 --2- 172.21.15.148:0/102550582 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f972806c530 0x7f972806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.527+0000 7f9743fff640 1 --2- 172.21.15.148:0/102550582 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f972806c530 0x7f972806e9f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9730036000 tx=0x7f973003b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:53.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.531+0000 7f97417fa640 1 -- 172.21.15.148:0/102550582 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f973805a0f0 con 0x7f97440ba030 2024-03-29T04:09:53.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.708+0000 7f974afc8640 1 -- 172.21.15.148:0/102550582 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f97440d3de0 con 0x7f97440ba030 2024-03-29T04:09:53.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.709+0000 7f97417fa640 1 -- 172.21.15.148:0/102550582 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12896 (secure 0 0 0) 0x7f97380599f0 con 0x7f97440ba030 2024-03-29T04:09:53.689 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:53.689 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":16,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:09:49.904034+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:53.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.711+0000 7f974afc8640 1 -- 172.21.15.148:0/102550582 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f972806c530 msgr2=0x7f972806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.711+0000 7f974afc8640 1 --2- 172.21.15.148:0/102550582 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f972806c530 0x7f972806e9f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9730036000 tx=0x7f973003b040 comp rx=0 tx=0).stop 2024-03-29T04:09:53.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.711+0000 7f974afc8640 1 -- 172.21.15.148:0/102550582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97440ba030 msgr2=0x7f974415d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.711+0000 7f974afc8640 1 --2- 172.21.15.148:0/102550582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97440ba030 0x7f974415d490 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f973800d620 tx=0x7f973800dae0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.711+0000 7f974afc8640 1 -- 172.21.15.148:0/102550582 shutdown_connections 2024-03-29T04:09:53.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.711+0000 7f974afc8640 1 --2- 172.21.15.148:0/102550582 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f972806c530 0x7f972806e9f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.711+0000 7f974afc8640 1 --2- 172.21.15.148:0/102550582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f974415e3e0 0x7f974414b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.711+0000 7f974afc8640 1 --2- 172.21.15.148:0/102550582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97440d10d0 0x7f97441f7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.711+0000 7f974afc8640 1 --2- 172.21.15.148:0/102550582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97440ba030 0x7f974415d490 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.711+0000 7f974afc8640 1 -- 172.21.15.148:0/102550582 >> 172.21.15.148:0/102550582 conn(0x7f97440b6210 msgr2=0x7f97440b82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:53.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.712+0000 7f974afc8640 1 -- 172.21.15.148:0/102550582 shutdown_connections 2024-03-29T04:09:53.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.712+0000 7f974afc8640 1 -- 172.21.15.148:0/102550582 wait complete. 2024-03-29T04:09:53.703 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:47.898351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:09:53.703 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:09:53.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.826+0000 7fa805104640 1 Processor -- start 2024-03-29T04:09:53.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.826+0000 7fa805104640 1 -- start start 2024-03-29T04:09:53.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.827+0000 7fa805104640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa80015f200 0x7fa80015f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.827+0000 7fa805104640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa8000ccec0 0x7fa8000b6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.827+0000 7fa805104640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8000cfdd0 con 0x7fa8000b4fb0 2024-03-29T04:09:53.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.827+0000 7fa805104640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fa8000d09e0 con 0x7fa8000ccec0 2024-03-29T04:09:53.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.827+0000 7fa805104640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa8000d00b0 con 0x7fa80015f200 2024-03-29T04:09:53.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.827+0000 7fa7ff59e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa8000ccec0 0x7fa8000b6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.827+0000 7fa7ff59e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa8000ccec0 0x7fa8000b6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38588/0 (socket says 172.21.15.148:38588) 2024-03-29T04:09:53.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.827+0000 7fa7ff59e640 1 -- 172.21.15.148:0/881652169 learned_addr learned my addr 172.21.15.148:0/881652169 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:09:53.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.827+0000 7fa7ff59e640 1 -- 172.21.15.148:0/881652169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa80015f200 msgr2=0x7fa80015f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:53.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.827+0000 7fa7fed9d640 1 --2- 172.21.15.148:0/881652169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa80015f200 0x7fa80015f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.827+0000 7fa7ff59e640 1 --2- 172.21.15.148:0/881652169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa80015f200 0x7fa80015f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.827+0000 7fa7ff59e640 1 -- 172.21.15.148:0/881652169 >> v1:172.21.15.157:6789/0 conn(0x7fa8000b4fb0 legacy=0x7fa8000cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:53.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.827+0000 7fa7ff59e640 1 -- 172.21.15.148:0/881652169 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8000d0390 con 0x7fa8000ccec0 2024-03-29T04:09:53.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.828+0000 7fa7ff59e640 1 --2- 172.21.15.148:0/881652169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8000ccec0 0x7fa8000b6b30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8009e50 tx=0x7fa7f802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486b25b2c3448dfa server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:53.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.828+0000 7fa7fdd9b640 1 -- 172.21.15.148:0/881652169 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f802ca70 con 0x7fa8000ccec0 2024-03-29T04:09:53.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.828+0000 7fa7fdd9b640 1 -- 172.21.15.148:0/881652169 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f802cc10 con 0x7fa8000ccec0 2024-03-29T04:09:53.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.828+0000 7fa7fdd9b640 1 -- 172.21.15.148:0/881652169 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f80343f0 con 0x7fa8000ccec0 2024-03-29T04:09:53.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.829+0000 7fa805104640 1 -- 172.21.15.148:0/881652169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8000ccec0 msgr2=0x7fa8000b6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.829+0000 7fa805104640 1 --2- 172.21.15.148:0/881652169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8000ccec0 0x7fa8000b6b30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8009e50 tx=0x7fa7f802bed0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.829+0000 7fa805104640 1 -- 172.21.15.148:0/881652169 shutdown_connections 2024-03-29T04:09:53.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.829+0000 7fa805104640 1 --2- 172.21.15.148:0/881652169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8000ccec0 0x7fa8000b6b30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.829+0000 7fa805104640 1 --2- 172.21.15.148:0/881652169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa80015f200 0x7fa80015f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.829+0000 7fa805104640 1 -- 172.21.15.148:0/881652169 >> 172.21.15.148:0/881652169 conn(0x7fa8000b3c70 msgr2=0x7fa8000b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:53.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.829+0000 7fa805104640 1 -- 172.21.15.148:0/881652169 shutdown_connections 2024-03-29T04:09:53.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.829+0000 7fa805104640 1 -- 172.21.15.148:0/881652169 wait complete. 2024-03-29T04:09:53.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.830+0000 7fa805104640 1 Processor -- start 2024-03-29T04:09:53.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.830+0000 7fa805104640 1 -- start start 2024-03-29T04:09:53.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.831+0000 7fa805104640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8000b4fb0 0x7fa800147550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.831+0000 7fa805104640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8000ccec0 0x7fa800160e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.831+0000 7fa805104640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa80015f200 0x7fa800161350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.831+0000 7fa805104640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa8000b55b0 con 0x7fa80015f200 2024-03-29T04:09:53.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.831+0000 7fa805104640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa8000b5430 con 0x7fa8000b4fb0 2024-03-29T04:09:53.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.831+0000 7fa805104640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa8000b5730 con 0x7fa8000ccec0 2024-03-29T04:09:53.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.831+0000 7fa7ff59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa80015f200 0x7fa800161350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.831+0000 7fa7ff59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa80015f200 0x7fa800161350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38600/0 (socket says 172.21.15.148:38600) 2024-03-29T04:09:53.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.831+0000 7fa7ff59e640 1 -- 172.21.15.148:0/3525463355 learned_addr learned my addr 172.21.15.148:0/3525463355 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:09:53.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.831+0000 7fa7fed9d640 1 --2- 172.21.15.148:0/3525463355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8000b4fb0 0x7fa800147550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.831+0000 7fa7fe59c640 1 --2- 172.21.15.148:0/3525463355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8000ccec0 0x7fa800160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.831+0000 7fa7ff59e640 1 -- 172.21.15.148:0/3525463355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8000ccec0 msgr2=0x7fa800160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.831+0000 7fa7ff59e640 1 --2- 172.21.15.148:0/3525463355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8000ccec0 0x7fa800160e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.831+0000 7fa7ff59e640 1 -- 172.21.15.148:0/3525463355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8000b4fb0 msgr2=0x7fa800147550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.831+0000 7fa7ff59e640 1 --2- 172.21.15.148:0/3525463355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8000b4fb0 0x7fa800147550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.831+0000 7fa7ff59e640 1 -- 172.21.15.148:0/3525463355 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa800147a90 con 0x7fa80015f200 2024-03-29T04:09:53.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.831+0000 7fa7fed9d640 1 --2- 172.21.15.148:0/3525463355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8000b4fb0 0x7fa800147550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:53.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.832+0000 7fa7ff59e640 1 --2- 172.21.15.148:0/3525463355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa80015f200 0x7fa800161350 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8009e50 tx=0x7fa7f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:53.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.832+0000 7fa7e77fe640 1 -- 172.21.15.148:0/3525463355 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8043020 con 0x7fa80015f200 2024-03-29T04:09:53.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.832+0000 7fa7e77fe640 1 -- 172.21.15.148:0/3525463355 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f803e450 con 0x7fa80015f200 2024-03-29T04:09:53.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.832+0000 7fa7e77fe640 1 -- 172.21.15.148:0/3525463355 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8009510 con 0x7fa80015f200 2024-03-29T04:09:53.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.832+0000 7fa805104640 1 -- 172.21.15.148:0/3525463355 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8001619d0 con 0x7fa80015f200 2024-03-29T04:09:53.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.832+0000 7fa805104640 1 -- 172.21.15.148:0/3525463355 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa800161f90 con 0x7fa80015f200 2024-03-29T04:09:53.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.833+0000 7fa805104640 1 -- 172.21.15.148:0/3525463355 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8000758d0 con 0x7fa80015f200 2024-03-29T04:09:53.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.833+0000 7fa7e77fe640 1 -- 172.21.15.148:0/3525463355 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa7f8046020 con 0x7fa80015f200 2024-03-29T04:09:53.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.834+0000 7fa7e77fe640 1 --2- 172.21.15.148:0/3525463355 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa7ec06c560 0x7fa7ec06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.834+0000 7fa7fed9d640 1 --2- 172.21.15.148:0/3525463355 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa7ec06c560 0x7fa7ec06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.834+0000 7fa7e77fe640 1 -- 172.21.15.148:0/3525463355 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fa7f80adb90 con 0x7fa80015f200 2024-03-29T04:09:53.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.834+0000 7fa7fed9d640 1 --2- 172.21.15.148:0/3525463355 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa7ec06c560 0x7fa7ec06ea20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0006fd0 tx=0x7fa7f0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:53.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:53.838+0000 7fa7e77fe640 1 -- 172.21.15.148:0/3525463355 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa7f804d030 con 0x7fa80015f200 2024-03-29T04:09:53.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.012+0000 7fa805104640 1 -- 172.21.15.148:0/3525463355 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa8000d1630 con 0x7fa80015f200 2024-03-29T04:09:53.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.013+0000 7fa7e77fe640 1 -- 172.21.15.148:0/3525463355 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fa800147a90 con 0x7fa80015f200 2024-03-29T04:09:53.993 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-29T04:09:53.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.015+0000 7fa805104640 1 -- 172.21.15.148:0/3525463355 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa7ec06c560 msgr2=0x7fa7ec06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.015+0000 7fa805104640 1 --2- 172.21.15.148:0/3525463355 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa7ec06c560 0x7fa7ec06ea20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0006fd0 tx=0x7fa7f0008040 comp rx=0 tx=0).stop 2024-03-29T04:09:53.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.015+0000 7fa805104640 1 -- 172.21.15.148:0/3525463355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa80015f200 msgr2=0x7fa800161350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.015+0000 7fa805104640 1 --2- 172.21.15.148:0/3525463355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa80015f200 0x7fa800161350 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8009e50 tx=0x7fa7f8004a40 comp rx=0 tx=0).stop 2024-03-29T04:09:53.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.015+0000 7fa805104640 1 -- 172.21.15.148:0/3525463355 shutdown_connections 2024-03-29T04:09:53.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.015+0000 7fa805104640 1 --2- 172.21.15.148:0/3525463355 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa7ec06c560 0x7fa7ec06ea20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.015+0000 7fa805104640 1 --2- 172.21.15.148:0/3525463355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa80015f200 0x7fa800161350 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.015+0000 7fa805104640 1 --2- 172.21.15.148:0/3525463355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8000ccec0 0x7fa800160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.015+0000 7fa805104640 1 --2- 172.21.15.148:0/3525463355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8000b4fb0 0x7fa800147550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.015+0000 7fa805104640 1 -- 172.21.15.148:0/3525463355 >> 172.21.15.148:0/3525463355 conn(0x7fa8000b3c70 msgr2=0x7fa8000b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:53.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.015+0000 7fa805104640 1 -- 172.21.15.148:0/3525463355 shutdown_connections 2024-03-29T04:09:53.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.015+0000 7fa805104640 1 -- 172.21.15.148:0/3525463355 wait complete. 2024-03-29T04:09:54.006 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:09:54.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.126+0000 7f36b5847640 1 Processor -- start 2024-03-29T04:09:54.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.126+0000 7f36b5847640 1 -- start start 2024-03-29T04:09:54.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.126+0000 7f36b5847640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f36b00d17e0 0x7f36b00d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:54.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.126+0000 7f36b5847640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36b015e7c0 con 0x7f36b015e3e0 2024-03-29T04:09:54.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.126+0000 7f36b5847640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36b00d2c30 con 0x7f36b00ba030 2024-03-29T04:09:54.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.126+0000 7f36b5847640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f36b00d3840 con 0x7f36b00d17e0 2024-03-29T04:09:54.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.126+0000 7f36aeffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f36b015e3e0 0x7f36b00d2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42748/0 (socket says 172.21.15.148:42748) 2024-03-29T04:09:54.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.126+0000 7f36aeffd640 1 -- 172.21.15.148:0/3670094783 learned_addr learned my addr 172.21.15.148:0/3670094783 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:09:54.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.127+0000 7f36af7fe640 1 --2- 172.21.15.148:0/3670094783 >> v2:172.21.15.157:3300/0 conn(0x7f36b00d17e0 0x7f36b00d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:54.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.127+0000 7f36adffb640 1 -- 172.21.15.148:0/3670094783 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1032449893 0 0) 0x7f36b015e7c0 con 0x7f36b015e3e0 2024-03-29T04:09:54.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.127+0000 7f36adffb640 1 -- 172.21.15.148:0/3670094783 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3698003520 con 0x7f36b015e3e0 2024-03-29T04:09:54.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.127+0000 7f36adffb640 1 -- 172.21.15.148:0/3670094783 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 5674731 0 0) 0x7f3698003520 con 0x7f36b015e3e0 2024-03-29T04:09:54.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.127+0000 7f36adffb640 1 -- 172.21.15.148:0/3670094783 >> v2:172.21.15.157:3300/0 conn(0x7f36b00d17e0 msgr2=0x7f36b00d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:54.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.127+0000 7f36adffb640 1 --2- 172.21.15.148:0/3670094783 >> v2:172.21.15.157:3300/0 conn(0x7f36b00d17e0 0x7f36b00d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.127+0000 7f36adffb640 1 -- 172.21.15.148:0/3670094783 >> v1:172.21.15.157:6789/0 conn(0x7f36b00ba030 legacy=0x7f36b00d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:54.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.127+0000 7f36adffb640 1 -- 172.21.15.148:0/3670094783 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b00d2f10 con 0x7f36b015e3e0 2024-03-29T04:09:54.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.127+0000 7f36adffb640 1 -- 172.21.15.148:0/3670094783 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f36a4001670 con 0x7f36b015e3e0 2024-03-29T04:09:54.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.127+0000 7f36af7fe640 1 --2- 172.21.15.148:0/3670094783 >> v2:172.21.15.157:3300/0 conn(0x7f36b00d17e0 0x7f36b00d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:54.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.127+0000 7f36adffb640 1 -- 172.21.15.148:0/3670094783 >> v1:172.21.15.148:6789/0 conn(0x7f36b015e3e0 legacy=0x7f36b00d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:54.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.127+0000 7f36adffb640 1 --2- 172.21.15.148:0/3670094783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36980039b0 0x7f3698003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:54.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.127+0000 7f36adffb640 1 --2- 172.21.15.148:0/3670094783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3698004800 0x7f3698004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:54.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.127+0000 7f36adffb640 1 --2- 172.21.15.148:0/3670094783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36980051c0 0x7f369800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:54.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.127+0000 7f36adffb640 1 -- 172.21.15.148:0/3670094783 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f36b00d3840 con 0x7f3698004800 2024-03-29T04:09:54.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.127+0000 7f36adffb640 1 -- 172.21.15.148:0/3670094783 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f369800c070 con 0x7f36980051c0 2024-03-29T04:09:54.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.127+0000 7f36adffb640 1 -- 172.21.15.148:0/3670094783 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f369800c270 con 0x7f36980039b0 2024-03-29T04:09:54.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.127+0000 7f36ae7fc640 1 --2- 172.21.15.148:0/3670094783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3698004800 0x7f3698004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:54.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.127+0000 7f36af7fe640 1 --2- 172.21.15.148:0/3670094783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36980051c0 0x7f369800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:54.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.127+0000 7f36aeffd640 1 --2- 172.21.15.148:0/3670094783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36980039b0 0x7f3698003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:54.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.127+0000 7f36ae7fc640 1 -- 172.21.15.148:0/3670094783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36980039b0 msgr2=0x7f3698003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:54.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.127+0000 7f36ae7fc640 1 --2- 172.21.15.148:0/3670094783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36980039b0 0x7f3698003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.127+0000 7f36ae7fc640 1 -- 172.21.15.148:0/3670094783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36980051c0 msgr2=0x7f369800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:54.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.127+0000 7f36ae7fc640 1 --2- 172.21.15.148:0/3670094783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36980051c0 0x7f369800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.127+0000 7f36ae7fc640 1 -- 172.21.15.148:0/3670094783 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3698003520 con 0x7f3698004800 2024-03-29T04:09:54.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.127+0000 7f36af7fe640 1 -- 172.21.15.148:0/3670094783 reap_dead start 2024-03-29T04:09:54.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.128+0000 7f36ae7fc640 1 --2- 172.21.15.148:0/3670094783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3698004800 0x7f3698004c80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f36a0001750 tx=0x7f3698005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f6434f9da3eb51 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:54.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.128+0000 7f36adffb640 1 -- 172.21.15.148:0/3670094783 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0001370 con 0x7f3698004800 2024-03-29T04:09:54.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.128+0000 7f36adffb640 1 -- 172.21.15.148:0/3670094783 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36a00052d0 con 0x7f3698004800 2024-03-29T04:09:54.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.128+0000 7f36adffb640 1 -- 172.21.15.148:0/3670094783 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0003830 con 0x7f3698004800 2024-03-29T04:09:54.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.129+0000 7f36b5847640 1 -- 172.21.15.148:0/3670094783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3698004800 msgr2=0x7f3698004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:54.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.129+0000 7f36b5847640 1 --2- 172.21.15.148:0/3670094783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3698004800 0x7f3698004c80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f36a0001750 tx=0x7f3698005830 comp rx=0 tx=0).stop 2024-03-29T04:09:54.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.129+0000 7f36b5847640 1 -- 172.21.15.148:0/3670094783 shutdown_connections 2024-03-29T04:09:54.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.129+0000 7f36b5847640 1 --2- 172.21.15.148:0/3670094783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3698004800 0x7f3698004c80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.129+0000 7f36b5847640 1 -- 172.21.15.148:0/3670094783 >> 172.21.15.148:0/3670094783 conn(0x7f36b00b6210 msgr2=0x7f36b00b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:54.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.129+0000 7f36b5847640 1 -- 172.21.15.148:0/3670094783 shutdown_connections 2024-03-29T04:09:54.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.129+0000 7f36b5847640 1 -- 172.21.15.148:0/3670094783 wait complete. 2024-03-29T04:09:54.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.130+0000 7f36b5847640 1 Processor -- start 2024-03-29T04:09:54.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.130+0000 7f36b5847640 1 -- start start 2024-03-29T04:09:54.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.131+0000 7f36b5847640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3698004800 0x7f36b014a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:54.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.131+0000 7f36b5847640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36b01611b0 0x7f36b00cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:54.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.131+0000 7f36b5847640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36b01615f0 0x7f36b00cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:54.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.131+0000 7f36b5847640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f36b00d3cd0 con 0x7f36b01611b0 2024-03-29T04:09:54.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.131+0000 7f36b5847640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f36b00d34d0 con 0x7f3698004800 2024-03-29T04:09:54.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.131+0000 7f36b5847640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f36b00d3e50 con 0x7f36b01615f0 2024-03-29T04:09:54.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.131+0000 7f36ae7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36b01611b0 0x7f36b00cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:54.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.131+0000 7f36ae7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36b01611b0 0x7f36b00cd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38604/0 (socket says 172.21.15.148:38604) 2024-03-29T04:09:54.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.131+0000 7f36ae7fc640 1 -- 172.21.15.148:0/3615003133 learned_addr learned my addr 172.21.15.148:0/3615003133 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:09:54.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.131+0000 7f36aeffd640 1 --2- 172.21.15.148:0/3615003133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3698004800 0x7f36b014a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:54.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.131+0000 7f36af7fe640 1 --2- 172.21.15.148:0/3615003133 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36b01615f0 0x7f36b00cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:54.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.131+0000 7f36ae7fc640 1 -- 172.21.15.148:0/3615003133 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36b01615f0 msgr2=0x7f36b00cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:54.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.131+0000 7f36ae7fc640 1 --2- 172.21.15.148:0/3615003133 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36b01615f0 0x7f36b00cd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.131+0000 7f36ae7fc640 1 -- 172.21.15.148:0/3615003133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3698004800 msgr2=0x7f36b014a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:54.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.131+0000 7f36ae7fc640 1 --2- 172.21.15.148:0/3615003133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3698004800 0x7f36b014a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.131+0000 7f36ae7fc640 1 -- 172.21.15.148:0/3615003133 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b015f520 con 0x7f36b01611b0 2024-03-29T04:09:54.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.131+0000 7f36ae7fc640 1 --2- 172.21.15.148:0/3615003133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36b01611b0 0x7f36b00cd2a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f36a0000c00 tx=0x7f36a0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:54.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.132+0000 7f3697fff640 1 -- 172.21.15.148:0/3615003133 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0041020 con 0x7f36b01611b0 2024-03-29T04:09:54.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.132+0000 7f3697fff640 1 -- 172.21.15.148:0/3615003133 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36a0005480 con 0x7f36b01611b0 2024-03-29T04:09:54.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.132+0000 7f36b5847640 1 -- 172.21.15.148:0/3615003133 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b015f800 con 0x7f36b01611b0 2024-03-29T04:09:54.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.132+0000 7f3697fff640 1 -- 172.21.15.148:0/3615003133 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0007a30 con 0x7f36b01611b0 2024-03-29T04:09:54.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.132+0000 7f36b5847640 1 -- 172.21.15.148:0/3615003133 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b015fdc0 con 0x7f36b01611b0 2024-03-29T04:09:54.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.133+0000 7f36b5847640 1 -- 172.21.15.148:0/3615003133 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36b00687e0 con 0x7f36b01611b0 2024-03-29T04:09:54.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.133+0000 7f3697fff640 1 -- 172.21.15.148:0/3615003133 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f36a0044020 con 0x7f36b01611b0 2024-03-29T04:09:54.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.133+0000 7f3697fff640 1 --2- 172.21.15.148:0/3615003133 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f369c06c530 0x7f369c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:54.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.133+0000 7f3697fff640 1 -- 172.21.15.148:0/3615003133 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f36a00acd00 con 0x7f36b01611b0 2024-03-29T04:09:54.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.133+0000 7f36aeffd640 1 --2- 172.21.15.148:0/3615003133 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f369c06c530 0x7f369c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:54.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.134+0000 7f36aeffd640 1 --2- 172.21.15.148:0/3615003133 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f369c06c530 0x7f369c06e9f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f36a40011e0 tx=0x7f36a4000ed0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:54.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.138+0000 7f3697fff640 1 -- 172.21.15.148:0/3615003133 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f36a004b030 con 0x7f36b01611b0 2024-03-29T04:09:54.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.351+0000 7f36b5847640 1 -- 172.21.15.148:0/3615003133 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36b006d220 con 0x7f36b01611b0 2024-03-29T04:09:54.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.351+0000 7f3697fff640 1 -- 172.21.15.148:0/3615003133 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1395 (secure 0 0 0) 0x7f36a007af90 con 0x7f36b01611b0 2024-03-29T04:09:54.331 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:54.331 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3356449643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3356449643},{"type":"v1","addr":"172.21.15.148:6836","nonce":3356449643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-29T04:09:54.332 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 2 2024-03-29T04:09:54.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.353+0000 7f36b5847640 1 -- 172.21.15.148:0/3615003133 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f369c06c530 msgr2=0x7f369c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:54.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.353+0000 7f36b5847640 1 --2- 172.21.15.148:0/3615003133 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f369c06c530 0x7f369c06e9f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f36a40011e0 tx=0x7f36a4000ed0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.353+0000 7f36b5847640 1 -- 172.21.15.148:0/3615003133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36b01611b0 msgr2=0x7f36b00cd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:54.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.353+0000 7f36b5847640 1 --2- 172.21.15.148:0/3615003133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36b01611b0 0x7f36b00cd2a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f36a0000c00 tx=0x7f36a0001120 comp rx=0 tx=0).stop 2024-03-29T04:09:54.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.353+0000 7f36b5847640 1 -- 172.21.15.148:0/3615003133 shutdown_connections 2024-03-29T04:09:54.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.353+0000 7f36b5847640 1 --2- 172.21.15.148:0/3615003133 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f369c06c530 0x7f369c06e9f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.353+0000 7f36b5847640 1 --2- 172.21.15.148:0/3615003133 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36b01615f0 0x7f36b00cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.353+0000 7f36b5847640 1 --2- 172.21.15.148:0/3615003133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36b01611b0 0x7f36b00cd2a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.353+0000 7f36b5847640 1 --2- 172.21.15.148:0/3615003133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3698004800 0x7f36b014a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.353+0000 7f36b5847640 1 -- 172.21.15.148:0/3615003133 >> 172.21.15.148:0/3615003133 conn(0x7f36b00b6210 msgr2=0x7f36b00b8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:54.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.353+0000 7f36b5847640 1 -- 172.21.15.148:0/3615003133 shutdown_connections 2024-03-29T04:09:54.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:54.353+0000 7f36b5847640 1 -- 172.21.15.148:0/3615003133 wait complete. 2024-03-29T04:09:56.345 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:09:56.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.477+0000 7fecbe560640 1 Processor -- start 2024-03-29T04:09:56.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.477+0000 7fecbe560640 1 -- start start 2024-03-29T04:09:56.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.477+0000 7fecbe560640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fecb80b2ee0 0x7fecb815f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.477+0000 7fecbe560640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fecb806c5c0 0x7fecb806ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.477+0000 7fecbe560640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecb80b3630 con 0x7fecb815b060 2024-03-29T04:09:56.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.477+0000 7fecbe560640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fecb80b3910 con 0x7fecb806c5c0 2024-03-29T04:09:56.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.477+0000 7fecbe560640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fecb80b3bf0 con 0x7fecb80b2ee0 2024-03-29T04:09:56.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.478+0000 7fecbcafd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fecb806c5c0 0x7fecb806ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.478+0000 7fecbcafd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fecb806c5c0 0x7fecb806ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47208/0 (socket says 172.21.15.148:47208) 2024-03-29T04:09:56.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.478+0000 7fecbcafd640 1 -- 172.21.15.148:0/2905150853 learned_addr learned my addr 172.21.15.148:0/2905150853 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:09:56.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.478+0000 7fecb77fe640 1 --2- 172.21.15.148:0/2905150853 >> v2:172.21.15.157:3300/0 conn(0x7fecb80b2ee0 0x7fecb815f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.478+0000 7fecbcafd640 1 -- 172.21.15.148:0/2905150853 >> v2:172.21.15.157:3300/0 conn(0x7fecb80b2ee0 msgr2=0x7fecb815f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:56.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.478+0000 7fecbcafd640 1 --2- 172.21.15.148:0/2905150853 >> v2:172.21.15.157:3300/0 conn(0x7fecb80b2ee0 0x7fecb815f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.478+0000 7fecbcafd640 1 -- 172.21.15.148:0/2905150853 >> v1:172.21.15.157:6789/0 conn(0x7fecb815b060 legacy=0x7fecb80b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.478+0000 7fecbcafd640 1 -- 172.21.15.148:0/2905150853 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb80b3ed0 con 0x7fecb806c5c0 2024-03-29T04:09:56.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.479+0000 7fecbcafd640 1 --2- 172.21.15.148:0/2905150853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecb806c5c0 0x7fecb806ca50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fecb806d1b0 tx=0x7fecb002e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562f43f719726966 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:56.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.479+0000 7fecb6ffd640 1 -- 172.21.15.148:0/2905150853 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb002ea00 con 0x7fecb806c5c0 2024-03-29T04:09:56.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.479+0000 7fecb6ffd640 1 -- 172.21.15.148:0/2905150853 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecb002eba0 con 0x7fecb806c5c0 2024-03-29T04:09:56.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.479+0000 7fecb6ffd640 1 -- 172.21.15.148:0/2905150853 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0009600 con 0x7fecb806c5c0 2024-03-29T04:09:56.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.479+0000 7fecbe560640 1 -- 172.21.15.148:0/2905150853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecb806c5c0 msgr2=0x7fecb806ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.479+0000 7fecbe560640 1 --2- 172.21.15.148:0/2905150853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecb806c5c0 0x7fecb806ca50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fecb806d1b0 tx=0x7fecb002e060 comp rx=0 tx=0).stop 2024-03-29T04:09:56.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.479+0000 7fecbe560640 1 -- 172.21.15.148:0/2905150853 shutdown_connections 2024-03-29T04:09:56.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.479+0000 7fecbe560640 1 --2- 172.21.15.148:0/2905150853 >> v2:172.21.15.157:3300/0 conn(0x7fecb80b2ee0 0x7fecb815f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.479+0000 7fecbe560640 1 --2- 172.21.15.148:0/2905150853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecb806c5c0 0x7fecb806ca50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.479+0000 7fecbe560640 1 -- 172.21.15.148:0/2905150853 >> 172.21.15.148:0/2905150853 conn(0x7fecb80cfed0 msgr2=0x7fecb80d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:56.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.479+0000 7fecbe560640 1 -- 172.21.15.148:0/2905150853 shutdown_connections 2024-03-29T04:09:56.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.479+0000 7fecbe560640 1 -- 172.21.15.148:0/2905150853 wait complete. 2024-03-29T04:09:56.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.481+0000 7fecbe560640 1 Processor -- start 2024-03-29T04:09:56.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.481+0000 7fecbe560640 1 -- start start 2024-03-29T04:09:56.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.481+0000 7fecbe560640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecb806c5c0 0x7fecb814a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.482+0000 7fecbe560640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecb80b2ee0 0x7fecb80c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.482+0000 7fecbe560640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecb815b060 0x7fecb80c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.482+0000 7fecbe560640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fecb815b440 con 0x7fecb815b060 2024-03-29T04:09:56.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.482+0000 7fecbe560640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fecb80b3340 con 0x7fecb80b2ee0 2024-03-29T04:09:56.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.482+0000 7fecbe560640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fecb80cd530 con 0x7fecb806c5c0 2024-03-29T04:09:56.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.482+0000 7fecbcafd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecb815b060 0x7fecb80c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.482+0000 7fecbcafd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecb815b060 0x7fecb80c8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47224/0 (socket says 172.21.15.148:47224) 2024-03-29T04:09:56.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.482+0000 7fecbcafd640 1 -- 172.21.15.148:0/513198655 learned_addr learned my addr 172.21.15.148:0/513198655 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:09:56.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.482+0000 7fecb7fff640 1 --2- 172.21.15.148:0/513198655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecb806c5c0 0x7fecb814a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.482+0000 7fecb77fe640 1 --2- 172.21.15.148:0/513198655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecb80b2ee0 0x7fecb80c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.482+0000 7fecbcafd640 1 -- 172.21.15.148:0/513198655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecb806c5c0 msgr2=0x7fecb814a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.482+0000 7fecbcafd640 1 --2- 172.21.15.148:0/513198655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecb806c5c0 0x7fecb814a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.482+0000 7fecbcafd640 1 -- 172.21.15.148:0/513198655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecb80b2ee0 msgr2=0x7fecb80c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.482+0000 7fecbcafd640 1 --2- 172.21.15.148:0/513198655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecb80b2ee0 0x7fecb80c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.482+0000 7fecbcafd640 1 -- 172.21.15.148:0/513198655 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb815d5b0 con 0x7fecb815b060 2024-03-29T04:09:56.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.482+0000 7fecb77fe640 1 --2- 172.21.15.148:0/513198655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecb80b2ee0 0x7fecb80c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:56.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.482+0000 7fecbcafd640 1 --2- 172.21.15.148:0/513198655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecb815b060 0x7fecb80c8750 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fecb000d240 tx=0x7fecb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:56.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.482+0000 7fecb4ff9640 1 -- 172.21.15.148:0/513198655 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0045020 con 0x7fecb815b060 2024-03-29T04:09:56.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.482+0000 7fecb4ff9640 1 -- 172.21.15.148:0/513198655 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecb0009e30 con 0x7fecb815b060 2024-03-29T04:09:56.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.482+0000 7fecbe560640 1 -- 172.21.15.148:0/513198655 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecb815d890 con 0x7fecb815b060 2024-03-29T04:09:56.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.482+0000 7fecb4ff9640 1 -- 172.21.15.148:0/513198655 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb002ea00 con 0x7fecb815b060 2024-03-29T04:09:56.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.482+0000 7fecbe560640 1 -- 172.21.15.148:0/513198655 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecb815de50 con 0x7fecb815b060 2024-03-29T04:09:56.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.483+0000 7fecbe560640 1 -- 172.21.15.148:0/513198655 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecb80687e0 con 0x7fecb815b060 2024-03-29T04:09:56.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.484+0000 7fecb4ff9640 1 -- 172.21.15.148:0/513198655 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fecb0048020 con 0x7fecb815b060 2024-03-29T04:09:56.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.484+0000 7fecb4ff9640 1 --2- 172.21.15.148:0/513198655 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7feca406c530 0x7feca406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.484+0000 7fecb4ff9640 1 -- 172.21.15.148:0/513198655 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fecb00afda0 con 0x7fecb815b060 2024-03-29T04:09:56.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.484+0000 7fecb7fff640 1 --2- 172.21.15.148:0/513198655 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7feca406c530 0x7feca406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.485+0000 7fecb7fff640 1 --2- 172.21.15.148:0/513198655 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7feca406c530 0x7feca406e9f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7feca8007fd0 tx=0x7feca8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:56.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.489+0000 7fecb4ff9640 1 -- 172.21.15.148:0/513198655 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fecb004f030 con 0x7fecb815b060 2024-03-29T04:09:56.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.702+0000 7fecbe560640 1 -- 172.21.15.148:0/513198655 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fecb80721e0 con 0x7fecb815b060 2024-03-29T04:09:56.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.703+0000 7fecb4ff9640 1 -- 172.21.15.148:0/513198655 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3870 (secure 0 0 0) 0x7fecb007e720 con 0x7fecb815b060 2024-03-29T04:09:56.683 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:56.683 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/36660496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":36660496},{"type":"v1","addr":"172.21.15.157:6834","nonce":36660496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2674481055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2674481055},{"type":"v1","addr":"172.21.15.157:6835","nonce":2674481055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3356449643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3356449643},{"type":"v1","addr":"172.21.15.148:6836","nonce":3356449643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2453181817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2453181817},{"type":"v1","addr":"172.21.15.148:6837","nonce":2453181817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-29T04:09:56.685 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 3 2024-03-29T04:09:56.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.705+0000 7fecbe560640 1 -- 172.21.15.148:0/513198655 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7feca406c530 msgr2=0x7feca406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.705+0000 7fecbe560640 1 --2- 172.21.15.148:0/513198655 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7feca406c530 0x7feca406e9f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7feca8007fd0 tx=0x7feca8009040 comp rx=0 tx=0).stop 2024-03-29T04:09:56.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.705+0000 7fecbe560640 1 -- 172.21.15.148:0/513198655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecb815b060 msgr2=0x7fecb80c8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.705+0000 7fecbe560640 1 --2- 172.21.15.148:0/513198655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecb815b060 0x7fecb80c8750 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fecb000d240 tx=0x7fecb0004a40 comp rx=0 tx=0).stop 2024-03-29T04:09:56.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.705+0000 7fecbe560640 1 -- 172.21.15.148:0/513198655 shutdown_connections 2024-03-29T04:09:56.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.705+0000 7fecbe560640 1 --2- 172.21.15.148:0/513198655 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7feca406c530 0x7feca406e9f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.705+0000 7fecbe560640 1 --2- 172.21.15.148:0/513198655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecb815b060 0x7fecb80c8750 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.705+0000 7fecbe560640 1 --2- 172.21.15.148:0/513198655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecb80b2ee0 0x7fecb80c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.705+0000 7fecbe560640 1 --2- 172.21.15.148:0/513198655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecb806c5c0 0x7fecb814a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.705+0000 7fecbe560640 1 -- 172.21.15.148:0/513198655 >> 172.21.15.148:0/513198655 conn(0x7fecb80cfed0 msgr2=0x7fecb8156c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:56.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.705+0000 7fecbe560640 1 -- 172.21.15.148:0/513198655 shutdown_connections 2024-03-29T04:09:56.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.705+0000 7fecbe560640 1 -- 172.21.15.148:0/513198655 wait complete. 2024-03-29T04:09:56.696 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:56.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.816+0000 7f9914db5640 1 Processor -- start 2024-03-29T04:09:56.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.816+0000 7f9914db5640 1 -- start start 2024-03-29T04:09:56.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.817+0000 7f9914db5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f99100ba030 0x7f99100d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.817+0000 7f9914db5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f991015e7c0 con 0x7f991015e3e0 2024-03-29T04:09:56.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.817+0000 7f9914db5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99100d2c30 con 0x7f99100d10d0 2024-03-29T04:09:56.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.817+0000 7f9914db5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f99100d3840 con 0x7f99100ba030 2024-03-29T04:09:56.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.817+0000 7f990e59c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f991015e3e0 0x7f99100d2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51726/0 (socket says 172.21.15.148:51726) 2024-03-29T04:09:56.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.817+0000 7f990e59c640 1 -- 172.21.15.148:0/3663339099 learned_addr learned my addr 172.21.15.148:0/3663339099 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:09:56.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.818+0000 7f990dd9b640 1 --2- 172.21.15.148:0/3663339099 >> v2:172.21.15.157:3300/0 conn(0x7f99100ba030 0x7f99100d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.818+0000 7f990d59a640 1 -- 172.21.15.148:0/3663339099 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 778404390 0 0) 0x7f991015e7c0 con 0x7f991015e3e0 2024-03-29T04:09:56.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.818+0000 7f990d59a640 1 -- 172.21.15.148:0/3663339099 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98fc003520 con 0x7f991015e3e0 2024-03-29T04:09:56.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.818+0000 7f990dd9b640 1 -- 172.21.15.148:0/3663339099 >> v1:172.21.15.157:6789/0 conn(0x7f99100d10d0 legacy=0x7f9910167f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.818+0000 7f990d59a640 1 -- 172.21.15.148:0/3663339099 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2575676559 0 0) 0x7f98fc003520 con 0x7f991015e3e0 2024-03-29T04:09:56.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.818+0000 7f990dd9b640 1 -- 172.21.15.148:0/3663339099 >> v1:172.21.15.148:6789/0 conn(0x7f991015e3e0 legacy=0x7f99100d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.818+0000 7f990dd9b640 1 -- 172.21.15.148:0/3663339099 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99100d2f10 con 0x7f99100ba030 2024-03-29T04:09:56.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.819+0000 7f990dd9b640 1 --2- 172.21.15.148:0/3663339099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99100ba030 0x7f99100d3cd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f98f80097a0 tx=0x7f98f802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdff45158c270a71 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:56.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.819+0000 7f990d59a640 1 -- 172.21.15.148:0/3663339099 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99080019f0 con 0x7f99100ba030 2024-03-29T04:09:56.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.819+0000 7f990d59a640 1 -- 172.21.15.148:0/3663339099 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98f8037070 con 0x7f99100ba030 2024-03-29T04:09:56.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.819+0000 7f990d59a640 1 -- 172.21.15.148:0/3663339099 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f8031790 con 0x7f99100ba030 2024-03-29T04:09:56.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.819+0000 7f9914db5640 1 -- 172.21.15.148:0/3663339099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99100ba030 msgr2=0x7f99100d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.819+0000 7f9914db5640 1 --2- 172.21.15.148:0/3663339099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99100ba030 0x7f99100d3cd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f98f80097a0 tx=0x7f98f802b8b0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.820+0000 7f9914db5640 1 -- 172.21.15.148:0/3663339099 shutdown_connections 2024-03-29T04:09:56.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.820+0000 7f9914db5640 1 --2- 172.21.15.148:0/3663339099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99100ba030 0x7f99100d3cd0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.820+0000 7f9914db5640 1 -- 172.21.15.148:0/3663339099 >> 172.21.15.148:0/3663339099 conn(0x7f99100b6210 msgr2=0x7f99100b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:56.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.820+0000 7f9914db5640 1 -- 172.21.15.148:0/3663339099 shutdown_connections 2024-03-29T04:09:56.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.820+0000 7f9914db5640 1 -- 172.21.15.148:0/3663339099 wait complete. 2024-03-29T04:09:56.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.821+0000 7f9914db5640 1 Processor -- start 2024-03-29T04:09:56.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.821+0000 7f9914db5640 1 -- start start 2024-03-29T04:09:56.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.822+0000 7f9914db5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99100ba030 0x7f991014b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.822+0000 7f9914db5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99100d10d0 0x7f991015a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.822+0000 7f9914db5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f991015e3e0 0x7f991015bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.822+0000 7f9914db5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f99100d1530 con 0x7f99100d10d0 2024-03-29T04:09:56.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.822+0000 7f9914db5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f99100d34d0 con 0x7f99100ba030 2024-03-29T04:09:56.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.822+0000 7f9914db5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f99100d16b0 con 0x7f991015e3e0 2024-03-29T04:09:56.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.822+0000 7f990dd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99100d10d0 0x7f991015a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.822+0000 7f990dd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99100d10d0 0x7f991015a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47240/0 (socket says 172.21.15.148:47240) 2024-03-29T04:09:56.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.822+0000 7f990dd9b640 1 -- 172.21.15.148:0/1355240892 learned_addr learned my addr 172.21.15.148:0/1355240892 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:09:56.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.822+0000 7f990ed9d640 1 --2- 172.21.15.148:0/1355240892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f991015e3e0 0x7f991015bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.822+0000 7f990e59c640 1 --2- 172.21.15.148:0/1355240892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99100ba030 0x7f991014b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.822+0000 7f990dd9b640 1 -- 172.21.15.148:0/1355240892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f991015e3e0 msgr2=0x7f991015bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.822+0000 7f990dd9b640 1 --2- 172.21.15.148:0/1355240892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f991015e3e0 0x7f991015bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.822+0000 7f990dd9b640 1 -- 172.21.15.148:0/1355240892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99100ba030 msgr2=0x7f991014b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.822+0000 7f990dd9b640 1 --2- 172.21.15.148:0/1355240892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99100ba030 0x7f991014b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.822+0000 7f990dd9b640 1 -- 172.21.15.148:0/1355240892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991014bc60 con 0x7f99100d10d0 2024-03-29T04:09:56.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.822+0000 7f990ed9d640 1 --2- 172.21.15.148:0/1355240892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f991015e3e0 0x7f991015bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:56.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.822+0000 7f990e59c640 1 --2- 172.21.15.148:0/1355240892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99100ba030 0x7f991014b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:56.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.822+0000 7f990dd9b640 1 --2- 172.21.15.148:0/1355240892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99100d10d0 0x7f991015a0d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f98f8000c00 tx=0x7f98f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:56.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.822+0000 7f9906ffd640 1 -- 172.21.15.148:0/1355240892 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f8032620 con 0x7f99100d10d0 2024-03-29T04:09:56.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.822+0000 7f9906ffd640 1 -- 172.21.15.148:0/1355240892 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98f80327c0 con 0x7f99100d10d0 2024-03-29T04:09:56.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.822+0000 7f9906ffd640 1 -- 172.21.15.148:0/1355240892 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f8032bb0 con 0x7f99100d10d0 2024-03-29T04:09:56.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.822+0000 7f9914db5640 1 -- 172.21.15.148:0/1355240892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f991015ca60 con 0x7f99100d10d0 2024-03-29T04:09:56.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.822+0000 7f9914db5640 1 -- 172.21.15.148:0/1355240892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f991015d020 con 0x7f99100d10d0 2024-03-29T04:09:56.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.823+0000 7f9914db5640 1 -- 172.21.15.148:0/1355240892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99100687e0 con 0x7f99100d10d0 2024-03-29T04:09:56.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.824+0000 7f9906ffd640 1 -- 172.21.15.148:0/1355240892 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f98f804c060 con 0x7f99100d10d0 2024-03-29T04:09:56.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.824+0000 7f9906ffd640 1 --2- 172.21.15.148:0/1355240892 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f98f006c530 0x7f98f006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.824+0000 7f990e59c640 1 --2- 172.21.15.148:0/1355240892 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f98f006c530 0x7f98f006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.824+0000 7f9906ffd640 1 -- 172.21.15.148:0/1355240892 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f98f80acb90 con 0x7f99100d10d0 2024-03-29T04:09:56.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.825+0000 7f990e59c640 1 --2- 172.21.15.148:0/1355240892 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f98f006c530 0x7f98f006e9f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9908005c00 tx=0x7f9908009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:56.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:56.829+0000 7f9906ffd640 1 -- 172.21.15.148:0/1355240892 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f991014bc60 con 0x7f99100d10d0 2024-03-29T04:09:56.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.004+0000 7f9914db5640 1 -- 172.21.15.148:0/1355240892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99100b7500 con 0x7f99100d10d0 2024-03-29T04:09:56.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.005+0000 7f9906ffd640 1 -- 172.21.15.148:0/1355240892 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12896 (secure 0 0 0) 0x7f98f8040070 con 0x7f99100d10d0 2024-03-29T04:09:56.985 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:56.985 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":16,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:09:49.904034+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:56.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.007+0000 7f9914db5640 1 -- 172.21.15.148:0/1355240892 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f98f006c530 msgr2=0x7f98f006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.007+0000 7f9914db5640 1 --2- 172.21.15.148:0/1355240892 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f98f006c530 0x7f98f006e9f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9908005c00 tx=0x7f9908009040 comp rx=0 tx=0).stop 2024-03-29T04:09:56.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.007+0000 7f9914db5640 1 -- 172.21.15.148:0/1355240892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99100d10d0 msgr2=0x7f991015a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.007+0000 7f9914db5640 1 --2- 172.21.15.148:0/1355240892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99100d10d0 0x7f991015a0d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f98f8000c00 tx=0x7f98f80046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.007+0000 7f9914db5640 1 -- 172.21.15.148:0/1355240892 shutdown_connections 2024-03-29T04:09:56.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.007+0000 7f9914db5640 1 --2- 172.21.15.148:0/1355240892 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f98f006c530 0x7f98f006e9f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.007+0000 7f9914db5640 1 --2- 172.21.15.148:0/1355240892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f991015e3e0 0x7f991015bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.007+0000 7f9914db5640 1 --2- 172.21.15.148:0/1355240892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99100d10d0 0x7f991015a0d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.007+0000 7f9914db5640 1 --2- 172.21.15.148:0/1355240892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99100ba030 0x7f991014b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.007+0000 7f9914db5640 1 -- 172.21.15.148:0/1355240892 >> 172.21.15.148:0/1355240892 conn(0x7f99100b6210 msgr2=0x7f99100c9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:56.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.007+0000 7f9914db5640 1 -- 172.21.15.148:0/1355240892 shutdown_connections 2024-03-29T04:09:56.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.007+0000 7f9914db5640 1 -- 172.21.15.148:0/1355240892 wait complete. 2024-03-29T04:09:56.998 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:47.898351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:09:56.999 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:09:57.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.122+0000 7fecf994d640 1 Processor -- start 2024-03-29T04:09:57.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.122+0000 7fecf994d640 1 -- start start 2024-03-29T04:09:57.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.123+0000 7fecf994d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fecf415e3c0 0x7fecf4161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.123+0000 7fecf994d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf40b2f00 0x7fecf41624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.123+0000 7fecf994d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecf40cf330 con 0x7fecf40b3430 2024-03-29T04:09:57.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.123+0000 7fecf994d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fecf40cf960 con 0x7fecf415e3c0 2024-03-29T04:09:57.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.123+0000 7fecf994d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fecf4162cf0 con 0x7fecf40b2f00 2024-03-29T04:09:57.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.123+0000 7fecf2ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fecf415e3c0 0x7fecf4161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.123+0000 7fecf37fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fecf40b3430 0x7fecf40d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51736/0 (socket says 172.21.15.148:51736) 2024-03-29T04:09:57.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.123+0000 7fecf37fe640 1 -- 172.21.15.148:0/1891419835 learned_addr learned my addr 172.21.15.148:0/1891419835 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:09:57.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.123+0000 7fecf27fc640 1 --2- 172.21.15.148:0/1891419835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf40b2f00 0x7fecf41624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.123+0000 7fecf2ffd640 1 -- 172.21.15.148:0/1891419835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf40b2f00 msgr2=0x7fecf41624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:57.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.123+0000 7fecf2ffd640 1 --2- 172.21.15.148:0/1891419835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf40b2f00 0x7fecf41624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.123+0000 7fecf2ffd640 1 -- 172.21.15.148:0/1891419835 >> v1:172.21.15.148:6789/0 conn(0x7fecf40b3430 legacy=0x7fecf40d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.123+0000 7fecf2ffd640 1 -- 172.21.15.148:0/1891419835 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf40cf5f0 con 0x7fecf415e3c0 2024-03-29T04:09:57.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.124+0000 7fecf2ffd640 1 --2- 172.21.15.148:0/1891419835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecf415e3c0 0x7fecf4161f70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fece40097a0 tx=0x7fece402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808f2f5e6ea31a95 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:57.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.124+0000 7fecf1ffb640 1 -- 172.21.15.148:0/1891419835 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4035020 con 0x7fecf415e3c0 2024-03-29T04:09:57.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.124+0000 7fecf1ffb640 1 -- 172.21.15.148:0/1891419835 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fece4031810 con 0x7fecf415e3c0 2024-03-29T04:09:57.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.124+0000 7fecf1ffb640 1 -- 172.21.15.148:0/1891419835 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4031be0 con 0x7fecf415e3c0 2024-03-29T04:09:57.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.125+0000 7fecf994d640 1 -- 172.21.15.148:0/1891419835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecf415e3c0 msgr2=0x7fecf4161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.125+0000 7fecf994d640 1 --2- 172.21.15.148:0/1891419835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecf415e3c0 0x7fecf4161f70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fece40097a0 tx=0x7fece402b8b0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.125+0000 7fecf994d640 1 -- 172.21.15.148:0/1891419835 shutdown_connections 2024-03-29T04:09:57.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.125+0000 7fecf994d640 1 --2- 172.21.15.148:0/1891419835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf40b2f00 0x7fecf41624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.125+0000 7fecf994d640 1 --2- 172.21.15.148:0/1891419835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecf415e3c0 0x7fecf4161f70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.125+0000 7fecf994d640 1 -- 172.21.15.148:0/1891419835 >> 172.21.15.148:0/1891419835 conn(0x7fecf40b6430 msgr2=0x7fecf40d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:57.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.125+0000 7fecf994d640 1 -- 172.21.15.148:0/1891419835 shutdown_connections 2024-03-29T04:09:57.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.125+0000 7fecf994d640 1 -- 172.21.15.148:0/1891419835 wait complete. 2024-03-29T04:09:57.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.126+0000 7fecf994d640 1 Processor -- start 2024-03-29T04:09:57.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.126+0000 7fecf994d640 1 -- start start 2024-03-29T04:09:57.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.127+0000 7fecf994d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf40b3430 0x7fecf40cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.127+0000 7fecf994d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecf415e3c0 0x7fecf41e7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.127+0000 7fecf994d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecf40cd890 0x7fecf41e83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.127+0000 7fecf994d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fecf40b39a0 con 0x7fecf415e3c0 2024-03-29T04:09:57.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.127+0000 7fecf994d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fecf415e7a0 con 0x7fecf40cd890 2024-03-29T04:09:57.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.127+0000 7fecf994d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fecf40b3b20 con 0x7fecf40b3430 2024-03-29T04:09:57.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.127+0000 7fecf27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecf415e3c0 0x7fecf41e7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.127+0000 7fecf27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecf415e3c0 0x7fecf41e7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47258/0 (socket says 172.21.15.148:47258) 2024-03-29T04:09:57.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.127+0000 7fecf27fc640 1 -- 172.21.15.148:0/2690318290 learned_addr learned my addr 172.21.15.148:0/2690318290 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:09:57.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.127+0000 7fecf2ffd640 1 --2- 172.21.15.148:0/2690318290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf40b3430 0x7fecf40cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.127+0000 7fecf37fe640 1 --2- 172.21.15.148:0/2690318290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecf40cd890 0x7fecf41e83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.127+0000 7fecf27fc640 1 -- 172.21.15.148:0/2690318290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf40b3430 msgr2=0x7fecf40cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.127+0000 7fecf27fc640 1 --2- 172.21.15.148:0/2690318290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf40b3430 0x7fecf40cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.127+0000 7fecf27fc640 1 -- 172.21.15.148:0/2690318290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecf40cd890 msgr2=0x7fecf41e83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.127+0000 7fecf27fc640 1 --2- 172.21.15.148:0/2690318290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecf40cd890 0x7fecf41e83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.127+0000 7fecf27fc640 1 -- 172.21.15.148:0/2690318290 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf40cc7f0 con 0x7fecf415e3c0 2024-03-29T04:09:57.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.127+0000 7fecf37fe640 1 --2- 172.21.15.148:0/2690318290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecf40cd890 0x7fecf41e83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:57.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.128+0000 7fecf27fc640 1 --2- 172.21.15.148:0/2690318290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecf415e3c0 0x7fecf41e7e90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fece800d800 tx=0x7fece800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:57.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.128+0000 7fecdbfff640 1 -- 172.21.15.148:0/2690318290 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece800b6c0 con 0x7fecf415e3c0 2024-03-29T04:09:57.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.128+0000 7fecdbfff640 1 -- 172.21.15.148:0/2690318290 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fece8019070 con 0x7fecf415e3c0 2024-03-29T04:09:57.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.128+0000 7fecf994d640 1 -- 172.21.15.148:0/2690318290 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf40ccad0 con 0x7fecf415e3c0 2024-03-29T04:09:57.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.128+0000 7fecdbfff640 1 -- 172.21.15.148:0/2690318290 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece800ba80 con 0x7fecf415e3c0 2024-03-29T04:09:57.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.128+0000 7fecf994d640 1 -- 172.21.15.148:0/2690318290 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf40cd090 con 0x7fecf415e3c0 2024-03-29T04:09:57.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.129+0000 7fecf994d640 1 -- 172.21.15.148:0/2690318290 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecf4068850 con 0x7fecf415e3c0 2024-03-29T04:09:57.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.129+0000 7fecdbfff640 1 -- 172.21.15.148:0/2690318290 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fece800bc20 con 0x7fecf415e3c0 2024-03-29T04:09:57.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.129+0000 7fecdbfff640 1 --2- 172.21.15.148:0/2690318290 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fece006c530 0x7fece006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.129+0000 7fecdbfff640 1 -- 172.21.15.148:0/2690318290 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fece808b7c0 con 0x7fecf415e3c0 2024-03-29T04:09:57.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.129+0000 7fecf2ffd640 1 --2- 172.21.15.148:0/2690318290 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fece006c530 0x7fece006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.130+0000 7fecf2ffd640 1 --2- 172.21.15.148:0/2690318290 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fece006c530 0x7fece006e9f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fece4036000 tx=0x7fece403c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:57.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.134+0000 7fecdbfff640 1 -- 172.21.15.148:0/2690318290 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fece805a130 con 0x7fecf415e3c0 2024-03-29T04:09:57.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.310+0000 7fecf994d640 1 -- 172.21.15.148:0/2690318290 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fecf40cf110 con 0x7fecf415e3c0 2024-03-29T04:09:57.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.310+0000 7fecdbfff640 1 -- 172.21.15.148:0/2690318290 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fece8059a30 con 0x7fecf415e3c0 2024-03-29T04:09:57.290 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-29T04:09:57.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.312+0000 7fecf994d640 1 -- 172.21.15.148:0/2690318290 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fece006c530 msgr2=0x7fece006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.312+0000 7fecf994d640 1 --2- 172.21.15.148:0/2690318290 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fece006c530 0x7fece006e9f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fece4036000 tx=0x7fece403c040 comp rx=0 tx=0).stop 2024-03-29T04:09:57.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.312+0000 7fecf994d640 1 -- 172.21.15.148:0/2690318290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecf415e3c0 msgr2=0x7fecf41e7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.312+0000 7fecf994d640 1 --2- 172.21.15.148:0/2690318290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecf415e3c0 0x7fecf41e7e90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fece800d800 tx=0x7fece800dcc0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.312+0000 7fecf994d640 1 -- 172.21.15.148:0/2690318290 shutdown_connections 2024-03-29T04:09:57.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.312+0000 7fecf994d640 1 --2- 172.21.15.148:0/2690318290 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fece006c530 0x7fece006e9f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.312+0000 7fecf994d640 1 --2- 172.21.15.148:0/2690318290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecf40cd890 0x7fecf41e83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.312+0000 7fecf994d640 1 --2- 172.21.15.148:0/2690318290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecf415e3c0 0x7fecf41e7e90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.312+0000 7fecf994d640 1 --2- 172.21.15.148:0/2690318290 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecf40b3430 0x7fecf40cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.312+0000 7fecf994d640 1 -- 172.21.15.148:0/2690318290 >> 172.21.15.148:0/2690318290 conn(0x7fecf40b6430 msgr2=0x7fecf40b5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:57.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.312+0000 7fecf994d640 1 -- 172.21.15.148:0/2690318290 shutdown_connections 2024-03-29T04:09:57.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.312+0000 7fecf994d640 1 -- 172.21.15.148:0/2690318290 wait complete. 2024-03-29T04:09:57.303 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-29T04:09:57.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.424+0000 7f7aa767b640 1 Processor -- start 2024-03-29T04:09:57.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.424+0000 7f7aa767b640 1 -- start start 2024-03-29T04:09:57.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.425+0000 7f7aa767b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7aa015ed50 0x7f7aa00cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.425+0000 7f7aa767b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7aa006d010 con 0x7f7aa015e800 2024-03-29T04:09:57.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.425+0000 7f7aa767b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7aa006d2f0 con 0x7f7aa00d0220 2024-03-29T04:09:57.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.425+0000 7f7aa767b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7aa00d0690 con 0x7f7aa015ed50 2024-03-29T04:09:57.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.425+0000 7f7aa5417640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7aa015e800 0x7f7aa006c6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51752/0 (socket says 172.21.15.148:51752) 2024-03-29T04:09:57.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.425+0000 7f7aa5417640 1 -- 172.21.15.148:0/280640131 learned_addr learned my addr 172.21.15.148:0/280640131 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:09:57.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.425+0000 7f7aa4c16640 1 --2- 172.21.15.148:0/280640131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7aa015ed50 0x7f7aa00cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.425+0000 7f7a97fff640 1 -- 172.21.15.148:0/280640131 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3203913015 0 0) 0x7f7aa006d010 con 0x7f7aa015e800 2024-03-29T04:09:57.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.425+0000 7f7a97fff640 1 -- 172.21.15.148:0/280640131 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a8c003520 con 0x7f7aa015e800 2024-03-29T04:09:57.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.425+0000 7f7a97fff640 1 -- 172.21.15.148:0/280640131 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1965379155 0 0) 0x7f7a8c003520 con 0x7f7aa015e800 2024-03-29T04:09:57.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.425+0000 7f7a97fff640 1 -- 172.21.15.148:0/280640131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7aa015ed50 msgr2=0x7f7aa00cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:57.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.425+0000 7f7a97fff640 1 --2- 172.21.15.148:0/280640131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7aa015ed50 0x7f7aa00cfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.425+0000 7f7a97fff640 1 -- 172.21.15.148:0/280640131 >> v1:172.21.15.157:6789/0 conn(0x7f7aa00d0220 legacy=0x7f7aa00b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.425+0000 7f7a97fff640 1 -- 172.21.15.148:0/280640131 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa006d5d0 con 0x7f7aa015e800 2024-03-29T04:09:57.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.425+0000 7f7a97fff640 1 -- 172.21.15.148:0/280640131 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f7a9c001670 con 0x7f7aa015e800 2024-03-29T04:09:57.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.425+0000 7f7aa4c16640 1 --2- 172.21.15.148:0/280640131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7aa015ed50 0x7f7aa00cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:57.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.425+0000 7f7a97fff640 1 -- 172.21.15.148:0/280640131 >> v1:172.21.15.148:6789/0 conn(0x7f7aa015e800 legacy=0x7f7aa006c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.425+0000 7f7a97fff640 1 --2- 172.21.15.148:0/280640131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a8c0039e0 0x7f7a8c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.425+0000 7f7a97fff640 1 --2- 172.21.15.148:0/280640131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a8c004390 0x7f7a8c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.425+0000 7f7a97fff640 1 --2- 172.21.15.148:0/280640131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7aa015ed50 0x7f7a8c005120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.425+0000 7f7a97fff640 1 -- 172.21.15.148:0/280640131 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa00d0690 con 0x7f7a8c0039e0 2024-03-29T04:09:57.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.425+0000 7f7a97fff640 1 -- 172.21.15.148:0/280640131 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c00ba40 con 0x7f7a8c004390 2024-03-29T04:09:57.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.425+0000 7f7a97fff640 1 -- 172.21.15.148:0/280640131 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7a8c00bc40 con 0x7f7aa015ed50 2024-03-29T04:09:57.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.425+0000 7f7aa5417640 1 --2- 172.21.15.148:0/280640131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a8c0039e0 0x7f7a8c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.426+0000 7f7aa4c16640 1 --2- 172.21.15.148:0/280640131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a8c004390 0x7f7a8c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.426+0000 7f7aa5c18640 1 --2- 172.21.15.148:0/280640131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7aa015ed50 0x7f7a8c005120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.426+0000 7f7aa5417640 1 -- 172.21.15.148:0/280640131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7aa015ed50 msgr2=0x7f7a8c005120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:57.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.426+0000 7f7aa5417640 1 --2- 172.21.15.148:0/280640131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7aa015ed50 0x7f7a8c005120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.426+0000 7f7aa5417640 1 -- 172.21.15.148:0/280640131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a8c004390 msgr2=0x7f7a8c004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:57.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.426+0000 7f7aa5417640 1 --2- 172.21.15.148:0/280640131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a8c004390 0x7f7a8c004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.426+0000 7f7aa5417640 1 -- 172.21.15.148:0/280640131 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c003520 con 0x7f7a8c0039e0 2024-03-29T04:09:57.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.426+0000 7f7aa4c16640 1 --2- 172.21.15.148:0/280640131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a8c004390 0x7f7a8c004860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:57.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.426+0000 7f7aa5c18640 1 --2- 172.21.15.148:0/280640131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7aa015ed50 0x7f7a8c005120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:57.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.426+0000 7f7aa5417640 1 --2- 172.21.15.148:0/280640131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a8c0039e0 0x7f7a8c003dc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00c240 tx=0x7f7a8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ed5409529be2e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:57.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.427+0000 7f7a97fff640 1 -- 172.21.15.148:0/280640131 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c007d30 con 0x7f7a8c0039e0 2024-03-29T04:09:57.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.427+0000 7f7a97fff640 1 -- 172.21.15.148:0/280640131 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a9c030070 con 0x7f7a8c0039e0 2024-03-29T04:09:57.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.427+0000 7f7a97fff640 1 -- 172.21.15.148:0/280640131 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c02cd60 con 0x7f7a8c0039e0 2024-03-29T04:09:57.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.427+0000 7f7aa767b640 1 -- 172.21.15.148:0/280640131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a8c0039e0 msgr2=0x7f7a8c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.427+0000 7f7aa767b640 1 --2- 172.21.15.148:0/280640131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a8c0039e0 0x7f7a8c003dc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00c240 tx=0x7f7a8c005830 comp rx=0 tx=0).stop 2024-03-29T04:09:57.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.427+0000 7f7aa5c18640 1 -- 172.21.15.148:0/280640131 reap_dead start 2024-03-29T04:09:57.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.427+0000 7f7aa767b640 1 -- 172.21.15.148:0/280640131 shutdown_connections 2024-03-29T04:09:57.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.427+0000 7f7aa767b640 1 -- 172.21.15.148:0/280640131 >> 172.21.15.148:0/280640131 conn(0x7f7aa00d2110 msgr2=0x7f7aa00d24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:57.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.427+0000 7f7aa767b640 1 -- 172.21.15.148:0/280640131 shutdown_connections 2024-03-29T04:09:57.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.427+0000 7f7aa767b640 1 -- 172.21.15.148:0/280640131 wait complete. 2024-03-29T04:09:57.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.428+0000 7f7aa767b640 1 Processor -- start 2024-03-29T04:09:57.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.428+0000 7f7aa767b640 1 -- start start 2024-03-29T04:09:57.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.429+0000 7f7aa767b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a8c0039e0 0x7f7aa0144f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.429+0000 7f7aa767b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7aa00b7270 0x7f7aa0156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.429+0000 7f7aa767b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa00b76d0 0x7f7aa00c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.429+0000 7f7aa767b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa00b1b30 con 0x7f7a8c0039e0 2024-03-29T04:09:57.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.429+0000 7f7aa767b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa006cdd0 con 0x7f7aa00b76d0 2024-03-29T04:09:57.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.429+0000 7f7aa767b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7aa00cfce0 con 0x7f7aa00b7270 2024-03-29T04:09:57.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.429+0000 7f7aa5417640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a8c0039e0 0x7f7aa0144f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.429+0000 7f7aa5417640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a8c0039e0 0x7f7aa0144f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47276/0 (socket says 172.21.15.148:47276) 2024-03-29T04:09:57.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.429+0000 7f7aa5417640 1 -- 172.21.15.148:0/2300984503 learned_addr learned my addr 172.21.15.148:0/2300984503 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:09:57.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.429+0000 7f7aa5c18640 1 --2- 172.21.15.148:0/2300984503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa00b76d0 0x7f7aa00c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.429+0000 7f7aa5417640 1 -- 172.21.15.148:0/2300984503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7aa00b7270 msgr2=0x7f7aa0156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.429+0000 7f7aa4c16640 1 --2- 172.21.15.148:0/2300984503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7aa00b7270 0x7f7aa0156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.429+0000 7f7aa5417640 1 --2- 172.21.15.148:0/2300984503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7aa00b7270 0x7f7aa0156830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.429+0000 7f7aa5417640 1 -- 172.21.15.148:0/2300984503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa00b76d0 msgr2=0x7f7aa00c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.429+0000 7f7aa5417640 1 --2- 172.21.15.148:0/2300984503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa00b76d0 0x7f7aa00c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.429+0000 7f7aa5417640 1 -- 172.21.15.148:0/2300984503 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa0157280 con 0x7f7a8c0039e0 2024-03-29T04:09:57.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.429+0000 7f7aa5c18640 1 --2- 172.21.15.148:0/2300984503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa00b76d0 0x7f7aa00c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:57.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.430+0000 7f7aa5417640 1 --2- 172.21.15.148:0/2300984503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a8c0039e0 0x7f7aa0144f20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c001970 tx=0x7f7aa006d1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:57.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.430+0000 7f7a95ffb640 1 -- 172.21.15.148:0/2300984503 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c007d30 con 0x7f7a8c0039e0 2024-03-29T04:09:57.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.430+0000 7f7a95ffb640 1 -- 172.21.15.148:0/2300984503 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a9c004bf0 con 0x7f7a8c0039e0 2024-03-29T04:09:57.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.430+0000 7f7a95ffb640 1 -- 172.21.15.148:0/2300984503 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c007070 con 0x7f7a8c0039e0 2024-03-29T04:09:57.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.430+0000 7f7aa767b640 1 -- 172.21.15.148:0/2300984503 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa0161500 con 0x7f7a8c0039e0 2024-03-29T04:09:57.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.430+0000 7f7aa767b640 1 -- 172.21.15.148:0/2300984503 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa0161ac0 con 0x7f7a8c0039e0 2024-03-29T04:09:57.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.431+0000 7f7aa767b640 1 -- 172.21.15.148:0/2300984503 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aa00687e0 con 0x7f7a8c0039e0 2024-03-29T04:09:57.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.431+0000 7f7a95ffb640 1 -- 172.21.15.148:0/2300984503 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7a9c048020 con 0x7f7a8c0039e0 2024-03-29T04:09:57.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.432+0000 7f7a95ffb640 1 --2- 172.21.15.148:0/2300984503 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7a8406c560 0x7f7a8406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.432+0000 7f7a95ffb640 1 -- 172.21.15.148:0/2300984503 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f7a9c0ac600 con 0x7f7a8c0039e0 2024-03-29T04:09:57.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.432+0000 7f7aa4c16640 1 --2- 172.21.15.148:0/2300984503 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7a8406c560 0x7f7a8406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.432+0000 7f7aa4c16640 1 --2- 172.21.15.148:0/2300984503 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7a8406c560 0x7f7a8406ea20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009560 tx=0x7f7a90008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:57.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.436+0000 7f7a95ffb640 1 -- 172.21.15.148:0/2300984503 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7a9c07af70 con 0x7f7a8c0039e0 2024-03-29T04:09:57.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.613+0000 7f7aa767b640 1 -- 172.21.15.148:0/2300984503 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7aa00d2ff0 con 0x7f7a8c0039e0 2024-03-29T04:09:57.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.613+0000 7f7a95ffb640 1 -- 172.21.15.148:0/2300984503 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f7a9c07a870 con 0x7f7a8c0039e0 2024-03-29T04:09:57.594 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:09:57.594 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2024-03-29T04:09:57.594 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-29T04:09:57.594 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2024-03-29T04:09:57.594 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2024-03-29T04:09:57.594 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:09:57.594 INFO:teuthology.orchestra.run.smithi148.stdout:] 2024-03-29T04:09:57.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.615+0000 7f7aa767b640 1 -- 172.21.15.148:0/2300984503 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7a8406c560 msgr2=0x7f7a8406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.615+0000 7f7aa767b640 1 --2- 172.21.15.148:0/2300984503 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7a8406c560 0x7f7a8406ea20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009560 tx=0x7f7a90008040 comp rx=0 tx=0).stop 2024-03-29T04:09:57.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.615+0000 7f7aa767b640 1 -- 172.21.15.148:0/2300984503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a8c0039e0 msgr2=0x7f7aa0144f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.615+0000 7f7aa767b640 1 --2- 172.21.15.148:0/2300984503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a8c0039e0 0x7f7aa0144f20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c001970 tx=0x7f7aa006d1e0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.615+0000 7f7aa767b640 1 -- 172.21.15.148:0/2300984503 shutdown_connections 2024-03-29T04:09:57.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.615+0000 7f7aa767b640 1 --2- 172.21.15.148:0/2300984503 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7a8406c560 0x7f7a8406ea20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.615+0000 7f7aa767b640 1 --2- 172.21.15.148:0/2300984503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa00b76d0 0x7f7aa00c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.615+0000 7f7aa767b640 1 --2- 172.21.15.148:0/2300984503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7aa00b7270 0x7f7aa0156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.615+0000 7f7aa767b640 1 --2- 172.21.15.148:0/2300984503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a8c0039e0 0x7f7aa0144f20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.615+0000 7f7aa767b640 1 -- 172.21.15.148:0/2300984503 >> 172.21.15.148:0/2300984503 conn(0x7f7aa00d2110 msgr2=0x7f7aa015c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:57.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.615+0000 7f7aa767b640 1 -- 172.21.15.148:0/2300984503 shutdown_connections 2024-03-29T04:09:57.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.615+0000 7f7aa767b640 1 -- 172.21.15.148:0/2300984503 wait complete. 2024-03-29T04:09:57.606 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-29T04:09:57.606 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-29T04:09:57.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.730+0000 7fb8995f4640 1 Processor -- start 2024-03-29T04:09:57.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.730+0000 7fb8995f4640 1 -- start start 2024-03-29T04:09:57.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.731+0000 7fb8995f4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb8940ce9a0 0x7fb89406e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.731+0000 7fb8995f4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb89415ed70 0x7fb89415f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.731+0000 7fb8995f4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8940b2e50 con 0x7fb8940ced80 2024-03-29T04:09:57.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.731+0000 7fb8995f4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fb89406e730 con 0x7fb8940ce9a0 2024-03-29T04:09:57.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.731+0000 7fb8995f4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb8940cfcc0 con 0x7fb89415ed70 2024-03-29T04:09:57.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.731+0000 7fb892ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb8940ce9a0 0x7fb89406e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.731+0000 7fb8927fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb8940ced80 0x7fb8940b1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51760/0 (socket says 172.21.15.148:51760) 2024-03-29T04:09:57.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.731+0000 7fb8927fc640 1 -- 172.21.15.148:0/4117011826 learned_addr learned my addr 172.21.15.148:0/4117011826 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:09:57.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.732+0000 7fb8937fe640 1 --2- 172.21.15.148:0/4117011826 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb89415ed70 0x7fb89415f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.732+0000 7fb892ffd640 1 -- 172.21.15.148:0/4117011826 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb89415ed70 msgr2=0x7fb89415f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:57.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.732+0000 7fb892ffd640 1 --2- 172.21.15.148:0/4117011826 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb89415ed70 0x7fb89415f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.732+0000 7fb892ffd640 1 -- 172.21.15.148:0/4117011826 >> v1:172.21.15.148:6789/0 conn(0x7fb8940ced80 legacy=0x7fb8940b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.732+0000 7fb891ffb640 1 -- 172.21.15.148:0/4117011826 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 57598733 0 0) 0x7fb8940b2e50 con 0x7fb8940ced80 2024-03-29T04:09:57.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.732+0000 7fb892ffd640 1 -- 172.21.15.148:0/4117011826 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8940d0520 con 0x7fb8940ce9a0 2024-03-29T04:09:57.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.732+0000 7fb892ffd640 1 --2- 172.21.15.148:0/4117011826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8940ce9a0 0x7fb89406e1f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb884009d90 tx=0x7fb88402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6333214cd6d3739c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:57.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.732+0000 7fb891ffb640 1 -- 172.21.15.148:0/4117011826 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb884039070 con 0x7fb8940ce9a0 2024-03-29T04:09:57.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.732+0000 7fb891ffb640 1 -- 172.21.15.148:0/4117011826 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb88402cc50 con 0x7fb8940ce9a0 2024-03-29T04:09:57.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.733+0000 7fb891ffb640 1 -- 172.21.15.148:0/4117011826 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb884034440 con 0x7fb8940ce9a0 2024-03-29T04:09:57.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.733+0000 7fb8995f4640 1 -- 172.21.15.148:0/4117011826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8940ce9a0 msgr2=0x7fb89406e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.733+0000 7fb8995f4640 1 --2- 172.21.15.148:0/4117011826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8940ce9a0 0x7fb89406e1f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb884009d90 tx=0x7fb88402bea0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.733+0000 7fb8995f4640 1 -- 172.21.15.148:0/4117011826 shutdown_connections 2024-03-29T04:09:57.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.733+0000 7fb8995f4640 1 --2- 172.21.15.148:0/4117011826 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb89415ed70 0x7fb89415f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.733+0000 7fb8995f4640 1 --2- 172.21.15.148:0/4117011826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8940ce9a0 0x7fb89406e1f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.733+0000 7fb8995f4640 1 -- 172.21.15.148:0/4117011826 >> 172.21.15.148:0/4117011826 conn(0x7fb894071b00 msgr2=0x7fb8940b3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:57.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.733+0000 7fb8995f4640 1 -- 172.21.15.148:0/4117011826 shutdown_connections 2024-03-29T04:09:57.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.733+0000 7fb8995f4640 1 -- 172.21.15.148:0/4117011826 wait complete. 2024-03-29T04:09:57.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.734+0000 7fb8995f4640 1 Processor -- start 2024-03-29T04:09:57.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.734+0000 7fb8995f4640 1 -- start start 2024-03-29T04:09:57.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.735+0000 7fb8995f4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8940ce9a0 0x7fb8941549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.735+0000 7fb8995f4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8940ced80 0x7fb89415a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.735+0000 7fb8995f4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb89415ed70 0x7fb89415aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.735+0000 7fb8995f4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb8940d0240 con 0x7fb89415ed70 2024-03-29T04:09:57.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.735+0000 7fb8995f4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb8940cff60 con 0x7fb8940ce9a0 2024-03-29T04:09:57.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.735+0000 7fb8995f4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb8940d1cc0 con 0x7fb8940ced80 2024-03-29T04:09:57.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.735+0000 7fb8937fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb89415ed70 0x7fb89415aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.735+0000 7fb8937fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb89415ed70 0x7fb89415aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47290/0 (socket says 172.21.15.148:47290) 2024-03-29T04:09:57.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.735+0000 7fb8937fe640 1 -- 172.21.15.148:0/4243974923 learned_addr learned my addr 172.21.15.148:0/4243974923 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:09:57.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.735+0000 7fb8927fc640 1 --2- 172.21.15.148:0/4243974923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8940ced80 0x7fb89415a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.735+0000 7fb892ffd640 1 --2- 172.21.15.148:0/4243974923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8940ce9a0 0x7fb8941549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.735+0000 7fb8937fe640 1 -- 172.21.15.148:0/4243974923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8940ced80 msgr2=0x7fb89415a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.735+0000 7fb8937fe640 1 --2- 172.21.15.148:0/4243974923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8940ced80 0x7fb89415a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.735+0000 7fb8937fe640 1 -- 172.21.15.148:0/4243974923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8940ce9a0 msgr2=0x7fb8941549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.735+0000 7fb8937fe640 1 --2- 172.21.15.148:0/4243974923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8940ce9a0 0x7fb8941549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.735+0000 7fb8937fe640 1 -- 172.21.15.148:0/4243974923 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8940cc0d0 con 0x7fb89415ed70 2024-03-29T04:09:57.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.735+0000 7fb892ffd640 1 --2- 172.21.15.148:0/4243974923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8940ce9a0 0x7fb8941549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:57.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.736+0000 7fb8937fe640 1 --2- 172.21.15.148:0/4243974923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb89415ed70 0x7fb89415aaa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00d800 tx=0x7fb88c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:57.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.736+0000 7fb87b7fe640 1 -- 172.21.15.148:0/4243974923 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88c00b6c0 con 0x7fb89415ed70 2024-03-29T04:09:57.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.736+0000 7fb87b7fe640 1 -- 172.21.15.148:0/4243974923 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb88c019070 con 0x7fb89415ed70 2024-03-29T04:09:57.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.736+0000 7fb8995f4640 1 -- 172.21.15.148:0/4243974923 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8940cc3b0 con 0x7fb89415ed70 2024-03-29T04:09:57.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.736+0000 7fb87b7fe640 1 -- 172.21.15.148:0/4243974923 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88c00bac0 con 0x7fb89415ed70 2024-03-29T04:09:57.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.736+0000 7fb8995f4640 1 -- 172.21.15.148:0/4243974923 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8941f5090 con 0x7fb89415ed70 2024-03-29T04:09:57.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.737+0000 7fb8995f4640 1 -- 172.21.15.148:0/4243974923 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb894068740 con 0x7fb89415ed70 2024-03-29T04:09:57.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.737+0000 7fb87b7fe640 1 -- 172.21.15.148:0/4243974923 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb88c00bc60 con 0x7fb89415ed70 2024-03-29T04:09:57.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.737+0000 7fb87b7fe640 1 --2- 172.21.15.148:0/4243974923 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb88006c530 0x7fb88006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.738+0000 7fb892ffd640 1 --2- 172.21.15.148:0/4243974923 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb88006c530 0x7fb88006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.738+0000 7fb87b7fe640 1 -- 172.21.15.148:0/4243974923 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fb88c08b6b0 con 0x7fb89415ed70 2024-03-29T04:09:57.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.738+0000 7fb892ffd640 1 --2- 172.21.15.148:0/4243974923 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb88006c530 0x7fb88006e9f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb884036000 tx=0x7fb884030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:57.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.742+0000 7fb87b7fe640 1 -- 172.21.15.148:0/4243974923 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb88c05a0b0 con 0x7fb89415ed70 2024-03-29T04:09:57.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:57.917+0000 7fb8995f4640 1 -- 172.21.15.148:0/4243974923 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb8940b32c0 con 0x7fb89415ed70 2024-03-29T04:09:58.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.512+0000 7fb87b7fe640 1 -- 172.21.15.148:0/4243974923 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7fb88c0599b0 con 0x7fb89415ed70 2024-03-29T04:09:58.492 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2024-03-29T04:09:58.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.515+0000 7fb8995f4640 1 -- 172.21.15.148:0/4243974923 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb88006c530 msgr2=0x7fb88006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:58.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.515+0000 7fb8995f4640 1 --2- 172.21.15.148:0/4243974923 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb88006c530 0x7fb88006e9f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb884036000 tx=0x7fb884030000 comp rx=0 tx=0).stop 2024-03-29T04:09:58.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.515+0000 7fb8995f4640 1 -- 172.21.15.148:0/4243974923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb89415ed70 msgr2=0x7fb89415aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:58.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.515+0000 7fb8995f4640 1 --2- 172.21.15.148:0/4243974923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb89415ed70 0x7fb89415aaa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00d800 tx=0x7fb88c00dcc0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.515+0000 7fb8995f4640 1 -- 172.21.15.148:0/4243974923 shutdown_connections 2024-03-29T04:09:58.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.515+0000 7fb8995f4640 1 --2- 172.21.15.148:0/4243974923 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb88006c530 0x7fb88006e9f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.515+0000 7fb8995f4640 1 --2- 172.21.15.148:0/4243974923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb89415ed70 0x7fb89415aaa0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.515+0000 7fb8995f4640 1 --2- 172.21.15.148:0/4243974923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8940ced80 0x7fb89415a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.515+0000 7fb8995f4640 1 --2- 172.21.15.148:0/4243974923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8940ce9a0 0x7fb8941549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.515+0000 7fb8995f4640 1 -- 172.21.15.148:0/4243974923 >> 172.21.15.148:0/4243974923 conn(0x7fb894071b00 msgr2=0x7fb89415c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:58.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.515+0000 7fb8995f4640 1 -- 172.21.15.148:0/4243974923 shutdown_connections 2024-03-29T04:09:58.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.515+0000 7fb8995f4640 1 -- 172.21.15.148:0/4243974923 wait complete. 2024-03-29T04:09:58.510 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-29T04:09:58.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.638+0000 7f519efc3640 1 Processor -- start 2024-03-29T04:09:58.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.638+0000 7f519efc3640 1 -- start start 2024-03-29T04:09:58.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.639+0000 7f519efc3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f519815e760 0x7f51980d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:58.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.639+0000 7f519efc3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51980d2090 0x7f51980cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:58.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.639+0000 7f519efc3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51980d1110 con 0x7f51980d4240 2024-03-29T04:09:58.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.639+0000 7f519efc3640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f519815eb40 con 0x7f519815e760 2024-03-29T04:09:58.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.639+0000 7f519efc3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f51980d13f0 con 0x7f51980d2090 2024-03-29T04:09:58.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.639+0000 7f519cd5f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f519815e760 0x7f51980d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:58.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.639+0000 7f519d560640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f51980d4240 0x7f5198162200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51764/0 (socket says 172.21.15.148:51764) 2024-03-29T04:09:58.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.639+0000 7f519d560640 1 -- 172.21.15.148:0/512710589 learned_addr learned my addr 172.21.15.148:0/512710589 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:09:58.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.639+0000 7f5197fff640 1 --2- 172.21.15.148:0/512710589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51980d2090 0x7f51980cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:58.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.639+0000 7f519cd5f640 1 -- 172.21.15.148:0/512710589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51980d2090 msgr2=0x7f51980cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:58.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.639+0000 7f519cd5f640 1 --2- 172.21.15.148:0/512710589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51980d2090 0x7f51980cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.639+0000 7f519cd5f640 1 -- 172.21.15.148:0/512710589 >> v1:172.21.15.148:6789/0 conn(0x7f51980d4240 legacy=0x7f5198162200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:58.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.639+0000 7f519cd5f640 1 -- 172.21.15.148:0/512710589 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51980d16d0 con 0x7f519815e760 2024-03-29T04:09:58.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.639+0000 7f51977fe640 1 -- 172.21.15.148:0/512710589 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 940283415 0 0) 0x7f51980d1110 con 0x7f51980d4240 2024-03-29T04:09:58.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.640+0000 7f519cd5f640 1 --2- 172.21.15.148:0/512710589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f519815e760 0x7f51980d0b30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f518c0097a0 tx=0x7f518c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5588cf53bbe2d44 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:58.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.640+0000 7f51977fe640 1 -- 172.21.15.148:0/512710589 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c034070 con 0x7f519815e760 2024-03-29T04:09:58.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.640+0000 7f51977fe640 1 -- 172.21.15.148:0/512710589 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f518c030610 con 0x7f519815e760 2024-03-29T04:09:58.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.640+0000 7f51977fe640 1 -- 172.21.15.148:0/512710589 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c0309e0 con 0x7f519815e760 2024-03-29T04:09:58.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.640+0000 7f519efc3640 1 -- 172.21.15.148:0/512710589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f519815e760 msgr2=0x7f51980d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:58.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.640+0000 7f519efc3640 1 --2- 172.21.15.148:0/512710589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f519815e760 0x7f51980d0b30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f518c0097a0 tx=0x7f518c02b960 comp rx=0 tx=0).stop 2024-03-29T04:09:58.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.640+0000 7f519efc3640 1 -- 172.21.15.148:0/512710589 shutdown_connections 2024-03-29T04:09:58.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.640+0000 7f519efc3640 1 --2- 172.21.15.148:0/512710589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51980d2090 0x7f51980cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.640+0000 7f519efc3640 1 --2- 172.21.15.148:0/512710589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f519815e760 0x7f51980d0b30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.641+0000 7f519efc3640 1 -- 172.21.15.148:0/512710589 >> 172.21.15.148:0/512710589 conn(0x7f51980b7200 msgr2=0x7f51980b7610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:58.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.641+0000 7f519efc3640 1 -- 172.21.15.148:0/512710589 shutdown_connections 2024-03-29T04:09:58.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.641+0000 7f519efc3640 1 -- 172.21.15.148:0/512710589 wait complete. 2024-03-29T04:09:58.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.642+0000 7f519efc3640 1 Processor -- start 2024-03-29T04:09:58.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.642+0000 7f519efc3640 1 -- start start 2024-03-29T04:09:58.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.642+0000 7f519efc3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51980d2090 0x7f519815d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:58.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.642+0000 7f519efc3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51980d4240 0x7f51980c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:58.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.643+0000 7f519efc3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f519815e760 0x7f519806d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:58.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.643+0000 7f519efc3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f51980d46d0 con 0x7f519815e760 2024-03-29T04:09:58.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.643+0000 7f519efc3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f51980d3350 con 0x7f51980d2090 2024-03-29T04:09:58.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.643+0000 7f519efc3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f51980d4850 con 0x7f51980d4240 2024-03-29T04:09:58.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.643+0000 7f519d560640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f519815e760 0x7f519806d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:58.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.643+0000 7f519d560640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f519815e760 0x7f519806d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47306/0 (socket says 172.21.15.148:47306) 2024-03-29T04:09:58.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.643+0000 7f519d560640 1 -- 172.21.15.148:0/3146507186 learned_addr learned my addr 172.21.15.148:0/3146507186 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:09:58.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.643+0000 7f5197fff640 1 --2- 172.21.15.148:0/3146507186 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51980d4240 0x7f51980c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:58.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.643+0000 7f519cd5f640 1 --2- 172.21.15.148:0/3146507186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51980d2090 0x7f519815d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:58.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.643+0000 7f519d560640 1 -- 172.21.15.148:0/3146507186 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51980d4240 msgr2=0x7f51980c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:58.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.643+0000 7f519d560640 1 --2- 172.21.15.148:0/3146507186 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51980d4240 0x7f51980c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.643+0000 7f519d560640 1 -- 172.21.15.148:0/3146507186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51980d2090 msgr2=0x7f519815d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:58.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.643+0000 7f519d560640 1 --2- 172.21.15.148:0/3146507186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51980d2090 0x7f519815d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.643+0000 7f519d560640 1 -- 172.21.15.148:0/3146507186 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51980c84c0 con 0x7f519815e760 2024-03-29T04:09:58.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.643+0000 7f519d560640 1 --2- 172.21.15.148:0/3146507186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f519815e760 0x7f519806d830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f519000c7a0 tx=0x7f519000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:58.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.643+0000 7f51957fa640 1 -- 172.21.15.148:0/3146507186 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190018070 con 0x7f519815e760 2024-03-29T04:09:58.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.643+0000 7f51957fa640 1 -- 172.21.15.148:0/3146507186 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f519000a640 con 0x7f519815e760 2024-03-29T04:09:58.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.643+0000 7f51957fa640 1 -- 172.21.15.148:0/3146507186 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519000aab0 con 0x7f519815e760 2024-03-29T04:09:58.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.643+0000 7f519efc3640 1 -- 172.21.15.148:0/3146507186 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51980c87a0 con 0x7f519815e760 2024-03-29T04:09:58.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.643+0000 7f519efc3640 1 -- 172.21.15.148:0/3146507186 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51980c8d60 con 0x7f519815e760 2024-03-29T04:09:58.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.644+0000 7f519efc3640 1 -- 172.21.15.148:0/3146507186 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5198068850 con 0x7f519815e760 2024-03-29T04:09:58.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.645+0000 7f51957fa640 1 -- 172.21.15.148:0/3146507186 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f519001b070 con 0x7f519815e760 2024-03-29T04:09:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.645+0000 7f51957fa640 1 --2- 172.21.15.148:0/3146507186 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f517c06c530 0x7f517c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.645+0000 7f51957fa640 1 -- 172.21.15.148:0/3146507186 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f519008aa90 con 0x7f519815e760 2024-03-29T04:09:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.645+0000 7f519cd5f640 1 --2- 172.21.15.148:0/3146507186 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f517c06c530 0x7f517c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.646+0000 7f519cd5f640 1 --2- 172.21.15.148:0/3146507186 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f517c06c530 0x7f517c06e9f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f518c036000 tx=0x7f518c03b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:58.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.650+0000 7f51957fa640 1 -- 172.21.15.148:0/3146507186 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5190058c40 con 0x7f519815e760 2024-03-29T04:09:58.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:58.824+0000 7f519efc3640 1 -- 172.21.15.148:0/3146507186 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f51980b4c90 con 0x7f519815e760 2024-03-29T04:09:59.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.520+0000 7f51957fa640 1 -- 172.21.15.148:0/3146507186 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) ==== 171+0+0 (secure 0 0 0) 0x7f519005cf40 con 0x7f519815e760 2024-03-29T04:09:59.500 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2024-03-29T04:09:59.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.525+0000 7f519efc3640 1 -- 172.21.15.148:0/3146507186 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f517c06c530 msgr2=0x7f517c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:59.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.525+0000 7f519efc3640 1 --2- 172.21.15.148:0/3146507186 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f517c06c530 0x7f517c06e9f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f518c036000 tx=0x7f518c03b040 comp rx=0 tx=0).stop 2024-03-29T04:09:59.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.525+0000 7f519efc3640 1 -- 172.21.15.148:0/3146507186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f519815e760 msgr2=0x7f519806d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:59.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.525+0000 7f519efc3640 1 --2- 172.21.15.148:0/3146507186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f519815e760 0x7f519806d830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f519000c7a0 tx=0x7f519000cc60 comp rx=0 tx=0).stop 2024-03-29T04:09:59.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.525+0000 7f519efc3640 1 -- 172.21.15.148:0/3146507186 shutdown_connections 2024-03-29T04:09:59.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.525+0000 7f519efc3640 1 --2- 172.21.15.148:0/3146507186 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f517c06c530 0x7f517c06e9f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:59.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.525+0000 7f519efc3640 1 --2- 172.21.15.148:0/3146507186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f519815e760 0x7f519806d830 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:59.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.525+0000 7f519efc3640 1 --2- 172.21.15.148:0/3146507186 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51980d4240 0x7f51980c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:59.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.525+0000 7f519efc3640 1 --2- 172.21.15.148:0/3146507186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51980d2090 0x7f519815d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:59.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.525+0000 7f519efc3640 1 -- 172.21.15.148:0/3146507186 >> 172.21.15.148:0/3146507186 conn(0x7f51980b7200 msgr2=0x7f51980b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:59.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.526+0000 7f519efc3640 1 -- 172.21.15.148:0/3146507186 shutdown_connections 2024-03-29T04:09:59.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.527+0000 7f519efc3640 1 -- 172.21.15.148:0/3146507186 wait complete. 2024-03-29T04:09:59.531 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-29T04:09:59.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.659+0000 7f49de1d4640 1 Processor -- start 2024-03-29T04:09:59.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.659+0000 7f49de1d4640 1 -- start start 2024-03-29T04:09:59.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.660+0000 7f49de1d4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f49d815f200 0x7f49d815f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:59.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.660+0000 7f49de1d4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49d80cfdd0 con 0x7f49d80ccec0 2024-03-29T04:09:59.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.660+0000 7f49de1d4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49d80d00b0 con 0x7f49d80b4fb0 2024-03-29T04:09:59.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.660+0000 7f49de1d4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f49d80d09e0 con 0x7f49d815f200 2024-03-29T04:09:59.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.660+0000 7f49d77fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f49d815f200 0x7f49d815f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:59.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.660+0000 7f49d7fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f49d80ccec0 0x7f49d80b6b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51768/0 (socket says 172.21.15.148:51768) 2024-03-29T04:09:59.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.660+0000 7f49d77fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f49d815f200 0x7f49d815f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47322/0 (socket says 172.21.15.148:47322) 2024-03-29T04:09:59.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.660+0000 7f49d7fff640 1 -- 172.21.15.148:0/2573933248 learned_addr learned my addr 172.21.15.148:0/2573933248 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:09:59.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.660+0000 7f49d77fe640 1 -- 172.21.15.148:0/2573933248 >> v1:172.21.15.157:6789/0 conn(0x7f49d80b4fb0 legacy=0x7f49d80cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:59.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.660+0000 7f49d77fe640 1 -- 172.21.15.148:0/2573933248 >> v1:172.21.15.148:6789/0 conn(0x7f49d80ccec0 legacy=0x7f49d80b6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:59.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.660+0000 7f49d77fe640 1 -- 172.21.15.148:0/2573933248 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d80d0390 con 0x7f49d815f200 2024-03-29T04:09:59.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.660+0000 7f49d67fc640 1 -- 172.21.15.148:0/2573933248 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1490041445 0 0) 0x7f49d80cfdd0 con 0x7f49d80ccec0 2024-03-29T04:09:59.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.661+0000 7f49d77fe640 1 --2- 172.21.15.148:0/2573933248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49d815f200 0x7f49d815f5e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f49c80097a0 tx=0x7f49c802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4671dabc9fd8fed8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:59.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.661+0000 7f49d67fc640 1 -- 172.21.15.148:0/2573933248 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c8035020 con 0x7f49d815f200 2024-03-29T04:09:59.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.661+0000 7f49d67fc640 1 -- 172.21.15.148:0/2573933248 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49c8031830 con 0x7f49d815f200 2024-03-29T04:09:59.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.661+0000 7f49d67fc640 1 -- 172.21.15.148:0/2573933248 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c8031c00 con 0x7f49d815f200 2024-03-29T04:09:59.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.661+0000 7f49de1d4640 1 -- 172.21.15.148:0/2573933248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49d815f200 msgr2=0x7f49d815f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:59.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.661+0000 7f49de1d4640 1 --2- 172.21.15.148:0/2573933248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49d815f200 0x7f49d815f5e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f49c80097a0 tx=0x7f49c802b8d0 comp rx=0 tx=0).stop 2024-03-29T04:09:59.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.661+0000 7f49de1d4640 1 -- 172.21.15.148:0/2573933248 shutdown_connections 2024-03-29T04:09:59.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.661+0000 7f49de1d4640 1 --2- 172.21.15.148:0/2573933248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49d815f200 0x7f49d815f5e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:59.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.661+0000 7f49de1d4640 1 -- 172.21.15.148:0/2573933248 >> 172.21.15.148:0/2573933248 conn(0x7f49d80b3c70 msgr2=0x7f49d80b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:59.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.661+0000 7f49de1d4640 1 -- 172.21.15.148:0/2573933248 shutdown_connections 2024-03-29T04:09:59.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.661+0000 7f49de1d4640 1 -- 172.21.15.148:0/2573933248 wait complete. 2024-03-29T04:09:59.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.663+0000 7f49de1d4640 1 Processor -- start 2024-03-29T04:09:59.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.663+0000 7f49de1d4640 1 -- start start 2024-03-29T04:09:59.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.663+0000 7f49de1d4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49d80b4fb0 0x7f49d80c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:59.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.663+0000 7f49de1d4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49d80ccec0 0x7f49d8147420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:59.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.663+0000 7f49de1d4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49d815f200 0x7f49d806fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:59.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.663+0000 7f49de1d4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f49d80b55b0 con 0x7f49d815f200 2024-03-29T04:09:59.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.663+0000 7f49de1d4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f49d80b5430 con 0x7f49d80ccec0 2024-03-29T04:09:59.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.663+0000 7f49de1d4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f49d80b5730 con 0x7f49d80b4fb0 2024-03-29T04:09:59.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.664+0000 7f49d7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49d815f200 0x7f49d806fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:59.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.664+0000 7f49d7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49d815f200 0x7f49d806fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47328/0 (socket says 172.21.15.148:47328) 2024-03-29T04:09:59.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.664+0000 7f49d7fff640 1 -- 172.21.15.148:0/1352735240 learned_addr learned my addr 172.21.15.148:0/1352735240 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:09:59.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.664+0000 7f49d77fe640 1 --2- 172.21.15.148:0/1352735240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49d80b4fb0 0x7f49d80c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:59.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.664+0000 7f49d6ffd640 1 --2- 172.21.15.148:0/1352735240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49d80ccec0 0x7f49d8147420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:59.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.664+0000 7f49d7fff640 1 -- 172.21.15.148:0/1352735240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49d80b4fb0 msgr2=0x7f49d80c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:59.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.664+0000 7f49d7fff640 1 --2- 172.21.15.148:0/1352735240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49d80b4fb0 0x7f49d80c8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:59.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.664+0000 7f49d7fff640 1 -- 172.21.15.148:0/1352735240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49d80ccec0 msgr2=0x7f49d8147420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:59.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.664+0000 7f49d7fff640 1 --2- 172.21.15.148:0/1352735240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49d80ccec0 0x7f49d8147420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:59.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.664+0000 7f49d7fff640 1 -- 172.21.15.148:0/1352735240 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d806c920 con 0x7f49d815f200 2024-03-29T04:09:59.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.664+0000 7f49d6ffd640 1 --2- 172.21.15.148:0/1352735240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49d80ccec0 0x7f49d8147420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:59.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.664+0000 7f49d7fff640 1 --2- 172.21.15.148:0/1352735240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49d815f200 0x7f49d806fd60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f49d000c730 tx=0x7f49d000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:59.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.664+0000 7f49bffff640 1 -- 172.21.15.148:0/1352735240 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0018070 con 0x7f49d815f200 2024-03-29T04:09:59.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.664+0000 7f49bffff640 1 -- 172.21.15.148:0/1352735240 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d000ce40 con 0x7f49d815f200 2024-03-29T04:09:59.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.664+0000 7f49de1d4640 1 -- 172.21.15.148:0/1352735240 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d806ecc0 con 0x7f49d815f200 2024-03-29T04:09:59.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.664+0000 7f49bffff640 1 -- 172.21.15.148:0/1352735240 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d000a800 con 0x7f49d815f200 2024-03-29T04:09:59.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.664+0000 7f49de1d4640 1 -- 172.21.15.148:0/1352735240 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d806f280 con 0x7f49d815f200 2024-03-29T04:09:59.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.665+0000 7f49de1d4640 1 -- 172.21.15.148:0/1352735240 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d806fa40 con 0x7f49d815f200 2024-03-29T04:09:59.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.666+0000 7f49bffff640 1 -- 172.21.15.148:0/1352735240 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f49d001b070 con 0x7f49d815f200 2024-03-29T04:09:59.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.666+0000 7f49bffff640 1 --2- 172.21.15.148:0/1352735240 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f49c406c530 0x7f49c406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:59.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.666+0000 7f49bffff640 1 -- 172.21.15.148:0/1352735240 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f49d008ab70 con 0x7f49d815f200 2024-03-29T04:09:59.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.666+0000 7f49d77fe640 1 --2- 172.21.15.148:0/1352735240 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f49c406c530 0x7f49c406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:59.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.667+0000 7f49d77fe640 1 --2- 172.21.15.148:0/1352735240 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f49c406c530 0x7f49c406e9f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f49c8036000 tx=0x7f49c803c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:59.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.671+0000 7f49bffff640 1 -- 172.21.15.148:0/1352735240 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f49d806fa40 con 0x7f49d815f200 2024-03-29T04:09:59.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:09:59.884+0000 7f49de1d4640 1 -- 172.21.15.148:0/1352735240 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f49d80d1630 con 0x7f49d815f200 2024-03-29T04:10:00.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.536+0000 7f49bffff640 1 -- 172.21.15.148:0/1352735240 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f49d0058ba0 con 0x7f49d815f200 2024-03-29T04:10:00.517 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 2 and data pool 3 2024-03-29T04:10:00.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.547+0000 7f49de1d4640 1 -- 172.21.15.148:0/1352735240 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f49c406c530 msgr2=0x7f49c406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:00.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.547+0000 7f49de1d4640 1 --2- 172.21.15.148:0/1352735240 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f49c406c530 0x7f49c406e9f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f49c8036000 tx=0x7f49c803c040 comp rx=0 tx=0).stop 2024-03-29T04:10:00.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.547+0000 7f49de1d4640 1 -- 172.21.15.148:0/1352735240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49d815f200 msgr2=0x7f49d806fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:00.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.547+0000 7f49de1d4640 1 --2- 172.21.15.148:0/1352735240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49d815f200 0x7f49d806fd60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f49d000c730 tx=0x7f49d000cbf0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.547+0000 7f49de1d4640 1 -- 172.21.15.148:0/1352735240 shutdown_connections 2024-03-29T04:10:00.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.547+0000 7f49de1d4640 1 --2- 172.21.15.148:0/1352735240 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f49c406c530 0x7f49c406e9f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.547+0000 7f49de1d4640 1 --2- 172.21.15.148:0/1352735240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49d815f200 0x7f49d806fd60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.547+0000 7f49de1d4640 1 --2- 172.21.15.148:0/1352735240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49d80ccec0 0x7f49d8147420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.547+0000 7f49de1d4640 1 --2- 172.21.15.148:0/1352735240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f49d80b4fb0 0x7f49d80c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.547+0000 7f49de1d4640 1 -- 172.21.15.148:0/1352735240 >> 172.21.15.148:0/1352735240 conn(0x7f49d80b3c70 msgr2=0x7f49d80d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:00.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.550+0000 7f49de1d4640 1 -- 172.21.15.148:0/1352735240 shutdown_connections 2024-03-29T04:10:00.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.551+0000 7f49de1d4640 1 -- 172.21.15.148:0/1352735240 wait complete. 2024-03-29T04:10:00.548 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-29T04:10:00.548 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-29T04:10:00.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.680+0000 7f043fe4a640 1 Processor -- start 2024-03-29T04:10:00.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.681+0000 7f043fe4a640 1 -- start start 2024-03-29T04:10:00.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.681+0000 7f043fe4a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f043815e220 0x7f04380cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:00.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.682+0000 7f043fe4a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04380cf0d0 0x7f04380b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:00.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.682+0000 7f043dbe6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f043815e220 0x7f04380cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:00.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.682+0000 7f043d3e5640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f043806e1f0 0x7f04380ce9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51776/0 (socket says 172.21.15.148:51776) 2024-03-29T04:10:00.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.682+0000 7f043d3e5640 1 -- 172.21.15.148:0/787504675 learned_addr learned my addr 172.21.15.148:0/787504675 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:00.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.682+0000 7f043fe4a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0438162920 con 0x7f043806e1f0 2024-03-29T04:10:00.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.682+0000 7f043fe4a640 1 -- 172.21.15.148:0/787504675 --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f04380d04b0 con 0x7f043815e220 2024-03-29T04:10:00.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.682+0000 7f043fe4a640 1 -- 172.21.15.148:0/787504675 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0438162c00 con 0x7f04380cf0d0 2024-03-29T04:10:00.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.682+0000 7f043dbe6640 1 -- 172.21.15.148:0/787504675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04380cf0d0 msgr2=0x7f04380b1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:00.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.682+0000 7f043e3e7640 1 --2- 172.21.15.148:0/787504675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04380cf0d0 0x7f04380b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:00.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.682+0000 7f043dbe6640 1 --2- 172.21.15.148:0/787504675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04380cf0d0 0x7f04380b1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.682+0000 7f043cbe4640 1 -- 172.21.15.148:0/787504675 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3879102609 0 0) 0x7f0438162920 con 0x7f043806e1f0 2024-03-29T04:10:00.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.682+0000 7f043dbe6640 1 -- 172.21.15.148:0/787504675 >> v1:172.21.15.148:6789/0 conn(0x7f043806e1f0 legacy=0x7f04380ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:00.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.682+0000 7f043dbe6640 1 -- 172.21.15.148:0/787504675 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04380cfe60 con 0x7f043815e220 2024-03-29T04:10:00.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.682+0000 7f043e3e7640 1 --2- 172.21.15.148:0/787504675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04380cf0d0 0x7f04380b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:00.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.683+0000 7f043dbe6640 1 --2- 172.21.15.148:0/787504675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043815e220 0x7f04380cc760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f04340097a0 tx=0x7f043402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c736baca98dc83 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:00.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.683+0000 7f043cbe4640 1 -- 172.21.15.148:0/787504675 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04340315a0 con 0x7f043815e220 2024-03-29T04:10:00.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.683+0000 7f043cbe4640 1 -- 172.21.15.148:0/787504675 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0434037070 con 0x7f043815e220 2024-03-29T04:10:00.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.683+0000 7f043cbe4640 1 -- 172.21.15.148:0/787504675 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0434031960 con 0x7f043815e220 2024-03-29T04:10:00.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.683+0000 7f043fe4a640 1 -- 172.21.15.148:0/787504675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043815e220 msgr2=0x7f04380cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:00.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.683+0000 7f043fe4a640 1 --2- 172.21.15.148:0/787504675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043815e220 0x7f04380cc760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f04340097a0 tx=0x7f043402b840 comp rx=0 tx=0).stop 2024-03-29T04:10:00.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.683+0000 7f043fe4a640 1 -- 172.21.15.148:0/787504675 shutdown_connections 2024-03-29T04:10:00.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.683+0000 7f043fe4a640 1 --2- 172.21.15.148:0/787504675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04380cf0d0 0x7f04380b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.683+0000 7f043fe4a640 1 --2- 172.21.15.148:0/787504675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043815e220 0x7f04380cc760 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.683+0000 7f043fe4a640 1 -- 172.21.15.148:0/787504675 >> 172.21.15.148:0/787504675 conn(0x7f04380b3ce0 msgr2=0x7f04380b40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:00.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.683+0000 7f043fe4a640 1 -- 172.21.15.148:0/787504675 shutdown_connections 2024-03-29T04:10:00.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.683+0000 7f043fe4a640 1 -- 172.21.15.148:0/787504675 wait complete. 2024-03-29T04:10:00.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.685+0000 7f043fe4a640 1 Processor -- start 2024-03-29T04:10:00.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.685+0000 7f043fe4a640 1 -- start start 2024-03-29T04:10:00.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.685+0000 7f043fe4a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f043806e1f0 0x7f0438155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:00.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.685+0000 7f043fe4a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04380cf0d0 0x7f0438158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:00.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.685+0000 7f043fe4a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043815e220 0x7f04380ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:00.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.685+0000 7f043fe4a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f043815e600 con 0x7f043815e220 2024-03-29T04:10:00.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.685+0000 7f043fe4a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f043806e650 con 0x7f043806e1f0 2024-03-29T04:10:00.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.685+0000 7f043fe4a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f043815e780 con 0x7f04380cf0d0 2024-03-29T04:10:00.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.686+0000 7f043e3e7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043815e220 0x7f04380ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:00.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.686+0000 7f043e3e7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043815e220 0x7f04380ca8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47346/0 (socket says 172.21.15.148:47346) 2024-03-29T04:10:00.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.686+0000 7f043e3e7640 1 -- 172.21.15.148:0/4140057718 learned_addr learned my addr 172.21.15.148:0/4140057718 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:00.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.686+0000 7f043d3e5640 1 --2- 172.21.15.148:0/4140057718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04380cf0d0 0x7f0438158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:00.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.686+0000 7f043dbe6640 1 --2- 172.21.15.148:0/4140057718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f043806e1f0 0x7f0438155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:00.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.686+0000 7f043e3e7640 1 -- 172.21.15.148:0/4140057718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04380cf0d0 msgr2=0x7f0438158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:00.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.686+0000 7f043e3e7640 1 --2- 172.21.15.148:0/4140057718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04380cf0d0 0x7f0438158a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.686+0000 7f043e3e7640 1 -- 172.21.15.148:0/4140057718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f043806e1f0 msgr2=0x7f0438155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:00.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.686+0000 7f043e3e7640 1 --2- 172.21.15.148:0/4140057718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f043806e1f0 0x7f0438155390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.686+0000 7f043e3e7640 1 -- 172.21.15.148:0/4140057718 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04381593e0 con 0x7f043815e220 2024-03-29T04:10:00.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.686+0000 7f043e3e7640 1 --2- 172.21.15.148:0/4140057718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043815e220 0x7f04380ca8a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f043000cf20 tx=0x7f043000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:00.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.686+0000 7f042e7fc640 1 -- 172.21.15.148:0/4140057718 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043001a070 con 0x7f043815e220 2024-03-29T04:10:00.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.686+0000 7f042e7fc640 1 -- 172.21.15.148:0/4140057718 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f043000d840 con 0x7f043815e220 2024-03-29T04:10:00.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.686+0000 7f043fe4a640 1 -- 172.21.15.148:0/4140057718 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0438156c00 con 0x7f043815e220 2024-03-29T04:10:00.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.686+0000 7f042e7fc640 1 -- 172.21.15.148:0/4140057718 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043000dc30 con 0x7f043815e220 2024-03-29T04:10:00.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.686+0000 7f043fe4a640 1 -- 172.21.15.148:0/4140057718 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04381571c0 con 0x7f043815e220 2024-03-29T04:10:00.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.687+0000 7f043fe4a640 1 -- 172.21.15.148:0/4140057718 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0438068740 con 0x7f043815e220 2024-03-29T04:10:00.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.688+0000 7f042e7fc640 1 -- 172.21.15.148:0/4140057718 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f043001d070 con 0x7f043815e220 2024-03-29T04:10:00.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.688+0000 7f042e7fc640 1 --2- 172.21.15.148:0/4140057718 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f041c06c530 0x7f041c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:00.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.688+0000 7f043dbe6640 1 --2- 172.21.15.148:0/4140057718 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f041c06c530 0x7f041c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:00.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.688+0000 7f042e7fc640 1 -- 172.21.15.148:0/4140057718 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f043008bfa0 con 0x7f043815e220 2024-03-29T04:10:00.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.688+0000 7f043dbe6640 1 --2- 172.21.15.148:0/4140057718 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f041c06c530 0x7f041c06e9f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0434036000 tx=0x7f043403d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:00.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.693+0000 7f042e7fc640 1 -- 172.21.15.148:0/4140057718 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f043005a5c0 con 0x7f043815e220 2024-03-29T04:10:00.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:00.874+0000 7f043fe4a640 1 -- 172.21.15.148:0/4140057718 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f04380b3260 con 0x7f043815e220 2024-03-29T04:10:01.406 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:10:01.410 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:10:01.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.523+0000 7f042e7fc640 1 -- 172.21.15.148:0/4140057718 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) ==== 157+0+0 (secure 0 0 0) 0x7f0430059ec0 con 0x7f043815e220 2024-03-29T04:10:01.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.526+0000 7f043fe4a640 1 -- 172.21.15.148:0/4140057718 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f041c06c530 msgr2=0x7f041c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:01.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.526+0000 7f043fe4a640 1 --2- 172.21.15.148:0/4140057718 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f041c06c530 0x7f041c06e9f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0434036000 tx=0x7f043403d040 comp rx=0 tx=0).stop 2024-03-29T04:10:01.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.526+0000 7f043fe4a640 1 -- 172.21.15.148:0/4140057718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043815e220 msgr2=0x7f04380ca8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:01.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.526+0000 7f043fe4a640 1 --2- 172.21.15.148:0/4140057718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043815e220 0x7f04380ca8a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f043000cf20 tx=0x7f043000a7d0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.526+0000 7f043fe4a640 1 -- 172.21.15.148:0/4140057718 shutdown_connections 2024-03-29T04:10:01.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.526+0000 7f043fe4a640 1 --2- 172.21.15.148:0/4140057718 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f041c06c530 0x7f041c06e9f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.526+0000 7f043fe4a640 1 --2- 172.21.15.148:0/4140057718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043815e220 0x7f04380ca8a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.526+0000 7f043fe4a640 1 --2- 172.21.15.148:0/4140057718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04380cf0d0 0x7f0438158a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.526+0000 7f043fe4a640 1 --2- 172.21.15.148:0/4140057718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f043806e1f0 0x7f0438155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.526+0000 7f043fe4a640 1 -- 172.21.15.148:0/4140057718 >> 172.21.15.148:0/4140057718 conn(0x7f04380b3ce0 msgr2=0x7f043815d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:01.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.526+0000 7f043fe4a640 1 -- 172.21.15.148:0/4140057718 shutdown_connections 2024-03-29T04:10:01.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.526+0000 7f043fe4a640 1 -- 172.21.15.148:0/4140057718 wait complete. 2024-03-29T04:10:01.518 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-29T04:10:01.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.649+0000 7f33a4d00640 1 Processor -- start 2024-03-29T04:10:01.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.649+0000 7f33a4d00640 1 -- start start 2024-03-29T04:10:01.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.650+0000 7f33a4d00640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f33a00d2460 0x7f33a00d3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:01.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.650+0000 7f33a4d00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33a00cf060 0x7f33a00cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:01.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.650+0000 7f33a4d00640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33a00d2a40 con 0x7f33a015e4f0 2024-03-29T04:10:01.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.650+0000 7f33a4d00640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f33a015e8d0 con 0x7f33a00d2460 2024-03-29T04:10:01.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.650+0000 7f33a4d00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f33a00d2d20 con 0x7f33a00cf060 2024-03-29T04:10:01.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.650+0000 7f3397fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f33a00d2460 0x7f33a00d3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:01.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.650+0000 7f339e59c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f33a015e4f0 0x7f33a00b9d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51782/0 (socket says 172.21.15.148:51782) 2024-03-29T04:10:01.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.650+0000 7f339e59c640 1 -- 172.21.15.148:0/4182101140 learned_addr learned my addr 172.21.15.148:0/4182101140 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:01.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.650+0000 7f339ed9d640 1 --2- 172.21.15.148:0/4182101140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33a00cf060 0x7f33a00cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:01.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.650+0000 7f3397fff640 1 -- 172.21.15.148:0/4182101140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33a00cf060 msgr2=0x7f33a00cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:01.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.650+0000 7f3397fff640 1 --2- 172.21.15.148:0/4182101140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33a00cf060 0x7f33a00cf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.650+0000 7f3397fff640 1 -- 172.21.15.148:0/4182101140 >> v1:172.21.15.148:6789/0 conn(0x7f33a015e4f0 legacy=0x7f33a00b9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:01.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.650+0000 7f3397fff640 1 -- 172.21.15.148:0/4182101140 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a00d3000 con 0x7f33a00d2460 2024-03-29T04:10:01.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.651+0000 7f3397fff640 1 --2- 172.21.15.148:0/4182101140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33a00d2460 0x7f33a00d3ae0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f33880097a0 tx=0x7f338802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc703b0f26e1824 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:01.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.651+0000 7f339dd9b640 1 -- 172.21.15.148:0/4182101140 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33880315a0 con 0x7f33a00d2460 2024-03-29T04:10:01.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.651+0000 7f339dd9b640 1 -- 172.21.15.148:0/4182101140 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3388037070 con 0x7f33a00d2460 2024-03-29T04:10:01.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.651+0000 7f339dd9b640 1 -- 172.21.15.148:0/4182101140 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388031960 con 0x7f33a00d2460 2024-03-29T04:10:01.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.651+0000 7f33a4d00640 1 -- 172.21.15.148:0/4182101140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33a00d2460 msgr2=0x7f33a00d3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:01.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.651+0000 7f33a4d00640 1 --2- 172.21.15.148:0/4182101140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33a00d2460 0x7f33a00d3ae0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f33880097a0 tx=0x7f338802b840 comp rx=0 tx=0).stop 2024-03-29T04:10:01.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.652+0000 7f33a4d00640 1 -- 172.21.15.148:0/4182101140 shutdown_connections 2024-03-29T04:10:01.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.652+0000 7f33a4d00640 1 --2- 172.21.15.148:0/4182101140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33a00cf060 0x7f33a00cf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.652+0000 7f33a4d00640 1 --2- 172.21.15.148:0/4182101140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33a00d2460 0x7f33a00d3ae0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.652+0000 7f33a4d00640 1 -- 172.21.15.148:0/4182101140 >> 172.21.15.148:0/4182101140 conn(0x7f33a00d1230 msgr2=0x7f33a00d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:01.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.652+0000 7f33a4d00640 1 -- 172.21.15.148:0/4182101140 shutdown_connections 2024-03-29T04:10:01.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.652+0000 7f33a4d00640 1 -- 172.21.15.148:0/4182101140 wait complete. 2024-03-29T04:10:01.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.653+0000 7f33a4d00640 1 Processor -- start 2024-03-29T04:10:01.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.653+0000 7f33a4d00640 1 -- start start 2024-03-29T04:10:01.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.654+0000 7f33a4d00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a00cf060 0x7f33a00cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:01.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.654+0000 7f33a4d00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33a00d2460 0x7f33a0160ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:01.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.654+0000 7f33a4d00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33a015e4f0 0x7f33a00ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:01.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.654+0000 7f33a4d00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f33a00d4020 con 0x7f33a00d2460 2024-03-29T04:10:01.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.654+0000 7f33a4d00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f33a00d35c0 con 0x7f33a00cf060 2024-03-29T04:10:01.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.654+0000 7f33a4d00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f33a00d41a0 con 0x7f33a015e4f0 2024-03-29T04:10:01.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.654+0000 7f3397fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33a00d2460 0x7f33a0160ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:01.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.654+0000 7f3397fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33a00d2460 0x7f33a0160ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47372/0 (socket says 172.21.15.148:47372) 2024-03-29T04:10:01.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.654+0000 7f3397fff640 1 -- 172.21.15.148:0/161275591 learned_addr learned my addr 172.21.15.148:0/161275591 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:01.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.654+0000 7f339ed9d640 1 --2- 172.21.15.148:0/161275591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33a015e4f0 0x7f33a00ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:01.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.654+0000 7f339e59c640 1 --2- 172.21.15.148:0/161275591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a00cf060 0x7f33a00cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:01.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.654+0000 7f3397fff640 1 -- 172.21.15.148:0/161275591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33a015e4f0 msgr2=0x7f33a00ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:01.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.654+0000 7f3397fff640 1 --2- 172.21.15.148:0/161275591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33a015e4f0 0x7f33a00ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.654+0000 7f3397fff640 1 -- 172.21.15.148:0/161275591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a00cf060 msgr2=0x7f33a00cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:01.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.654+0000 7f3397fff640 1 --2- 172.21.15.148:0/161275591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a00cf060 0x7f33a00cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.654+0000 7f3397fff640 1 -- 172.21.15.148:0/161275591 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a01f4400 con 0x7f33a00d2460 2024-03-29T04:10:01.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.654+0000 7f3397fff640 1 --2- 172.21.15.148:0/161275591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33a00d2460 0x7f33a0160ea0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3388034040 tx=0x7f33880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:01.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.654+0000 7f3396ffd640 1 -- 172.21.15.148:0/161275591 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388032630 con 0x7f33a00d2460 2024-03-29T04:10:01.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.654+0000 7f3396ffd640 1 -- 172.21.15.148:0/161275591 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33880327d0 con 0x7f33a00d2460 2024-03-29T04:10:01.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.654+0000 7f33a4d00640 1 -- 172.21.15.148:0/161275591 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a01f46e0 con 0x7f33a00d2460 2024-03-29T04:10:01.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.654+0000 7f3396ffd640 1 -- 172.21.15.148:0/161275591 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388032ba0 con 0x7f33a00d2460 2024-03-29T04:10:01.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.654+0000 7f33a4d00640 1 -- 172.21.15.148:0/161275591 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a01f4ca0 con 0x7f33a00d2460 2024-03-29T04:10:01.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.655+0000 7f33a4d00640 1 -- 172.21.15.148:0/161275591 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33a0068880 con 0x7f33a00d2460 2024-03-29T04:10:01.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.656+0000 7f3396ffd640 1 -- 172.21.15.148:0/161275591 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f338804e430 con 0x7f33a00d2460 2024-03-29T04:10:01.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.656+0000 7f3396ffd640 1 --2- 172.21.15.148:0/161275591 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f338006c530 0x7f338006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:01.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.657+0000 7f339e59c640 1 --2- 172.21.15.148:0/161275591 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f338006c530 0x7f338006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:01.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.657+0000 7f3396ffd640 1 -- 172.21.15.148:0/161275591 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 6007+0+0 (secure 0 0 0) 0x7f338802f030 con 0x7f33a00d2460 2024-03-29T04:10:01.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.657+0000 7f339e59c640 1 --2- 172.21.15.148:0/161275591 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f338006c530 0x7f338006e9f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3390007fd0 tx=0x7f3390007e80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:01.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.661+0000 7f3396ffd640 1 -- 172.21.15.148:0/161275591 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3388040070 con 0x7f33a00d2460 2024-03-29T04:10:01.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:01.844+0000 7f33a4d00640 1 -- 172.21.15.148:0/161275591 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f33a01f7630 con 0x7f33a00d2460 2024-03-29T04:10:03.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.547+0000 7f3396ffd640 1 -- 172.21.15.148:0/161275591 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) ==== 228+0+0 (secure 0 0 0) 0x7f338807b490 con 0x7f33a00d2460 2024-03-29T04:10:03.527 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data_ec' created 2024-03-29T04:10:03.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.550+0000 7f33a4d00640 1 -- 172.21.15.148:0/161275591 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f338006c530 msgr2=0x7f338006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.550+0000 7f33a4d00640 1 --2- 172.21.15.148:0/161275591 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f338006c530 0x7f338006e9f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3390007fd0 tx=0x7f3390007e80 comp rx=0 tx=0).stop 2024-03-29T04:10:03.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.551+0000 7f33a4d00640 1 -- 172.21.15.148:0/161275591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33a00d2460 msgr2=0x7f33a0160ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.551+0000 7f33a4d00640 1 --2- 172.21.15.148:0/161275591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33a00d2460 0x7f33a0160ea0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3388034040 tx=0x7f33880046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.551+0000 7f33a4d00640 1 -- 172.21.15.148:0/161275591 shutdown_connections 2024-03-29T04:10:03.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.551+0000 7f33a4d00640 1 --2- 172.21.15.148:0/161275591 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f338006c530 0x7f338006e9f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.551+0000 7f33a4d00640 1 --2- 172.21.15.148:0/161275591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33a015e4f0 0x7f33a00ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.551+0000 7f33a4d00640 1 --2- 172.21.15.148:0/161275591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33a00d2460 0x7f33a0160ea0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.551+0000 7f33a4d00640 1 --2- 172.21.15.148:0/161275591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a00cf060 0x7f33a00cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.551+0000 7f33a4d00640 1 -- 172.21.15.148:0/161275591 >> 172.21.15.148:0/161275591 conn(0x7f33a00d1230 msgr2=0x7f33a006cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:03.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.551+0000 7f33a4d00640 1 -- 172.21.15.148:0/161275591 shutdown_connections 2024-03-29T04:10:03.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.551+0000 7f33a4d00640 1 -- 172.21.15.148:0/161275591 wait complete. 2024-03-29T04:10:03.547 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-29T04:10:03.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.673+0000 7f299cce4640 1 Processor -- start 2024-03-29T04:10:03.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.673+0000 7f299cce4640 1 -- start start 2024-03-29T04:10:03.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.674+0000 7f299cce4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2998161860 0x7f29980d3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.674+0000 7f299cce4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f29980b9fc0 0x7f29980d2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.674+0000 7f299cce4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29980d2c10 con 0x7f29980cefe0 2024-03-29T04:10:03.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.674+0000 7f299cce4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f29980d2ef0 con 0x7f29980b9fc0 2024-03-29T04:10:03.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.674+0000 7f299cce4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f29980d31d0 con 0x7f2998161860 2024-03-29T04:10:03.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.674+0000 7f2995d9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f29980b9fc0 0x7f29980d2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.674+0000 7f2995d9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f29980b9fc0 0x7f29980d2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47388/0 (socket says 172.21.15.148:47388) 2024-03-29T04:10:03.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.674+0000 7f2995d9b640 1 -- 172.21.15.148:0/1107265200 learned_addr learned my addr 172.21.15.148:0/1107265200 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:03.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.674+0000 7f299659c640 1 --2- 172.21.15.148:0/1107265200 >> v2:172.21.15.157:3300/0 conn(0x7f2998161860 0x7f29980d3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.675+0000 7f2995d9b640 1 -- 172.21.15.148:0/1107265200 >> v2:172.21.15.157:3300/0 conn(0x7f2998161860 msgr2=0x7f29980d3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:03.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.675+0000 7f2995d9b640 1 --2- 172.21.15.148:0/1107265200 >> v2:172.21.15.157:3300/0 conn(0x7f2998161860 0x7f29980d3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.675+0000 7f2995d9b640 1 -- 172.21.15.148:0/1107265200 >> v1:172.21.15.157:6789/0 conn(0x7f29980cefe0 legacy=0x7f29980cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.675+0000 7f2995d9b640 1 -- 172.21.15.148:0/1107265200 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29980d34b0 con 0x7f29980b9fc0 2024-03-29T04:10:03.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.676+0000 7f2995d9b640 1 --2- 172.21.15.148:0/1107265200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29980b9fc0 0x7f29980d2350 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2988009870 tx=0x7f298802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534d644caeb37fa8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:03.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.676+0000 7f299559a640 1 -- 172.21.15.148:0/1107265200 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298802f840 con 0x7f29980b9fc0 2024-03-29T04:10:03.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.676+0000 7f299559a640 1 -- 172.21.15.148:0/1107265200 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f298802f9e0 con 0x7f29980b9fc0 2024-03-29T04:10:03.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.676+0000 7f299559a640 1 -- 172.21.15.148:0/1107265200 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298802fdb0 con 0x7f29980b9fc0 2024-03-29T04:10:03.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.677+0000 7f299cce4640 1 -- 172.21.15.148:0/1107265200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29980b9fc0 msgr2=0x7f29980d2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.677+0000 7f299cce4640 1 --2- 172.21.15.148:0/1107265200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29980b9fc0 0x7f29980d2350 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2988009870 tx=0x7f298802ba20 comp rx=0 tx=0).stop 2024-03-29T04:10:03.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.677+0000 7f299cce4640 1 -- 172.21.15.148:0/1107265200 shutdown_connections 2024-03-29T04:10:03.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.677+0000 7f299cce4640 1 --2- 172.21.15.148:0/1107265200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29980b9fc0 0x7f29980d2350 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.677+0000 7f299cce4640 1 --2- 172.21.15.148:0/1107265200 >> v2:172.21.15.157:3300/0 conn(0x7f2998161860 0x7f29980d3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.677+0000 7f299cce4640 1 -- 172.21.15.148:0/1107265200 >> 172.21.15.148:0/1107265200 conn(0x7f29980b6210 msgr2=0x7f29980b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:03.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.677+0000 7f299cce4640 1 -- 172.21.15.148:0/1107265200 shutdown_connections 2024-03-29T04:10:03.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.677+0000 7f299cce4640 1 -- 172.21.15.148:0/1107265200 wait complete. 2024-03-29T04:10:03.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.678+0000 7f299cce4640 1 Processor -- start 2024-03-29T04:10:03.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.678+0000 7f299cce4640 1 -- start start 2024-03-29T04:10:03.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.679+0000 7f299cce4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29980b9fc0 0x7f29980ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.679+0000 7f299cce4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29980cefe0 0x7f29981e8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.679+0000 7f299cce4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2998161860 0x7f29981f5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.679+0000 7f299cce4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f29980d2890 con 0x7f29980b9fc0 2024-03-29T04:10:03.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.679+0000 7f299cce4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2998161c40 con 0x7f29980cefe0 2024-03-29T04:10:03.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.679+0000 7f299cce4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f29980b7380 con 0x7f2998161860 2024-03-29T04:10:03.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.679+0000 7f299659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29980b9fc0 0x7f29980ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.679+0000 7f299659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29980b9fc0 0x7f29980ca720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47390/0 (socket says 172.21.15.148:47390) 2024-03-29T04:10:03.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.679+0000 7f299659c640 1 -- 172.21.15.148:0/1552698544 learned_addr learned my addr 172.21.15.148:0/1552698544 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:03.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.679+0000 7f2995d9b640 1 --2- 172.21.15.148:0/1552698544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29980cefe0 0x7f29981e8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.679+0000 7f2996d9d640 1 --2- 172.21.15.148:0/1552698544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2998161860 0x7f29981f5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.679+0000 7f299659c640 1 -- 172.21.15.148:0/1552698544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2998161860 msgr2=0x7f29981f5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.679+0000 7f299659c640 1 --2- 172.21.15.148:0/1552698544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2998161860 0x7f29981f5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.679+0000 7f299659c640 1 -- 172.21.15.148:0/1552698544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29980cefe0 msgr2=0x7f29981e8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.679+0000 7f299659c640 1 --2- 172.21.15.148:0/1552698544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29980cefe0 0x7f29981e8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.679+0000 7f299659c640 1 -- 172.21.15.148:0/1552698544 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29981e8a60 con 0x7f29980b9fc0 2024-03-29T04:10:03.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.679+0000 7f2995d9b640 1 --2- 172.21.15.148:0/1552698544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29980cefe0 0x7f29981e8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:03.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.679+0000 7f2996d9d640 1 --2- 172.21.15.148:0/1552698544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2998161860 0x7f29981f5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:03.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.679+0000 7f299659c640 1 --2- 172.21.15.148:0/1552698544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29980b9fc0 0x7f29980ca720 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f298c00e870 tx=0x7f298c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:03.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.680+0000 7f2986ffd640 1 -- 172.21.15.148:0/1552698544 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c017070 con 0x7f29980b9fc0 2024-03-29T04:10:03.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.680+0000 7f2986ffd640 1 -- 172.21.15.148:0/1552698544 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f298c00c940 con 0x7f29980b9fc0 2024-03-29T04:10:03.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.680+0000 7f299cce4640 1 -- 172.21.15.148:0/1552698544 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29981e8d40 con 0x7f29980b9fc0 2024-03-29T04:10:03.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.680+0000 7f2986ffd640 1 -- 172.21.15.148:0/1552698544 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c00cd30 con 0x7f29980b9fc0 2024-03-29T04:10:03.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.680+0000 7f299cce4640 1 -- 172.21.15.148:0/1552698544 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29981e8770 con 0x7f29980b9fc0 2024-03-29T04:10:03.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.681+0000 7f299cce4640 1 -- 172.21.15.148:0/1552698544 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2998068740 con 0x7f29980b9fc0 2024-03-29T04:10:03.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.681+0000 7f2986ffd640 1 -- 172.21.15.148:0/1552698544 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f298c01a070 con 0x7f29980b9fc0 2024-03-29T04:10:03.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.682+0000 7f2986ffd640 1 --2- 172.21.15.148:0/1552698544 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f297806c560 0x7f297806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.682+0000 7f2995d9b640 1 --2- 172.21.15.148:0/1552698544 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f297806c560 0x7f297806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.682+0000 7f2986ffd640 1 -- 172.21.15.148:0/1552698544 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6500+0+0 (secure 0 0 0) 0x7f298c08bfd0 con 0x7f29980b9fc0 2024-03-29T04:10:03.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.682+0000 7f2995d9b640 1 --2- 172.21.15.148:0/1552698544 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f297806c560 0x7f297806ea20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2988036000 tx=0x7f2988039040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:03.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.686+0000 7f2986ffd640 1 -- 172.21.15.148:0/1552698544 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f298c05a3b0 con 0x7f29980b9fc0 2024-03-29T04:10:03.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:03.865+0000 7f299cce4640 1 -- 172.21.15.148:0/1552698544 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f29981e8010 con 0x7f29980b9fc0 2024-03-29T04:10:04.477 INFO:tasks.ceph.mds.a.smithi148.stderr:2024-03-29T04:10:04.495+0000 7fded2601640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T04:10:04.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.551+0000 7f2986ffd640 1 -- 172.21.15.148:0/1552698544 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) ==== 169+0+0 (secure 0 0 0) 0x7f298c059cb0 con 0x7f29980b9fc0 2024-03-29T04:10:04.531 INFO:teuthology.orchestra.run.smithi148.stderr:set pool 4 allow_ec_overwrites to true 2024-03-29T04:10:04.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.554+0000 7f299cce4640 1 -- 172.21.15.148:0/1552698544 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f297806c560 msgr2=0x7f297806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.554+0000 7f299cce4640 1 --2- 172.21.15.148:0/1552698544 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f297806c560 0x7f297806ea20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2988036000 tx=0x7f2988039040 comp rx=0 tx=0).stop 2024-03-29T04:10:04.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.554+0000 7f299cce4640 1 -- 172.21.15.148:0/1552698544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29980b9fc0 msgr2=0x7f29980ca720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.554+0000 7f299cce4640 1 --2- 172.21.15.148:0/1552698544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29980b9fc0 0x7f29980ca720 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f298c00e870 tx=0x7f298c00ed30 comp rx=0 tx=0).stop 2024-03-29T04:10:04.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.554+0000 7f299cce4640 1 -- 172.21.15.148:0/1552698544 shutdown_connections 2024-03-29T04:10:04.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.554+0000 7f299cce4640 1 --2- 172.21.15.148:0/1552698544 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f297806c560 0x7f297806ea20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.554+0000 7f299cce4640 1 --2- 172.21.15.148:0/1552698544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2998161860 0x7f29981f5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.554+0000 7f299cce4640 1 --2- 172.21.15.148:0/1552698544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29980cefe0 0x7f29981e8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.554+0000 7f299cce4640 1 --2- 172.21.15.148:0/1552698544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29980b9fc0 0x7f29980ca720 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.555+0000 7f299cce4640 1 -- 172.21.15.148:0/1552698544 >> 172.21.15.148:0/1552698544 conn(0x7f29980b6210 msgr2=0x7f29980b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:04.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.555+0000 7f299cce4640 1 -- 172.21.15.148:0/1552698544 shutdown_connections 2024-03-29T04:10:04.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.555+0000 7f299cce4640 1 -- 172.21.15.148:0/1552698544 wait complete. 2024-03-29T04:10:04.556 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-29T04:10:04.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.684+0000 7f20ea0d9640 1 Processor -- start 2024-03-29T04:10:04.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.684+0000 7f20ea0d9640 1 -- start start 2024-03-29T04:10:04.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.685+0000 7f20ea0d9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f20e40cc0f0 0x7f20e40cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.685+0000 7f20ea0d9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f20e40cfbd0 0x7f20e406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.685+0000 7f20ea0d9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20e40cc660 con 0x7f20e4144980 2024-03-29T04:10:04.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.685+0000 7f20ea0d9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f20e40701c0 con 0x7f20e40cfbd0 2024-03-29T04:10:04.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.685+0000 7f20ea0d9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f20e40cd270 con 0x7f20e40cc0f0 2024-03-29T04:10:04.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.685+0000 7f20e3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f20e40cfbd0 0x7f20e406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.685+0000 7f20e3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f20e40cfbd0 0x7f20e406c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47396/0 (socket says 172.21.15.148:47396) 2024-03-29T04:10:04.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.685+0000 7f20e3fff640 1 -- 172.21.15.148:0/2973981694 learned_addr learned my addr 172.21.15.148:0/2973981694 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:04.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.685+0000 7f20e2ffd640 1 --2- 172.21.15.148:0/2973981694 >> v2:172.21.15.157:3300/0 conn(0x7f20e40cc0f0 0x7f20e40cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.685+0000 7f20e3fff640 1 -- 172.21.15.148:0/2973981694 >> v2:172.21.15.157:3300/0 conn(0x7f20e40cc0f0 msgr2=0x7f20e40cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:04.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.685+0000 7f20e3fff640 1 --2- 172.21.15.148:0/2973981694 >> v2:172.21.15.157:3300/0 conn(0x7f20e40cc0f0 0x7f20e40cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.685+0000 7f20e3fff640 1 -- 172.21.15.148:0/2973981694 >> v1:172.21.15.157:6789/0 conn(0x7f20e4144980 legacy=0x7f20e406fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:04.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.685+0000 7f20e3fff640 1 -- 172.21.15.148:0/2973981694 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e40cc940 con 0x7f20e40cfbd0 2024-03-29T04:10:04.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.686+0000 7f20e3fff640 1 --2- 172.21.15.148:0/2973981694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20e40cfbd0 0x7f20e406c6c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f20e40d0f10 tx=0x7f20dc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a40296e2bdb734 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:04.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.686+0000 7f20e27fc640 1 -- 172.21.15.148:0/2973981694 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc02ca80 con 0x7f20e40cfbd0 2024-03-29T04:10:04.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.686+0000 7f20e27fc640 1 -- 172.21.15.148:0/2973981694 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20dc02cc20 con 0x7f20e40cfbd0 2024-03-29T04:10:04.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.686+0000 7f20e27fc640 1 -- 172.21.15.148:0/2973981694 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc0343f0 con 0x7f20e40cfbd0 2024-03-29T04:10:04.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.687+0000 7f20ea0d9640 1 -- 172.21.15.148:0/2973981694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20e40cfbd0 msgr2=0x7f20e406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.687+0000 7f20ea0d9640 1 --2- 172.21.15.148:0/2973981694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20e40cfbd0 0x7f20e406c6c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f20e40d0f10 tx=0x7f20dc02bee0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.687+0000 7f20ea0d9640 1 -- 172.21.15.148:0/2973981694 shutdown_connections 2024-03-29T04:10:04.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.687+0000 7f20ea0d9640 1 --2- 172.21.15.148:0/2973981694 >> v2:172.21.15.157:3300/0 conn(0x7f20e40cc0f0 0x7f20e40cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.687+0000 7f20ea0d9640 1 --2- 172.21.15.148:0/2973981694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20e40cfbd0 0x7f20e406c6c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.687+0000 7f20ea0d9640 1 -- 172.21.15.148:0/2973981694 >> 172.21.15.148:0/2973981694 conn(0x7f20e40b2c60 msgr2=0x7f20e40b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:04.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.687+0000 7f20ea0d9640 1 -- 172.21.15.148:0/2973981694 shutdown_connections 2024-03-29T04:10:04.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.687+0000 7f20ea0d9640 1 -- 172.21.15.148:0/2973981694 wait complete. 2024-03-29T04:10:04.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.688+0000 7f20ea0d9640 1 Processor -- start 2024-03-29T04:10:04.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.689+0000 7f20ea0d9640 1 -- start start 2024-03-29T04:10:04.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.690+0000 7f20ea0d9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20e40cc0f0 0x7f20e41f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.690+0000 7f20ea0d9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20e40cfbd0 0x7f20e413f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.690+0000 7f20ea0d9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20e4144980 0x7f20e40b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.690+0000 7f20ea0d9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f20e4144d60 con 0x7f20e40cc0f0 2024-03-29T04:10:04.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.690+0000 7f20ea0d9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f20e40ccf00 con 0x7f20e40cfbd0 2024-03-29T04:10:04.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.690+0000 7f20ea0d9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f20e40cd700 con 0x7f20e4144980 2024-03-29T04:10:04.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.690+0000 7f20e37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20e40cc0f0 0x7f20e41f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.690+0000 7f20e37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20e40cc0f0 0x7f20e41f4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47410/0 (socket says 172.21.15.148:47410) 2024-03-29T04:10:04.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.690+0000 7f20e37fe640 1 -- 172.21.15.148:0/1209825261 learned_addr learned my addr 172.21.15.148:0/1209825261 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:04.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.690+0000 7f20e3fff640 1 --2- 172.21.15.148:0/1209825261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20e4144980 0x7f20e40b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.690+0000 7f20e2ffd640 1 --2- 172.21.15.148:0/1209825261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20e40cfbd0 0x7f20e413f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.690+0000 7f20e37fe640 1 -- 172.21.15.148:0/1209825261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20e4144980 msgr2=0x7f20e40b6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.690+0000 7f20e37fe640 1 --2- 172.21.15.148:0/1209825261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20e4144980 0x7f20e40b6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.690+0000 7f20e37fe640 1 -- 172.21.15.148:0/1209825261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20e40cfbd0 msgr2=0x7f20e413f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.690+0000 7f20e37fe640 1 --2- 172.21.15.148:0/1209825261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20e40cfbd0 0x7f20e413f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.690+0000 7f20e37fe640 1 -- 172.21.15.148:0/1209825261 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e41f51c0 con 0x7f20e40cc0f0 2024-03-29T04:10:04.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.691+0000 7f20e37fe640 1 --2- 172.21.15.148:0/1209825261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20e40cc0f0 0x7f20e41f4c80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f20d400d990 tx=0x7f20d400de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:04.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.691+0000 7f20cbfff640 1 -- 172.21.15.148:0/1209825261 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d401b070 con 0x7f20e40cc0f0 2024-03-29T04:10:04.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.691+0000 7f20cbfff640 1 -- 172.21.15.148:0/1209825261 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20d400ed00 con 0x7f20e40cc0f0 2024-03-29T04:10:04.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.691+0000 7f20ea0d9640 1 -- 172.21.15.148:0/1209825261 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e40b52e0 con 0x7f20e40cc0f0 2024-03-29T04:10:04.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.691+0000 7f20cbfff640 1 -- 172.21.15.148:0/1209825261 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d4016680 con 0x7f20e40cc0f0 2024-03-29T04:10:04.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.691+0000 7f20ea0d9640 1 -- 172.21.15.148:0/1209825261 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e40b58a0 con 0x7f20e40cc0f0 2024-03-29T04:10:04.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.692+0000 7f20ea0d9640 1 -- 172.21.15.148:0/1209825261 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20e4068850 con 0x7f20e40cc0f0 2024-03-29T04:10:04.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.692+0000 7f20cbfff640 1 -- 172.21.15.148:0/1209825261 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f20d401e070 con 0x7f20e40cc0f0 2024-03-29T04:10:04.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.692+0000 7f20cbfff640 1 --2- 172.21.15.148:0/1209825261 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f20d006c530 0x7f20d006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.692+0000 7f20cbfff640 1 -- 172.21.15.148:0/1209825261 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7f20d408d550 con 0x7f20e40cc0f0 2024-03-29T04:10:04.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.693+0000 7f20e2ffd640 1 --2- 172.21.15.148:0/1209825261 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f20d006c530 0x7f20d006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.693+0000 7f20e2ffd640 1 --2- 172.21.15.148:0/1209825261 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f20d006c530 0x7f20d006e9f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f20e415b260 tx=0x7f20d8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:04.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:04.697+0000 7f20cbfff640 1 -- 172.21.15.148:0/1209825261 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f20e41f51c0 con 0x7f20e40cc0f0 2024-03-29T04:10:05.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.025+0000 7f20ea0d9640 1 -- 172.21.15.148:0/1209825261 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f20e4074cd0 con 0x7f20e40cc0f0 2024-03-29T04:10:05.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.561+0000 7f20cbfff640 1 -- 172.21.15.148:0/1209825261 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f20d405b290 con 0x7f20e40cc0f0 2024-03-29T04:10:05.540 INFO:teuthology.orchestra.run.smithi148.stderr:added data pool 4 to fsmap 2024-03-29T04:10:05.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.564+0000 7f20ea0d9640 1 -- 172.21.15.148:0/1209825261 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f20d006c530 msgr2=0x7f20d006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:05.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.564+0000 7f20ea0d9640 1 --2- 172.21.15.148:0/1209825261 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f20d006c530 0x7f20d006e9f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f20e415b260 tx=0x7f20d8008040 comp rx=0 tx=0).stop 2024-03-29T04:10:05.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.564+0000 7f20ea0d9640 1 -- 172.21.15.148:0/1209825261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20e40cc0f0 msgr2=0x7f20e41f4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:05.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.564+0000 7f20ea0d9640 1 --2- 172.21.15.148:0/1209825261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20e40cc0f0 0x7f20e41f4c80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f20d400d990 tx=0x7f20d400de50 comp rx=0 tx=0).stop 2024-03-29T04:10:05.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.564+0000 7f20ea0d9640 1 -- 172.21.15.148:0/1209825261 shutdown_connections 2024-03-29T04:10:05.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.564+0000 7f20ea0d9640 1 --2- 172.21.15.148:0/1209825261 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f20d006c530 0x7f20d006e9f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.564+0000 7f20ea0d9640 1 --2- 172.21.15.148:0/1209825261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20e4144980 0x7f20e40b6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.564+0000 7f20ea0d9640 1 --2- 172.21.15.148:0/1209825261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20e40cfbd0 0x7f20e413f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.564+0000 7f20ea0d9640 1 --2- 172.21.15.148:0/1209825261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20e40cc0f0 0x7f20e41f4c80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.564+0000 7f20ea0d9640 1 -- 172.21.15.148:0/1209825261 >> 172.21.15.148:0/1209825261 conn(0x7f20e40b2c60 msgr2=0x7f20e40cf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:05.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.564+0000 7f20ea0d9640 1 -- 172.21.15.148:0/1209825261 shutdown_connections 2024-03-29T04:10:05.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.564+0000 7f20ea0d9640 1 -- 172.21.15.148:0/1209825261 wait complete. 2024-03-29T04:10:05.553 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:10:05.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.678+0000 7f7abe07b640 1 Processor -- start 2024-03-29T04:10:05.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.678+0000 7f7abe07b640 1 -- start start 2024-03-29T04:10:05.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.679+0000 7f7abe07b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7ab815c5f0 0x7f7ab80cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:05.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.679+0000 7f7abe07b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ab81608b0 0x7f7ab815cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:05.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.679+0000 7f7abe07b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ab80d19d0 con 0x7f7ab80ceb50 2024-03-29T04:10:05.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.679+0000 7f7abe07b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f7ab80cf470 con 0x7f7ab815c5f0 2024-03-29T04:10:05.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.679+0000 7f7abe07b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7ab80ccde0 con 0x7f7ab81608b0 2024-03-29T04:10:05.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.679+0000 7f7aaeffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7ab815c5f0 0x7f7ab80cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:05.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.679+0000 7f7aaeffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7ab815c5f0 0x7f7ab80cef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51156/0 (socket says 172.21.15.148:51156) 2024-03-29T04:10:05.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.679+0000 7f7aaeffd640 1 -- 172.21.15.148:0/1827357278 learned_addr learned my addr 172.21.15.148:0/1827357278 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:05.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.680+0000 7f7ab7fff640 1 --2- 172.21.15.148:0/1827357278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ab81608b0 0x7f7ab815cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:05.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.680+0000 7f7aaeffd640 1 -- 172.21.15.148:0/1827357278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ab81608b0 msgr2=0x7f7ab815cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:05.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.680+0000 7f7aaeffd640 1 --2- 172.21.15.148:0/1827357278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ab81608b0 0x7f7ab815cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.680+0000 7f7aaeffd640 1 -- 172.21.15.148:0/1827357278 >> v1:172.21.15.157:6789/0 conn(0x7f7ab80ceb50 legacy=0x7f7ab80cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:05.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.680+0000 7f7aaeffd640 1 -- 172.21.15.148:0/1827357278 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab80ccb00 con 0x7f7ab815c5f0 2024-03-29T04:10:05.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.680+0000 7f7aaeffd640 1 --2- 172.21.15.148:0/1827357278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ab815c5f0 0x7f7ab80cef30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80b8280 tx=0x7f7aa002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651b95d3d7a63997 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:05.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.680+0000 7f7ab6ffd640 1 -- 172.21.15.148:0/1827357278 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0035020 con 0x7f7ab815c5f0 2024-03-29T04:10:05.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.681+0000 7f7ab6ffd640 1 -- 172.21.15.148:0/1827357278 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa0031640 con 0x7f7ab815c5f0 2024-03-29T04:10:05.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.681+0000 7f7ab6ffd640 1 -- 172.21.15.148:0/1827357278 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa002d040 con 0x7f7ab815c5f0 2024-03-29T04:10:05.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.681+0000 7f7abe07b640 1 -- 172.21.15.148:0/1827357278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ab815c5f0 msgr2=0x7f7ab80cef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:05.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.681+0000 7f7abe07b640 1 --2- 172.21.15.148:0/1827357278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ab815c5f0 0x7f7ab80cef30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80b8280 tx=0x7f7aa002b8f0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.681+0000 7f7abe07b640 1 -- 172.21.15.148:0/1827357278 shutdown_connections 2024-03-29T04:10:05.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.681+0000 7f7abe07b640 1 --2- 172.21.15.148:0/1827357278 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ab81608b0 0x7f7ab815cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.681+0000 7f7abe07b640 1 --2- 172.21.15.148:0/1827357278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ab815c5f0 0x7f7ab80cef30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.681+0000 7f7abe07b640 1 -- 172.21.15.148:0/1827357278 >> 172.21.15.148:0/1827357278 conn(0x7f7ab80b3c70 msgr2=0x7f7ab80b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:05.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.681+0000 7f7abe07b640 1 -- 172.21.15.148:0/1827357278 shutdown_connections 2024-03-29T04:10:05.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.681+0000 7f7abe07b640 1 -- 172.21.15.148:0/1827357278 wait complete. 2024-03-29T04:10:05.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.682+0000 7f7abe07b640 1 Processor -- start 2024-03-29T04:10:05.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.682+0000 7f7abe07b640 1 -- start start 2024-03-29T04:10:05.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.683+0000 7f7abe07b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ab80ceb50 0x7f7ab80cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:05.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.683+0000 7f7abe07b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ab81608b0 0x7f7ab806e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:05.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.683+0000 7f7abe07b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ab806c920 0x7f7ab81575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:05.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.683+0000 7f7abe07b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab80ce540 con 0x7f7ab81608b0 2024-03-29T04:10:05.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.683+0000 7f7abe07b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab80cc790 con 0x7f7ab806c920 2024-03-29T04:10:05.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.683+0000 7f7abe07b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7ab80ce6c0 con 0x7f7ab80ceb50 2024-03-29T04:10:05.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.683+0000 7f7aaeffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ab81608b0 0x7f7ab806e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:05.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.683+0000 7f7aaeffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ab81608b0 0x7f7ab806e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51168/0 (socket says 172.21.15.148:51168) 2024-03-29T04:10:05.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.683+0000 7f7aaeffd640 1 -- 172.21.15.148:0/590551506 learned_addr learned my addr 172.21.15.148:0/590551506 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:05.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.683+0000 7f7ab77fe640 1 --2- 172.21.15.148:0/590551506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ab80ceb50 0x7f7ab80cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:05.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.683+0000 7f7ab7fff640 1 --2- 172.21.15.148:0/590551506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ab806c920 0x7f7ab81575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:05.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.684+0000 7f7aaeffd640 1 -- 172.21.15.148:0/590551506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ab80ceb50 msgr2=0x7f7ab80cbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:05.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.684+0000 7f7aaeffd640 1 --2- 172.21.15.148:0/590551506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ab80ceb50 0x7f7ab80cbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.684+0000 7f7aaeffd640 1 -- 172.21.15.148:0/590551506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ab806c920 msgr2=0x7f7ab81575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:05.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.684+0000 7f7aaeffd640 1 --2- 172.21.15.148:0/590551506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ab806c920 0x7f7ab81575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.684+0000 7f7aaeffd640 1 -- 172.21.15.148:0/590551506 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab806d160 con 0x7f7ab81608b0 2024-03-29T04:10:05.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.684+0000 7f7aaeffd640 1 --2- 172.21.15.148:0/590551506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ab81608b0 0x7f7ab806e200 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0000c00 tx=0x7f7aa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:05.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.684+0000 7f7ab4ff9640 1 -- 172.21.15.148:0/590551506 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0040070 con 0x7f7ab81608b0 2024-03-29T04:10:05.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.684+0000 7f7ab4ff9640 1 -- 172.21.15.148:0/590551506 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa0039c30 con 0x7f7ab81608b0 2024-03-29T04:10:05.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.684+0000 7f7ab4ff9640 1 -- 172.21.15.148:0/590551506 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0035040 con 0x7f7ab81608b0 2024-03-29T04:10:05.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.684+0000 7f7abe07b640 1 -- 172.21.15.148:0/590551506 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab806d440 con 0x7f7ab81608b0 2024-03-29T04:10:05.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.684+0000 7f7abe07b640 1 -- 172.21.15.148:0/590551506 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab806da00 con 0x7f7ab81608b0 2024-03-29T04:10:05.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.685+0000 7f7abe07b640 1 -- 172.21.15.148:0/590551506 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ab80687e0 con 0x7f7ab81608b0 2024-03-29T04:10:05.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.685+0000 7f7ab4ff9640 1 -- 172.21.15.148:0/590551506 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7aa004b060 con 0x7f7ab81608b0 2024-03-29T04:10:05.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.686+0000 7f7ab4ff9640 1 --2- 172.21.15.148:0/590551506 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7a9806c530 0x7f7a9806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:05.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.686+0000 7f7ab77fe640 1 --2- 172.21.15.148:0/590551506 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7a9806c530 0x7f7a9806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:05.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.686+0000 7f7ab4ff9640 1 -- 172.21.15.148:0/590551506 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7aa00ad570 con 0x7f7ab81608b0 2024-03-29T04:10:05.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.686+0000 7f7ab77fe640 1 --2- 172.21.15.148:0/590551506 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7a9806c530 0x7f7a9806e9f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8007fd0 tx=0x7f7aa8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:05.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.690+0000 7f7ab4ff9640 1 -- 172.21.15.148:0/590551506 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7aa0031610 con 0x7f7ab81608b0 2024-03-29T04:10:05.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.912+0000 7f7abe07b640 1 -- 172.21.15.148:0/590551506 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ab80701d0 con 0x7f7ab81608b0 2024-03-29T04:10:05.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.913+0000 7f7ab4ff9640 1 -- 172.21.15.148:0/590551506 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5183 (secure 0 0 0) 0x7f7aa007b850 con 0x7f7ab81608b0 2024-03-29T04:10:05.892 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:05.892 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/36660496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":36660496},{"type":"v1","addr":"172.21.15.157:6834","nonce":36660496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2674481055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2674481055},{"type":"v1","addr":"172.21.15.157:6835","nonce":2674481055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3356449643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3356449643},{"type":"v1","addr":"172.21.15.148:6836","nonce":3356449643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:09:59.886367+0000","modified":"2024-03-29T04:10:05.026739+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/2453181817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2453181817},{"type":"v1","addr":"172.21.15.148:6837","nonce":2453181817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-03-29T04:10:05.892 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 7 2024-03-29T04:10:05.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.915+0000 7f7abe07b640 1 -- 172.21.15.148:0/590551506 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7a9806c530 msgr2=0x7f7a9806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:05.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.915+0000 7f7abe07b640 1 --2- 172.21.15.148:0/590551506 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7a9806c530 0x7f7a9806e9f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8007fd0 tx=0x7f7aa8009040 comp rx=0 tx=0).stop 2024-03-29T04:10:05.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.915+0000 7f7abe07b640 1 -- 172.21.15.148:0/590551506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ab81608b0 msgr2=0x7f7ab806e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:05.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.916+0000 7f7abe07b640 1 --2- 172.21.15.148:0/590551506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ab81608b0 0x7f7ab806e200 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0000c00 tx=0x7f7aa00046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.916+0000 7f7abe07b640 1 -- 172.21.15.148:0/590551506 shutdown_connections 2024-03-29T04:10:05.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.916+0000 7f7abe07b640 1 --2- 172.21.15.148:0/590551506 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7a9806c530 0x7f7a9806e9f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.916+0000 7f7abe07b640 1 --2- 172.21.15.148:0/590551506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ab806c920 0x7f7ab81575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.916+0000 7f7abe07b640 1 --2- 172.21.15.148:0/590551506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ab81608b0 0x7f7ab806e200 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.916+0000 7f7abe07b640 1 --2- 172.21.15.148:0/590551506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ab80ceb50 0x7f7ab80cbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.916+0000 7f7abe07b640 1 -- 172.21.15.148:0/590551506 >> 172.21.15.148:0/590551506 conn(0x7f7ab80b3c70 msgr2=0x7f7ab80b4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:05.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.916+0000 7f7abe07b640 1 -- 172.21.15.148:0/590551506 shutdown_connections 2024-03-29T04:10:05.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:05.916+0000 7f7abe07b640 1 -- 172.21.15.148:0/590551506 wait complete. 2024-03-29T04:10:05.911 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:06.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.035+0000 7f857f87b640 1 Processor -- start 2024-03-29T04:10:06.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.035+0000 7f857f87b640 1 -- start start 2024-03-29T04:10:06.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.036+0000 7f857f87b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85780b05c0 0x7f857806e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.036+0000 7f857f87b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f85780b5e90 0x7f857806ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.036+0000 7f857f87b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f857806b930 con 0x7f857815e380 2024-03-29T04:10:06.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.036+0000 7f857f87b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f857806bc10 con 0x7f85780b5e90 2024-03-29T04:10:06.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.036+0000 7f857f87b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f857806bef0 con 0x7f85780b05c0 2024-03-29T04:10:06.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.036+0000 7f857de18640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f85780b5e90 0x7f857806ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.036+0000 7f857d617640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f857815e380 0x7f85780afeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50166/0 (socket says 172.21.15.148:50166) 2024-03-29T04:10:06.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.036+0000 7f857d617640 1 -- 172.21.15.148:0/1442842778 learned_addr learned my addr 172.21.15.148:0/1442842778 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:06.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.037+0000 7f857ce16640 1 --2- 172.21.15.148:0/1442842778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85780b05c0 0x7f857806e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.037+0000 7f857de18640 1 -- 172.21.15.148:0/1442842778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85780b05c0 msgr2=0x7f857806e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:06.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.037+0000 7f857de18640 1 --2- 172.21.15.148:0/1442842778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85780b05c0 0x7f857806e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.037+0000 7f857de18640 1 -- 172.21.15.148:0/1442842778 >> v1:172.21.15.148:6789/0 conn(0x7f857815e380 legacy=0x7f85780afeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.037+0000 7f856ffff640 1 -- 172.21.15.148:0/1442842778 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1303695890 0 0) 0x7f857806b930 con 0x7f857815e380 2024-03-29T04:10:06.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.037+0000 7f857de18640 1 -- 172.21.15.148:0/1442842778 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f857806c1d0 con 0x7f85780b5e90 2024-03-29T04:10:06.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.038+0000 7f857de18640 1 --2- 172.21.15.148:0/1442842778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85780b5e90 0x7f857806ec10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8570009d90 tx=0x7f857002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3efa12ea2c228b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:06.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.038+0000 7f856ffff640 1 -- 172.21.15.148:0/1442842778 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8570039070 con 0x7f85780b5e90 2024-03-29T04:10:06.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.038+0000 7f856ffff640 1 -- 172.21.15.148:0/1442842778 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f857002cbc0 con 0x7f85780b5e90 2024-03-29T04:10:06.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.038+0000 7f856ffff640 1 -- 172.21.15.148:0/1442842778 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8570034440 con 0x7f85780b5e90 2024-03-29T04:10:06.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.038+0000 7f857f87b640 1 -- 172.21.15.148:0/1442842778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85780b5e90 msgr2=0x7f857806ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.038+0000 7f857f87b640 1 --2- 172.21.15.148:0/1442842778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85780b5e90 0x7f857806ec10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8570009d90 tx=0x7f857002be10 comp rx=0 tx=0).stop 2024-03-29T04:10:06.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.038+0000 7f857f87b640 1 -- 172.21.15.148:0/1442842778 shutdown_connections 2024-03-29T04:10:06.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.038+0000 7f857f87b640 1 --2- 172.21.15.148:0/1442842778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85780b5e90 0x7f857806ec10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.038+0000 7f857f87b640 1 --2- 172.21.15.148:0/1442842778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85780b05c0 0x7f857806e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.038+0000 7f857f87b640 1 -- 172.21.15.148:0/1442842778 >> 172.21.15.148:0/1442842778 conn(0x7f85780b2c60 msgr2=0x7f85780b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:06.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.039+0000 7f857f87b640 1 -- 172.21.15.148:0/1442842778 shutdown_connections 2024-03-29T04:10:06.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.039+0000 7f857f87b640 1 -- 172.21.15.148:0/1442842778 wait complete. 2024-03-29T04:10:06.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.040+0000 7f857f87b640 1 Processor -- start 2024-03-29T04:10:06.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.040+0000 7f857f87b640 1 -- start start 2024-03-29T04:10:06.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.040+0000 7f857f87b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85780b05c0 0x7f8578156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.040+0000 7f857f87b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85780b5e90 0x7f85781593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.041+0000 7f857f87b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f857815e380 0x7f8578158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.041+0000 7f857f87b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f85780b4fa0 con 0x7f85780b5e90 2024-03-29T04:10:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.041+0000 7f857f87b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f857806b5b0 con 0x7f85780b05c0 2024-03-29T04:10:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.041+0000 7f857f87b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f85780b5120 con 0x7f857815e380 2024-03-29T04:10:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.041+0000 7f857ce16640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85780b5e90 0x7f85781593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.041+0000 7f857ce16640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85780b5e90 0x7f85781593c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51188/0 (socket says 172.21.15.148:51188) 2024-03-29T04:10:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.041+0000 7f857ce16640 1 -- 172.21.15.148:0/4142894714 learned_addr learned my addr 172.21.15.148:0/4142894714 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.041+0000 7f857de18640 1 --2- 172.21.15.148:0/4142894714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f857815e380 0x7f8578158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.041+0000 7f857d617640 1 --2- 172.21.15.148:0/4142894714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85780b05c0 0x7f8578156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.041+0000 7f857ce16640 1 -- 172.21.15.148:0/4142894714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f857815e380 msgr2=0x7f8578158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.041+0000 7f857ce16640 1 --2- 172.21.15.148:0/4142894714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f857815e380 0x7f8578158160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.041+0000 7f857ce16640 1 -- 172.21.15.148:0/4142894714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85780b05c0 msgr2=0x7f8578156560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.041+0000 7f857ce16640 1 --2- 172.21.15.148:0/4142894714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85780b05c0 0x7f8578156560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.041+0000 7f857ce16640 1 -- 172.21.15.148:0/4142894714 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85781542a0 con 0x7f85780b5e90 2024-03-29T04:10:06.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.041+0000 7f857ce16640 1 --2- 172.21.15.148:0/4142894714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85780b5e90 0x7f85781593c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f857400d800 tx=0x7f857400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:06.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.041+0000 7f856dffb640 1 -- 172.21.15.148:0/4142894714 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857400b6c0 con 0x7f85780b5e90 2024-03-29T04:10:06.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.041+0000 7f856dffb640 1 -- 172.21.15.148:0/4142894714 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8574019070 con 0x7f85780b5e90 2024-03-29T04:10:06.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.041+0000 7f857f87b640 1 -- 172.21.15.148:0/4142894714 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8578154580 con 0x7f85780b5e90 2024-03-29T04:10:06.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.041+0000 7f856dffb640 1 -- 172.21.15.148:0/4142894714 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857400ba80 con 0x7f85780b5e90 2024-03-29T04:10:06.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.041+0000 7f857f87b640 1 -- 172.21.15.148:0/4142894714 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8578156bf0 con 0x7f85780b5e90 2024-03-29T04:10:06.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.043+0000 7f856dffb640 1 -- 172.21.15.148:0/4142894714 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f857400bc20 con 0x7f85780b5e90 2024-03-29T04:10:06.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.043+0000 7f857f87b640 1 -- 172.21.15.148:0/4142894714 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85780687e0 con 0x7f85780b5e90 2024-03-29T04:10:06.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.043+0000 7f856dffb640 1 --2- 172.21.15.148:0/4142894714 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f855c06c530 0x7f855c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.043+0000 7f857d617640 1 --2- 172.21.15.148:0/4142894714 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f855c06c530 0x7f855c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.043+0000 7f856dffb640 1 -- 172.21.15.148:0/4142894714 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f857408bdc0 con 0x7f85780b5e90 2024-03-29T04:10:06.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.044+0000 7f857d617640 1 --2- 172.21.15.148:0/4142894714 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f855c06c530 0x7f855c06e9f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8568007fd0 tx=0x7f8568007de0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:06.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.048+0000 7f856dffb640 1 -- 172.21.15.148:0/4142894714 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f85740599c0 con 0x7f85780b5e90 2024-03-29T04:10:06.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.230+0000 7f857f87b640 1 -- 172.21.15.148:0/4142894714 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85780b4830 con 0x7f85780b5e90 2024-03-29T04:10:06.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.231+0000 7f856dffb640 1 -- 172.21.15.148:0/4142894714 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7f857405dcc0 con 0x7f85780b5e90 2024-03-29T04:10:06.210 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:06.211 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:05.026481+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:06.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.233+0000 7f857f87b640 1 -- 172.21.15.148:0/4142894714 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f855c06c530 msgr2=0x7f855c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.233+0000 7f857f87b640 1 --2- 172.21.15.148:0/4142894714 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f855c06c530 0x7f855c06e9f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8568007fd0 tx=0x7f8568007de0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.233+0000 7f857f87b640 1 -- 172.21.15.148:0/4142894714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85780b5e90 msgr2=0x7f85781593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.233+0000 7f857f87b640 1 --2- 172.21.15.148:0/4142894714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85780b5e90 0x7f85781593c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f857400d800 tx=0x7f857400dcc0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.233+0000 7f857f87b640 1 -- 172.21.15.148:0/4142894714 shutdown_connections 2024-03-29T04:10:06.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.233+0000 7f857f87b640 1 --2- 172.21.15.148:0/4142894714 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f855c06c530 0x7f855c06e9f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.233+0000 7f857f87b640 1 --2- 172.21.15.148:0/4142894714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f857815e380 0x7f8578158160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.233+0000 7f857f87b640 1 --2- 172.21.15.148:0/4142894714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85780b5e90 0x7f85781593c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.233+0000 7f857f87b640 1 --2- 172.21.15.148:0/4142894714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85780b05c0 0x7f8578156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.233+0000 7f857f87b640 1 -- 172.21.15.148:0/4142894714 >> 172.21.15.148:0/4142894714 conn(0x7f85780b2c60 msgr2=0x7f857815c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:06.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.233+0000 7f857f87b640 1 -- 172.21.15.148:0/4142894714 shutdown_connections 2024-03-29T04:10:06.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.233+0000 7f857f87b640 1 -- 172.21.15.148:0/4142894714 wait complete. 2024-03-29T04:10:06.223 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:06.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.346+0000 7efc95d40640 1 Processor -- start 2024-03-29T04:10:06.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.346+0000 7efc95d40640 1 -- start start 2024-03-29T04:10:06.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.347+0000 7efc95d40640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc900b05c0 0x7efc9006e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.347+0000 7efc95d40640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efc900b5e90 0x7efc9006ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.347+0000 7efc95d40640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc9006b930 con 0x7efc9015e380 2024-03-29T04:10:06.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.347+0000 7efc95d40640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7efc9006bc10 con 0x7efc900b5e90 2024-03-29T04:10:06.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.347+0000 7efc95d40640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efc9006bef0 con 0x7efc900b05c0 2024-03-29T04:10:06.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.347+0000 7efc8f7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7efc9015e380 0x7efc900afeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50178/0 (socket says 172.21.15.148:50178) 2024-03-29T04:10:06.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.347+0000 7efc8f7fe640 1 -- 172.21.15.148:0/1373982105 learned_addr learned my addr 172.21.15.148:0/1373982105 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:06.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.347+0000 7efc86ffd640 1 --2- 172.21.15.148:0/1373982105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc900b05c0 0x7efc9006e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.347+0000 7efc8ffff640 1 --2- 172.21.15.148:0/1373982105 >> v2:172.21.15.157:3300/0 conn(0x7efc900b5e90 0x7efc9006ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.347+0000 7efc8effd640 1 -- 172.21.15.148:0/1373982105 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 345046161 0 0) 0x7efc9006b930 con 0x7efc9015e380 2024-03-29T04:10:06.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.347+0000 7efc8effd640 1 -- 172.21.15.148:0/1373982105 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc7c003520 con 0x7efc9015e380 2024-03-29T04:10:06.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.348+0000 7efc86ffd640 1 -- 172.21.15.148:0/1373982105 >> v2:172.21.15.157:3300/0 conn(0x7efc900b5e90 msgr2=0x7efc9006ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:06.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.348+0000 7efc8effd640 1 -- 172.21.15.148:0/1373982105 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1397244426 0 0) 0x7efc7c003520 con 0x7efc9015e380 2024-03-29T04:10:06.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.348+0000 7efc86ffd640 1 --2- 172.21.15.148:0/1373982105 >> v2:172.21.15.157:3300/0 conn(0x7efc900b5e90 0x7efc9006ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.348+0000 7efc86ffd640 1 -- 172.21.15.148:0/1373982105 >> v1:172.21.15.148:6789/0 conn(0x7efc9015e380 legacy=0x7efc900afeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.348+0000 7efc86ffd640 1 -- 172.21.15.148:0/1373982105 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9006c1d0 con 0x7efc900b05c0 2024-03-29T04:10:06.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.348+0000 7efc8ffff640 1 --2- 172.21.15.148:0/1373982105 >> v2:172.21.15.157:3300/0 conn(0x7efc900b5e90 0x7efc9006ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:06.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.348+0000 7efc86ffd640 1 --2- 172.21.15.148:0/1373982105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc900b05c0 0x7efc9006e6d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efc780097a0 tx=0x7efc7802b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7794e4d64225ac69 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:06.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.349+0000 7efc8effd640 1 -- 172.21.15.148:0/1373982105 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc78037070 con 0x7efc900b05c0 2024-03-29T04:10:06.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.349+0000 7efc8effd640 1 -- 172.21.15.148:0/1373982105 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc78030510 con 0x7efc900b05c0 2024-03-29T04:10:06.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.349+0000 7efc8effd640 1 -- 172.21.15.148:0/1373982105 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc80001670 con 0x7efc900b05c0 2024-03-29T04:10:06.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.349+0000 7efc95d40640 1 -- 172.21.15.148:0/1373982105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc900b05c0 msgr2=0x7efc9006e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.349+0000 7efc95d40640 1 --2- 172.21.15.148:0/1373982105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc900b05c0 0x7efc9006e6d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efc780097a0 tx=0x7efc7802b710 comp rx=0 tx=0).stop 2024-03-29T04:10:06.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.349+0000 7efc95d40640 1 -- 172.21.15.148:0/1373982105 shutdown_connections 2024-03-29T04:10:06.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.349+0000 7efc95d40640 1 --2- 172.21.15.148:0/1373982105 >> v2:172.21.15.157:3300/0 conn(0x7efc900b5e90 0x7efc9006ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.349+0000 7efc95d40640 1 --2- 172.21.15.148:0/1373982105 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc900b05c0 0x7efc9006e6d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.349+0000 7efc95d40640 1 -- 172.21.15.148:0/1373982105 >> 172.21.15.148:0/1373982105 conn(0x7efc900b2c60 msgr2=0x7efc900b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:06.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.349+0000 7efc95d40640 1 -- 172.21.15.148:0/1373982105 shutdown_connections 2024-03-29T04:10:06.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.349+0000 7efc95d40640 1 -- 172.21.15.148:0/1373982105 wait complete. 2024-03-29T04:10:06.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.350+0000 7efc95d40640 1 Processor -- start 2024-03-29T04:10:06.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.350+0000 7efc95d40640 1 -- start start 2024-03-29T04:10:06.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.351+0000 7efc95d40640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc900b05c0 0x7efc90155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.351+0000 7efc95d40640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc900b5e90 0x7efc900cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.351+0000 7efc95d40640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc9015e380 0x7efc900d0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.351+0000 7efc95d40640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efc900b4fa0 con 0x7efc900b5e90 2024-03-29T04:10:06.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.351+0000 7efc95d40640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efc9006b5b0 con 0x7efc9015e380 2024-03-29T04:10:06.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.351+0000 7efc95d40640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efc900b5120 con 0x7efc900b05c0 2024-03-29T04:10:06.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.351+0000 7efc86ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc900b5e90 0x7efc900cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.351+0000 7efc86ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc900b5e90 0x7efc900cfc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51190/0 (socket says 172.21.15.148:51190) 2024-03-29T04:10:06.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.351+0000 7efc86ffd640 1 -- 172.21.15.148:0/3923616767 learned_addr learned my addr 172.21.15.148:0/3923616767 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:06.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.351+0000 7efc8f7fe640 1 --2- 172.21.15.148:0/3923616767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc900b05c0 0x7efc90155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.351+0000 7efc8ffff640 1 --2- 172.21.15.148:0/3923616767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc9015e380 0x7efc900d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.352+0000 7efc86ffd640 1 -- 172.21.15.148:0/3923616767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc900b05c0 msgr2=0x7efc90155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.352+0000 7efc86ffd640 1 --2- 172.21.15.148:0/3923616767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc900b05c0 0x7efc90155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.352+0000 7efc86ffd640 1 -- 172.21.15.148:0/3923616767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc9015e380 msgr2=0x7efc900d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.352+0000 7efc86ffd640 1 --2- 172.21.15.148:0/3923616767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc9015e380 0x7efc900d0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.352+0000 7efc86ffd640 1 -- 172.21.15.148:0/3923616767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9015feb0 con 0x7efc900b5e90 2024-03-29T04:10:06.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.352+0000 7efc86ffd640 1 --2- 172.21.15.148:0/3923616767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc900b5e90 0x7efc900cfc00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efc7802b750 tx=0x7efc780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:06.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.352+0000 7efc8cff9640 1 -- 172.21.15.148:0/3923616767 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc78041020 con 0x7efc900b5e90 2024-03-29T04:10:06.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.352+0000 7efc8cff9640 1 -- 172.21.15.148:0/3923616767 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc7803c870 con 0x7efc900b5e90 2024-03-29T04:10:06.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.352+0000 7efc95d40640 1 -- 172.21.15.148:0/3923616767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc900cd170 con 0x7efc900b5e90 2024-03-29T04:10:06.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.352+0000 7efc8cff9640 1 -- 172.21.15.148:0/3923616767 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc78037040 con 0x7efc900b5e90 2024-03-29T04:10:06.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.352+0000 7efc95d40640 1 -- 172.21.15.148:0/3923616767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc90160410 con 0x7efc900b5e90 2024-03-29T04:10:06.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.353+0000 7efc95d40640 1 -- 172.21.15.148:0/3923616767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc900687e0 con 0x7efc900b5e90 2024-03-29T04:10:06.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.354+0000 7efc8cff9640 1 -- 172.21.15.148:0/3923616767 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efc78044020 con 0x7efc900b5e90 2024-03-29T04:10:06.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.354+0000 7efc8cff9640 1 --2- 172.21.15.148:0/3923616767 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efc7006c530 0x7efc7006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.354+0000 7efc8cff9640 1 -- 172.21.15.148:0/3923616767 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7efc780acfd0 con 0x7efc900b5e90 2024-03-29T04:10:06.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.354+0000 7efc8f7fe640 1 --2- 172.21.15.148:0/3923616767 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efc7006c530 0x7efc7006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.355+0000 7efc8f7fe640 1 --2- 172.21.15.148:0/3923616767 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efc7006c530 0x7efc7006e9f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efc80001320 tx=0x7efc800012b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:06.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.358+0000 7efc8cff9640 1 -- 172.21.15.148:0/3923616767 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efc7804b070 con 0x7efc900b5e90 2024-03-29T04:10:06.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.537+0000 7efc95d40640 1 -- 172.21.15.148:0/3923616767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc900b4830 con 0x7efc900b5e90 2024-03-29T04:10:06.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.538+0000 7efc8cff9640 1 -- 172.21.15.148:0/3923616767 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7efc9015feb0 con 0x7efc900b5e90 2024-03-29T04:10:06.518 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:06.518 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:05.026481+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:06.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.540+0000 7efc95d40640 1 -- 172.21.15.148:0/3923616767 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efc7006c530 msgr2=0x7efc7006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.540+0000 7efc95d40640 1 --2- 172.21.15.148:0/3923616767 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efc7006c530 0x7efc7006e9f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efc80001320 tx=0x7efc800012b0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.540+0000 7efc95d40640 1 -- 172.21.15.148:0/3923616767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc900b5e90 msgr2=0x7efc900cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.540+0000 7efc95d40640 1 --2- 172.21.15.148:0/3923616767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc900b5e90 0x7efc900cfc00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efc7802b750 tx=0x7efc780046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.540+0000 7efc95d40640 1 -- 172.21.15.148:0/3923616767 shutdown_connections 2024-03-29T04:10:06.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.540+0000 7efc95d40640 1 --2- 172.21.15.148:0/3923616767 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efc7006c530 0x7efc7006e9f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.540+0000 7efc95d40640 1 --2- 172.21.15.148:0/3923616767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc9015e380 0x7efc900d0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.540+0000 7efc95d40640 1 --2- 172.21.15.148:0/3923616767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc900b5e90 0x7efc900cfc00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.540+0000 7efc95d40640 1 --2- 172.21.15.148:0/3923616767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc900b05c0 0x7efc90155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.540+0000 7efc95d40640 1 -- 172.21.15.148:0/3923616767 >> 172.21.15.148:0/3923616767 conn(0x7efc900b2c60 msgr2=0x7efc9006e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:06.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.540+0000 7efc95d40640 1 -- 172.21.15.148:0/3923616767 shutdown_connections 2024-03-29T04:10:06.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.540+0000 7efc95d40640 1 -- 172.21.15.148:0/3923616767 wait complete. 2024-03-29T04:10:06.545 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:10:06.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.669+0000 7fada8655640 1 Processor -- start 2024-03-29T04:10:06.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.669+0000 7fada8655640 1 -- start start 2024-03-29T04:10:06.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.670+0000 7fada8655640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fada0073390 0x7fada015c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.670+0000 7fada8655640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fada00b5e20 0x7fada00cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.670+0000 7fada8655640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fada00739d0 con 0x7fada015aaa0 2024-03-29T04:10:06.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.670+0000 7fada8655640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fada015b9a0 con 0x7fada0073390 2024-03-29T04:10:06.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.670+0000 7fada8655640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fada015bc80 con 0x7fada00b5e20 2024-03-29T04:10:06.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.671+0000 7fada6bf2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fada00b5e20 0x7fada00cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.671+0000 7fada5bf0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fada0073390 0x7fada015c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.671+0000 7fada6bf2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fada00b5e20 0x7fada00cc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.148:54758/0 (socket says 172.21.15.148:54758) 2024-03-29T04:10:06.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.671+0000 7fada6bf2640 1 -- 172.21.15.148:0/4060825450 learned_addr learned my addr 172.21.15.148:0/4060825450 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:06.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.671+0000 7fada6bf2640 1 -- 172.21.15.148:0/4060825450 >> v2:172.21.15.157:3300/0 conn(0x7fada0073390 msgr2=0x7fada015c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:06.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.671+0000 7fada6bf2640 1 --2- 172.21.15.148:0/4060825450 >> v2:172.21.15.157:3300/0 conn(0x7fada0073390 0x7fada015c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.671+0000 7fada6bf2640 1 -- 172.21.15.148:0/4060825450 >> v1:172.21.15.157:6789/0 conn(0x7fada015aaa0 legacy=0x7fada0072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.671+0000 7fada6bf2640 1 -- 172.21.15.148:0/4060825450 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada006b730 con 0x7fada00b5e20 2024-03-29T04:10:06.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.671+0000 7fada5bf0640 1 --2- 172.21.15.148:0/4060825450 >> v2:172.21.15.157:3300/0 conn(0x7fada0073390 0x7fada015c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:06.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.672+0000 7fada6bf2640 1 --2- 172.21.15.148:0/4060825450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fada00b5e20 0x7fada00cc4e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fad98009e50 tx=0x7fad9802bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbb4b4183a686ce8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:06.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.672+0000 7fada53ef640 1 -- 172.21.15.148:0/4060825450 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9802ca70 con 0x7fada00b5e20 2024-03-29T04:10:06.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.672+0000 7fada53ef640 1 -- 172.21.15.148:0/4060825450 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad9802cc10 con 0x7fada00b5e20 2024-03-29T04:10:06.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.672+0000 7fada53ef640 1 -- 172.21.15.148:0/4060825450 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad980343f0 con 0x7fada00b5e20 2024-03-29T04:10:06.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.673+0000 7fada8655640 1 -- 172.21.15.148:0/4060825450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fada00b5e20 msgr2=0x7fada00cc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.673+0000 7fada8655640 1 --2- 172.21.15.148:0/4060825450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fada00b5e20 0x7fada00cc4e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fad98009e50 tx=0x7fad9802bed0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.673+0000 7fada8655640 1 -- 172.21.15.148:0/4060825450 shutdown_connections 2024-03-29T04:10:06.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.673+0000 7fada8655640 1 --2- 172.21.15.148:0/4060825450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fada00b5e20 0x7fada00cc4e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.673+0000 7fada8655640 1 --2- 172.21.15.148:0/4060825450 >> v2:172.21.15.157:3300/0 conn(0x7fada0073390 0x7fada015c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.673+0000 7fada8655640 1 -- 172.21.15.148:0/4060825450 >> 172.21.15.148:0/4060825450 conn(0x7fada00b5050 msgr2=0x7fada00b5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:06.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.673+0000 7fada8655640 1 -- 172.21.15.148:0/4060825450 shutdown_connections 2024-03-29T04:10:06.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.673+0000 7fada8655640 1 -- 172.21.15.148:0/4060825450 wait complete. 2024-03-29T04:10:06.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.674+0000 7fada8655640 1 Processor -- start 2024-03-29T04:10:06.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.674+0000 7fada8655640 1 -- start start 2024-03-29T04:10:06.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.675+0000 7fada8655640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fada0073390 0x7fada01563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.675+0000 7fada8655640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fada00b5e20 0x7fada014d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.675+0000 7fada8655640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fada015aaa0 0x7fada0159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.675+0000 7fada8655640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fada00d0690 con 0x7fada00b5e20 2024-03-29T04:10:06.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.675+0000 7fada8655640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fada00d0510 con 0x7fada015aaa0 2024-03-29T04:10:06.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.675+0000 7fada8655640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fada00d0810 con 0x7fada0073390 2024-03-29T04:10:06.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.675+0000 7fada5bf0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fada00b5e20 0x7fada014d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.675+0000 7fada5bf0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fada00b5e20 0x7fada014d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51206/0 (socket says 172.21.15.148:51206) 2024-03-29T04:10:06.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.675+0000 7fada5bf0640 1 -- 172.21.15.148:0/1027484957 learned_addr learned my addr 172.21.15.148:0/1027484957 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:06.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.675+0000 7fada63f1640 1 --2- 172.21.15.148:0/1027484957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fada0073390 0x7fada01563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.675+0000 7fada6bf2640 1 --2- 172.21.15.148:0/1027484957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fada015aaa0 0x7fada0159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.675+0000 7fada5bf0640 1 -- 172.21.15.148:0/1027484957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fada0073390 msgr2=0x7fada01563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.675+0000 7fada5bf0640 1 --2- 172.21.15.148:0/1027484957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fada0073390 0x7fada01563a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.675+0000 7fada5bf0640 1 -- 172.21.15.148:0/1027484957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fada015aaa0 msgr2=0x7fada0159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.675+0000 7fada5bf0640 1 --2- 172.21.15.148:0/1027484957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fada015aaa0 0x7fada0159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.675+0000 7fada5bf0640 1 -- 172.21.15.148:0/1027484957 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada014e5a0 con 0x7fada00b5e20 2024-03-29T04:10:06.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.675+0000 7fada5bf0640 1 --2- 172.21.15.148:0/1027484957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fada00b5e20 0x7fada014d010 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fad9400ebe0 tx=0x7fad9400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:06.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.675+0000 7fad9effd640 1 -- 172.21.15.148:0/1027484957 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94017070 con 0x7fada00b5e20 2024-03-29T04:10:06.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.675+0000 7fad9effd640 1 -- 172.21.15.148:0/1027484957 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad9400cc30 con 0x7fada00b5e20 2024-03-29T04:10:06.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.675+0000 7fada8655640 1 -- 172.21.15.148:0/1027484957 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada014e880 con 0x7fada00b5e20 2024-03-29T04:10:06.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.675+0000 7fad9effd640 1 -- 172.21.15.148:0/1027484957 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94013470 con 0x7fada00b5e20 2024-03-29T04:10:06.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.675+0000 7fada8655640 1 -- 172.21.15.148:0/1027484957 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada014ee40 con 0x7fada00b5e20 2024-03-29T04:10:06.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.677+0000 7fada8655640 1 -- 172.21.15.148:0/1027484957 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fada00687e0 con 0x7fada00b5e20 2024-03-29T04:10:06.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.677+0000 7fad9effd640 1 -- 172.21.15.148:0/1027484957 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fad9401a070 con 0x7fada00b5e20 2024-03-29T04:10:06.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.677+0000 7fad9effd640 1 --2- 172.21.15.148:0/1027484957 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fad8c06c530 0x7fad8c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.677+0000 7fada63f1640 1 --2- 172.21.15.148:0/1027484957 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fad8c06c530 0x7fad8c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.677+0000 7fad9effd640 1 -- 172.21.15.148:0/1027484957 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fad9408bcd0 con 0x7fada00b5e20 2024-03-29T04:10:06.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.678+0000 7fada63f1640 1 --2- 172.21.15.148:0/1027484957 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fad8c06c530 0x7fad8c06e9f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fad900016e0 tx=0x7fad90009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:06.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:06.682+0000 7fad9effd640 1 -- 172.21.15.148:0/1027484957 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fad940598e0 con 0x7fada00b5e20 2024-03-29T04:10:06.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.001+0000 7fada8655640 1 -- 172.21.15.148:0/1027484957 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fada006d5b0 con 0x7fada00b5e20 2024-03-29T04:10:06.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.001+0000 7fad9effd640 1 -- 172.21.15.148:0/1027484957 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5183 (secure 0 0 0) 0x7fad9405dbe0 con 0x7fada00b5e20 2024-03-29T04:10:06.981 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:06.981 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/36660496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":36660496},{"type":"v1","addr":"172.21.15.157:6834","nonce":36660496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2674481055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2674481055},{"type":"v1","addr":"172.21.15.157:6835","nonce":2674481055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3356449643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3356449643},{"type":"v1","addr":"172.21.15.148:6836","nonce":3356449643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:09:59.886367+0000","modified":"2024-03-29T04:10:05.026739+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/2453181817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2453181817},{"type":"v1","addr":"172.21.15.148:6837","nonce":2453181817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-03-29T04:10:06.981 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 7 2024-03-29T04:10:06.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.003+0000 7fada8655640 1 -- 172.21.15.148:0/1027484957 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fad8c06c530 msgr2=0x7fad8c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.003+0000 7fada8655640 1 --2- 172.21.15.148:0/1027484957 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fad8c06c530 0x7fad8c06e9f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fad900016e0 tx=0x7fad90009040 comp rx=0 tx=0).stop 2024-03-29T04:10:06.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.003+0000 7fada8655640 1 -- 172.21.15.148:0/1027484957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fada00b5e20 msgr2=0x7fada014d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.003+0000 7fada8655640 1 --2- 172.21.15.148:0/1027484957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fada00b5e20 0x7fada014d010 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fad9400ebe0 tx=0x7fad9400c520 comp rx=0 tx=0).stop 2024-03-29T04:10:06.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.003+0000 7fada8655640 1 -- 172.21.15.148:0/1027484957 shutdown_connections 2024-03-29T04:10:06.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.003+0000 7fada8655640 1 --2- 172.21.15.148:0/1027484957 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fad8c06c530 0x7fad8c06e9f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.003+0000 7fada8655640 1 --2- 172.21.15.148:0/1027484957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fada015aaa0 0x7fada0159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.003+0000 7fada8655640 1 --2- 172.21.15.148:0/1027484957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fada00b5e20 0x7fada014d010 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.003+0000 7fada8655640 1 --2- 172.21.15.148:0/1027484957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fada0073390 0x7fada01563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.003+0000 7fada8655640 1 -- 172.21.15.148:0/1027484957 >> 172.21.15.148:0/1027484957 conn(0x7fada00b5050 msgr2=0x7fada0154b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:06.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.004+0000 7fada8655640 1 -- 172.21.15.148:0/1027484957 shutdown_connections 2024-03-29T04:10:06.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.004+0000 7fada8655640 1 -- 172.21.15.148:0/1027484957 wait complete. 2024-03-29T04:10:06.997 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:07.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.119+0000 7f903fe74640 1 Processor -- start 2024-03-29T04:10:07.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.119+0000 7f903fe74640 1 -- start start 2024-03-29T04:10:07.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.120+0000 7f903fe74640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f90380d17e0 0x7f90380d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.120+0000 7f903fe74640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f903815e7c0 con 0x7f903815e3e0 2024-03-29T04:10:07.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.120+0000 7f903fe74640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90380d2c30 con 0x7f90380ba030 2024-03-29T04:10:07.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.120+0000 7f903fe74640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f90380d3840 con 0x7f90380d17e0 2024-03-29T04:10:07.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.120+0000 7f903e411640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f90380d17e0 0x7f90380d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.120+0000 7f903dc10640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f903815e3e0 0x7f90380d2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50184/0 (socket says 172.21.15.148:50184) 2024-03-29T04:10:07.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.120+0000 7f903dc10640 1 -- 172.21.15.148:0/4138990364 learned_addr learned my addr 172.21.15.148:0/4138990364 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:07.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.121+0000 7f903cc0e640 1 -- 172.21.15.148:0/4138990364 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3228628032 0 0) 0x7f903815e7c0 con 0x7f903815e3e0 2024-03-29T04:10:07.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.121+0000 7f903cc0e640 1 -- 172.21.15.148:0/4138990364 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9024003520 con 0x7f903815e3e0 2024-03-29T04:10:07.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.121+0000 7f903e411640 1 -- 172.21.15.148:0/4138990364 >> v1:172.21.15.157:6789/0 conn(0x7f90380ba030 legacy=0x7f90380d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:07.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.121+0000 7f903e411640 1 -- 172.21.15.148:0/4138990364 >> v1:172.21.15.148:6789/0 conn(0x7f903815e3e0 legacy=0x7f90380d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.121+0000 7f903e411640 1 -- 172.21.15.148:0/4138990364 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90380d2f10 con 0x7f90380d17e0 2024-03-29T04:10:07.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.121+0000 7f903e411640 1 --2- 172.21.15.148:0/4138990364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90380d17e0 0x7f90380d3cd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9030009d90 tx=0x7f903002bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509a374d36e1bc88 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:07.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.121+0000 7f903cc0e640 1 -- 172.21.15.148:0/4138990364 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9030039070 con 0x7f90380d17e0 2024-03-29T04:10:07.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.122+0000 7f903cc0e640 1 -- 172.21.15.148:0/4138990364 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f903002cd80 con 0x7f90380d17e0 2024-03-29T04:10:07.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.122+0000 7f903cc0e640 1 -- 172.21.15.148:0/4138990364 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9030034660 con 0x7f90380d17e0 2024-03-29T04:10:07.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.122+0000 7f903fe74640 1 -- 172.21.15.148:0/4138990364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90380d17e0 msgr2=0x7f90380d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.122+0000 7f903fe74640 1 --2- 172.21.15.148:0/4138990364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90380d17e0 0x7f90380d3cd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9030009d90 tx=0x7f903002bfd0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.122+0000 7f903fe74640 1 -- 172.21.15.148:0/4138990364 shutdown_connections 2024-03-29T04:10:07.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.122+0000 7f903fe74640 1 --2- 172.21.15.148:0/4138990364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90380d17e0 0x7f90380d3cd0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.122+0000 7f903fe74640 1 -- 172.21.15.148:0/4138990364 >> 172.21.15.148:0/4138990364 conn(0x7f90380b6210 msgr2=0x7f90380b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:07.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.122+0000 7f903fe74640 1 -- 172.21.15.148:0/4138990364 shutdown_connections 2024-03-29T04:10:07.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.122+0000 7f903fe74640 1 -- 172.21.15.148:0/4138990364 wait complete. 2024-03-29T04:10:07.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.124+0000 7f903fe74640 1 Processor -- start 2024-03-29T04:10:07.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.124+0000 7f903fe74640 1 -- start start 2024-03-29T04:10:07.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.125+0000 7f903fe74640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90380ba030 0x7f9038154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.125+0000 7f903fe74640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90380d17e0 0x7f90380c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.125+0000 7f903fe74640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f903815e3e0 0x7f903806c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.125+0000 7f903fe74640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9038168f60 con 0x7f90380d17e0 2024-03-29T04:10:07.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.125+0000 7f903fe74640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f90380d34d0 con 0x7f90380ba030 2024-03-29T04:10:07.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.125+0000 7f903fe74640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f90381690e0 con 0x7f903815e3e0 2024-03-29T04:10:07.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.125+0000 7f903d40f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90380d17e0 0x7f90380c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.125+0000 7f903d40f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90380d17e0 0x7f90380c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51220/0 (socket says 172.21.15.148:51220) 2024-03-29T04:10:07.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.125+0000 7f903d40f640 1 -- 172.21.15.148:0/2240746911 learned_addr learned my addr 172.21.15.148:0/2240746911 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:07.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.125+0000 7f903dc10640 1 --2- 172.21.15.148:0/2240746911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90380ba030 0x7f9038154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.125+0000 7f903e411640 1 --2- 172.21.15.148:0/2240746911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f903815e3e0 0x7f903806c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.126+0000 7f903d40f640 1 -- 172.21.15.148:0/2240746911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f903815e3e0 msgr2=0x7f903806c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.126+0000 7f903d40f640 1 --2- 172.21.15.148:0/2240746911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f903815e3e0 0x7f903806c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.126+0000 7f903d40f640 1 -- 172.21.15.148:0/2240746911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90380ba030 msgr2=0x7f9038154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.126+0000 7f903d40f640 1 --2- 172.21.15.148:0/2240746911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90380ba030 0x7f9038154340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.126+0000 7f903d40f640 1 -- 172.21.15.148:0/2240746911 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90381572d0 con 0x7f90380d17e0 2024-03-29T04:10:07.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.126+0000 7f903e411640 1 --2- 172.21.15.148:0/2240746911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f903815e3e0 0x7f903806c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:07.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.126+0000 7f903dc10640 1 --2- 172.21.15.148:0/2240746911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90380ba030 0x7f9038154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:07.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.126+0000 7f903d40f640 1 --2- 172.21.15.148:0/2240746911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90380d17e0 0x7f90380c9770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f903400c400 tx=0x7f903400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:07.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.126+0000 7f902e7fc640 1 -- 172.21.15.148:0/2240746911 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90340052d0 con 0x7f90380d17e0 2024-03-29T04:10:07.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.126+0000 7f902e7fc640 1 -- 172.21.15.148:0/2240746911 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9034005490 con 0x7f90380d17e0 2024-03-29T04:10:07.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.126+0000 7f902e7fc640 1 -- 172.21.15.148:0/2240746911 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034007260 con 0x7f90380d17e0 2024-03-29T04:10:07.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.126+0000 7f903fe74640 1 -- 172.21.15.148:0/2240746911 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90381575b0 con 0x7f90380d17e0 2024-03-29T04:10:07.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.126+0000 7f903fe74640 1 -- 172.21.15.148:0/2240746911 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9038154990 con 0x7f90380d17e0 2024-03-29T04:10:07.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.127+0000 7f903fe74640 1 -- 172.21.15.148:0/2240746911 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90380687e0 con 0x7f90380d17e0 2024-03-29T04:10:07.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.127+0000 7f902e7fc640 1 -- 172.21.15.148:0/2240746911 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9034007400 con 0x7f90380d17e0 2024-03-29T04:10:07.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.128+0000 7f902e7fc640 1 --2- 172.21.15.148:0/2240746911 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f901c06c530 0x7f901c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.128+0000 7f902e7fc640 1 -- 172.21.15.148:0/2240746911 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f903408af00 con 0x7f90380d17e0 2024-03-29T04:10:07.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.128+0000 7f903dc10640 1 --2- 172.21.15.148:0/2240746911 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f901c06c530 0x7f901c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.128+0000 7f903dc10640 1 --2- 172.21.15.148:0/2240746911 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f901c06c530 0x7f901c06e9f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f90280011e0 tx=0x7f9028009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:07.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.132+0000 7f902e7fc640 1 -- 172.21.15.148:0/2240746911 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9034059270 con 0x7f90380d17e0 2024-03-29T04:10:07.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.313+0000 7f903fe74640 1 -- 172.21.15.148:0/2240746911 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f90380b8590 con 0x7f90380d17e0 2024-03-29T04:10:07.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.314+0000 7f902e7fc640 1 -- 172.21.15.148:0/2240746911 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7f9034058b70 con 0x7f90380d17e0 2024-03-29T04:10:07.293 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:07.293 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:05.026481+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:07.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.316+0000 7f903fe74640 1 -- 172.21.15.148:0/2240746911 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f901c06c530 msgr2=0x7f901c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.316+0000 7f903fe74640 1 --2- 172.21.15.148:0/2240746911 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f901c06c530 0x7f901c06e9f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f90280011e0 tx=0x7f9028009040 comp rx=0 tx=0).stop 2024-03-29T04:10:07.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.316+0000 7f903fe74640 1 -- 172.21.15.148:0/2240746911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90380d17e0 msgr2=0x7f90380c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.316+0000 7f903fe74640 1 --2- 172.21.15.148:0/2240746911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90380d17e0 0x7f90380c9770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f903400c400 tx=0x7f903400c8c0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.316+0000 7f903fe74640 1 -- 172.21.15.148:0/2240746911 shutdown_connections 2024-03-29T04:10:07.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.316+0000 7f903fe74640 1 --2- 172.21.15.148:0/2240746911 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f901c06c530 0x7f901c06e9f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.316+0000 7f903fe74640 1 --2- 172.21.15.148:0/2240746911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f903815e3e0 0x7f903806c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.316+0000 7f903fe74640 1 --2- 172.21.15.148:0/2240746911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90380d17e0 0x7f90380c9770 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.316+0000 7f903fe74640 1 --2- 172.21.15.148:0/2240746911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90380ba030 0x7f9038154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.316+0000 7f903fe74640 1 -- 172.21.15.148:0/2240746911 >> 172.21.15.148:0/2240746911 conn(0x7f90380b6210 msgr2=0x7f903815a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:07.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.316+0000 7f903fe74640 1 -- 172.21.15.148:0/2240746911 shutdown_connections 2024-03-29T04:10:07.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.316+0000 7f903fe74640 1 -- 172.21.15.148:0/2240746911 wait complete. 2024-03-29T04:10:07.709 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T04:09:59.886367+0000', 'modified': '2024-03-29T04:10:05.026739+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4290}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4290': {'gid': 4290, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6837/2453181817', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 2453181817}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 2453181817}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4290, 'qdb_cluster': [4290]} 2024-03-29T04:10:07.714 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T04:10:07.715 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-29T04:10:07.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.842+0000 7fd6e1e55640 1 Processor -- start 2024-03-29T04:10:07.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.842+0000 7fd6e1e55640 1 -- start start 2024-03-29T04:10:07.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.844+0000 7fd6e1e55640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd6dc06c6c0 0x7fd6dc06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.844+0000 7fd6e1e55640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6dc0b5280 0x7fd6dc0b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.844+0000 7fd6e1e55640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6dc0cd9e0 con 0x7fd6dc0cd600 2024-03-29T04:10:07.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.844+0000 7fd6e1e55640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd6dc0b5c10 con 0x7fd6dc06c6c0 2024-03-29T04:10:07.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.844+0000 7fd6e1e55640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd6dc06d800 con 0x7fd6dc0b5280 2024-03-29T04:10:07.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.844+0000 7fd6dbfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6dc0b5280 0x7fd6dc0b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.844+0000 7fd6daffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd6dc06c6c0 0x7fd6dc06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.844+0000 7fd6daffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd6dc06c6c0 0x7fd6dc06dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:37426/0 (socket says 172.21.15.148:37426) 2024-03-29T04:10:07.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.844+0000 7fd6daffd640 1 -- 172.21.15.148:0/3444511352 learned_addr learned my addr 172.21.15.148:0/3444511352 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:07.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.844+0000 7fd6dbfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6dc0b5280 0x7fd6dc0b56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.148:54790/0 (socket says 172.21.15.148:54790) 2024-03-29T04:10:07.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.844+0000 7fd6dbfff640 1 -- 172.21.15.148:0/3444511352 >> v2:172.21.15.157:3300/0 conn(0x7fd6dc06c6c0 msgr2=0x7fd6dc06dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:07.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.844+0000 7fd6dbfff640 1 --2- 172.21.15.148:0/3444511352 >> v2:172.21.15.157:3300/0 conn(0x7fd6dc06c6c0 0x7fd6dc06dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.844+0000 7fd6dbfff640 1 -- 172.21.15.148:0/3444511352 >> v1:172.21.15.157:6789/0 conn(0x7fd6dc0cd600 legacy=0x7fd6dc15c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.844+0000 7fd6dbfff640 1 -- 172.21.15.148:0/3444511352 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6dc06cbf0 con 0x7fd6dc0b5280 2024-03-29T04:10:07.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.844+0000 7fd6daffd640 1 --2- 172.21.15.148:0/3444511352 >> v2:172.21.15.157:3300/0 conn(0x7fd6dc06c6c0 0x7fd6dc06dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:07.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.844+0000 7fd6da7fc640 1 -- 172.21.15.148:0/3444511352 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2004634921 0 0) 0x7fd6dc0cd9e0 con 0x7fd6dc0cd600 2024-03-29T04:10:07.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.845+0000 7fd6dbfff640 1 --2- 172.21.15.148:0/3444511352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6dc0b5280 0x7fd6dc0b56d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009d90 tx=0x7fd6d402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c622eb4524a3938 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:07.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.845+0000 7fd6da7fc640 1 -- 172.21.15.148:0/3444511352 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4039070 con 0x7fd6dc0b5280 2024-03-29T04:10:07.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.845+0000 7fd6da7fc640 1 -- 172.21.15.148:0/3444511352 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6d402cc50 con 0x7fd6dc0b5280 2024-03-29T04:10:07.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.845+0000 7fd6da7fc640 1 -- 172.21.15.148:0/3444511352 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4034440 con 0x7fd6dc0b5280 2024-03-29T04:10:07.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.846+0000 7fd6e1e55640 1 -- 172.21.15.148:0/3444511352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6dc0b5280 msgr2=0x7fd6dc0b56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.846+0000 7fd6e1e55640 1 --2- 172.21.15.148:0/3444511352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6dc0b5280 0x7fd6dc0b56d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009d90 tx=0x7fd6d402bea0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.846+0000 7fd6e1e55640 1 -- 172.21.15.148:0/3444511352 shutdown_connections 2024-03-29T04:10:07.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.846+0000 7fd6e1e55640 1 --2- 172.21.15.148:0/3444511352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6dc0b5280 0x7fd6dc0b56d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.846+0000 7fd6e1e55640 1 --2- 172.21.15.148:0/3444511352 >> v2:172.21.15.157:3300/0 conn(0x7fd6dc06c6c0 0x7fd6dc06dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.846+0000 7fd6e1e55640 1 -- 172.21.15.148:0/3444511352 >> 172.21.15.148:0/3444511352 conn(0x7fd6dc0b2c60 msgr2=0x7fd6dc0b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:07.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.846+0000 7fd6e1e55640 1 -- 172.21.15.148:0/3444511352 shutdown_connections 2024-03-29T04:10:07.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.846+0000 7fd6e1e55640 1 -- 172.21.15.148:0/3444511352 wait complete. 2024-03-29T04:10:07.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.847+0000 7fd6e1e55640 1 Processor -- start 2024-03-29T04:10:07.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.847+0000 7fd6e1e55640 1 -- start start 2024-03-29T04:10:07.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.848+0000 7fd6e1e55640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6dc06c6c0 0x7fd6dc14c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.848+0000 7fd6e1e55640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6dc0b5280 0x7fd6dc145a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.848+0000 7fd6e1e55640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6dc0cd600 0x7fd6dc06b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.848+0000 7fd6e1e55640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc06d490 con 0x7fd6dc0b5280 2024-03-29T04:10:07.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.848+0000 7fd6e1e55640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc06d1b0 con 0x7fd6dc06c6c0 2024-03-29T04:10:07.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.848+0000 7fd6e1e55640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd6dc15c880 con 0x7fd6dc0cd600 2024-03-29T04:10:07.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.848+0000 7fd6daffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6dc0b5280 0x7fd6dc145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.848+0000 7fd6daffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6dc0b5280 0x7fd6dc145a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51226/0 (socket says 172.21.15.148:51226) 2024-03-29T04:10:07.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.848+0000 7fd6daffd640 1 -- 172.21.15.148:0/1284903205 learned_addr learned my addr 172.21.15.148:0/1284903205 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:07.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.848+0000 7fd6dbfff640 1 --2- 172.21.15.148:0/1284903205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6dc0cd600 0x7fd6dc06b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.848+0000 7fd6db7fe640 1 --2- 172.21.15.148:0/1284903205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6dc06c6c0 0x7fd6dc14c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.848+0000 7fd6daffd640 1 -- 172.21.15.148:0/1284903205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6dc0cd600 msgr2=0x7fd6dc06b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.848+0000 7fd6daffd640 1 --2- 172.21.15.148:0/1284903205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6dc0cd600 0x7fd6dc06b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.848+0000 7fd6daffd640 1 -- 172.21.15.148:0/1284903205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6dc06c6c0 msgr2=0x7fd6dc14c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.848+0000 7fd6daffd640 1 --2- 172.21.15.148:0/1284903205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6dc06c6c0 0x7fd6dc14c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.848+0000 7fd6daffd640 1 -- 172.21.15.148:0/1284903205 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6dc14cc70 con 0x7fd6dc0b5280 2024-03-29T04:10:07.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.848+0000 7fd6db7fe640 1 --2- 172.21.15.148:0/1284903205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6dc06c6c0 0x7fd6dc14c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:07.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.849+0000 7fd6daffd640 1 --2- 172.21.15.148:0/1284903205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6dc0b5280 0x7fd6dc145a90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd6d000e9a0 tx=0x7fd6d000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:07.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.849+0000 7fd6c3fff640 1 -- 172.21.15.148:0/1284903205 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0017070 con 0x7fd6dc0b5280 2024-03-29T04:10:07.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.849+0000 7fd6c3fff640 1 -- 172.21.15.148:0/1284903205 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6d000ca00 con 0x7fd6dc0b5280 2024-03-29T04:10:07.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.849+0000 7fd6e1e55640 1 -- 172.21.15.148:0/1284903205 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6dc1f8350 con 0x7fd6dc0b5280 2024-03-29T04:10:07.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.849+0000 7fd6c3fff640 1 -- 172.21.15.148:0/1284903205 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d000cdb0 con 0x7fd6dc0b5280 2024-03-29T04:10:07.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.849+0000 7fd6e1e55640 1 -- 172.21.15.148:0/1284903205 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6dc1f7870 con 0x7fd6dc0b5280 2024-03-29T04:10:07.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.850+0000 7fd6c3fff640 1 -- 172.21.15.148:0/1284903205 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd6d001a070 con 0x7fd6dc0b5280 2024-03-29T04:10:07.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.851+0000 7fd6c3fff640 1 --2- 172.21.15.148:0/1284903205 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd6c806c530 0x7fd6c806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.851+0000 7fd6c3fff640 1 -- 172.21.15.148:0/1284903205 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd6d008b7f0 con 0x7fd6dc0b5280 2024-03-29T04:10:07.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.851+0000 7fd6db7fe640 1 --2- 172.21.15.148:0/1284903205 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd6c806c530 0x7fd6c806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.851+0000 7fd6db7fe640 1 --2- 172.21.15.148:0/1284903205 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd6c806c530 0x7fd6c806e9f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc000f60 tx=0x7fd6cc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:07.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.852+0000 7fd6e4a4a740 1 Processor -- start 2024-03-29T04:10:07.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.852+0000 7fd6e4a4a740 1 -- start start 2024-03-29T04:10:07.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.853+0000 7fd6e4a4a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e113b860 0x5608e110e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.853+0000 7fd6e4a4a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608e113bc40 0x5608e113e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.853+0000 7fd6e4a4a740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5608e110ed40 0x5608e113eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.853+0000 7fd6e4a4a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5608e1078020 con 0x5608e113bc40 2024-03-29T04:10:07.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.853+0000 7fd6e4a4a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5608e10205f0 con 0x5608e113b860 2024-03-29T04:10:07.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.853+0000 7fd6e4a4a740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x5608e1073fd0 con 0x5608e110ed40 2024-03-29T04:10:07.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.853+0000 7fd6daffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608e113bc40 0x5608e113e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.853+0000 7fd6daffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608e113bc40 0x5608e113e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51242/0 (socket says 172.21.15.148:51242) 2024-03-29T04:10:07.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.853+0000 7fd6daffd640 1 -- 172.21.15.148:0/1926038120 learned_addr learned my addr 172.21.15.148:0/1926038120 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:07.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.853+0000 7fd6daffd640 1 -- 172.21.15.148:0/1926038120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5608e110ed40 msgr2=0x5608e113eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:07.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.853+0000 7fd6daffd640 1 --2- 172.21.15.148:0/1926038120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5608e110ed40 0x5608e113eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.853+0000 7fd6daffd640 1 -- 172.21.15.148:0/1926038120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e113b860 msgr2=0x5608e110e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:07.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.853+0000 7fd6daffd640 1 --2- 172.21.15.148:0/1926038120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e113b860 0x5608e110e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.853+0000 7fd6daffd640 1 -- 172.21.15.148:0/1926038120 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608e0d8c2d0 con 0x5608e113bc40 2024-03-29T04:10:07.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.854+0000 7fd6daffd640 1 --2- 172.21.15.148:0/1926038120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608e113bc40 0x5608e113e5f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0061b10 tx=0x7fd6d00595d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acea9156042fc36b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:07.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.854+0000 7fd6c17fa640 1 -- 172.21.15.148:0/1926038120 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0021040 con 0x5608e113bc40 2024-03-29T04:10:07.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.854+0000 7fd6c17fa640 1 -- 172.21.15.148:0/1926038120 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6d00575b0 con 0x5608e113bc40 2024-03-29T04:10:07.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.854+0000 7fd6c17fa640 1 -- 172.21.15.148:0/1926038120 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0055b80 con 0x5608e113bc40 2024-03-29T04:10:07.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.855+0000 7fd6e4a4a740 1 -- 172.21.15.148:0/1926038120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608e113bc40 msgr2=0x5608e113e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.855+0000 7fd6e4a4a740 1 --2- 172.21.15.148:0/1926038120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608e113bc40 0x5608e113e5f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0061b10 tx=0x7fd6d00595d0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.855+0000 7fd6e4a4a740 1 -- 172.21.15.148:0/1926038120 shutdown_connections 2024-03-29T04:10:07.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.855+0000 7fd6e4a4a740 1 --2- 172.21.15.148:0/1926038120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5608e110ed40 0x5608e113eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.855+0000 7fd6e4a4a740 1 --2- 172.21.15.148:0/1926038120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608e113bc40 0x5608e113e5f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.855+0000 7fd6e4a4a740 1 --2- 172.21.15.148:0/1926038120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e113b860 0x5608e110e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.855+0000 7fd6e4a4a740 1 -- 172.21.15.148:0/1926038120 >> 172.21.15.148:0/1926038120 conn(0x5608e1110a70 msgr2=0x5608e11595f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:07.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.855+0000 7fd6e4a4a740 1 -- 172.21.15.148:0/1926038120 shutdown_connections 2024-03-29T04:10:07.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.855+0000 7fd6e4a4a740 1 -- 172.21.15.148:0/1926038120 wait complete. 2024-03-29T04:10:07.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.856+0000 7fd6e4a4a740 1 -- start start 2024-03-29T04:10:07.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.856+0000 7fd6e4a4a740 1 Processor -- start 2024-03-29T04:10:07.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.857+0000 7fd6e4a4a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608e113b860 0x5608e1169cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.857+0000 7fd6e4a4a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e113bc40 0x5608e116a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.857+0000 7fd6e4a4a740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5608e11588c0 0x5608e1158d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.857+0000 7fd6e4a4a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5608e1113110 con 0x5608e113b860 2024-03-29T04:10:07.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.857+0000 7fd6e4a4a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5608e10339c0 con 0x5608e113bc40 2024-03-29T04:10:07.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.857+0000 7fd6e4a4a740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x5608e1115950 con 0x5608e11588c0 2024-03-29T04:10:07.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.857+0000 7fd6e4a4a740 10 client.0 _ll_register_callbacks cb 0x5608e1017fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:10:07.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.857+0000 7fd6db7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608e113b860 0x5608e1169cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.857+0000 7fd6db7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608e113b860 0x5608e1169cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51244/0 (socket says 172.21.15.148:51244) 2024-03-29T04:10:07.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.857+0000 7fd6db7fe640 1 -- 172.21.15.148:0/4053595897 learned_addr learned my addr 172.21.15.148:0/4053595897 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:07.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.857+0000 7fd6daffd640 1 --2- 172.21.15.148:0/4053595897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e113bc40 0x5608e116a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.857+0000 7fd6dbfff640 1 --2- 172.21.15.148:0/4053595897 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5608e11588c0 0x5608e1158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.857+0000 7fd6db7fe640 1 -- 172.21.15.148:0/4053595897 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5608e11588c0 msgr2=0x5608e1158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:07.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.857+0000 7fd6db7fe640 1 --2- 172.21.15.148:0/4053595897 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5608e11588c0 0x5608e1158d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.857+0000 7fd6db7fe640 1 -- 172.21.15.148:0/4053595897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e113bc40 msgr2=0x5608e116a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:07.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.857+0000 7fd6db7fe640 1 --2- 172.21.15.148:0/4053595897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e113bc40 0x5608e116a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.857+0000 7fd6db7fe640 1 -- 172.21.15.148:0/4053595897 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608e0e566b0 con 0x5608e113b860 2024-03-29T04:10:07.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.858+0000 7fd6db7fe640 1 --2- 172.21.15.148:0/4053595897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608e113b860 0x5608e1169cc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc001ea0 tx=0x7fd6cc00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3567628ad7505742 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:07.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.858+0000 7fd6ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-03-29T04:10:07.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.858+0000 7fd6ba7fc640 1 -- 172.21.15.148:0/4053595897 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc003af0 con 0x5608e113b860 2024-03-29T04:10:07.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.858+0000 7fd6e4a4a740 1 -- 172.21.15.148:0/4053595897 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6d00575b0 con 0x5608e113b860 2024-03-29T04:10:07.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.858+0000 7fd6ba7fc640 1 -- 172.21.15.148:0/4053595897 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd6cc003c90 con 0x5608e113b860 2024-03-29T04:10:07.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.858+0000 7fd6ba7fc640 1 -- 172.21.15.148:0/4053595897 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fd6cc013420 con 0x5608e113b860 2024-03-29T04:10:07.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.858+0000 7fd6e4a4a740 10 client.4443 fetch_fsmap learned FSMap version 7 2024-03-29T04:10:07.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.858+0000 7fd6e4a4a740 1 -- 172.21.15.148:0/4053595897 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x5608e1141610 con 0x5608e113b860 2024-03-29T04:10:07.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.858+0000 7fd6ba7fc640 1 -- 172.21.15.148:0/4053595897 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6d00575b0 con 0x5608e113b860 2024-03-29T04:10:07.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.858+0000 7fd6ba7fc640 1 -- 172.21.15.148:0/4053595897 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7fd6cc02a050 con 0x5608e113b860 2024-03-29T04:10:07.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.858+0000 7fd6e4a4a740 10 client.4443 fetch_fsmap finished waiting for FSMap version 7 2024-03-29T04:10:07.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.858+0000 7fd6e4a4a740 10 client.4443 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-29T04:10:07.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.858+0000 7fd6e4a4a740 20 client.4443 populate_metadata read hostname 'smithi148' 2024-03-29T04:10:07.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.858+0000 7fd6e4a4a740 1 --2- 172.21.15.148:0/4053595897 >> [v2:172.21.15.148:6835/2453181817,v1:172.21.15.148:6837/2453181817] conn(0x5608e113fd90 0x5608e1140170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.858+0000 7fd6e4a4a740 4 client.4443 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T04:10:07.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.858+0000 7fd6e4a4a740 1 -- 172.21.15.148:0/4053595897 --> [v2:172.21.15.148:6835/2453181817,v1:172.21.15.148:6837/2453181817] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608e104c4e0 con 0x5608e113fd90 2024-03-29T04:10:07.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.858+0000 7fd6daffd640 1 --2- 172.21.15.148:0/4053595897 >> [v2:172.21.15.148:6835/2453181817,v1:172.21.15.148:6837/2453181817] conn(0x5608e113fd90 0x5608e1140170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.859+0000 7fd6ba7fc640 10 client.4443 ms_handle_connect on v2:172.21.15.148:6835/2453181817 2024-03-29T04:10:07.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.859+0000 7fd6daffd640 1 --2- 172.21.15.148:0/4053595897 >> [v2:172.21.15.148:6835/2453181817,v1:172.21.15.148:6837/2453181817] conn(0x5608e113fd90 0x5608e1140170 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=967bbb25d08af0e3 server_cookie=b0773127809e2e1e in_seq=0 out_seq=0 2024-03-29T04:10:07.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.860+0000 7fd6ba7fc640 1 -- 172.21.15.148:0/4053595897 <== mds.0 v2:172.21.15.148:6835/2453181817 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x5608e104c4e0 con 0x5608e113fd90 2024-03-29T04:10:07.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.860+0000 7fd6ba7fc640 10 client.4443 handle_command_reply: tid=0 2024-03-29T04:10:07.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.860+0000 7fd6e4a4a740 1 client.4443 shutdown 2024-03-29T04:10:07.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.860+0000 7fd6e4a4a740 1 -- 172.21.15.148:0/4053595897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608e113b860 msgr2=0x5608e1169cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.860+0000 7fd6e4a4a740 1 --2- 172.21.15.148:0/4053595897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608e113b860 0x5608e1169cc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc001ea0 tx=0x7fd6cc00eb80 comp rx=0 tx=0).stop 2024-03-29T04:10:07.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.860+0000 7fd6e4a4a740 1 -- 172.21.15.148:0/4053595897 shutdown_connections 2024-03-29T04:10:07.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.860+0000 7fd6e4a4a740 1 --2- 172.21.15.148:0/4053595897 >> [v2:172.21.15.148:6835/2453181817,v1:172.21.15.148:6837/2453181817] conn(0x5608e113fd90 0x5608e1140170 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.860+0000 7fd6e4a4a740 1 --2- 172.21.15.148:0/4053595897 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5608e11588c0 0x5608e1158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.860+0000 7fd6e4a4a740 1 --2- 172.21.15.148:0/4053595897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e113bc40 0x5608e116a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.860+0000 7fd6e4a4a740 1 --2- 172.21.15.148:0/4053595897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608e113b860 0x5608e1169cc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.860+0000 7fd6e4a4a740 1 -- 172.21.15.148:0/4053595897 >> 172.21.15.148:0/4053595897 conn(0x5608e110ed40 msgr2=0x5608e1112710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:07.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.860+0000 7fd6ba7fc640 0 client.4443 ms_handle_reset on v2:172.21.15.148:6835/2453181817 2024-03-29T04:10:07.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.860+0000 7fd6e4a4a740 1 -- 172.21.15.148:0/4053595897 shutdown_connections 2024-03-29T04:10:07.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.860+0000 7fd6e4a4a740 1 -- 172.21.15.148:0/4053595897 wait complete. 2024-03-29T04:10:07.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.860+0000 7fd6e4a4a740 20 client.4443 trim_cache size 0 max 16384 2024-03-29T04:10:07.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.878+0000 7fd6e4a4a740 1 Processor -- start 2024-03-29T04:10:07.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.878+0000 7fd6e4a4a740 1 -- start start 2024-03-29T04:10:07.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.879+0000 7fd6e4a4a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608e113b860 0x5608e1156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.879+0000 7fd6e4a4a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e113bc40 0x5608e1140170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.879+0000 7fd6e4a4a740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5608e11588c0 0x5608e11406b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.879+0000 7fd6e4a4a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc200da0 con 0x5608e113b860 2024-03-29T04:10:07.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.879+0000 7fd6e4a4a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc200150 con 0x5608e113bc40 2024-03-29T04:10:07.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.879+0000 7fd6e4a4a740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd6dc200750 con 0x5608e11588c0 2024-03-29T04:10:07.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.879+0000 7fd6db7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608e113b860 0x5608e1156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.879+0000 7fd6db7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608e113b860 0x5608e1156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51260/0 (socket says 172.21.15.148:51260) 2024-03-29T04:10:07.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.879+0000 7fd6db7fe640 1 -- 172.21.15.148:0/3518620806 learned_addr learned my addr 172.21.15.148:0/3518620806 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:07.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.879+0000 7fd6db7fe640 1 -- 172.21.15.148:0/3518620806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5608e11588c0 msgr2=0x5608e11406b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:07.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.879+0000 7fd6dbfff640 1 --2- 172.21.15.148:0/3518620806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5608e11588c0 0x5608e11406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.879+0000 7fd6daffd640 1 --2- 172.21.15.148:0/3518620806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e113bc40 0x5608e1140170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.879+0000 7fd6db7fe640 1 --2- 172.21.15.148:0/3518620806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5608e11588c0 0x5608e11406b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.879+0000 7fd6db7fe640 1 -- 172.21.15.148:0/3518620806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e113bc40 msgr2=0x5608e1140170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:07.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.879+0000 7fd6db7fe640 1 --2- 172.21.15.148:0/3518620806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e113bc40 0x5608e1140170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.879+0000 7fd6db7fe640 1 -- 172.21.15.148:0/3518620806 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608e116ae40 con 0x5608e113b860 2024-03-29T04:10:07.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.880+0000 7fd6db7fe640 1 --2- 172.21.15.148:0/3518620806 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608e113b860 0x5608e1156910 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc003f90 tx=0x7fd6cc01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda9af0787a13496 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:07.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.880+0000 7fd6c0ff9640 1 -- 172.21.15.148:0/3518620806 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc0135c0 con 0x5608e113b860 2024-03-29T04:10:07.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.880+0000 7fd6c0ff9640 1 -- 172.21.15.148:0/3518620806 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6cc0039c0 con 0x5608e113b860 2024-03-29T04:10:07.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.880+0000 7fd6c0ff9640 1 -- 172.21.15.148:0/3518620806 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc0138c0 con 0x5608e113b860 2024-03-29T04:10:07.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.880+0000 7fd6e4a4a740 1 -- 172.21.15.148:0/3518620806 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608e113b860 msgr2=0x5608e1156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.880+0000 7fd6e4a4a740 1 --2- 172.21.15.148:0/3518620806 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608e113b860 0x5608e1156910 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc003f90 tx=0x7fd6cc01d000 comp rx=0 tx=0).stop 2024-03-29T04:10:07.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.880+0000 7fd6e4a4a740 1 -- 172.21.15.148:0/3518620806 shutdown_connections 2024-03-29T04:10:07.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.880+0000 7fd6e4a4a740 1 --2- 172.21.15.148:0/3518620806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5608e11588c0 0x5608e11406b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.880+0000 7fd6e4a4a740 1 --2- 172.21.15.148:0/3518620806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e113bc40 0x5608e1140170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.881+0000 7fd6e4a4a740 1 --2- 172.21.15.148:0/3518620806 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608e113b860 0x5608e1156910 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.881+0000 7fd6e4a4a740 1 -- 172.21.15.148:0/3518620806 >> 172.21.15.148:0/3518620806 conn(0x5608e110ed40 msgr2=0x5608e1152940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:07.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.881+0000 7fd6e4a4a740 1 -- 172.21.15.148:0/3518620806 shutdown_connections 2024-03-29T04:10:07.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.881+0000 7fd6e4a4a740 1 -- 172.21.15.148:0/3518620806 wait complete. 2024-03-29T04:10:07.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.881+0000 7fd6e4a4a740 1 -- start start 2024-03-29T04:10:07.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.881+0000 7fd6e4a4a740 1 Processor -- start 2024-03-29T04:10:07.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.882+0000 7fd6e4a4a740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5608e113b860 0x5608e1140170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.882+0000 7fd6e4a4a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e113bc40 0x5608e11406b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.882+0000 7fd6e4a4a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608e11588c0 0x5608e115aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.882+0000 7fd6e4a4a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5608e113b250 con 0x5608e11588c0 2024-03-29T04:10:07.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.882+0000 7fd6e4a4a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5608e116b020 con 0x5608e113bc40 2024-03-29T04:10:07.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.882+0000 7fd6e4a4a740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x5608e11420f0 con 0x5608e113b860 2024-03-29T04:10:07.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.882+0000 7fd6e4a4a740 10 client.0 _ll_register_callbacks cb 0x7fd6cc001380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:10:07.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.882+0000 7fd6dbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608e11588c0 0x5608e115aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.882+0000 7fd6dbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608e11588c0 0x5608e115aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51262/0 (socket says 172.21.15.148:51262) 2024-03-29T04:10:07.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.882+0000 7fd6dbfff640 1 -- 172.21.15.148:0/1976529717 learned_addr learned my addr 172.21.15.148:0/1976529717 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:07.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.882+0000 7fd6daffd640 1 --2- 172.21.15.148:0/1976529717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e113bc40 0x5608e11406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.882+0000 7fd6db7fe640 1 --2- 172.21.15.148:0/1976529717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5608e113b860 0x5608e1140170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.882+0000 7fd6dbfff640 1 -- 172.21.15.148:0/1976529717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5608e113b860 msgr2=0x5608e1140170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:07.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.882+0000 7fd6dbfff640 1 --2- 172.21.15.148:0/1976529717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5608e113b860 0x5608e1140170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.882+0000 7fd6dbfff640 1 -- 172.21.15.148:0/1976529717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e113bc40 msgr2=0x5608e11406b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:07.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.882+0000 7fd6dbfff640 1 --2- 172.21.15.148:0/1976529717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e113bc40 0x5608e11406b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.882+0000 7fd6dbfff640 1 -- 172.21.15.148:0/1976529717 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608e1173550 con 0x5608e11588c0 2024-03-29T04:10:07.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.883+0000 7fd6dbfff640 1 --2- 172.21.15.148:0/1976529717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608e11588c0 0x5608e115aa00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4037040 tx=0x7fd6d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b605b0998707a91 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:07.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.883+0000 7fd6ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-03-29T04:10:07.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.883+0000 7fd6ba7fc640 1 -- 172.21.15.148:0/1976529717 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4039040 con 0x5608e11588c0 2024-03-29T04:10:07.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.883+0000 7fd6ba7fc640 1 -- 172.21.15.148:0/1976529717 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd6d4034b30 con 0x5608e11588c0 2024-03-29T04:10:07.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.883+0000 7fd6e4a4a740 1 -- 172.21.15.148:0/1976529717 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6cc0039c0 con 0x5608e11588c0 2024-03-29T04:10:07.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.883+0000 7fd6ba7fc640 1 -- 172.21.15.148:0/1976529717 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fd6d4053050 con 0x5608e11588c0 2024-03-29T04:10:07.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.883+0000 7fd6e4a4a740 10 client.4449 fetch_fsmap learned FSMap version 7 2024-03-29T04:10:07.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.883+0000 7fd6e4a4a740 1 -- 172.21.15.148:0/1976529717 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x5608e1138d50 con 0x5608e11588c0 2024-03-29T04:10:07.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.883+0000 7fd6ba7fc640 1 -- 172.21.15.148:0/1976529717 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6cc0039c0 con 0x5608e11588c0 2024-03-29T04:10:07.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.883+0000 7fd6ba7fc640 1 -- 172.21.15.148:0/1976529717 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7fd6d4056020 con 0x5608e11588c0 2024-03-29T04:10:07.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.883+0000 7fd6e4a4a740 10 client.4449 fetch_fsmap finished waiting for FSMap version 7 2024-03-29T04:10:07.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.883+0000 7fd6e4a4a740 10 client.4449 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-29T04:10:07.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.883+0000 7fd6e4a4a740 20 client.4449 populate_metadata read hostname 'smithi148' 2024-03-29T04:10:07.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.883+0000 7fd6e4a4a740 1 --2- 172.21.15.148:0/1976529717 >> [v2:172.21.15.148:6835/2453181817,v1:172.21.15.148:6837/2453181817] conn(0x5608e114ef80 0x5608e114f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.883+0000 7fd6e4a4a740 4 client.4449 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T04:10:07.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.883+0000 7fd6e4a4a740 1 -- 172.21.15.148:0/1976529717 --> [v2:172.21.15.148:6835/2453181817,v1:172.21.15.148:6837/2453181817] -- command(tid 0: {"prefix": "status"}) -- 0x5608e1085560 con 0x5608e114ef80 2024-03-29T04:10:07.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.883+0000 7fd6db7fe640 1 --2- 172.21.15.148:0/1976529717 >> [v2:172.21.15.148:6835/2453181817,v1:172.21.15.148:6837/2453181817] conn(0x5608e114ef80 0x5608e114f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.884+0000 7fd6ba7fc640 10 client.4449 ms_handle_connect on v2:172.21.15.148:6835/2453181817 2024-03-29T04:10:07.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.884+0000 7fd6db7fe640 1 --2- 172.21.15.148:0/1976529717 >> [v2:172.21.15.148:6835/2453181817,v1:172.21.15.148:6837/2453181817] conn(0x5608e114ef80 0x5608e114f360 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5f9ac98bd81ab9d server_cookie=25810573dd1657a7 in_seq=0 out_seq=0 2024-03-29T04:10:07.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.884+0000 7fd6ba7fc640 1 -- 172.21.15.148:0/1976529717 <== mds.0 v2:172.21.15.148:6835/2453181817 1 ==== command_reply(tid 0: 0 ) ==== 8+0+317 (crc 0 0 0) 0x5608e1085560 con 0x5608e114ef80 2024-03-29T04:10:07.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.884+0000 7fd6ba7fc640 10 client.4449 handle_command_reply: tid=0 2024-03-29T04:10:07.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.884+0000 7fd6e4a4a740 1 client.4449 shutdown 2024-03-29T04:10:07.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.884+0000 7fd6e4a4a740 1 -- 172.21.15.148:0/1976529717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608e11588c0 msgr2=0x5608e115aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.884+0000 7fd6e4a4a740 1 --2- 172.21.15.148:0/1976529717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608e11588c0 0x5608e115aa00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4037040 tx=0x7fd6d4004a40 comp rx=0 tx=0).stop 2024-03-29T04:10:07.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.884+0000 7fd6e4a4a740 1 -- 172.21.15.148:0/1976529717 shutdown_connections 2024-03-29T04:10:07.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.884+0000 7fd6e4a4a740 1 --2- 172.21.15.148:0/1976529717 >> [v2:172.21.15.148:6835/2453181817,v1:172.21.15.148:6837/2453181817] conn(0x5608e114ef80 0x5608e114f360 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.884+0000 7fd6e4a4a740 1 --2- 172.21.15.148:0/1976529717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5608e11588c0 0x5608e115aa00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.884+0000 7fd6ba7fc640 0 client.4449 ms_handle_reset on v2:172.21.15.148:6835/2453181817 2024-03-29T04:10:07.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.884+0000 7fd6e4a4a740 1 --2- 172.21.15.148:0/1976529717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608e113bc40 0x5608e11406b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.884+0000 7fd6e4a4a740 1 --2- 172.21.15.148:0/1976529717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5608e113b860 0x5608e1140170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.884+0000 7fd6e4a4a740 1 -- 172.21.15.148:0/1976529717 >> 172.21.15.148:0/1976529717 conn(0x5608e110ed40 msgr2=0x5608e11517d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:07.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.884+0000 7fd6e4a4a740 1 -- 172.21.15.148:0/1976529717 shutdown_connections 2024-03-29T04:10:07.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.884+0000 7fd6e4a4a740 1 -- 172.21.15.148:0/1976529717 wait complete. 2024-03-29T04:10:07.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.884+0000 7fd6e4a4a740 20 client.4449 trim_cache size 0 max 16384 2024-03-29T04:10:07.870 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-03-29T04:10:07.870 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "56b659bb-6db8-45e8-b612-135349c3fd6b", 2024-03-29T04:10:07.870 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2024-03-29T04:10:07.870 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 4290, 2024-03-29T04:10:07.870 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2024-03-29T04:10:07.870 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2024-03-29T04:10:07.870 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2024-03-29T04:10:07.870 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 7.347452627, 2024-03-29T04:10:07.870 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 7, 2024-03-29T04:10:07.870 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 24, 2024-03-29T04:10:07.870 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 24, 2024-03-29T04:10:07.870 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 14.398949385 2024-03-29T04:10:07.871 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-03-29T04:10:07.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.885+0000 7fd6e1e55640 1 -- 172.21.15.148:0/1284903205 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd6c806c530 msgr2=0x7fd6c806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.885+0000 7fd6e1e55640 1 --2- 172.21.15.148:0/1284903205 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd6c806c530 0x7fd6c806e9f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc000f60 tx=0x7fd6cc009040 comp rx=0 tx=0).stop 2024-03-29T04:10:07.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.885+0000 7fd6e1e55640 1 -- 172.21.15.148:0/1284903205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6dc0b5280 msgr2=0x7fd6dc145a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.885+0000 7fd6e1e55640 1 --2- 172.21.15.148:0/1284903205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6dc0b5280 0x7fd6dc145a90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd6d000e9a0 tx=0x7fd6d000ee60 comp rx=0 tx=0).stop 2024-03-29T04:10:07.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.885+0000 7fd6e1e55640 1 -- 172.21.15.148:0/1284903205 shutdown_connections 2024-03-29T04:10:07.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.885+0000 7fd6e1e55640 1 --2- 172.21.15.148:0/1284903205 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd6c806c530 0x7fd6c806e9f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.885+0000 7fd6e1e55640 1 --2- 172.21.15.148:0/1284903205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6dc0cd600 0x7fd6dc06b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.885+0000 7fd6e1e55640 1 --2- 172.21.15.148:0/1284903205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6dc0b5280 0x7fd6dc145a90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.885+0000 7fd6e1e55640 1 --2- 172.21.15.148:0/1284903205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6dc06c6c0 0x7fd6dc14c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.885+0000 7fd6e1e55640 1 -- 172.21.15.148:0/1284903205 >> 172.21.15.148:0/1284903205 conn(0x7fd6dc0b2c60 msgr2=0x7fd6dc0b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:07.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.885+0000 7fd6e1e55640 1 -- 172.21.15.148:0/1284903205 shutdown_connections 2024-03-29T04:10:07.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.885+0000 7fd6e1e55640 1 -- 172.21.15.148:0/1284903205 wait complete. 2024-03-29T04:10:07.874 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T04:10:07.874 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-29T04:10:07.875 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T04:10:07.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.997+0000 7f4843759640 1 Processor -- start 2024-03-29T04:10:07.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.997+0000 7f4843759640 1 -- start start 2024-03-29T04:10:07.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.998+0000 7f4843759640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f483c13f110 0x7f483c06d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.998+0000 7f4843759640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f483c13f4f0 con 0x7f483c160590 2024-03-29T04:10:07.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.998+0000 7f4843759640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f483c1419f0 con 0x7f483c06dc10 2024-03-29T04:10:07.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.998+0000 7f4843759640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f483c1427b0 con 0x7f483c13f110 2024-03-29T04:10:07.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.998+0000 7f4841cf6640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f483c160590 0x7f483c0cf820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50196/0 (socket says 172.21.15.148:50196) 2024-03-29T04:10:07.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.998+0000 7f4841cf6640 1 -- 172.21.15.148:0/3107156070 learned_addr learned my addr 172.21.15.148:0/3107156070 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:07.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.998+0000 7f48414f5640 1 --2- 172.21.15.148:0/3107156070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f483c13f110 0x7f483c06d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.998+0000 7f483bfff640 1 -- 172.21.15.148:0/3107156070 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 834375937 0 0) 0x7f483c13f4f0 con 0x7f483c160590 2024-03-29T04:10:07.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.998+0000 7f483bfff640 1 -- 172.21.15.148:0/3107156070 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4824003520 con 0x7f483c160590 2024-03-29T04:10:07.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.998+0000 7f483bfff640 1 -- 172.21.15.148:0/3107156070 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1462117750 0 0) 0x7f4824003520 con 0x7f483c160590 2024-03-29T04:10:07.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.998+0000 7f483bfff640 1 -- 172.21.15.148:0/3107156070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f483c13f110 msgr2=0x7f483c06d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:07.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.998+0000 7f483bfff640 1 --2- 172.21.15.148:0/3107156070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f483c13f110 0x7f483c06d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.998+0000 7f483bfff640 1 -- 172.21.15.148:0/3107156070 >> v1:172.21.15.157:6789/0 conn(0x7f483c06dc10 legacy=0x7f483c15fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.998+0000 7f483bfff640 1 -- 172.21.15.148:0/3107156070 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483c141cd0 con 0x7f483c160590 2024-03-29T04:10:07.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.998+0000 7f483bfff640 1 -- 172.21.15.148:0/3107156070 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f48340019f0 con 0x7f483c160590 2024-03-29T04:10:07.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.998+0000 7f483bfff640 1 -- 172.21.15.148:0/3107156070 >> v1:172.21.15.148:6789/0 conn(0x7f483c160590 legacy=0x7f483c0cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.999+0000 7f483bfff640 1 --2- 172.21.15.148:0/3107156070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f483c13f110 0x7f48240039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.999+0000 7f483bfff640 1 --2- 172.21.15.148:0/3107156070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4824003f60 0x7f4824004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.999+0000 7f483bfff640 1 --2- 172.21.15.148:0/3107156070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4824004d40 0x7f48240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.999+0000 7f483bfff640 1 -- 172.21.15.148:0/3107156070 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f483c1427b0 con 0x7f4824004d40 2024-03-29T04:10:07.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.999+0000 7f483bfff640 1 -- 172.21.15.148:0/3107156070 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f482400bb50 con 0x7f4824003f60 2024-03-29T04:10:07.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.999+0000 7f483bfff640 1 -- 172.21.15.148:0/3107156070 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f482400bd50 con 0x7f483c13f110 2024-03-29T04:10:07.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.999+0000 7f4841cf6640 1 --2- 172.21.15.148:0/3107156070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4824004d40 0x7f48240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.999+0000 7f48414f5640 1 --2- 172.21.15.148:0/3107156070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f483c13f110 0x7f48240039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.999+0000 7f4840cf4640 1 --2- 172.21.15.148:0/3107156070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4824003f60 0x7f4824004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.999+0000 7f4841cf6640 1 -- 172.21.15.148:0/3107156070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f483c13f110 msgr2=0x7f48240039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:07.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.999+0000 7f4841cf6640 1 --2- 172.21.15.148:0/3107156070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f483c13f110 0x7f48240039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.999+0000 7f4841cf6640 1 -- 172.21.15.148:0/3107156070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4824003f60 msgr2=0x7f4824004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:07.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.999+0000 7f4841cf6640 1 --2- 172.21.15.148:0/3107156070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4824003f60 0x7f4824004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:07.999+0000 7f4841cf6640 1 -- 172.21.15.148:0/3107156070 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4824003520 con 0x7f4824004d40 2024-03-29T04:10:07.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.000+0000 7f4841cf6640 1 --2- 172.21.15.148:0/3107156070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4824004d40 0x7f48240051c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f483400a850 tx=0x7f4824005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793bfc7b3d8bf2cd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:07.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.000+0000 7f483bfff640 1 -- 172.21.15.148:0/3107156070 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483400a4a0 con 0x7f4824004d40 2024-03-29T04:10:07.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.000+0000 7f483bfff640 1 -- 172.21.15.148:0/3107156070 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4834034650 con 0x7f4824004d40 2024-03-29T04:10:07.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.000+0000 7f483bfff640 1 -- 172.21.15.148:0/3107156070 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4834034a40 con 0x7f4824004d40 2024-03-29T04:10:07.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.000+0000 7f4843759640 1 -- 172.21.15.148:0/3107156070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4824004d40 msgr2=0x7f48240051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.000+0000 7f4843759640 1 --2- 172.21.15.148:0/3107156070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4824004d40 0x7f48240051c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f483400a850 tx=0x7f4824005830 comp rx=0 tx=0).stop 2024-03-29T04:10:07.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.000+0000 7f4841cf6640 1 -- 172.21.15.148:0/3107156070 reap_dead start 2024-03-29T04:10:07.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.000+0000 7f4843759640 1 -- 172.21.15.148:0/3107156070 shutdown_connections 2024-03-29T04:10:07.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.000+0000 7f4843759640 1 -- 172.21.15.148:0/3107156070 >> 172.21.15.148:0/3107156070 conn(0x7f483c072450 msgr2=0x7f483c072860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:07.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.000+0000 7f4843759640 1 -- 172.21.15.148:0/3107156070 shutdown_connections 2024-03-29T04:10:07.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.000+0000 7f4843759640 1 -- 172.21.15.148:0/3107156070 wait complete. 2024-03-29T04:10:07.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.002+0000 7f4843759640 1 Processor -- start 2024-03-29T04:10:07.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.002+0000 7f4843759640 1 -- start start 2024-03-29T04:10:07.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.002+0000 7f4843759640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f483c15ca20 0x7f483c0af430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.002+0000 7f4843759640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f483c0af970 0x7f483c13a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.002+0000 7f4843759640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f483c15ce00 0x7f483c13a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.002+0000 7f4843759640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f483c160bf0 con 0x7f483c0af970 2024-03-29T04:10:07.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.002+0000 7f4843759640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f483c06e050 con 0x7f483c15ca20 2024-03-29T04:10:07.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.002+0000 7f4843759640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f483c142290 con 0x7f483c15ce00 2024-03-29T04:10:07.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.002+0000 7f4840cf4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f483c0af970 0x7f483c13a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.002+0000 7f4840cf4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f483c0af970 0x7f483c13a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51282/0 (socket says 172.21.15.148:51282) 2024-03-29T04:10:07.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.002+0000 7f4840cf4640 1 -- 172.21.15.148:0/3874336894 learned_addr learned my addr 172.21.15.148:0/3874336894 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:07.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.003+0000 7f48414f5640 1 --2- 172.21.15.148:0/3874336894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f483c15ca20 0x7f483c0af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.003+0000 7f4841cf6640 1 --2- 172.21.15.148:0/3874336894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f483c15ce00 0x7f483c13a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.003+0000 7f4840cf4640 1 -- 172.21.15.148:0/3874336894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f483c15ce00 msgr2=0x7f483c13a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.003+0000 7f4840cf4640 1 --2- 172.21.15.148:0/3874336894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f483c15ce00 0x7f483c13a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.003+0000 7f4840cf4640 1 -- 172.21.15.148:0/3874336894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f483c15ca20 msgr2=0x7f483c0af430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.003+0000 7f4840cf4640 1 --2- 172.21.15.148:0/3874336894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f483c15ca20 0x7f483c0af430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.003+0000 7f4840cf4640 1 -- 172.21.15.148:0/3874336894 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483c13ae60 con 0x7f483c0af970 2024-03-29T04:10:07.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.003+0000 7f4841cf6640 1 --2- 172.21.15.148:0/3874336894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f483c15ce00 0x7f483c13a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:07.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.003+0000 7f4840cf4640 1 --2- 172.21.15.148:0/3874336894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f483c0af970 0x7f483c13a3e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f483000f850 tx=0x7f483000fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:07.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.003+0000 7f4839ffb640 1 -- 172.21.15.148:0/3874336894 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483000fe40 con 0x7f483c0af970 2024-03-29T04:10:07.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.003+0000 7f4839ffb640 1 -- 172.21.15.148:0/3874336894 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4830018070 con 0x7f483c0af970 2024-03-29T04:10:07.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.003+0000 7f4843759640 1 -- 172.21.15.148:0/3874336894 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f483c0afdf0 con 0x7f483c0af970 2024-03-29T04:10:07.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.003+0000 7f4839ffb640 1 -- 172.21.15.148:0/3874336894 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483000d730 con 0x7f483c0af970 2024-03-29T04:10:07.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.003+0000 7f4843759640 1 -- 172.21.15.148:0/3874336894 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f483c0b4540 con 0x7f483c0af970 2024-03-29T04:10:07.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.004+0000 7f4843759640 1 -- 172.21.15.148:0/3874336894 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f483c0687e0 con 0x7f483c0af970 2024-03-29T04:10:07.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.005+0000 7f4839ffb640 1 -- 172.21.15.148:0/3874336894 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f483000d8d0 con 0x7f483c0af970 2024-03-29T04:10:07.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.005+0000 7f4839ffb640 1 --2- 172.21.15.148:0/3874336894 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f482806c530 0x7f482806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.006+0000 7f4839ffb640 1 -- 172.21.15.148:0/3874336894 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f483008b8f0 con 0x7f483c0af970 2024-03-29T04:10:07.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.006+0000 7f48414f5640 1 --2- 172.21.15.148:0/3874336894 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f482806c530 0x7f482806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.006+0000 7f48414f5640 1 --2- 172.21.15.148:0/3874336894 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f482806c530 0x7f482806e9f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f482c009b00 tx=0x7f482c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:07.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.010+0000 7f4839ffb640 1 -- 172.21.15.148:0/3874336894 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4830059bd0 con 0x7f483c0af970 2024-03-29T04:10:08.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.233+0000 7f4843759640 1 -- 172.21.15.148:0/3874336894 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f483c06e880 con 0x7f483c0af970 2024-03-29T04:10:08.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.234+0000 7f4839ffb640 1 -- 172.21.15.148:0/3874336894 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f48300594d0 con 0x7f483c0af970 2024-03-29T04:10:08.213 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-29T04:10:08.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.236+0000 7f4843759640 1 -- 172.21.15.148:0/3874336894 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f482806c530 msgr2=0x7f482806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:08.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.236+0000 7f4843759640 1 --2- 172.21.15.148:0/3874336894 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f482806c530 0x7f482806e9f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f482c009b00 tx=0x7f482c008040 comp rx=0 tx=0).stop 2024-03-29T04:10:08.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.236+0000 7f4843759640 1 -- 172.21.15.148:0/3874336894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f483c0af970 msgr2=0x7f483c13a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:08.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.236+0000 7f4843759640 1 --2- 172.21.15.148:0/3874336894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f483c0af970 0x7f483c13a3e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f483000f850 tx=0x7f483000fd10 comp rx=0 tx=0).stop 2024-03-29T04:10:08.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.236+0000 7f4843759640 1 -- 172.21.15.148:0/3874336894 shutdown_connections 2024-03-29T04:10:08.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.236+0000 7f4843759640 1 --2- 172.21.15.148:0/3874336894 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f482806c530 0x7f482806e9f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.236+0000 7f4843759640 1 --2- 172.21.15.148:0/3874336894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f483c15ce00 0x7f483c13a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.236+0000 7f4843759640 1 --2- 172.21.15.148:0/3874336894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f483c0af970 0x7f483c13a3e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.236+0000 7f4843759640 1 --2- 172.21.15.148:0/3874336894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f483c15ca20 0x7f483c0af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.236+0000 7f4843759640 1 -- 172.21.15.148:0/3874336894 >> 172.21.15.148:0/3874336894 conn(0x7f483c072450 msgr2=0x7f483c15ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:08.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.236+0000 7f4843759640 1 -- 172.21.15.148:0/3874336894 shutdown_connections 2024-03-29T04:10:08.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08.236+0000 7f4843759640 1 -- 172.21.15.148:0/3874336894 wait complete. 2024-03-29T04:10:08.226 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-29T04:10:08.226 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:10:08.226 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-03-29T04:10:08.226 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-29T04:10:08.226 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-29T04:10:08.226 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-29T04:10:08.226 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T04:10:08.226 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:10:08.226 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-29T04:10:08.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:08.226 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-03-29T04:10:08.241 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:10:08.241 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:10:08.241 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:10:08.242 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:08.242 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-03-29T04:10:08.242 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:08.242 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:10:08.242 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:08.242 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-03-29T04:10:08.242 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:10:08.242 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:08.242 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-03-29T04:10:08.242 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:10:08.242 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:08.242 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:10:08.242 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:08.242 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T04:10:08.243 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42734sec preferred_lft 42734sec 2024-03-29T04:10:08.243 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link 2024-03-29T04:10:08.243 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:08.243 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-29T04:10:08.243 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:10:08.243 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-29T04:10:08.244 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add name ceph-brx type bridge 2024-03-29T04:10:08.244 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip addr flush dev ceph-brx 2024-03-29T04:10:08.244 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set ceph-brx up 2024-03-29T04:10:08.244 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-29T04:10:08.244 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-29T04:10:08.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:10:08.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:08.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:08.909 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:10:08.938 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-03-29T04:10:08.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:08.939 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-03-29T04:10:09.134 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-03-29T04:10:09.135 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:10:09.135 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T04:10:09.135 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T04:10:09.135 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:10:09.135 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:10:09.135 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-29T04:10:09.135 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T04:10:09.135 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T04:10:09.136 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T04:10:09.136 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-29T04:10:09.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:10:09.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:09.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:09.448 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-03-29T04:10:09.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:09.465 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list-id 2024-03-29T04:10:09.520 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:10:09.520 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-29T04:10:09.520 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-29T04:10:09.520 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-29T04:10:09.520 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-29T04:10:09.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:10:09.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:09.649 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-29T04:10:09.650 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:10:09.650 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-29T04:10:09.650 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-29T04:10:09.650 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-29T04:10:09.650 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-29T04:10:09.650 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-29T04:10:09.650 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-29T04:10:09.650 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-29T04:10:09.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:10:09.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:09.809 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:10:09.809 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-29T04:10:09.809 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2024-03-29T04:10:09.809 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set dev brx.0 master ceph-brx 2024-03-29T04:10:09.809 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-29T04:10:09.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:10:09.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:09.904 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-29T04:10:09.904 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:10:09.904 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-29T04:10:09.919 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-29T04:10:09.919 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:10:09.919 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-29T04:10:09.975 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2024-03-29T04:10:10.043 DEBUG:teuthology.orchestra.run.smithi148:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:10:10.099 INFO:teuthology.orchestra.run.smithi148.stdout:/proc 2024-03-29T04:10:10.100 INFO:teuthology.orchestra.run.smithi148.stdout:/sys 2024-03-29T04:10:10.100 INFO:teuthology.orchestra.run.smithi148.stdout:/dev 2024-03-29T04:10:10.100 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/security 2024-03-29T04:10:10.100 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/shm 2024-03-29T04:10:10.100 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/pts 2024-03-29T04:10:10.100 INFO:teuthology.orchestra.run.smithi148.stdout:/run 2024-03-29T04:10:10.100 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/cgroup 2024-03-29T04:10:10.100 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/pstore 2024-03-29T04:10:10.100 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/bpf 2024-03-29T04:10:10.100 INFO:teuthology.orchestra.run.smithi148.stdout:/ 2024-03-29T04:10:10.100 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/selinux 2024-03-29T04:10:10.100 INFO:teuthology.orchestra.run.smithi148.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:10:10.100 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/hugepages 2024-03-29T04:10:10.100 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/mqueue 2024-03-29T04:10:10.101 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/debug 2024-03-29T04:10:10.101 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/tracing 2024-03-29T04:10:10.101 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-29T04:10:10.101 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/fuse/connections 2024-03-29T04:10:10.101 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-sysctl.service 2024-03-29T04:10:10.101 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/config 2024-03-29T04:10:10.101 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-29T04:10:10.101 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/nfs/rpc_pipefs 2024-03-29T04:10:10.101 INFO:teuthology.orchestra.run.smithi148.stdout:/run/user/1000 2024-03-29T04:10:10.101 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph 2024-03-29T04:10:10.101 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-0 2024-03-29T04:10:10.101 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-1 2024-03-29T04:10:10.101 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-2 2024-03-29T04:10:10.101 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-3 2024-03-29T04:10:10.101 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns 2024-03-29T04:10:10.102 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-29T04:10:10.102 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-29T04:10:10.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:10.102 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2024-03-29T04:10:10.155 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T04:10:10.155 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-29T04:10:10.197 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2024-03-29T04:10:10.224 DEBUG:teuthology.orchestra.run.smithi148:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:10:10.281 INFO:teuthology.orchestra.run.smithi148.stdout:/proc 2024-03-29T04:10:10.281 INFO:teuthology.orchestra.run.smithi148.stdout:/sys 2024-03-29T04:10:10.281 INFO:teuthology.orchestra.run.smithi148.stdout:/dev 2024-03-29T04:10:10.281 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/security 2024-03-29T04:10:10.281 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/shm 2024-03-29T04:10:10.282 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/pts 2024-03-29T04:10:10.282 INFO:teuthology.orchestra.run.smithi148.stdout:/run 2024-03-29T04:10:10.282 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/cgroup 2024-03-29T04:10:10.282 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/pstore 2024-03-29T04:10:10.282 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/bpf 2024-03-29T04:10:10.282 INFO:teuthology.orchestra.run.smithi148.stdout:/ 2024-03-29T04:10:10.282 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/selinux 2024-03-29T04:10:10.282 INFO:teuthology.orchestra.run.smithi148.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:10:10.282 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/hugepages 2024-03-29T04:10:10.282 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/mqueue 2024-03-29T04:10:10.282 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/debug 2024-03-29T04:10:10.282 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/tracing 2024-03-29T04:10:10.282 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-29T04:10:10.282 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/fuse/connections 2024-03-29T04:10:10.283 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-sysctl.service 2024-03-29T04:10:10.283 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/config 2024-03-29T04:10:10.283 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-29T04:10:10.283 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/nfs/rpc_pipefs 2024-03-29T04:10:10.283 INFO:teuthology.orchestra.run.smithi148.stdout:/run/user/1000 2024-03-29T04:10:10.283 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph 2024-03-29T04:10:10.283 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-0 2024-03-29T04:10:10.283 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-1 2024-03-29T04:10:10.283 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-2 2024-03-29T04:10:10.283 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-3 2024-03-29T04:10:10.283 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns 2024-03-29T04:10:10.283 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-29T04:10:10.283 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-29T04:10:10.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:10.284 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2024-03-29T04:10:10.289 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi148.stderr:2024-03-29T04:10:10.310+0000 7f05db2bb480 -1 init, newargv = 0x563819898c60 newargc=15 2024-03-29T04:10:10.289 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi148.stderr:ceph-fuse[44699]: starting ceph client 2024-03-29T04:10:10.296 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi148.stderr:ceph-fuse[44699]: starting fuse 2024-03-29T04:10:10.305 INFO:teuthology.orchestra.run.smithi148.stdout:42 2024-03-29T04:10:10.305 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-03-29T04:10:10.305 DEBUG:teuthology.orchestra.run.smithi148:> sudo stdin-killer -- python3 -c ' 2024-03-29T04:10:10.306 DEBUG:teuthology.orchestra.run.smithi148:> import glob 2024-03-29T04:10:10.306 DEBUG:teuthology.orchestra.run.smithi148:> import re 2024-03-29T04:10:10.306 DEBUG:teuthology.orchestra.run.smithi148:> import os 2024-03-29T04:10:10.306 DEBUG:teuthology.orchestra.run.smithi148:> import subprocess 2024-03-29T04:10:10.306 DEBUG:teuthology.orchestra.run.smithi148:> 2024-03-29T04:10:10.306 DEBUG:teuthology.orchestra.run.smithi148:> def _find_admin_socket(client_name): 2024-03-29T04:10:10.306 DEBUG:teuthology.orchestra.run.smithi148:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-29T04:10:10.306 DEBUG:teuthology.orchestra.run.smithi148:> files = glob.glob(asok_path) 2024-03-29T04:10:10.306 DEBUG:teuthology.orchestra.run.smithi148:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-29T04:10:10.306 DEBUG:teuthology.orchestra.run.smithi148:> 2024-03-29T04:10:10.306 DEBUG:teuthology.orchestra.run.smithi148:> # Given a non-glob path, it better be there 2024-03-29T04:10:10.306 DEBUG:teuthology.orchestra.run.smithi148:> if "*" not in asok_path: 2024-03-29T04:10:10.306 DEBUG:teuthology.orchestra.run.smithi148:> assert(len(files) == 1) 2024-03-29T04:10:10.306 DEBUG:teuthology.orchestra.run.smithi148:> return files[0] 2024-03-29T04:10:10.306 DEBUG:teuthology.orchestra.run.smithi148:> 2024-03-29T04:10:10.306 DEBUG:teuthology.orchestra.run.smithi148:> for f in files: 2024-03-29T04:10:10.307 DEBUG:teuthology.orchestra.run.smithi148:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T04:10:10.307 DEBUG:teuthology.orchestra.run.smithi148:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T04:10:10.307 DEBUG:teuthology.orchestra.run.smithi148:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T04:10:10.307 DEBUG:teuthology.orchestra.run.smithi148:> contents = proc_f.read() 2024-03-29T04:10:10.307 DEBUG:teuthology.orchestra.run.smithi148:> if mountpoint in contents: 2024-03-29T04:10:10.307 DEBUG:teuthology.orchestra.run.smithi148:> return f 2024-03-29T04:10:10.307 DEBUG:teuthology.orchestra.run.smithi148:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T04:10:10.307 DEBUG:teuthology.orchestra.run.smithi148:> 2024-03-29T04:10:10.307 DEBUG:teuthology.orchestra.run.smithi148:> print(_find_admin_socket("client.admin")) 2024-03-29T04:10:10.307 DEBUG:teuthology.orchestra.run.smithi148:> ' 2024-03-29T04:10:10.427 INFO:teuthology.orchestra.run.smithi148.stdout:/var/run/ceph/ceph-client.admin.44699.asok 2024-03-29T04:10:10.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:10.437 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.44699.asok 2024-03-29T04:10:10.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:10.438 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.44699.asok status 2024-03-29T04:10:10.590 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-03-29T04:10:10.590 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": { 2024-03-29T04:10:10.590 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_sha1": "69e2cd73c52284dfd5096c66b60c6e6b26a3c447", 2024-03-29T04:10:10.590 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_version": "ceph version 19.0.0-2520-g69e2cd73 (69e2cd73c52284dfd5096c66b60c6e6b26a3c447) squid (dev)", 2024-03-29T04:10:10.590 INFO:teuthology.orchestra.run.smithi148.stdout: "entity_id": "admin", 2024-03-29T04:10:10.590 INFO:teuthology.orchestra.run.smithi148.stdout: "hostname": "smithi148", 2024-03-29T04:10:10.590 INFO:teuthology.orchestra.run.smithi148.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-29T04:10:10.590 INFO:teuthology.orchestra.run.smithi148.stdout: "pid": "44699", 2024-03-29T04:10:10.590 INFO:teuthology.orchestra.run.smithi148.stdout: "root": "/" 2024-03-29T04:10:10.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:10:10.591 INFO:teuthology.orchestra.run.smithi148.stdout: "dentry_count": 0, 2024-03-29T04:10:10.591 INFO:teuthology.orchestra.run.smithi148.stdout: "dentry_pinned_count": 0, 2024-03-29T04:10:10.591 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 4458, 2024-03-29T04:10:10.591 INFO:teuthology.orchestra.run.smithi148.stdout: "inst": { 2024-03-29T04:10:10.591 INFO:teuthology.orchestra.run.smithi148.stdout: "name": { 2024-03-29T04:10:10.591 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client", 2024-03-29T04:10:10.591 INFO:teuthology.orchestra.run.smithi148.stdout: "num": 4458 2024-03-29T04:10:10.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:10:10.591 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": { 2024-03-29T04:10:10.591 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2024-03-29T04:10:10.591 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "192.168.0.1:0", 2024-03-29T04:10:10.591 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 3432157100 2024-03-29T04:10:10.591 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:10:10.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:10:10.592 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": { 2024-03-29T04:10:10.592 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2024-03-29T04:10:10.592 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "192.168.0.1:0", 2024-03-29T04:10:10.592 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 3432157100 2024-03-29T04:10:10.592 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:10:10.592 INFO:teuthology.orchestra.run.smithi148.stdout: "inst_str": "client.4458 192.168.0.1:0/3432157100", 2024-03-29T04:10:10.592 INFO:teuthology.orchestra.run.smithi148.stdout: "addr_str": "192.168.0.1:0/3432157100", 2024-03-29T04:10:10.592 INFO:teuthology.orchestra.run.smithi148.stdout: "inode_count": 1, 2024-03-29T04:10:10.592 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_epoch": 7, 2024-03-29T04:10:10.592 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_epoch": 24, 2024-03-29T04:10:10.592 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_epoch_barrier": 0, 2024-03-29T04:10:10.592 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklisted": false, 2024-03-29T04:10:10.592 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs" 2024-03-29T04:10:10.592 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-03-29T04:10:10.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:10.599 DEBUG:teuthology.orchestra.run.smithi148:> stat --file-system '--printf=%T 2024-03-29T04:10:10.599 DEBUG:teuthology.orchestra.run.smithi148:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-29T04:10:10.613 INFO:teuthology.orchestra.run.smithi148.stdout:fuseblk 2024-03-29T04:10:10.613 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-29T04:10:10.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:10.614 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-29T04:10:10.683 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-29T04:10:10.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:10:10.797 INFO:teuthology.orchestra.run.smithi148.stdout:# file: . 2024-03-29T04:10:10.797 INFO:teuthology.orchestra.run.smithi148.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-29T04:10:10.797 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:10.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:10.803 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:10:10.803 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:10:10.820 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:10.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.983+0000 7ff52430e640 1 Processor -- start 2024-03-29T04:10:10.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.983+0000 7ff52430e640 1 -- start start 2024-03-29T04:10:10.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.984+0000 7ff52430e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff51c15e3e0 0x7ff51c0d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:10.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.984+0000 7ff52430e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51c0d10d0 0x7ff51c0d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:10.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.984+0000 7ff52430e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff51c15e7c0 con 0x7ff51c0ba030 2024-03-29T04:10:10.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.984+0000 7ff52430e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7ff51c0d3840 con 0x7ff51c15e3e0 2024-03-29T04:10:10.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.984+0000 7ff52430e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff51c0d2c30 con 0x7ff51c0d10d0 2024-03-29T04:10:10.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.985+0000 7ff5220aa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff51c15e3e0 0x7ff51c0d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:10.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.985+0000 7ff5220aa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff51c15e3e0 0x7ff51c0d3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51294/0 (socket says 172.21.15.148:51294) 2024-03-29T04:10:10.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.985+0000 7ff5220aa640 1 -- 172.21.15.148:0/2277428014 learned_addr learned my addr 172.21.15.148:0/2277428014 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:10.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.985+0000 7ff5228ab640 1 --2- 172.21.15.148:0/2277428014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51c0d10d0 0x7ff51c0d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:10.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.985+0000 7ff5220aa640 1 -- 172.21.15.148:0/2277428014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51c0d10d0 msgr2=0x7ff51c0d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:10.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.985+0000 7ff5220aa640 1 --2- 172.21.15.148:0/2277428014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51c0d10d0 0x7ff51c0d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.985+0000 7ff5220aa640 1 -- 172.21.15.148:0/2277428014 >> v1:172.21.15.157:6789/0 conn(0x7ff51c0ba030 legacy=0x7ff51c0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:10.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.985+0000 7ff5220aa640 1 -- 172.21.15.148:0/2277428014 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff51c0d2f10 con 0x7ff51c15e3e0 2024-03-29T04:10:10.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.986+0000 7ff5220aa640 1 --2- 172.21.15.148:0/2277428014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff51c15e3e0 0x7ff51c0d3cd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff51c0b4eb0 tx=0x7ff51402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9256848692d58407 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:10.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.986+0000 7ff5210a8640 1 -- 172.21.15.148:0/2277428014 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff514034070 con 0x7ff51c15e3e0 2024-03-29T04:10:10.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.986+0000 7ff5210a8640 1 -- 172.21.15.148:0/2277428014 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff514030610 con 0x7ff51c15e3e0 2024-03-29T04:10:10.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.986+0000 7ff5210a8640 1 -- 172.21.15.148:0/2277428014 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5140309e0 con 0x7ff51c15e3e0 2024-03-29T04:10:10.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.986+0000 7ff52430e640 1 -- 172.21.15.148:0/2277428014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff51c15e3e0 msgr2=0x7ff51c0d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:10.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.986+0000 7ff52430e640 1 --2- 172.21.15.148:0/2277428014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff51c15e3e0 0x7ff51c0d3cd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff51c0b4eb0 tx=0x7ff51402b960 comp rx=0 tx=0).stop 2024-03-29T04:10:10.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.987+0000 7ff52430e640 1 -- 172.21.15.148:0/2277428014 shutdown_connections 2024-03-29T04:10:10.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.987+0000 7ff52430e640 1 --2- 172.21.15.148:0/2277428014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51c0d10d0 0x7ff51c0d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.987+0000 7ff52430e640 1 --2- 172.21.15.148:0/2277428014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff51c15e3e0 0x7ff51c0d3cd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.987+0000 7ff52430e640 1 -- 172.21.15.148:0/2277428014 >> 172.21.15.148:0/2277428014 conn(0x7ff51c0b6210 msgr2=0x7ff51c0b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:10.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.987+0000 7ff52430e640 1 -- 172.21.15.148:0/2277428014 shutdown_connections 2024-03-29T04:10:10.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.987+0000 7ff52430e640 1 -- 172.21.15.148:0/2277428014 wait complete. 2024-03-29T04:10:10.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.988+0000 7ff52430e640 1 Processor -- start 2024-03-29T04:10:10.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.988+0000 7ff52430e640 1 -- start start 2024-03-29T04:10:10.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.988+0000 7ff52430e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff51c0ba030 0x7ff51c06ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:10.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.988+0000 7ff52430e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff51c0d10d0 0x7ff51c14eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:10.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.988+0000 7ff52430e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51c15e3e0 0x7ff51c14ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:10.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.988+0000 7ff52430e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c167f50 con 0x7ff51c0ba030 2024-03-29T04:10:10.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.988+0000 7ff52430e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c0d34d0 con 0x7ff51c0d10d0 2024-03-29T04:10:10.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.988+0000 7ff52430e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff51c1680d0 con 0x7ff51c15e3e0 2024-03-29T04:10:10.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.989+0000 7ff5220aa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff51c0ba030 0x7ff51c06ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:10.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.989+0000 7ff5220aa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff51c0ba030 0x7ff51c06ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51308/0 (socket says 172.21.15.148:51308) 2024-03-29T04:10:10.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.989+0000 7ff5220aa640 1 -- 172.21.15.148:0/3733210114 learned_addr learned my addr 172.21.15.148:0/3733210114 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:10.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.989+0000 7ff5228ab640 1 --2- 172.21.15.148:0/3733210114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51c15e3e0 0x7ff51c14ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:10.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.989+0000 7ff5218a9640 1 --2- 172.21.15.148:0/3733210114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff51c0d10d0 0x7ff51c14eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:10.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.989+0000 7ff5220aa640 1 -- 172.21.15.148:0/3733210114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51c15e3e0 msgr2=0x7ff51c14ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:10.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.989+0000 7ff5220aa640 1 --2- 172.21.15.148:0/3733210114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51c15e3e0 0x7ff51c14ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.989+0000 7ff5220aa640 1 -- 172.21.15.148:0/3733210114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff51c0d10d0 msgr2=0x7ff51c14eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:10.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.989+0000 7ff5220aa640 1 --2- 172.21.15.148:0/3733210114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff51c0d10d0 0x7ff51c14eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.989+0000 7ff5220aa640 1 -- 172.21.15.148:0/3733210114 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff51c14d100 con 0x7ff51c0ba030 2024-03-29T04:10:10.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.989+0000 7ff5228ab640 1 --2- 172.21.15.148:0/3733210114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51c15e3e0 0x7ff51c14ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:10.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.989+0000 7ff5218a9640 1 --2- 172.21.15.148:0/3733210114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff51c0d10d0 0x7ff51c14eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:10.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.989+0000 7ff5220aa640 1 --2- 172.21.15.148:0/3733210114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff51c0ba030 0x7ff51c06ba90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff514000c00 tx=0x7ff514004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:10.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.989+0000 7ff512ffd640 1 -- 172.21.15.148:0/3733210114 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff514048070 con 0x7ff51c0ba030 2024-03-29T04:10:10.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.989+0000 7ff512ffd640 1 -- 172.21.15.148:0/3733210114 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff514030410 con 0x7ff51c0ba030 2024-03-29T04:10:10.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.989+0000 7ff52430e640 1 -- 172.21.15.148:0/3733210114 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff51c14d3a0 con 0x7ff51c0ba030 2024-03-29T04:10:10.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.989+0000 7ff512ffd640 1 -- 172.21.15.148:0/3733210114 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff514034040 con 0x7ff51c0ba030 2024-03-29T04:10:10.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.989+0000 7ff52430e640 1 -- 172.21.15.148:0/3733210114 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff51c14d960 con 0x7ff51c0ba030 2024-03-29T04:10:10.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.990+0000 7ff52430e640 1 -- 172.21.15.148:0/3733210114 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff51c15dc40 con 0x7ff51c0ba030 2024-03-29T04:10:10.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.991+0000 7ff512ffd640 1 -- 172.21.15.148:0/3733210114 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff51404b070 con 0x7ff51c0ba030 2024-03-29T04:10:10.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.991+0000 7ff512ffd640 1 --2- 172.21.15.148:0/3733210114 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff50006c530 0x7ff50006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:10.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.991+0000 7ff512ffd640 1 -- 172.21.15.148:0/3733210114 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff5140ad700 con 0x7ff51c0ba030 2024-03-29T04:10:10.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.991+0000 7ff5218a9640 1 --2- 172.21.15.148:0/3733210114 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff50006c530 0x7ff50006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:10.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.992+0000 7ff5218a9640 1 --2- 172.21.15.148:0/3733210114 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff50006c530 0x7ff50006e9f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff51c06b620 tx=0x7ff50c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:10.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:10.996+0000 7ff512ffd640 1 -- 172.21.15.148:0/3733210114 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff51c15dc40 con 0x7ff51c0ba030 2024-03-29T04:10:11.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.177+0000 7ff52430e640 1 -- 172.21.15.148:0/3733210114 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff51c0687e0 con 0x7ff51c0ba030 2024-03-29T04:10:11.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.179+0000 7ff512ffd640 1 -- 172.21.15.148:0/3733210114 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7ff51407b9e0 con 0x7ff51c0ba030 2024-03-29T04:10:11.158 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:11.158 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:05.026481+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:11.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.181+0000 7ff52430e640 1 -- 172.21.15.148:0/3733210114 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff50006c530 msgr2=0x7ff50006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.181+0000 7ff52430e640 1 --2- 172.21.15.148:0/3733210114 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff50006c530 0x7ff50006e9f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff51c06b620 tx=0x7ff50c008040 comp rx=0 tx=0).stop 2024-03-29T04:10:11.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.181+0000 7ff52430e640 1 -- 172.21.15.148:0/3733210114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff51c0ba030 msgr2=0x7ff51c06ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.181+0000 7ff52430e640 1 --2- 172.21.15.148:0/3733210114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff51c0ba030 0x7ff51c06ba90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff514000c00 tx=0x7ff514004730 comp rx=0 tx=0).stop 2024-03-29T04:10:11.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.181+0000 7ff52430e640 1 -- 172.21.15.148:0/3733210114 shutdown_connections 2024-03-29T04:10:11.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.181+0000 7ff52430e640 1 --2- 172.21.15.148:0/3733210114 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff50006c530 0x7ff50006e9f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.181+0000 7ff52430e640 1 --2- 172.21.15.148:0/3733210114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff51c15e3e0 0x7ff51c14ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.181+0000 7ff52430e640 1 --2- 172.21.15.148:0/3733210114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff51c0d10d0 0x7ff51c14eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.181+0000 7ff52430e640 1 --2- 172.21.15.148:0/3733210114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff51c0ba030 0x7ff51c06ba90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.181+0000 7ff52430e640 1 -- 172.21.15.148:0/3733210114 >> 172.21.15.148:0/3733210114 conn(0x7ff51c0b6210 msgr2=0x7ff51c150680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:11.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.181+0000 7ff52430e640 1 -- 172.21.15.148:0/3733210114 shutdown_connections 2024-03-29T04:10:11.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.181+0000 7ff52430e640 1 -- 172.21.15.148:0/3733210114 wait complete. 2024-03-29T04:10:11.171 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:47.898351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:57.922313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:58.829405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:10:02.551763+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:10:11.171 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:10:11.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.297+0000 7ff50295f640 1 Processor -- start 2024-03-29T04:10:11.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.297+0000 7ff50295f640 1 -- start start 2024-03-29T04:10:11.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.298+0000 7ff50295f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff4fc0ca2e0 0x7ff4fc06fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.298+0000 7ff50295f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff4fc0d02f0 0x7ff4fc0d0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.298+0000 7ff50295f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4fc144f10 con 0x7ff4fc144b30 2024-03-29T04:10:11.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.298+0000 7ff50295f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7ff4fc0702c0 con 0x7ff4fc0ca2e0 2024-03-29T04:10:11.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.298+0000 7ff50295f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff4fc15f090 con 0x7ff4fc0d02f0 2024-03-29T04:10:11.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.298+0000 7ff4fb7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff4fc0ca2e0 0x7ff4fc06fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.298+0000 7ff4fbfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff4fc144b30 0x7ff4fc15f520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50210/0 (socket says 172.21.15.148:50210) 2024-03-29T04:10:11.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.298+0000 7ff4fbfff640 1 -- 172.21.15.148:0/3819556150 learned_addr learned my addr 172.21.15.148:0/3819556150 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:11.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.299+0000 7ff500efc640 1 --2- 172.21.15.148:0/3819556150 >> v2:172.21.15.157:3300/0 conn(0x7ff4fc0d02f0 0x7ff4fc0d0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.299+0000 7ff4fb7fe640 1 -- 172.21.15.148:0/3819556150 >> v2:172.21.15.157:3300/0 conn(0x7ff4fc0d02f0 msgr2=0x7ff4fc0d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:11.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.299+0000 7ff4fb7fe640 1 --2- 172.21.15.148:0/3819556150 >> v2:172.21.15.157:3300/0 conn(0x7ff4fc0d02f0 0x7ff4fc0d0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.299+0000 7ff4fb7fe640 1 -- 172.21.15.148:0/3819556150 >> v1:172.21.15.148:6789/0 conn(0x7ff4fc144b30 legacy=0x7ff4fc15f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.299+0000 7ff4faffd640 1 -- 172.21.15.148:0/3819556150 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1453409394 0 0) 0x7ff4fc144f10 con 0x7ff4fc144b30 2024-03-29T04:10:11.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.299+0000 7ff4fb7fe640 1 -- 172.21.15.148:0/3819556150 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4fc0ca9c0 con 0x7ff4fc0ca2e0 2024-03-29T04:10:11.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.300+0000 7ff4fb7fe640 1 --2- 172.21.15.148:0/3819556150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4fc0ca2e0 0x7ff4fc06fd80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0cc3e0 tx=0x7ff4f002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4e4e78a0cbad33 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:11.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.300+0000 7ff4faffd640 1 -- 172.21.15.148:0/3819556150 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0035020 con 0x7ff4fc0ca2e0 2024-03-29T04:10:11.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.300+0000 7ff4faffd640 1 -- 172.21.15.148:0/3819556150 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4f0031830 con 0x7ff4fc0ca2e0 2024-03-29T04:10:11.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.300+0000 7ff4faffd640 1 -- 172.21.15.148:0/3819556150 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0031c00 con 0x7ff4fc0ca2e0 2024-03-29T04:10:11.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.301+0000 7ff50295f640 1 -- 172.21.15.148:0/3819556150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4fc0ca2e0 msgr2=0x7ff4fc06fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.301+0000 7ff50295f640 1 --2- 172.21.15.148:0/3819556150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4fc0ca2e0 0x7ff4fc06fd80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0cc3e0 tx=0x7ff4f002b8d0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.301+0000 7ff50295f640 1 -- 172.21.15.148:0/3819556150 shutdown_connections 2024-03-29T04:10:11.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.301+0000 7ff50295f640 1 --2- 172.21.15.148:0/3819556150 >> v2:172.21.15.157:3300/0 conn(0x7ff4fc0d02f0 0x7ff4fc0d0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.301+0000 7ff50295f640 1 --2- 172.21.15.148:0/3819556150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4fc0ca2e0 0x7ff4fc06fd80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.301+0000 7ff50295f640 1 -- 172.21.15.148:0/3819556150 >> 172.21.15.148:0/3819556150 conn(0x7ff4fc0b2c60 msgr2=0x7ff4fc0b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:11.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.301+0000 7ff50295f640 1 -- 172.21.15.148:0/3819556150 shutdown_connections 2024-03-29T04:10:11.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.301+0000 7ff50295f640 1 -- 172.21.15.148:0/3819556150 wait complete. 2024-03-29T04:10:11.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.302+0000 7ff50295f640 1 Processor -- start 2024-03-29T04:10:11.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.302+0000 7ff50295f640 1 -- start start 2024-03-29T04:10:11.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.303+0000 7ff50295f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4fc0ca2e0 0x7ff4fc15dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.303+0000 7ff50295f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4fc0d02f0 0x7ff4fc15c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.303+0000 7ff50295f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4fc144b30 0x7ff4fc145330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.303+0000 7ff50295f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc15eac0 con 0x7ff4fc0ca2e0 2024-03-29T04:10:11.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.303+0000 7ff50295f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc0ca770 con 0x7ff4fc144b30 2024-03-29T04:10:11.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.303+0000 7ff50295f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff4fc15ed20 con 0x7ff4fc0d02f0 2024-03-29T04:10:11.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.303+0000 7ff4fbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4fc0ca2e0 0x7ff4fc15dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.303+0000 7ff4fbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4fc0ca2e0 0x7ff4fc15dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51320/0 (socket says 172.21.15.148:51320) 2024-03-29T04:10:11.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.303+0000 7ff4fbfff640 1 -- 172.21.15.148:0/634596018 learned_addr learned my addr 172.21.15.148:0/634596018 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:11.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.303+0000 7ff4fb7fe640 1 --2- 172.21.15.148:0/634596018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4fc0d02f0 0x7ff4fc15c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.303+0000 7ff4fbfff640 1 -- 172.21.15.148:0/634596018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4fc0d02f0 msgr2=0x7ff4fc15c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.303+0000 7ff4fbfff640 1 --2- 172.21.15.148:0/634596018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4fc0d02f0 0x7ff4fc15c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.303+0000 7ff4fbfff640 1 -- 172.21.15.148:0/634596018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4fc144b30 msgr2=0x7ff4fc145330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.303+0000 7ff4fbfff640 1 --2- 172.21.15.148:0/634596018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4fc144b30 0x7ff4fc145330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.303+0000 7ff4fbfff640 1 -- 172.21.15.148:0/634596018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4fc0b4dd0 con 0x7ff4fc0ca2e0 2024-03-29T04:10:11.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.303+0000 7ff4fbfff640 1 --2- 172.21.15.148:0/634596018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4fc0ca2e0 0x7ff4fc15dfa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00c430 tx=0x7ff4ec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:11.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.303+0000 7ff4f8ff9640 1 -- 172.21.15.148:0/634596018 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec0074e0 con 0x7ff4fc0ca2e0 2024-03-29T04:10:11.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.303+0000 7ff4f8ff9640 1 -- 172.21.15.148:0/634596018 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4ec0047e0 con 0x7ff4fc0ca2e0 2024-03-29T04:10:11.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.303+0000 7ff50295f640 1 -- 172.21.15.148:0/634596018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4fc0b50b0 con 0x7ff4fc0ca2e0 2024-03-29T04:10:11.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.303+0000 7ff4f8ff9640 1 -- 172.21.15.148:0/634596018 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec004bd0 con 0x7ff4fc0ca2e0 2024-03-29T04:10:11.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.304+0000 7ff50295f640 1 -- 172.21.15.148:0/634596018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4fc0b5670 con 0x7ff4fc0ca2e0 2024-03-29T04:10:11.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.304+0000 7ff50295f640 1 -- 172.21.15.148:0/634596018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4fc068850 con 0x7ff4fc0ca2e0 2024-03-29T04:10:11.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.305+0000 7ff4f8ff9640 1 -- 172.21.15.148:0/634596018 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff4ec011070 con 0x7ff4fc0ca2e0 2024-03-29T04:10:11.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.305+0000 7ff4f8ff9640 1 --2- 172.21.15.148:0/634596018 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff4e806c530 0x7ff4e806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.306+0000 7ff4f8ff9640 1 -- 172.21.15.148:0/634596018 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff4ec08b5b0 con 0x7ff4fc0ca2e0 2024-03-29T04:10:11.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.306+0000 7ff4fb7fe640 1 --2- 172.21.15.148:0/634596018 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff4e806c530 0x7ff4e806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.306+0000 7ff4fb7fe640 1 --2- 172.21.15.148:0/634596018 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff4e806c530 0x7ff4e806e9f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0036000 tx=0x7ff4f0008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:11.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.310+0000 7ff4f8ff9640 1 -- 172.21.15.148:0/634596018 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff4ec00e280 con 0x7ff4fc0ca2e0 2024-03-29T04:10:11.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.490+0000 7ff50295f640 1 -- 172.21.15.148:0/634596018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff4fc0b31e0 con 0x7ff4fc0ca2e0 2024-03-29T04:10:11.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.490+0000 7ff4f8ff9640 1 -- 172.21.15.148:0/634596018 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7ff4ec059920 con 0x7ff4fc0ca2e0 2024-03-29T04:10:11.470 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-29T04:10:11.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.493+0000 7ff50295f640 1 -- 172.21.15.148:0/634596018 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff4e806c530 msgr2=0x7ff4e806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.493+0000 7ff50295f640 1 --2- 172.21.15.148:0/634596018 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff4e806c530 0x7ff4e806e9f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0036000 tx=0x7ff4f0008030 comp rx=0 tx=0).stop 2024-03-29T04:10:11.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.493+0000 7ff50295f640 1 -- 172.21.15.148:0/634596018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4fc0ca2e0 msgr2=0x7ff4fc15dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.493+0000 7ff50295f640 1 --2- 172.21.15.148:0/634596018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4fc0ca2e0 0x7ff4fc15dfa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00c430 tx=0x7ff4ec00c8f0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.493+0000 7ff50295f640 1 -- 172.21.15.148:0/634596018 shutdown_connections 2024-03-29T04:10:11.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.493+0000 7ff50295f640 1 --2- 172.21.15.148:0/634596018 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff4e806c530 0x7ff4e806e9f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.493+0000 7ff50295f640 1 --2- 172.21.15.148:0/634596018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4fc144b30 0x7ff4fc145330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.493+0000 7ff50295f640 1 --2- 172.21.15.148:0/634596018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4fc0d02f0 0x7ff4fc15c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.493+0000 7ff50295f640 1 --2- 172.21.15.148:0/634596018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4fc0ca2e0 0x7ff4fc15dfa0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.493+0000 7ff50295f640 1 -- 172.21.15.148:0/634596018 >> 172.21.15.148:0/634596018 conn(0x7ff4fc0b2c60 msgr2=0x7ff4fc06d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:11.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.493+0000 7ff50295f640 1 -- 172.21.15.148:0/634596018 shutdown_connections 2024-03-29T04:10:11.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.493+0000 7ff50295f640 1 -- 172.21.15.148:0/634596018 wait complete. 2024-03-29T04:10:11.483 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:10:11.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.610+0000 7f45635d4640 1 Processor -- start 2024-03-29T04:10:11.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.610+0000 7f45635d4640 1 -- start start 2024-03-29T04:10:11.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.611+0000 7f45635d4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f455c15a120 0x7f455c0700d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.611+0000 7f45635d4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f455c0ce310 0x7f455c06e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.611+0000 7f45635d4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f455c15d460 con 0x7f455c06ee10 2024-03-29T04:10:11.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.611+0000 7f45635d4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f455c0d1b20 con 0x7f455c15a120 2024-03-29T04:10:11.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.611+0000 7f45635d4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f455c0d0f60 con 0x7f455c0ce310 2024-03-29T04:10:11.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.611+0000 7f4561370640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f455c15a120 0x7f455c0700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.611+0000 7f4561370640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f455c15a120 0x7f455c0700d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51330/0 (socket says 172.21.15.148:51330) 2024-03-29T04:10:11.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.611+0000 7f4561370640 1 -- 172.21.15.148:0/1649113458 learned_addr learned my addr 172.21.15.148:0/1649113458 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:11.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.611+0000 7f4560b6f640 1 --2- 172.21.15.148:0/1649113458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f455c0ce310 0x7f455c06e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.611+0000 7f4561370640 1 -- 172.21.15.148:0/1649113458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f455c0ce310 msgr2=0x7f455c06e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:11.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.611+0000 7f4561370640 1 --2- 172.21.15.148:0/1649113458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f455c0ce310 0x7f455c06e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.611+0000 7f4561370640 1 -- 172.21.15.148:0/1649113458 >> v1:172.21.15.157:6789/0 conn(0x7f455c06ee10 legacy=0x7f455c0b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.611+0000 7f4561370640 1 -- 172.21.15.148:0/1649113458 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455c15d700 con 0x7f455c15a120 2024-03-29T04:10:11.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.613+0000 7f4561370640 1 --2- 172.21.15.148:0/1649113458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f455c15a120 0x7f455c0700d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4554009870 tx=0x7f455402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653f7f8ac2a06da2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:11.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.613+0000 7f4553fff640 1 -- 172.21.15.148:0/1649113458 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554034070 con 0x7f455c15a120 2024-03-29T04:10:11.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.613+0000 7f4553fff640 1 -- 172.21.15.148:0/1649113458 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4554030840 con 0x7f455c15a120 2024-03-29T04:10:11.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.613+0000 7f4553fff640 1 -- 172.21.15.148:0/1649113458 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554030c10 con 0x7f455c15a120 2024-03-29T04:10:11.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.613+0000 7f45635d4640 1 -- 172.21.15.148:0/1649113458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f455c15a120 msgr2=0x7f455c0700d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.613+0000 7f45635d4640 1 --2- 172.21.15.148:0/1649113458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f455c15a120 0x7f455c0700d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4554009870 tx=0x7f455402b980 comp rx=0 tx=0).stop 2024-03-29T04:10:11.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.614+0000 7f45635d4640 1 -- 172.21.15.148:0/1649113458 shutdown_connections 2024-03-29T04:10:11.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.614+0000 7f45635d4640 1 --2- 172.21.15.148:0/1649113458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f455c0ce310 0x7f455c06e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.614+0000 7f45635d4640 1 --2- 172.21.15.148:0/1649113458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f455c15a120 0x7f455c0700d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.614+0000 7f45635d4640 1 -- 172.21.15.148:0/1649113458 >> 172.21.15.148:0/1649113458 conn(0x7f455c0cead0 msgr2=0x7f455c0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:11.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.614+0000 7f45635d4640 1 -- 172.21.15.148:0/1649113458 shutdown_connections 2024-03-29T04:10:11.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.614+0000 7f45635d4640 1 -- 172.21.15.148:0/1649113458 wait complete. 2024-03-29T04:10:11.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.615+0000 7f45635d4640 1 Processor -- start 2024-03-29T04:10:11.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.615+0000 7f45635d4640 1 -- start start 2024-03-29T04:10:11.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.617+0000 7f45635d4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f455c06ee10 0x7f455c0c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.617+0000 7f45635d4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f455c0ce310 0x7f455c0c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.617+0000 7f45635d4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f455c15a120 0x7f455c15e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.617+0000 7f45635d4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f455c06f5c0 con 0x7f455c06ee10 2024-03-29T04:10:11.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.617+0000 7f45635d4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f455c0ce7d0 con 0x7f455c0ce310 2024-03-29T04:10:11.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.617+0000 7f45635d4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f455c06f740 con 0x7f455c15a120 2024-03-29T04:10:11.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.617+0000 7f4561370640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f455c06ee10 0x7f455c0c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.617+0000 7f4561370640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f455c06ee10 0x7f455c0c7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51346/0 (socket says 172.21.15.148:51346) 2024-03-29T04:10:11.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.617+0000 7f4561370640 1 -- 172.21.15.148:0/2998835878 learned_addr learned my addr 172.21.15.148:0/2998835878 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:11.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.617+0000 7f4561b71640 1 --2- 172.21.15.148:0/2998835878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f455c15a120 0x7f455c15e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.617+0000 7f4560b6f640 1 --2- 172.21.15.148:0/2998835878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f455c0ce310 0x7f455c0c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.617+0000 7f4561370640 1 -- 172.21.15.148:0/2998835878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f455c15a120 msgr2=0x7f455c15e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.617+0000 7f4561370640 1 --2- 172.21.15.148:0/2998835878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f455c15a120 0x7f455c15e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.617+0000 7f4561370640 1 -- 172.21.15.148:0/2998835878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f455c0ce310 msgr2=0x7f455c0c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.617+0000 7f4561370640 1 --2- 172.21.15.148:0/2998835878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f455c0ce310 0x7f455c0c9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.617+0000 7f4561370640 1 -- 172.21.15.148:0/2998835878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455c0c83a0 con 0x7f455c06ee10 2024-03-29T04:10:11.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.617+0000 7f4561370640 1 --2- 172.21.15.148:0/2998835878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f455c06ee10 0x7f455c0c7e60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4554000c00 tx=0x7f4554004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:11.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.617+0000 7f4551ffb640 1 -- 172.21.15.148:0/2998835878 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554048070 con 0x7f455c06ee10 2024-03-29T04:10:11.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.617+0000 7f4551ffb640 1 -- 172.21.15.148:0/2998835878 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4554030640 con 0x7f455c06ee10 2024-03-29T04:10:11.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.617+0000 7f45635d4640 1 -- 172.21.15.148:0/2998835878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f455c0ca4a0 con 0x7f455c06ee10 2024-03-29T04:10:11.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.617+0000 7f4551ffb640 1 -- 172.21.15.148:0/2998835878 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554034040 con 0x7f455c06ee10 2024-03-29T04:10:11.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.618+0000 7f45635d4640 1 -- 172.21.15.148:0/2998835878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f455c0caa60 con 0x7f455c06ee10 2024-03-29T04:10:11.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.619+0000 7f45635d4640 1 -- 172.21.15.148:0/2998835878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f455c068850 con 0x7f455c06ee10 2024-03-29T04:10:11.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.619+0000 7f4551ffb640 1 -- 172.21.15.148:0/2998835878 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f455404b070 con 0x7f455c06ee10 2024-03-29T04:10:11.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.619+0000 7f4551ffb640 1 --2- 172.21.15.148:0/2998835878 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f454006c530 0x7f454006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.620+0000 7f4551ffb640 1 -- 172.21.15.148:0/2998835878 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f45540ad7e0 con 0x7f455c06ee10 2024-03-29T04:10:11.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.620+0000 7f4560b6f640 1 --2- 172.21.15.148:0/2998835878 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f454006c530 0x7f454006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.620+0000 7f4560b6f640 1 --2- 172.21.15.148:0/2998835878 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f454006c530 0x7f454006e9f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f455c1f59b0 tx=0x7f454c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:11.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.624+0000 7f4551ffb640 1 -- 172.21.15.148:0/2998835878 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f455c0c83a0 con 0x7f455c06ee10 2024-03-29T04:10:11.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.804+0000 7f45635d4640 1 -- 172.21.15.148:0/2998835878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f455c0d1cb0 con 0x7f455c06ee10 2024-03-29T04:10:11.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.804+0000 7f4551ffb640 1 -- 172.21.15.148:0/2998835878 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f4554030810 con 0x7f455c06ee10 2024-03-29T04:10:11.784 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:10:11.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.806+0000 7f45635d4640 1 -- 172.21.15.148:0/2998835878 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f454006c530 msgr2=0x7f454006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.806+0000 7f45635d4640 1 --2- 172.21.15.148:0/2998835878 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f454006c530 0x7f454006e9f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f455c1f59b0 tx=0x7f454c009190 comp rx=0 tx=0).stop 2024-03-29T04:10:11.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.806+0000 7f45635d4640 1 -- 172.21.15.148:0/2998835878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f455c06ee10 msgr2=0x7f455c0c7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.806+0000 7f45635d4640 1 --2- 172.21.15.148:0/2998835878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f455c06ee10 0x7f455c0c7e60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4554000c00 tx=0x7f4554004730 comp rx=0 tx=0).stop 2024-03-29T04:10:11.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.806+0000 7f45635d4640 1 -- 172.21.15.148:0/2998835878 shutdown_connections 2024-03-29T04:10:11.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.806+0000 7f45635d4640 1 --2- 172.21.15.148:0/2998835878 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f454006c530 0x7f454006e9f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.806+0000 7f45635d4640 1 --2- 172.21.15.148:0/2998835878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f455c15a120 0x7f455c15e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.806+0000 7f45635d4640 1 --2- 172.21.15.148:0/2998835878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f455c0ce310 0x7f455c0c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.806+0000 7f45635d4640 1 --2- 172.21.15.148:0/2998835878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f455c06ee10 0x7f455c0c7e60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.806+0000 7f45635d4640 1 -- 172.21.15.148:0/2998835878 >> 172.21.15.148:0/2998835878 conn(0x7f455c0cead0 msgr2=0x7f455c070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:11.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.807+0000 7f45635d4640 1 -- 172.21.15.148:0/2998835878 shutdown_connections 2024-03-29T04:10:11.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.807+0000 7f45635d4640 1 -- 172.21.15.148:0/2998835878 wait complete. 2024-03-29T04:10:11.796 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:10:11.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.923+0000 7f1355f8d640 1 Processor -- start 2024-03-29T04:10:11.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.923+0000 7f1355f8d640 1 -- start start 2024-03-29T04:10:11.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.925+0000 7f1355f8d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1350144980 0x7f135006fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.925+0000 7f1355f8d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13500cc0f0 0x7f13500cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.925+0000 7f1355f8d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13500cc660 con 0x7f13500cfbd0 2024-03-29T04:10:11.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.925+0000 7f1355f8d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f135006d6d0 con 0x7f1350144980 2024-03-29T04:10:11.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.925+0000 7f1355f8d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f13500cd270 con 0x7f13500cc0f0 2024-03-29T04:10:11.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.925+0000 7f134ffff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f13500cfbd0 0x7f135015edc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50220/0 (socket says 172.21.15.148:50220) 2024-03-29T04:10:11.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.925+0000 7f134ffff640 1 -- 172.21.15.148:0/2958479184 learned_addr learned my addr 172.21.15.148:0/2958479184 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:11.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.925+0000 7f134effd640 1 --2- 172.21.15.148:0/2958479184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13500cc0f0 0x7f13500cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.925+0000 7f134f7fe640 1 --2- 172.21.15.148:0/2958479184 >> v2:172.21.15.157:3300/0 conn(0x7f1350144980 0x7f135006fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.925+0000 7f134e7fc640 1 -- 172.21.15.148:0/2958479184 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3845008398 0 0) 0x7f13500cc660 con 0x7f13500cfbd0 2024-03-29T04:10:11.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.925+0000 7f134e7fc640 1 -- 172.21.15.148:0/2958479184 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1338003520 con 0x7f13500cfbd0 2024-03-29T04:10:11.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.925+0000 7f134e7fc640 1 -- 172.21.15.148:0/2958479184 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2224215759 0 0) 0x7f1338003520 con 0x7f13500cfbd0 2024-03-29T04:10:11.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.925+0000 7f134e7fc640 1 -- 172.21.15.148:0/2958479184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13500cc0f0 msgr2=0x7f13500cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:11.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.925+0000 7f134e7fc640 1 --2- 172.21.15.148:0/2958479184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13500cc0f0 0x7f13500cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.925+0000 7f134e7fc640 1 -- 172.21.15.148:0/2958479184 >> v2:172.21.15.157:3300/0 conn(0x7f1350144980 msgr2=0x7f135006fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:11.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.925+0000 7f134e7fc640 1 --2- 172.21.15.148:0/2958479184 >> v2:172.21.15.157:3300/0 conn(0x7f1350144980 0x7f135006fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.925+0000 7f134e7fc640 1 -- 172.21.15.148:0/2958479184 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13500cc940 con 0x7f13500cfbd0 2024-03-29T04:10:11.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.925+0000 7f134e7fc640 1 -- 172.21.15.148:0/2958479184 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f13480019f0 con 0x7f13500cfbd0 2024-03-29T04:10:11.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.925+0000 7f134e7fc640 1 -- 172.21.15.148:0/2958479184 >> v1:172.21.15.148:6789/0 conn(0x7f13500cfbd0 legacy=0x7f135015edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.925+0000 7f134e7fc640 1 --2- 172.21.15.148:0/2958479184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13500cc0f0 0x7f13380039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.925+0000 7f134e7fc640 1 --2- 172.21.15.148:0/2958479184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1338003f30 0x7f1338004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.925+0000 7f134e7fc640 1 --2- 172.21.15.148:0/2958479184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1338004d40 0x7f13380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.926+0000 7f134e7fc640 1 -- 172.21.15.148:0/2958479184 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f13500cd270 con 0x7f1338003f30 2024-03-29T04:10:11.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.926+0000 7f134e7fc640 1 -- 172.21.15.148:0/2958479184 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f133800b9c0 con 0x7f1338004d40 2024-03-29T04:10:11.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.926+0000 7f134e7fc640 1 -- 172.21.15.148:0/2958479184 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f133800bb40 con 0x7f13500cc0f0 2024-03-29T04:10:11.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.926+0000 7f134f7fe640 1 --2- 172.21.15.148:0/2958479184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13500cc0f0 0x7f13380039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.926+0000 7f134ffff640 1 --2- 172.21.15.148:0/2958479184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1338004d40 0x7f13380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.926+0000 7f134effd640 1 --2- 172.21.15.148:0/2958479184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1338003f30 0x7f1338004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.926+0000 7f134f7fe640 1 -- 172.21.15.148:0/2958479184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1338004d40 msgr2=0x7f13380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:11.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.926+0000 7f134f7fe640 1 --2- 172.21.15.148:0/2958479184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1338004d40 0x7f13380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.926+0000 7f134f7fe640 1 -- 172.21.15.148:0/2958479184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1338003f30 msgr2=0x7f1338004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:11.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.926+0000 7f134f7fe640 1 --2- 172.21.15.148:0/2958479184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1338003f30 0x7f1338004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.926+0000 7f134f7fe640 1 -- 172.21.15.148:0/2958479184 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1338003520 con 0x7f13500cc0f0 2024-03-29T04:10:11.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.926+0000 7f134effd640 1 --2- 172.21.15.148:0/2958479184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1338003f30 0x7f1338004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:11.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.926+0000 7f134ffff640 1 --2- 172.21.15.148:0/2958479184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1338004d40 0x7f13380051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:11.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.927+0000 7f134f7fe640 1 --2- 172.21.15.148:0/2958479184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13500cc0f0 0x7f13380039b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1340009ac0 tx=0x7f1338005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9bda14c1a5d3f45 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:11.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.927+0000 7f134e7fc640 1 -- 172.21.15.148:0/2958479184 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1340039070 con 0x7f13500cc0f0 2024-03-29T04:10:11.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.927+0000 7f134e7fc640 1 -- 172.21.15.148:0/2958479184 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f134002c710 con 0x7f13500cc0f0 2024-03-29T04:10:11.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.927+0000 7f134e7fc640 1 -- 172.21.15.148:0/2958479184 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1340009690 con 0x7f13500cc0f0 2024-03-29T04:10:11.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.927+0000 7f1355f8d640 1 -- 172.21.15.148:0/2958479184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13500cc0f0 msgr2=0x7f13380039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.927+0000 7f1355f8d640 1 --2- 172.21.15.148:0/2958479184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13500cc0f0 0x7f13380039b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1340009ac0 tx=0x7f1338005830 comp rx=0 tx=0).stop 2024-03-29T04:10:11.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.927+0000 7f134ffff640 1 -- 172.21.15.148:0/2958479184 reap_dead start 2024-03-29T04:10:11.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.927+0000 7f1355f8d640 1 -- 172.21.15.148:0/2958479184 shutdown_connections 2024-03-29T04:10:11.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.927+0000 7f1355f8d640 1 -- 172.21.15.148:0/2958479184 >> 172.21.15.148:0/2958479184 conn(0x7f13500b2c60 msgr2=0x7f13500b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:11.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.927+0000 7f1355f8d640 1 -- 172.21.15.148:0/2958479184 shutdown_connections 2024-03-29T04:10:11.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.927+0000 7f1355f8d640 1 -- 172.21.15.148:0/2958479184 wait complete. 2024-03-29T04:10:11.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.929+0000 7f1355f8d640 1 Processor -- start 2024-03-29T04:10:11.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.929+0000 7f1355f8d640 1 -- start start 2024-03-29T04:10:11.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.929+0000 7f1355f8d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13500cc0f0 0x7f135015b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.929+0000 7f1355f8d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f135015d010 0x7f135015a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.929+0000 7f1355f8d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f135015a950 0x7f135013ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.929+0000 7f1355f8d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1350144d60 con 0x7f135015d010 2024-03-29T04:10:11.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.929+0000 7f1355f8d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f13500ccf00 con 0x7f135015a950 2024-03-29T04:10:11.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.929+0000 7f1355f8d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f13500cd700 con 0x7f13500cc0f0 2024-03-29T04:10:11.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.930+0000 7f134effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f135015d010 0x7f135015a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.930+0000 7f134effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f135015d010 0x7f135015a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51350/0 (socket says 172.21.15.148:51350) 2024-03-29T04:10:11.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.930+0000 7f134effd640 1 -- 172.21.15.148:0/435481671 learned_addr learned my addr 172.21.15.148:0/435481671 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:11.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.930+0000 7f134f7fe640 1 --2- 172.21.15.148:0/435481671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13500cc0f0 0x7f135015b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.930+0000 7f134ffff640 1 --2- 172.21.15.148:0/435481671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f135015a950 0x7f135013ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.930+0000 7f134effd640 1 -- 172.21.15.148:0/435481671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13500cc0f0 msgr2=0x7f135015b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.930+0000 7f134effd640 1 --2- 172.21.15.148:0/435481671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13500cc0f0 0x7f135015b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.930+0000 7f134effd640 1 -- 172.21.15.148:0/435481671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f135015a950 msgr2=0x7f135013ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.930+0000 7f134effd640 1 --2- 172.21.15.148:0/435481671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f135015a950 0x7f135013ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.930+0000 7f134effd640 1 -- 172.21.15.148:0/435481671 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135015bf70 con 0x7f135015d010 2024-03-29T04:10:11.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.930+0000 7f134ffff640 1 --2- 172.21.15.148:0/435481671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f135015a950 0x7f135013ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:11.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.930+0000 7f134effd640 1 --2- 172.21.15.148:0/435481671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f135015d010 0x7f135015a120 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1344011ac0 tx=0x7f1344011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:11.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.930+0000 7f1337fff640 1 -- 172.21.15.148:0/435481671 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134400fc10 con 0x7f135015d010 2024-03-29T04:10:11.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.930+0000 7f1337fff640 1 -- 172.21.15.148:0/435481671 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f134400fdb0 con 0x7f135015d010 2024-03-29T04:10:11.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.930+0000 7f1337fff640 1 -- 172.21.15.148:0/435481671 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344014660 con 0x7f135015d010 2024-03-29T04:10:11.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.930+0000 7f1355f8d640 1 -- 172.21.15.148:0/435481671 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f135015c250 con 0x7f135015d010 2024-03-29T04:10:11.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.930+0000 7f1355f8d640 1 -- 172.21.15.148:0/435481671 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f135015c810 con 0x7f135015d010 2024-03-29T04:10:11.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.931+0000 7f1355f8d640 1 -- 172.21.15.148:0/435481671 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1350068850 con 0x7f135015d010 2024-03-29T04:10:11.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.932+0000 7f1337fff640 1 -- 172.21.15.148:0/435481671 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1344012040 con 0x7f135015d010 2024-03-29T04:10:11.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.932+0000 7f1337fff640 1 --2- 172.21.15.148:0/435481671 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f133c06c530 0x7f133c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.932+0000 7f1337fff640 1 -- 172.21.15.148:0/435481671 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f134408c050 con 0x7f135015d010 2024-03-29T04:10:11.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.932+0000 7f134f7fe640 1 --2- 172.21.15.148:0/435481671 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f133c06c530 0x7f133c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.933+0000 7f134f7fe640 1 --2- 172.21.15.148:0/435481671 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f133c06c530 0x7f133c06e9f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1340036000 tx=0x7f1340030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:11.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:11.937+0000 7f1337fff640 1 -- 172.21.15.148:0/435481671 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1344019030 con 0x7f135015d010 2024-03-29T04:10:12.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.117+0000 7f1355f8d640 1 -- 172.21.15.148:0/435481671 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f135006eb30 con 0x7f135015d010 2024-03-29T04:10:12.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.117+0000 7f1337fff640 1 -- 172.21.15.148:0/435481671 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f134405a480 con 0x7f135015d010 2024-03-29T04:10:12.096 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-29T04:10:12.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.119+0000 7f1355f8d640 1 -- 172.21.15.148:0/435481671 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f133c06c530 msgr2=0x7f133c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:12.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.119+0000 7f1355f8d640 1 --2- 172.21.15.148:0/435481671 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f133c06c530 0x7f133c06e9f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1340036000 tx=0x7f1340030000 comp rx=0 tx=0).stop 2024-03-29T04:10:12.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.119+0000 7f1355f8d640 1 -- 172.21.15.148:0/435481671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f135015d010 msgr2=0x7f135015a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:12.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.119+0000 7f1355f8d640 1 --2- 172.21.15.148:0/435481671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f135015d010 0x7f135015a120 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1344011ac0 tx=0x7f1344011f80 comp rx=0 tx=0).stop 2024-03-29T04:10:12.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.119+0000 7f1355f8d640 1 -- 172.21.15.148:0/435481671 shutdown_connections 2024-03-29T04:10:12.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.119+0000 7f1355f8d640 1 --2- 172.21.15.148:0/435481671 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f133c06c530 0x7f133c06e9f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.119+0000 7f1355f8d640 1 --2- 172.21.15.148:0/435481671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f135015a950 0x7f135013ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.119+0000 7f1355f8d640 1 --2- 172.21.15.148:0/435481671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f135015d010 0x7f135015a120 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.119+0000 7f1355f8d640 1 --2- 172.21.15.148:0/435481671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13500cc0f0 0x7f135015b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.119+0000 7f1355f8d640 1 -- 172.21.15.148:0/435481671 >> 172.21.15.148:0/435481671 conn(0x7f13500b2c60 msgr2=0x7f13500ca2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:12.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.119+0000 7f1355f8d640 1 -- 172.21.15.148:0/435481671 shutdown_connections 2024-03-29T04:10:12.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.119+0000 7f1355f8d640 1 -- 172.21.15.148:0/435481671 wait complete. 2024-03-29T04:10:12.110 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:10:12.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.235+0000 7f744bd0e640 1 Processor -- start 2024-03-29T04:10:12.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.236+0000 7f744bd0e640 1 -- start start 2024-03-29T04:10:12.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.236+0000 7f744bd0e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f74440b3fd0 0x7f74440ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.236+0000 7f744bd0e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f744406f930 0x7f744406fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.236+0000 7f744bd0e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74440b4520 con 0x7f7444147f20 2024-03-29T04:10:12.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.236+0000 7f744bd0e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f744406cb80 con 0x7f74440b3fd0 2024-03-29T04:10:12.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.236+0000 7f744bd0e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f74440b4800 con 0x7f744406f930 2024-03-29T04:10:12.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.236+0000 7f7449aaa640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7444147f20 0x7f744406c470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50230/0 (socket says 172.21.15.148:50230) 2024-03-29T04:10:12.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.236+0000 7f7449aaa640 1 -- 172.21.15.148:0/370802359 learned_addr learned my addr 172.21.15.148:0/370802359 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:12.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.237+0000 7f744a2ab640 1 --2- 172.21.15.148:0/370802359 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f744406f930 0x7f744406fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:12.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.237+0000 7f7448aa8640 1 -- 172.21.15.148:0/370802359 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2478108242 0 0) 0x7f74440b4520 con 0x7f7444147f20 2024-03-29T04:10:12.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.237+0000 7f7448aa8640 1 -- 172.21.15.148:0/370802359 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7430003520 con 0x7f7444147f20 2024-03-29T04:10:12.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.237+0000 7f74492a9640 1 --2- 172.21.15.148:0/370802359 >> v2:172.21.15.157:3300/0 conn(0x7f74440b3fd0 0x7f74440ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:12.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.237+0000 7f7448aa8640 1 -- 172.21.15.148:0/370802359 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1326140453 0 0) 0x7f7430003520 con 0x7f7444147f20 2024-03-29T04:10:12.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.237+0000 7f7448aa8640 1 -- 172.21.15.148:0/370802359 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f744406f930 msgr2=0x7f744406fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:12.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.237+0000 7f7448aa8640 1 --2- 172.21.15.148:0/370802359 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f744406f930 0x7f744406fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.237+0000 7f7448aa8640 1 -- 172.21.15.148:0/370802359 >> v2:172.21.15.157:3300/0 conn(0x7f74440b3fd0 msgr2=0x7f74440ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:12.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.237+0000 7f7448aa8640 1 --2- 172.21.15.148:0/370802359 >> v2:172.21.15.157:3300/0 conn(0x7f74440b3fd0 0x7f74440ad8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.237+0000 7f7448aa8640 1 -- 172.21.15.148:0/370802359 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74440b4ae0 con 0x7f7444147f20 2024-03-29T04:10:12.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.237+0000 7f7448aa8640 1 -- 172.21.15.148:0/370802359 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f7440001670 con 0x7f7444147f20 2024-03-29T04:10:12.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.237+0000 7f7448aa8640 1 -- 172.21.15.148:0/370802359 >> v1:172.21.15.148:6789/0 conn(0x7f7444147f20 legacy=0x7f744406c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:12.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.237+0000 7f7448aa8640 1 --2- 172.21.15.148:0/370802359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74300039b0 0x7f7430003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.237+0000 7f7448aa8640 1 --2- 172.21.15.148:0/370802359 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f744406f930 0x7f7430004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.237+0000 7f7448aa8640 1 --2- 172.21.15.148:0/370802359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7430004d40 0x7f74300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.237+0000 7f7448aa8640 1 -- 172.21.15.148:0/370802359 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f74440b4800 con 0x7f74300039b0 2024-03-29T04:10:12.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.237+0000 7f7448aa8640 1 -- 172.21.15.148:0/370802359 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f743000b9c0 con 0x7f7430004d40 2024-03-29T04:10:12.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.237+0000 7f7448aa8640 1 -- 172.21.15.148:0/370802359 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f743000bb40 con 0x7f744406f930 2024-03-29T04:10:12.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.237+0000 7f7449aaa640 1 --2- 172.21.15.148:0/370802359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74300039b0 0x7f7430003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:12.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.237+0000 7f74492a9640 1 --2- 172.21.15.148:0/370802359 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f744406f930 0x7f7430004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:12.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.237+0000 7f744a2ab640 1 --2- 172.21.15.148:0/370802359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7430004d40 0x7f74300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:12.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.238+0000 7f7449aaa640 1 -- 172.21.15.148:0/370802359 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f744406f930 msgr2=0x7f7430004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:12.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.238+0000 7f7449aaa640 1 --2- 172.21.15.148:0/370802359 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f744406f930 0x7f7430004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.238+0000 7f7449aaa640 1 -- 172.21.15.148:0/370802359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7430004d40 msgr2=0x7f74300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:12.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.238+0000 7f7449aaa640 1 --2- 172.21.15.148:0/370802359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7430004d40 0x7f74300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.238+0000 7f7449aaa640 1 -- 172.21.15.148:0/370802359 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7430003520 con 0x7f74300039b0 2024-03-29T04:10:12.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.238+0000 7f7449aaa640 1 --2- 172.21.15.148:0/370802359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74300039b0 0x7f7430003dd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f744000c210 tx=0x7f7430005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd86dfdf4d8ee4b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:12.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.238+0000 7f7448aa8640 1 -- 172.21.15.148:0/370802359 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7440007d70 con 0x7f74300039b0 2024-03-29T04:10:12.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.238+0000 7f7448aa8640 1 -- 172.21.15.148:0/370802359 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7440030070 con 0x7f74300039b0 2024-03-29T04:10:12.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.238+0000 7f7448aa8640 1 -- 172.21.15.148:0/370802359 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744002cd60 con 0x7f74300039b0 2024-03-29T04:10:12.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.239+0000 7f744bd0e640 1 -- 172.21.15.148:0/370802359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74300039b0 msgr2=0x7f7430003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:12.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.239+0000 7f744bd0e640 1 --2- 172.21.15.148:0/370802359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74300039b0 0x7f7430003dd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f744000c210 tx=0x7f7430005830 comp rx=0 tx=0).stop 2024-03-29T04:10:12.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.239+0000 7f744a2ab640 1 -- 172.21.15.148:0/370802359 reap_dead start 2024-03-29T04:10:12.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.239+0000 7f744bd0e640 1 -- 172.21.15.148:0/370802359 shutdown_connections 2024-03-29T04:10:12.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.239+0000 7f744bd0e640 1 -- 172.21.15.148:0/370802359 >> 172.21.15.148:0/370802359 conn(0x7f74440d1060 msgr2=0x7f74440d1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:12.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.239+0000 7f744bd0e640 1 -- 172.21.15.148:0/370802359 shutdown_connections 2024-03-29T04:10:12.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.239+0000 7f744bd0e640 1 -- 172.21.15.148:0/370802359 wait complete. 2024-03-29T04:10:12.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.240+0000 7f744bd0e640 1 Processor -- start 2024-03-29T04:10:12.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.240+0000 7f744bd0e640 1 -- start start 2024-03-29T04:10:12.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.241+0000 7f744bd0e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74300039b0 0x7f7444152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.241+0000 7f744bd0e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74441f89f0 0x7f74440afa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.241+0000 7f744bd0e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74440af030 0x7f74441f7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.241+0000 7f744bd0e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f744406ff40 con 0x7f74300039b0 2024-03-29T04:10:12.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.241+0000 7f744bd0e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f744406fdc0 con 0x7f74441f89f0 2024-03-29T04:10:12.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.241+0000 7f744bd0e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f74440700c0 con 0x7f74440af030 2024-03-29T04:10:12.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.241+0000 7f7449aaa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74300039b0 0x7f7444152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:12.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.241+0000 7f7449aaa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74300039b0 0x7f7444152610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51354/0 (socket says 172.21.15.148:51354) 2024-03-29T04:10:12.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.241+0000 7f7449aaa640 1 -- 172.21.15.148:0/2501489395 learned_addr learned my addr 172.21.15.148:0/2501489395 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:12.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.241+0000 7f744a2ab640 1 --2- 172.21.15.148:0/2501489395 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74440af030 0x7f74441f7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:12.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.241+0000 7f7449aaa640 1 -- 172.21.15.148:0/2501489395 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74440af030 msgr2=0x7f74441f7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:12.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.241+0000 7f7449aaa640 1 --2- 172.21.15.148:0/2501489395 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74440af030 0x7f74441f7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.241+0000 7f7449aaa640 1 -- 172.21.15.148:0/2501489395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74441f89f0 msgr2=0x7f74440afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:12.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.241+0000 7f7449aaa640 1 --2- 172.21.15.148:0/2501489395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74441f89f0 0x7f74440afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.241+0000 7f7449aaa640 1 -- 172.21.15.148:0/2501489395 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74440affb0 con 0x7f74300039b0 2024-03-29T04:10:12.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.242+0000 7f7449aaa640 1 --2- 172.21.15.148:0/2501489395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74300039b0 0x7f7444152610 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7440000c00 tx=0x7f7440000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:12.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.242+0000 7f743a7fc640 1 -- 172.21.15.148:0/2501489395 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7440004bf0 con 0x7f74300039b0 2024-03-29T04:10:12.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.242+0000 7f743a7fc640 1 -- 172.21.15.148:0/2501489395 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7440004d90 con 0x7f74300039b0 2024-03-29T04:10:12.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.242+0000 7f744bd0e640 1 -- 172.21.15.148:0/2501489395 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7444140840 con 0x7f74300039b0 2024-03-29T04:10:12.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.242+0000 7f743a7fc640 1 -- 172.21.15.148:0/2501489395 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74400072b0 con 0x7f74300039b0 2024-03-29T04:10:12.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.242+0000 7f744bd0e640 1 -- 172.21.15.148:0/2501489395 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7444140e00 con 0x7f74300039b0 2024-03-29T04:10:12.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.243+0000 7f744bd0e640 1 -- 172.21.15.148:0/2501489395 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74440b50a0 con 0x7f74300039b0 2024-03-29T04:10:12.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.243+0000 7f743a7fc640 1 -- 172.21.15.148:0/2501489395 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7440007450 con 0x7f74300039b0 2024-03-29T04:10:12.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.244+0000 7f743a7fc640 1 --2- 172.21.15.148:0/2501489395 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f742806c530 0x7f742806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.244+0000 7f743a7fc640 1 -- 172.21.15.148:0/2501489395 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f74400ad030 con 0x7f74300039b0 2024-03-29T04:10:12.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.244+0000 7f74492a9640 1 --2- 172.21.15.148:0/2501489395 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f742806c530 0x7f742806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:12.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.244+0000 7f74492a9640 1 --2- 172.21.15.148:0/2501489395 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f742806c530 0x7f742806e9f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7434006b10 tx=0x7f7434006aa0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:12.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.248+0000 7f743a7fc640 1 -- 172.21.15.148:0/2501489395 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f74440b50a0 con 0x7f74300039b0 2024-03-29T04:10:12.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.428+0000 7f744bd0e640 1 -- 172.21.15.148:0/2501489395 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f74440d0ac0 con 0x7f74300039b0 2024-03-29T04:10:12.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.429+0000 7f743a7fc640 1 -- 172.21.15.148:0/2501489395 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f74440d0ac0 con 0x7f74300039b0 2024-03-29T04:10:12.408 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:10:12.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.431+0000 7f744bd0e640 1 -- 172.21.15.148:0/2501489395 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f742806c530 msgr2=0x7f742806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:12.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.431+0000 7f744bd0e640 1 --2- 172.21.15.148:0/2501489395 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f742806c530 0x7f742806e9f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7434006b10 tx=0x7f7434006aa0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.431+0000 7f744bd0e640 1 -- 172.21.15.148:0/2501489395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74300039b0 msgr2=0x7f7444152610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:12.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.431+0000 7f744bd0e640 1 --2- 172.21.15.148:0/2501489395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74300039b0 0x7f7444152610 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7440000c00 tx=0x7f7440000fe0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.431+0000 7f744bd0e640 1 -- 172.21.15.148:0/2501489395 shutdown_connections 2024-03-29T04:10:12.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.431+0000 7f744bd0e640 1 --2- 172.21.15.148:0/2501489395 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f742806c530 0x7f742806e9f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.431+0000 7f744bd0e640 1 --2- 172.21.15.148:0/2501489395 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74440af030 0x7f74441f7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.431+0000 7f744bd0e640 1 --2- 172.21.15.148:0/2501489395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74441f89f0 0x7f74440afa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.431+0000 7f744bd0e640 1 --2- 172.21.15.148:0/2501489395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74300039b0 0x7f7444152610 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.431+0000 7f744bd0e640 1 -- 172.21.15.148:0/2501489395 >> 172.21.15.148:0/2501489395 conn(0x7f74440d1060 msgr2=0x7f744415f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:12.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.431+0000 7f744bd0e640 1 -- 172.21.15.148:0/2501489395 shutdown_connections 2024-03-29T04:10:12.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.431+0000 7f744bd0e640 1 -- 172.21.15.148:0/2501489395 wait complete. 2024-03-29T04:10:12.421 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:10:12.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.562+0000 7f4da75bd640 1 Processor -- start 2024-03-29T04:10:12.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.562+0000 7f4da75bd640 1 -- start start 2024-03-29T04:10:12.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.563+0000 7f4da75bd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4da00ba030 0x7f4da00d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.563+0000 7f4da75bd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4da00d10d0 0x7f4da00d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.563+0000 7f4da75bd640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4da015e7c0 con 0x7f4da015e3e0 2024-03-29T04:10:12.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.563+0000 7f4da75bd640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f4da00d3840 con 0x7f4da00d10d0 2024-03-29T04:10:12.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.563+0000 7f4da75bd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4da00d2c30 con 0x7f4da00ba030 2024-03-29T04:10:12.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.563+0000 7f4da5359640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4da015e3e0 0x7f4da00d2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50244/0 (socket says 172.21.15.148:50244) 2024-03-29T04:10:12.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.563+0000 7f4da5b5a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4da00d10d0 0x7f4da00d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:12.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.563+0000 7f4da5359640 1 -- 172.21.15.148:0/2504276649 learned_addr learned my addr 172.21.15.148:0/2504276649 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:12.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.563+0000 7f4da4b58640 1 --2- 172.21.15.148:0/2504276649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4da00ba030 0x7f4da00d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:12.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.563+0000 7f4da5b5a640 1 -- 172.21.15.148:0/2504276649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4da00ba030 msgr2=0x7f4da00d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:12.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.563+0000 7f4da5b5a640 1 --2- 172.21.15.148:0/2504276649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4da00ba030 0x7f4da00d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.563+0000 7f4da5b5a640 1 -- 172.21.15.148:0/2504276649 >> v1:172.21.15.148:6789/0 conn(0x7f4da015e3e0 legacy=0x7f4da00d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:12.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.563+0000 7f4da5b5a640 1 -- 172.21.15.148:0/2504276649 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da00d2f10 con 0x7f4da00d10d0 2024-03-29T04:10:12.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.564+0000 7f4da5b5a640 1 --2- 172.21.15.148:0/2504276649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4da00d10d0 0x7f4da00d1580 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4d98009d90 tx=0x7f4d9802bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decb6a87fe53d855 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:12.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.565+0000 7f4d93fff640 1 -- 172.21.15.148:0/2504276649 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d98039070 con 0x7f4da00d10d0 2024-03-29T04:10:12.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.565+0000 7f4d93fff640 1 -- 172.21.15.148:0/2504276649 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d9802cce0 con 0x7f4da00d10d0 2024-03-29T04:10:12.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.565+0000 7f4d93fff640 1 -- 172.21.15.148:0/2504276649 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d98034660 con 0x7f4da00d10d0 2024-03-29T04:10:12.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.566+0000 7f4da75bd640 1 -- 172.21.15.148:0/2504276649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4da00d10d0 msgr2=0x7f4da00d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:12.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.566+0000 7f4da75bd640 1 --2- 172.21.15.148:0/2504276649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4da00d10d0 0x7f4da00d1580 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4d98009d90 tx=0x7f4d9802bf30 comp rx=0 tx=0).stop 2024-03-29T04:10:12.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.566+0000 7f4da75bd640 1 -- 172.21.15.148:0/2504276649 shutdown_connections 2024-03-29T04:10:12.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.566+0000 7f4da75bd640 1 --2- 172.21.15.148:0/2504276649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4da00d10d0 0x7f4da00d1580 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.566+0000 7f4da75bd640 1 --2- 172.21.15.148:0/2504276649 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4da00ba030 0x7f4da00d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.566+0000 7f4da75bd640 1 -- 172.21.15.148:0/2504276649 >> 172.21.15.148:0/2504276649 conn(0x7f4da00b6210 msgr2=0x7f4da00b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:12.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.566+0000 7f4da75bd640 1 -- 172.21.15.148:0/2504276649 shutdown_connections 2024-03-29T04:10:12.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.566+0000 7f4da75bd640 1 -- 172.21.15.148:0/2504276649 wait complete. 2024-03-29T04:10:12.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.567+0000 7f4da75bd640 1 Processor -- start 2024-03-29T04:10:12.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.567+0000 7f4da75bd640 1 -- start start 2024-03-29T04:10:12.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.568+0000 7f4da75bd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4da00ba030 0x7f4da00cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.568+0000 7f4da75bd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4da00d10d0 0x7f4da0157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.568+0000 7f4da75bd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4da015e3e0 0x7f4da01594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.568+0000 7f4da75bd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4da0167f50 con 0x7f4da00ba030 2024-03-29T04:10:12.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.568+0000 7f4da75bd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4da00d34d0 con 0x7f4da00d10d0 2024-03-29T04:10:12.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.568+0000 7f4da75bd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4da01680d0 con 0x7f4da015e3e0 2024-03-29T04:10:12.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.568+0000 7f4da5359640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4da00ba030 0x7f4da00cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:12.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.568+0000 7f4da5359640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4da00ba030 0x7f4da00cbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51364/0 (socket says 172.21.15.148:51364) 2024-03-29T04:10:12.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.568+0000 7f4da5359640 1 -- 172.21.15.148:0/4161823511 learned_addr learned my addr 172.21.15.148:0/4161823511 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:12.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.568+0000 7f4da5b5a640 1 --2- 172.21.15.148:0/4161823511 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4da015e3e0 0x7f4da01594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:12.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.568+0000 7f4da4b58640 1 --2- 172.21.15.148:0/4161823511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4da00d10d0 0x7f4da0157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:12.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.569+0000 7f4da5359640 1 -- 172.21.15.148:0/4161823511 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4da015e3e0 msgr2=0x7f4da01594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:12.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.569+0000 7f4da5359640 1 --2- 172.21.15.148:0/4161823511 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4da015e3e0 0x7f4da01594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.569+0000 7f4da5359640 1 -- 172.21.15.148:0/4161823511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4da00d10d0 msgr2=0x7f4da0157830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:12.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.569+0000 7f4da5359640 1 --2- 172.21.15.148:0/4161823511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4da00d10d0 0x7f4da0157830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.569+0000 7f4da5359640 1 -- 172.21.15.148:0/4161823511 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da015b6a0 con 0x7f4da00ba030 2024-03-29T04:10:12.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.569+0000 7f4da5359640 1 --2- 172.21.15.148:0/4161823511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4da00ba030 0x7f4da00cbdf0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4d9400c3c0 tx=0x7f4d9400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:12.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.569+0000 7f4d91ffb640 1 -- 172.21.15.148:0/4161823511 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d94017070 con 0x7f4da00ba030 2024-03-29T04:10:12.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.569+0000 7f4d91ffb640 1 -- 172.21.15.148:0/4161823511 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d94005ce0 con 0x7f4da00ba030 2024-03-29T04:10:12.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.569+0000 7f4d91ffb640 1 -- 172.21.15.148:0/4161823511 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d940072f0 con 0x7f4da00ba030 2024-03-29T04:10:12.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.569+0000 7f4da75bd640 1 -- 172.21.15.148:0/4161823511 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da0157d70 con 0x7f4da00ba030 2024-03-29T04:10:12.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.569+0000 7f4da75bd640 1 -- 172.21.15.148:0/4161823511 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da00cc840 con 0x7f4da00ba030 2024-03-29T04:10:12.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.570+0000 7f4da75bd640 1 -- 172.21.15.148:0/4161823511 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da00687e0 con 0x7f4da00ba030 2024-03-29T04:10:12.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.570+0000 7f4d91ffb640 1 -- 172.21.15.148:0/4161823511 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4d9401a070 con 0x7f4da00ba030 2024-03-29T04:10:12.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.571+0000 7f4d91ffb640 1 --2- 172.21.15.148:0/4161823511 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4d8406c530 0x7f4d8406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.571+0000 7f4d91ffb640 1 -- 172.21.15.148:0/4161823511 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4d9408ace0 con 0x7f4da00ba030 2024-03-29T04:10:12.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.571+0000 7f4da4b58640 1 --2- 172.21.15.148:0/4161823511 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4d8406c530 0x7f4d8406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:12.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.571+0000 7f4da4b58640 1 --2- 172.21.15.148:0/4161823511 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4d8406c530 0x7f4d8406e9f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4da00cdd90 tx=0x7f4d9c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:12.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.576+0000 7f4d91ffb640 1 -- 172.21.15.148:0/4161823511 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4da015b6a0 con 0x7f4da00ba030 2024-03-29T04:10:12.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.771+0000 7f4da75bd640 1 -- 172.21.15.148:0/4161823511 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4da00c7f10 con 0x7f4da00ba030 2024-03-29T04:10:12.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.772+0000 7f4d91ffb640 1 -- 172.21.15.148:0/4161823511 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f4d94058970 con 0x7f4da00ba030 2024-03-29T04:10:12.752 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-03-29T04:10:12.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.774+0000 7f4da75bd640 1 -- 172.21.15.148:0/4161823511 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4d8406c530 msgr2=0x7f4d8406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:12.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.774+0000 7f4da75bd640 1 --2- 172.21.15.148:0/4161823511 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4d8406c530 0x7f4d8406e9f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4da00cdd90 tx=0x7f4d9c009190 comp rx=0 tx=0).stop 2024-03-29T04:10:12.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.774+0000 7f4da75bd640 1 -- 172.21.15.148:0/4161823511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4da00ba030 msgr2=0x7f4da00cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:12.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.774+0000 7f4da75bd640 1 --2- 172.21.15.148:0/4161823511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4da00ba030 0x7f4da00cbdf0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4d9400c3c0 tx=0x7f4d9400c880 comp rx=0 tx=0).stop 2024-03-29T04:10:12.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.774+0000 7f4da75bd640 1 -- 172.21.15.148:0/4161823511 shutdown_connections 2024-03-29T04:10:12.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.774+0000 7f4da75bd640 1 --2- 172.21.15.148:0/4161823511 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4d8406c530 0x7f4d8406e9f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.774+0000 7f4da75bd640 1 --2- 172.21.15.148:0/4161823511 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4da015e3e0 0x7f4da01594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.774+0000 7f4da75bd640 1 --2- 172.21.15.148:0/4161823511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4da00d10d0 0x7f4da0157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.774+0000 7f4da75bd640 1 --2- 172.21.15.148:0/4161823511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4da00ba030 0x7f4da00cbdf0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.774+0000 7f4da75bd640 1 -- 172.21.15.148:0/4161823511 >> 172.21.15.148:0/4161823511 conn(0x7f4da00b6210 msgr2=0x7f4da00b8640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:12.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.774+0000 7f4da75bd640 1 -- 172.21.15.148:0/4161823511 shutdown_connections 2024-03-29T04:10:12.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.774+0000 7f4da75bd640 1 -- 172.21.15.148:0/4161823511 wait complete. 2024-03-29T04:10:12.764 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:10:12.764 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:10:12.780 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:12.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.942+0000 7f6aabbdb640 1 Processor -- start 2024-03-29T04:10:12.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.942+0000 7f6aabbdb640 1 -- start start 2024-03-29T04:10:12.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.944+0000 7f6aabbdb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6aa40b5990 0x7f6aa406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.944+0000 7f6aabbdb640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6aa40cd9e0 con 0x7f6aa40cd600 2024-03-29T04:10:12.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.944+0000 7f6aabbdb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6aa406cbf0 con 0x7f6aa406c6c0 2024-03-29T04:10:12.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.944+0000 7f6aabbdb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f6aa406d800 con 0x7f6aa40b5990 2024-03-29T04:10:12.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.944+0000 7f6aa9977640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6aa40cd600 0x7f6aa415c140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50260/0 (socket says 172.21.15.148:50260) 2024-03-29T04:10:12.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.944+0000 7f6aa9977640 1 -- 172.21.15.148:0/1751904981 learned_addr learned my addr 172.21.15.148:0/1751904981 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:12.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.944+0000 7f6aaa178640 1 --2- 172.21.15.148:0/1751904981 >> v2:172.21.15.157:3300/0 conn(0x7f6aa40b5990 0x7f6aa406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:12.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.944+0000 7f6aa8975640 1 -- 172.21.15.148:0/1751904981 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3752450380 0 0) 0x7f6aa40cd9e0 con 0x7f6aa40cd600 2024-03-29T04:10:12.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.944+0000 7f6aa8975640 1 -- 172.21.15.148:0/1751904981 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a90003520 con 0x7f6aa40cd600 2024-03-29T04:10:12.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.944+0000 7f6aa8975640 1 -- 172.21.15.148:0/1751904981 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3085162014 0 0) 0x7f6a90003520 con 0x7f6aa40cd600 2024-03-29T04:10:12.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.944+0000 7f6aa8975640 1 -- 172.21.15.148:0/1751904981 >> v2:172.21.15.157:3300/0 conn(0x7f6aa40b5990 msgr2=0x7f6aa406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:12.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.944+0000 7f6aa8975640 1 --2- 172.21.15.148:0/1751904981 >> v2:172.21.15.157:3300/0 conn(0x7f6aa40b5990 0x7f6aa406dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.944+0000 7f6aa8975640 1 -- 172.21.15.148:0/1751904981 >> v1:172.21.15.157:6789/0 conn(0x7f6aa406c6c0 legacy=0x7f6aa40b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:12.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.944+0000 7f6aa8975640 1 -- 172.21.15.148:0/1751904981 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa406ced0 con 0x7f6aa40cd600 2024-03-29T04:10:12.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.944+0000 7f6aa8975640 1 -- 172.21.15.148:0/1751904981 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f6a9c0019f0 con 0x7f6aa40cd600 2024-03-29T04:10:12.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.944+0000 7f6aa8975640 1 -- 172.21.15.148:0/1751904981 >> v1:172.21.15.148:6789/0 conn(0x7f6aa40cd600 legacy=0x7f6aa415c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:12.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.944+0000 7f6aa8975640 1 --2- 172.21.15.148:0/1751904981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a900039b0 0x7f6a90003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.944+0000 7f6aa8975640 1 --2- 172.21.15.148:0/1751904981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a90004800 0x7f6a90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.944+0000 7f6aa8975640 1 --2- 172.21.15.148:0/1751904981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a900051c0 0x7f6a9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.944+0000 7f6aa8975640 1 -- 172.21.15.148:0/1751904981 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa406d800 con 0x7f6a900039b0 2024-03-29T04:10:12.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.944+0000 7f6aa8975640 1 -- 172.21.15.148:0/1751904981 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9000c060 con 0x7f6a90004800 2024-03-29T04:10:12.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.944+0000 7f6aa8975640 1 -- 172.21.15.148:0/1751904981 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6a9000c260 con 0x7f6a900051c0 2024-03-29T04:10:12.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.944+0000 7f6aa9977640 1 --2- 172.21.15.148:0/1751904981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a900039b0 0x7f6a90003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:12.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.944+0000 7f6aa9176640 1 --2- 172.21.15.148:0/1751904981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a90004800 0x7f6a90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:12.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.945+0000 7f6aaa178640 1 --2- 172.21.15.148:0/1751904981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a900051c0 0x7f6a9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:12.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.945+0000 7f6aa9977640 1 -- 172.21.15.148:0/1751904981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a900051c0 msgr2=0x7f6a9000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:12.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.945+0000 7f6aa9977640 1 --2- 172.21.15.148:0/1751904981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a900051c0 0x7f6a9000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.945+0000 7f6aa9977640 1 -- 172.21.15.148:0/1751904981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a90004800 msgr2=0x7f6a90004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:12.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.945+0000 7f6aa9977640 1 --2- 172.21.15.148:0/1751904981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a90004800 0x7f6a90004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.945+0000 7f6aa9977640 1 -- 172.21.15.148:0/1751904981 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a90003520 con 0x7f6a900039b0 2024-03-29T04:10:12.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.945+0000 7f6aaa178640 1 -- 172.21.15.148:0/1751904981 reap_dead start 2024-03-29T04:10:12.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.945+0000 7f6aa9977640 1 --2- 172.21.15.148:0/1751904981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a900039b0 0x7f6a90003dd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00a840 tx=0x7f6a90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ca0ec4bbf9dca2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:12.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.946+0000 7f6aa8975640 1 -- 172.21.15.148:0/1751904981 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c00a460 con 0x7f6a900039b0 2024-03-29T04:10:12.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.946+0000 7f6aa8975640 1 -- 172.21.15.148:0/1751904981 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a9c034630 con 0x7f6a900039b0 2024-03-29T04:10:12.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.946+0000 7f6aa8975640 1 -- 172.21.15.148:0/1751904981 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c034a60 con 0x7f6a900039b0 2024-03-29T04:10:12.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.946+0000 7f6aabbdb640 1 -- 172.21.15.148:0/1751904981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a900039b0 msgr2=0x7f6a90003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:12.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.946+0000 7f6aabbdb640 1 --2- 172.21.15.148:0/1751904981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a900039b0 0x7f6a90003dd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00a840 tx=0x7f6a90005830 comp rx=0 tx=0).stop 2024-03-29T04:10:12.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.946+0000 7f6aabbdb640 1 -- 172.21.15.148:0/1751904981 shutdown_connections 2024-03-29T04:10:12.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.946+0000 7f6aabbdb640 1 --2- 172.21.15.148:0/1751904981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a900039b0 0x7f6a90003dd0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.946+0000 7f6aabbdb640 1 -- 172.21.15.148:0/1751904981 >> 172.21.15.148:0/1751904981 conn(0x7f6aa40b2c60 msgr2=0x7f6aa40b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:12.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.946+0000 7f6aabbdb640 1 -- 172.21.15.148:0/1751904981 shutdown_connections 2024-03-29T04:10:12.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.946+0000 7f6aabbdb640 1 -- 172.21.15.148:0/1751904981 wait complete. 2024-03-29T04:10:12.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.947+0000 7f6aabbdb640 1 Processor -- start 2024-03-29T04:10:12.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.947+0000 7f6aabbdb640 1 -- start start 2024-03-29T04:10:12.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.948+0000 7f6aabbdb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a900039b0 0x7f6aa4147a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.948+0000 7f6aabbdb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6aa4155040 0x7f6aa4154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.948+0000 7f6aabbdb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6aa415a190 0x7f6aa415a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.948+0000 7f6aabbdb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa406dc90 con 0x7f6a900039b0 2024-03-29T04:10:12.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.948+0000 7f6aabbdb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa406d490 con 0x7f6aa415a190 2024-03-29T04:10:12.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.948+0000 7f6aabbdb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6aa406de10 con 0x7f6aa4155040 2024-03-29T04:10:12.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.948+0000 7f6aa9977640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a900039b0 0x7f6aa4147a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:12.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.948+0000 7f6aa9977640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a900039b0 0x7f6aa4147a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51384/0 (socket says 172.21.15.148:51384) 2024-03-29T04:10:12.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.948+0000 7f6aa9977640 1 -- 172.21.15.148:0/766163332 learned_addr learned my addr 172.21.15.148:0/766163332 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:12.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.948+0000 7f6aaa178640 1 --2- 172.21.15.148:0/766163332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6aa415a190 0x7f6aa415a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:12.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.948+0000 7f6aa9176640 1 --2- 172.21.15.148:0/766163332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6aa4155040 0x7f6aa4154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:12.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.949+0000 7f6aa9977640 1 -- 172.21.15.148:0/766163332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6aa4155040 msgr2=0x7f6aa4154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:12.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.949+0000 7f6aa9977640 1 --2- 172.21.15.148:0/766163332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6aa4155040 0x7f6aa4154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.949+0000 7f6aa9977640 1 -- 172.21.15.148:0/766163332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6aa415a190 msgr2=0x7f6aa415a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:12.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.949+0000 7f6aa9977640 1 --2- 172.21.15.148:0/766163332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6aa415a190 0x7f6aa415a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.949+0000 7f6aa9977640 1 -- 172.21.15.148:0/766163332 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa4147f80 con 0x7f6a900039b0 2024-03-29T04:10:12.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.949+0000 7f6aa9977640 1 --2- 172.21.15.148:0/766163332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a900039b0 0x7f6aa4147a40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c001cf0 tx=0x7f6aa406cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:12.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.949+0000 7f6a9a7fc640 1 -- 172.21.15.148:0/766163332 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c039040 con 0x7f6a900039b0 2024-03-29T04:10:12.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.949+0000 7f6a9a7fc640 1 -- 172.21.15.148:0/766163332 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a9c043070 con 0x7f6a900039b0 2024-03-29T04:10:12.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.949+0000 7f6aabbdb640 1 -- 172.21.15.148:0/766163332 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa4154990 con 0x7f6a900039b0 2024-03-29T04:10:12.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.949+0000 7f6a9a7fc640 1 -- 172.21.15.148:0/766163332 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c004d60 con 0x7f6a900039b0 2024-03-29T04:10:12.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.949+0000 7f6aabbdb640 1 -- 172.21.15.148:0/766163332 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa40a99f0 con 0x7f6a900039b0 2024-03-29T04:10:12.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.950+0000 7f6aabbdb640 1 -- 172.21.15.148:0/766163332 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aa40687e0 con 0x7f6a900039b0 2024-03-29T04:10:12.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.951+0000 7f6a9a7fc640 1 -- 172.21.15.148:0/766163332 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6a9c034630 con 0x7f6a900039b0 2024-03-29T04:10:12.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.951+0000 7f6a9a7fc640 1 --2- 172.21.15.148:0/766163332 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6a8806c530 0x7f6a8806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.951+0000 7f6a9a7fc640 1 -- 172.21.15.148:0/766163332 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6a9c0ada80 con 0x7f6a900039b0 2024-03-29T04:10:12.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.951+0000 7f6aa9176640 1 --2- 172.21.15.148:0/766163332 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6a8806c530 0x7f6a8806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:12.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.952+0000 7f6aa9176640 1 --2- 172.21.15.148:0/766163332 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6a8806c530 0x7f6a8806e9f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40b0500 tx=0x7f6a94009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:12.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:12.956+0000 7f6a9a7fc640 1 -- 172.21.15.148:0/766163332 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6aa4147f80 con 0x7f6a900039b0 2024-03-29T04:10:13.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.140+0000 7f6aabbdb640 1 -- 172.21.15.148:0/766163332 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6aa40a94c0 con 0x7f6a900039b0 2024-03-29T04:10:13.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.142+0000 7f6a9a7fc640 1 -- 172.21.15.148:0/766163332 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7f6a9c07bdf0 con 0x7f6a900039b0 2024-03-29T04:10:13.121 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:13.121 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:05.026481+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:13.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.143+0000 7f6aabbdb640 1 -- 172.21.15.148:0/766163332 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6a8806c530 msgr2=0x7f6a8806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:13.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.143+0000 7f6aabbdb640 1 --2- 172.21.15.148:0/766163332 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6a8806c530 0x7f6a8806e9f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40b0500 tx=0x7f6a94009040 comp rx=0 tx=0).stop 2024-03-29T04:10:13.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.144+0000 7f6aabbdb640 1 -- 172.21.15.148:0/766163332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a900039b0 msgr2=0x7f6aa4147a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:13.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.144+0000 7f6aabbdb640 1 --2- 172.21.15.148:0/766163332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a900039b0 0x7f6aa4147a40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c001cf0 tx=0x7f6aa406cdc0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.144+0000 7f6aabbdb640 1 -- 172.21.15.148:0/766163332 shutdown_connections 2024-03-29T04:10:13.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.144+0000 7f6aabbdb640 1 --2- 172.21.15.148:0/766163332 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6a8806c530 0x7f6a8806e9f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.144+0000 7f6aabbdb640 1 --2- 172.21.15.148:0/766163332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6aa415a190 0x7f6aa415a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.144+0000 7f6aabbdb640 1 --2- 172.21.15.148:0/766163332 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6aa4155040 0x7f6aa4154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.144+0000 7f6aabbdb640 1 --2- 172.21.15.148:0/766163332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a900039b0 0x7f6aa4147a40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.144+0000 7f6aabbdb640 1 -- 172.21.15.148:0/766163332 >> 172.21.15.148:0/766163332 conn(0x7f6aa40b2c60 msgr2=0x7f6aa40ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:13.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.144+0000 7f6aabbdb640 1 -- 172.21.15.148:0/766163332 shutdown_connections 2024-03-29T04:10:13.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.144+0000 7f6aabbdb640 1 -- 172.21.15.148:0/766163332 wait complete. 2024-03-29T04:10:13.134 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:47.898351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:57.922313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:58.829405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:10:02.551763+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:10:13.134 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:10:13.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.261+0000 7f83193af640 1 Processor -- start 2024-03-29T04:10:13.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.261+0000 7f83193af640 1 -- start start 2024-03-29T04:10:13.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.262+0000 7f83193af640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83140b3430 0x7f8314162680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:13.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.262+0000 7f83193af640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83140cf330 con 0x7f831415e3c0 2024-03-29T04:10:13.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.262+0000 7f83193af640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8314162cf0 con 0x7f83140b2f00 2024-03-29T04:10:13.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.262+0000 7f83193af640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f83140cf960 con 0x7f83140b3430 2024-03-29T04:10:13.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.262+0000 7f8312ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f831415e3c0 0x7f8314161f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50268/0 (socket says 172.21.15.148:50268) 2024-03-29T04:10:13.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.262+0000 7f8312ffd640 1 -- 172.21.15.148:0/2030228489 learned_addr learned my addr 172.21.15.148:0/2030228489 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:13.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.262+0000 7f83137fe640 1 --2- 172.21.15.148:0/2030228489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83140b3430 0x7f8314162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:13.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.262+0000 7f83127fc640 1 -- 172.21.15.148:0/2030228489 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2857834628 0 0) 0x7f83140cf330 con 0x7f831415e3c0 2024-03-29T04:10:13.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.262+0000 7f83127fc640 1 -- 172.21.15.148:0/2030228489 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8300003520 con 0x7f831415e3c0 2024-03-29T04:10:13.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.262+0000 7f83127fc640 1 -- 172.21.15.148:0/2030228489 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1696487201 0 0) 0x7f8300003520 con 0x7f831415e3c0 2024-03-29T04:10:13.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.263+0000 7f83127fc640 1 -- 172.21.15.148:0/2030228489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83140b3430 msgr2=0x7f8314162680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:13.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.263+0000 7f83127fc640 1 --2- 172.21.15.148:0/2030228489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83140b3430 0x7f8314162680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.263+0000 7f83127fc640 1 -- 172.21.15.148:0/2030228489 >> v1:172.21.15.157:6789/0 conn(0x7f83140b2f00 legacy=0x7f831406b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:13.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.263+0000 7f83127fc640 1 -- 172.21.15.148:0/2030228489 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83140cf5f0 con 0x7f831415e3c0 2024-03-29T04:10:13.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.263+0000 7f83127fc640 1 -- 172.21.15.148:0/2030228489 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f8304001670 con 0x7f831415e3c0 2024-03-29T04:10:13.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.263+0000 7f83137fe640 1 --2- 172.21.15.148:0/2030228489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83140b3430 0x7f8314162680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:13.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.263+0000 7f83127fc640 1 -- 172.21.15.148:0/2030228489 >> v1:172.21.15.148:6789/0 conn(0x7f831415e3c0 legacy=0x7f8314161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:13.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.263+0000 7f83127fc640 1 --2- 172.21.15.148:0/2030228489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83000039e0 0x7f8300003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:13.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.263+0000 7f83127fc640 1 --2- 172.21.15.148:0/2030228489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83000043b0 0x7f8300004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:13.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.263+0000 7f83127fc640 1 --2- 172.21.15.148:0/2030228489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83140b3430 0x7f8300005140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:13.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.263+0000 7f83127fc640 1 -- 172.21.15.148:0/2030228489 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f83140cf960 con 0x7f83000043b0 2024-03-29T04:10:13.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.263+0000 7f83127fc640 1 -- 172.21.15.148:0/2030228489 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f830000ba70 con 0x7f83000039e0 2024-03-29T04:10:13.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.263+0000 7f83127fc640 1 -- 172.21.15.148:0/2030228489 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f830000bc40 con 0x7f83140b3430 2024-03-29T04:10:13.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.263+0000 7f8312ffd640 1 --2- 172.21.15.148:0/2030228489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83000039e0 0x7f8300003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:13.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.263+0000 7f830bfff640 1 --2- 172.21.15.148:0/2030228489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83000043b0 0x7f8300004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:13.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.263+0000 7f83137fe640 1 --2- 172.21.15.148:0/2030228489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83140b3430 0x7f8300005140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:13.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.263+0000 7f8312ffd640 1 -- 172.21.15.148:0/2030228489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83140b3430 msgr2=0x7f8300005140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:13.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.263+0000 7f8312ffd640 1 --2- 172.21.15.148:0/2030228489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83140b3430 0x7f8300005140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.263+0000 7f8312ffd640 1 -- 172.21.15.148:0/2030228489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83000043b0 msgr2=0x7f8300004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:13.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.263+0000 7f8312ffd640 1 --2- 172.21.15.148:0/2030228489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83000043b0 0x7f8300004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.263+0000 7f8312ffd640 1 -- 172.21.15.148:0/2030228489 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8300003520 con 0x7f83000039e0 2024-03-29T04:10:13.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.263+0000 7f830bfff640 1 --2- 172.21.15.148:0/2030228489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83000043b0 0x7f8300004880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:13.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.264+0000 7f8312ffd640 1 --2- 172.21.15.148:0/2030228489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83000039e0 0x7f8300003de0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f830400c240 tx=0x7f8300005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9b9f8d73a1bb507 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:13.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.264+0000 7f83127fc640 1 -- 172.21.15.148:0/2030228489 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304007d10 con 0x7f83000039e0 2024-03-29T04:10:13.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.264+0000 7f83127fc640 1 -- 172.21.15.148:0/2030228489 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8304030070 con 0x7f83000039e0 2024-03-29T04:10:13.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.264+0000 7f83127fc640 1 -- 172.21.15.148:0/2030228489 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830402cd60 con 0x7f83000039e0 2024-03-29T04:10:13.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.265+0000 7f83193af640 1 -- 172.21.15.148:0/2030228489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83000039e0 msgr2=0x7f8300003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:13.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.265+0000 7f83193af640 1 --2- 172.21.15.148:0/2030228489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83000039e0 0x7f8300003de0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f830400c240 tx=0x7f8300005830 comp rx=0 tx=0).stop 2024-03-29T04:10:13.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.265+0000 7f83137fe640 1 -- 172.21.15.148:0/2030228489 reap_dead start 2024-03-29T04:10:13.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.265+0000 7f83193af640 1 -- 172.21.15.148:0/2030228489 shutdown_connections 2024-03-29T04:10:13.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.265+0000 7f83193af640 1 -- 172.21.15.148:0/2030228489 >> 172.21.15.148:0/2030228489 conn(0x7f83140b6430 msgr2=0x7f83140d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:13.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.265+0000 7f83193af640 1 -- 172.21.15.148:0/2030228489 shutdown_connections 2024-03-29T04:10:13.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.265+0000 7f83193af640 1 -- 172.21.15.148:0/2030228489 wait complete. 2024-03-29T04:10:13.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.266+0000 7f83193af640 1 Processor -- start 2024-03-29T04:10:13.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.266+0000 7f83193af640 1 -- start start 2024-03-29T04:10:13.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.267+0000 7f83193af640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83000039e0 0x7f8314151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:13.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.267+0000 7f83193af640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8314151f80 0x7f83140cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:13.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.267+0000 7f83193af640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83140c7e00 0x7f83140b8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:13.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.267+0000 7f83193af640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f83140b3b20 con 0x7f83000039e0 2024-03-29T04:10:13.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.267+0000 7f83193af640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f83140b39a0 con 0x7f8314151f80 2024-03-29T04:10:13.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.267+0000 7f83193af640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8314162680 con 0x7f83140c7e00 2024-03-29T04:10:13.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.267+0000 7f8312ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83000039e0 0x7f8314151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:13.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.267+0000 7f8312ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83000039e0 0x7f8314151840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51402/0 (socket says 172.21.15.148:51402) 2024-03-29T04:10:13.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.267+0000 7f8312ffd640 1 -- 172.21.15.148:0/2204925707 learned_addr learned my addr 172.21.15.148:0/2204925707 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:13.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.267+0000 7f830bfff640 1 --2- 172.21.15.148:0/2204925707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8314151f80 0x7f83140cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:13.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.267+0000 7f83137fe640 1 --2- 172.21.15.148:0/2204925707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83140c7e00 0x7f83140b8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:13.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.267+0000 7f8312ffd640 1 -- 172.21.15.148:0/2204925707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83140c7e00 msgr2=0x7f83140b8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:13.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.267+0000 7f8312ffd640 1 --2- 172.21.15.148:0/2204925707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83140c7e00 0x7f83140b8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.267+0000 7f8312ffd640 1 -- 172.21.15.148:0/2204925707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8314151f80 msgr2=0x7f83140cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:13.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.267+0000 7f8312ffd640 1 --2- 172.21.15.148:0/2204925707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8314151f80 0x7f83140cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.267+0000 7f8312ffd640 1 -- 172.21.15.148:0/2204925707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8314151d80 con 0x7f83000039e0 2024-03-29T04:10:13.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.267+0000 7f830bfff640 1 --2- 172.21.15.148:0/2204925707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8314151f80 0x7f83140cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:13.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.267+0000 7f83137fe640 1 --2- 172.21.15.148:0/2204925707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83140c7e00 0x7f83140b8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:13.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.267+0000 7f8312ffd640 1 --2- 172.21.15.148:0/2204925707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83000039e0 0x7f8314151840 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f830400c360 tx=0x7f83140cf7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:13.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.267+0000 7f830b7fe640 1 -- 172.21.15.148:0/2204925707 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304045070 con 0x7f83000039e0 2024-03-29T04:10:13.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.267+0000 7f830b7fe640 1 -- 172.21.15.148:0/2204925707 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83040049e0 con 0x7f83000039e0 2024-03-29T04:10:13.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.267+0000 7f83193af640 1 -- 172.21.15.148:0/2204925707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8314152540 con 0x7f83000039e0 2024-03-29T04:10:13.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.267+0000 7f830b7fe640 1 -- 172.21.15.148:0/2204925707 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304001c50 con 0x7f83000039e0 2024-03-29T04:10:13.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.267+0000 7f83193af640 1 -- 172.21.15.148:0/2204925707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8314152b00 con 0x7f83000039e0 2024-03-29T04:10:13.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.268+0000 7f83193af640 1 -- 172.21.15.148:0/2204925707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8314152d00 con 0x7f83000039e0 2024-03-29T04:10:13.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.269+0000 7f830b7fe640 1 -- 172.21.15.148:0/2204925707 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8304007070 con 0x7f83000039e0 2024-03-29T04:10:13.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.269+0000 7f830b7fe640 1 --2- 172.21.15.148:0/2204925707 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f82f406c560 0x7f82f406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:13.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.270+0000 7f830b7fe640 1 -- 172.21.15.148:0/2204925707 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f83040aca40 con 0x7f83000039e0 2024-03-29T04:10:13.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.270+0000 7f830bfff640 1 --2- 172.21.15.148:0/2204925707 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f82f406c560 0x7f82f406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:13.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.270+0000 7f830bfff640 1 --2- 172.21.15.148:0/2204925707 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f82f406c560 0x7f82f406ea20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f83140ceb30 tx=0x7f82fc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:13.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.274+0000 7f830b7fe640 1 -- 172.21.15.148:0/2204925707 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8314152d00 con 0x7f83000039e0 2024-03-29T04:10:13.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.454+0000 7f83193af640 1 -- 172.21.15.148:0/2204925707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8314068850 con 0x7f83000039e0 2024-03-29T04:10:13.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.454+0000 7f830b7fe640 1 -- 172.21.15.148:0/2204925707 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f83040aae20 con 0x7f83000039e0 2024-03-29T04:10:13.433 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-29T04:10:13.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.456+0000 7f83193af640 1 -- 172.21.15.148:0/2204925707 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f82f406c560 msgr2=0x7f82f406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:13.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.456+0000 7f83193af640 1 --2- 172.21.15.148:0/2204925707 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f82f406c560 0x7f82f406ea20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f83140ceb30 tx=0x7f82fc009040 comp rx=0 tx=0).stop 2024-03-29T04:10:13.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.456+0000 7f83193af640 1 -- 172.21.15.148:0/2204925707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83000039e0 msgr2=0x7f8314151840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:13.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.456+0000 7f83193af640 1 --2- 172.21.15.148:0/2204925707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83000039e0 0x7f8314151840 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f830400c360 tx=0x7f83140cf7c0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.456+0000 7f83193af640 1 -- 172.21.15.148:0/2204925707 shutdown_connections 2024-03-29T04:10:13.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.456+0000 7f83193af640 1 --2- 172.21.15.148:0/2204925707 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f82f406c560 0x7f82f406ea20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.456+0000 7f83193af640 1 --2- 172.21.15.148:0/2204925707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83140c7e00 0x7f83140b8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.456+0000 7f83193af640 1 --2- 172.21.15.148:0/2204925707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8314151f80 0x7f83140cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.456+0000 7f83193af640 1 --2- 172.21.15.148:0/2204925707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83000039e0 0x7f8314151840 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.456+0000 7f83193af640 1 -- 172.21.15.148:0/2204925707 >> 172.21.15.148:0/2204925707 conn(0x7f83140b6430 msgr2=0x7f831415d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:13.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.456+0000 7f83193af640 1 -- 172.21.15.148:0/2204925707 shutdown_connections 2024-03-29T04:10:13.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.456+0000 7f83193af640 1 -- 172.21.15.148:0/2204925707 wait complete. 2024-03-29T04:10:13.446 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:10:13.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.588+0000 7f9857cb1640 1 Processor -- start 2024-03-29T04:10:13.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.589+0000 7f9857cb1640 1 -- start start 2024-03-29T04:10:13.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.589+0000 7f9857cb1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985015c230 0x7f985006f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:13.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.589+0000 7f9857cb1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f985006fe70 0x7f98500b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:13.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.589+0000 7f9857cb1640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f985006cf10 con 0x7f9850158be0 2024-03-29T04:10:13.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.589+0000 7f9857cb1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f98500702a0 con 0x7f985006fe70 2024-03-29T04:10:13.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.589+0000 7f9857cb1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f985006d1f0 con 0x7f985015c230 2024-03-29T04:10:13.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.590+0000 7f9855a4d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9850158be0 0x7f985006c6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50278/0 (socket says 172.21.15.148:50278) 2024-03-29T04:10:13.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.590+0000 7f9855a4d640 1 -- 172.21.15.148:0/1739503889 learned_addr learned my addr 172.21.15.148:0/1739503889 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:13.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.590+0000 7f985524c640 1 --2- 172.21.15.148:0/1739503889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985015c230 0x7f985006f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:13.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.590+0000 7f985624e640 1 --2- 172.21.15.148:0/1739503889 >> v2:172.21.15.157:3300/0 conn(0x7f985006fe70 0x7f98500b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:13.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.590+0000 7f9854a4b640 1 -- 172.21.15.148:0/1739503889 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3419159803 0 0) 0x7f985006cf10 con 0x7f9850158be0 2024-03-29T04:10:13.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.590+0000 7f9854a4b640 1 -- 172.21.15.148:0/1739503889 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f983c003520 con 0x7f9850158be0 2024-03-29T04:10:13.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.590+0000 7f985524c640 1 -- 172.21.15.148:0/1739503889 >> v2:172.21.15.157:3300/0 conn(0x7f985006fe70 msgr2=0x7f98500b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:13.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.590+0000 7f985524c640 1 --2- 172.21.15.148:0/1739503889 >> v2:172.21.15.157:3300/0 conn(0x7f985006fe70 0x7f98500b4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.590+0000 7f985524c640 1 -- 172.21.15.148:0/1739503889 >> v1:172.21.15.148:6789/0 conn(0x7f9850158be0 legacy=0x7f985006c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:13.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.590+0000 7f985524c640 1 -- 172.21.15.148:0/1739503889 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f985006d4d0 con 0x7f985015c230 2024-03-29T04:10:13.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.591+0000 7f985524c640 1 --2- 172.21.15.148:0/1739503889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985015c230 0x7f985006f930 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f984c0097a0 tx=0x7f984c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df5980bddcc3e683 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:13.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.591+0000 7f9854a4b640 1 -- 172.21.15.148:0/1739503889 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c035020 con 0x7f985015c230 2024-03-29T04:10:13.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.591+0000 7f9854a4b640 1 -- 172.21.15.148:0/1739503889 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f984c031830 con 0x7f985015c230 2024-03-29T04:10:13.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.591+0000 7f9854a4b640 1 -- 172.21.15.148:0/1739503889 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c031be0 con 0x7f985015c230 2024-03-29T04:10:13.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.592+0000 7f9857cb1640 1 -- 172.21.15.148:0/1739503889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985015c230 msgr2=0x7f985006f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:13.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.592+0000 7f9857cb1640 1 --2- 172.21.15.148:0/1739503889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985015c230 0x7f985006f930 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f984c0097a0 tx=0x7f984c02b8d0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.592+0000 7f9857cb1640 1 -- 172.21.15.148:0/1739503889 shutdown_connections 2024-03-29T04:10:13.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.592+0000 7f9857cb1640 1 --2- 172.21.15.148:0/1739503889 >> v2:172.21.15.157:3300/0 conn(0x7f985006fe70 0x7f98500b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.592+0000 7f9857cb1640 1 --2- 172.21.15.148:0/1739503889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985015c230 0x7f985006f930 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.592+0000 7f9857cb1640 1 -- 172.21.15.148:0/1739503889 >> 172.21.15.148:0/1739503889 conn(0x7f98500d1060 msgr2=0x7f98500d1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:13.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.592+0000 7f9857cb1640 1 -- 172.21.15.148:0/1739503889 shutdown_connections 2024-03-29T04:10:13.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.592+0000 7f9857cb1640 1 -- 172.21.15.148:0/1739503889 wait complete. 2024-03-29T04:10:13.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.594+0000 7f9857cb1640 1 Processor -- start 2024-03-29T04:10:13.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.594+0000 7f9857cb1640 1 -- start start 2024-03-29T04:10:13.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.595+0000 7f9857cb1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f985006fe70 0x7f9850153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:13.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.595+0000 7f9857cb1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9850158be0 0x7f98501f2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:13.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.595+0000 7f9857cb1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985015c230 0x7f98501f2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:13.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.595+0000 7f9857cb1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f98500b4bd0 con 0x7f9850158be0 2024-03-29T04:10:13.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.595+0000 7f9857cb1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9850158fc0 con 0x7f985006fe70 2024-03-29T04:10:13.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.595+0000 7f9857cb1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f98500b4d50 con 0x7f985015c230 2024-03-29T04:10:13.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.595+0000 7f985524c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9850158be0 0x7f98501f2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:13.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.595+0000 7f985524c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9850158be0 0x7f98501f2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51406/0 (socket says 172.21.15.148:51406) 2024-03-29T04:10:13.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.595+0000 7f985524c640 1 -- 172.21.15.148:0/4238545137 learned_addr learned my addr 172.21.15.148:0/4238545137 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:13.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.595+0000 7f985624e640 1 --2- 172.21.15.148:0/4238545137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985015c230 0x7f98501f2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:13.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.595+0000 7f9855a4d640 1 --2- 172.21.15.148:0/4238545137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f985006fe70 0x7f9850153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:13.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.595+0000 7f985524c640 1 -- 172.21.15.148:0/4238545137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985015c230 msgr2=0x7f98501f2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:13.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.595+0000 7f985524c640 1 --2- 172.21.15.148:0/4238545137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985015c230 0x7f98501f2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.595+0000 7f985524c640 1 -- 172.21.15.148:0/4238545137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f985006fe70 msgr2=0x7f9850153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:13.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.595+0000 7f985524c640 1 --2- 172.21.15.148:0/4238545137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f985006fe70 0x7f9850153db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.595+0000 7f985524c640 1 -- 172.21.15.148:0/4238545137 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9850154fb0 con 0x7f9850158be0 2024-03-29T04:10:13.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.595+0000 7f985524c640 1 --2- 172.21.15.148:0/4238545137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9850158be0 0x7f98501f2230 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f984c02b910 tx=0x7f984c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:13.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.596+0000 7f98467fc640 1 -- 172.21.15.148:0/4238545137 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c040070 con 0x7f9850158be0 2024-03-29T04:10:13.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.596+0000 7f98467fc640 1 -- 172.21.15.148:0/4238545137 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f984c03e070 con 0x7f9850158be0 2024-03-29T04:10:13.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.596+0000 7f98467fc640 1 -- 172.21.15.148:0/4238545137 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c035040 con 0x7f9850158be0 2024-03-29T04:10:13.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.596+0000 7f9857cb1640 1 -- 172.21.15.148:0/4238545137 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9850152a30 con 0x7f9850158be0 2024-03-29T04:10:13.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.596+0000 7f9857cb1640 1 -- 172.21.15.148:0/4238545137 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98501532d0 con 0x7f9850158be0 2024-03-29T04:10:13.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.597+0000 7f9857cb1640 1 -- 172.21.15.148:0/4238545137 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9850068850 con 0x7f9850158be0 2024-03-29T04:10:13.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.597+0000 7f98467fc640 1 -- 172.21.15.148:0/4238545137 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f984c038020 con 0x7f9850158be0 2024-03-29T04:10:13.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.598+0000 7f98467fc640 1 --2- 172.21.15.148:0/4238545137 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f983406c530 0x7f983406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:13.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.598+0000 7f9855a4d640 1 --2- 172.21.15.148:0/4238545137 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f983406c530 0x7f983406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:13.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.598+0000 7f98467fc640 1 -- 172.21.15.148:0/4238545137 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f984c0ad0e0 con 0x7f9850158be0 2024-03-29T04:10:13.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.598+0000 7f9855a4d640 1 --2- 172.21.15.148:0/4238545137 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f983406c530 0x7f983406e9f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9840005880 tx=0x7f9840009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:13.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.602+0000 7f98467fc640 1 -- 172.21.15.148:0/4238545137 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9850154fb0 con 0x7f9850158be0 2024-03-29T04:10:13.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.880+0000 7f9857cb1640 1 -- 172.21.15.148:0/4238545137 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9850166e00 con 0x7f9850158be0 2024-03-29T04:10:13.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.880+0000 7f98467fc640 1 -- 172.21.15.148:0/4238545137 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f984c07ae00 con 0x7f9850158be0 2024-03-29T04:10:13.860 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:10:13.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.882+0000 7f9857cb1640 1 -- 172.21.15.148:0/4238545137 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f983406c530 msgr2=0x7f983406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:13.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.882+0000 7f9857cb1640 1 --2- 172.21.15.148:0/4238545137 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f983406c530 0x7f983406e9f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9840005880 tx=0x7f9840009040 comp rx=0 tx=0).stop 2024-03-29T04:10:13.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.882+0000 7f9857cb1640 1 -- 172.21.15.148:0/4238545137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9850158be0 msgr2=0x7f98501f2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:13.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.882+0000 7f9857cb1640 1 --2- 172.21.15.148:0/4238545137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9850158be0 0x7f98501f2230 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f984c02b910 tx=0x7f984c0046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.882+0000 7f9857cb1640 1 -- 172.21.15.148:0/4238545137 shutdown_connections 2024-03-29T04:10:13.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.882+0000 7f9857cb1640 1 --2- 172.21.15.148:0/4238545137 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f983406c530 0x7f983406e9f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.882+0000 7f9857cb1640 1 --2- 172.21.15.148:0/4238545137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f985015c230 0x7f98501f2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.882+0000 7f9857cb1640 1 --2- 172.21.15.148:0/4238545137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9850158be0 0x7f98501f2230 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.882+0000 7f9857cb1640 1 --2- 172.21.15.148:0/4238545137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f985006fe70 0x7f9850153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.882+0000 7f9857cb1640 1 -- 172.21.15.148:0/4238545137 >> 172.21.15.148:0/4238545137 conn(0x7f98500d1060 msgr2=0x7f985015e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:13.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.882+0000 7f9857cb1640 1 -- 172.21.15.148:0/4238545137 shutdown_connections 2024-03-29T04:10:13.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:13.882+0000 7f9857cb1640 1 -- 172.21.15.148:0/4238545137 wait complete. 2024-03-29T04:10:13.873 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:10:13.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.002+0000 7fb587ffd640 1 Processor -- start 2024-03-29T04:10:13.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.003+0000 7fb587ffd640 1 -- start start 2024-03-29T04:10:13.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.003+0000 7fb587ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb58015a120 0x7fb5800700d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:13.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.003+0000 7fb587ffd640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb58015d460 con 0x7fb58006efe0 2024-03-29T04:10:13.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.003+0000 7fb587ffd640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5800d0f60 con 0x7fb5800ce310 2024-03-29T04:10:13.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.003+0000 7fb587ffd640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb5800d1b20 con 0x7fb58015a120 2024-03-29T04:10:13.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.004+0000 7fb58659a640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb58006efe0 0x7fb5800b6f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50282/0 (socket says 172.21.15.148:50282) 2024-03-29T04:10:13.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.004+0000 7fb58659a640 1 -- 172.21.15.148:0/1570657269 learned_addr learned my addr 172.21.15.148:0/1570657269 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:13.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.004+0000 7fb585d99640 1 --2- 172.21.15.148:0/1570657269 >> v2:172.21.15.157:3300/0 conn(0x7fb58015a120 0x7fb5800700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:13.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.004+0000 7fb584d97640 1 -- 172.21.15.148:0/1570657269 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2624875526 0 0) 0x7fb58015d460 con 0x7fb58006efe0 2024-03-29T04:10:13.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.004+0000 7fb584d97640 1 -- 172.21.15.148:0/1570657269 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb568003520 con 0x7fb58006efe0 2024-03-29T04:10:13.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.004+0000 7fb584d97640 1 -- 172.21.15.148:0/1570657269 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1305193244 0 0) 0x7fb568003520 con 0x7fb58006efe0 2024-03-29T04:10:13.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.004+0000 7fb584d97640 1 -- 172.21.15.148:0/1570657269 >> v2:172.21.15.157:3300/0 conn(0x7fb58015a120 msgr2=0x7fb5800700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:13.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.004+0000 7fb584d97640 1 --2- 172.21.15.148:0/1570657269 >> v2:172.21.15.157:3300/0 conn(0x7fb58015a120 0x7fb5800700d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.004+0000 7fb584d97640 1 -- 172.21.15.148:0/1570657269 >> v1:172.21.15.157:6789/0 conn(0x7fb5800ce310 legacy=0x7fb58006e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:13.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.004+0000 7fb584d97640 1 -- 172.21.15.148:0/1570657269 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb58015d700 con 0x7fb58006efe0 2024-03-29T04:10:13.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.004+0000 7fb584d97640 1 -- 172.21.15.148:0/1570657269 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7fb570001690 con 0x7fb58006efe0 2024-03-29T04:10:13.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.004+0000 7fb584d97640 1 -- 172.21.15.148:0/1570657269 >> v1:172.21.15.148:6789/0 conn(0x7fb58006efe0 legacy=0x7fb5800b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:13.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.004+0000 7fb584d97640 1 --2- 172.21.15.148:0/1570657269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5680039b0 0x7fb568003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:13.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.004+0000 7fb584d97640 1 --2- 172.21.15.148:0/1570657269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb568004800 0x7fb568004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.004+0000 7fb584d97640 1 --2- 172.21.15.148:0/1570657269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5680051c0 0x7fb56800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.004+0000 7fb584d97640 1 -- 172.21.15.148:0/1570657269 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb56800c030 con 0x7fb5680051c0 2024-03-29T04:10:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.004+0000 7fb584d97640 1 -- 172.21.15.148:0/1570657269 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb56800c1e0 con 0x7fb568004800 2024-03-29T04:10:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.004+0000 7fb584d97640 1 -- 172.21.15.148:0/1570657269 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb56800c3e0 con 0x7fb5680039b0 2024-03-29T04:10:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.004+0000 7fb58659a640 1 --2- 172.21.15.148:0/1570657269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5680051c0 0x7fb56800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.004+0000 7fb585d99640 1 --2- 172.21.15.148:0/1570657269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5680039b0 0x7fb568003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.004+0000 7fb585598640 1 --2- 172.21.15.148:0/1570657269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb568004800 0x7fb568004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.005+0000 7fb58659a640 1 -- 172.21.15.148:0/1570657269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5680039b0 msgr2=0x7fb568003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.005+0000 7fb58659a640 1 --2- 172.21.15.148:0/1570657269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5680039b0 0x7fb568003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.005+0000 7fb58659a640 1 -- 172.21.15.148:0/1570657269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb568004800 msgr2=0x7fb568004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.005+0000 7fb58659a640 1 --2- 172.21.15.148:0/1570657269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb568004800 0x7fb568004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.005+0000 7fb58659a640 1 -- 172.21.15.148:0/1570657269 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb568003520 con 0x7fb5680051c0 2024-03-29T04:10:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.005+0000 7fb58659a640 1 -- 172.21.15.148:0/1570657269 reap_dead start 2024-03-29T04:10:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.005+0000 7fb58659a640 1 --2- 172.21.15.148:0/1570657269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5680051c0 0x7fb56800b9c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb57000c210 tx=0x7fb568005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39fcc7491d786a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.005+0000 7fb584d97640 1 -- 172.21.15.148:0/1570657269 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570007d30 con 0x7fb5680051c0 2024-03-29T04:10:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.005+0000 7fb584d97640 1 -- 172.21.15.148:0/1570657269 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb570030070 con 0x7fb5680051c0 2024-03-29T04:10:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.005+0000 7fb584d97640 1 -- 172.21.15.148:0/1570657269 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570005d30 con 0x7fb5680051c0 2024-03-29T04:10:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.006+0000 7fb587ffd640 1 -- 172.21.15.148:0/1570657269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5680051c0 msgr2=0x7fb56800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.006+0000 7fb587ffd640 1 --2- 172.21.15.148:0/1570657269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5680051c0 0x7fb56800b9c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb57000c210 tx=0x7fb568005830 comp rx=0 tx=0).stop 2024-03-29T04:10:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.006+0000 7fb587ffd640 1 -- 172.21.15.148:0/1570657269 shutdown_connections 2024-03-29T04:10:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.006+0000 7fb587ffd640 1 --2- 172.21.15.148:0/1570657269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5680051c0 0x7fb56800b9c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.006+0000 7fb587ffd640 1 -- 172.21.15.148:0/1570657269 >> 172.21.15.148:0/1570657269 conn(0x7fb5800cead0 msgr2=0x7fb5800ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.006+0000 7fb587ffd640 1 -- 172.21.15.148:0/1570657269 shutdown_connections 2024-03-29T04:10:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.006+0000 7fb587ffd640 1 -- 172.21.15.148:0/1570657269 wait complete. 2024-03-29T04:10:13.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.007+0000 7fb587ffd640 1 Processor -- start 2024-03-29T04:10:13.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.007+0000 7fb587ffd640 1 -- start start 2024-03-29T04:10:13.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.008+0000 7fb587ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5680051c0 0x7fb5801f1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:13.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.008+0000 7fb587ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb58014e010 0x7fb5800c6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:13.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.008+0000 7fb587ffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb58015b5b0 0x7fb58015ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:13.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.008+0000 7fb587ffd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb58006f5c0 con 0x7fb58014e010 2024-03-29T04:10:13.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.008+0000 7fb587ffd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb5800ce7d0 con 0x7fb5680051c0 2024-03-29T04:10:13.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.008+0000 7fb587ffd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb58006f740 con 0x7fb58015b5b0 2024-03-29T04:10:13.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.008+0000 7fb585598640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb58014e010 0x7fb5800c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:13.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.008+0000 7fb585598640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb58014e010 0x7fb5800c6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51430/0 (socket says 172.21.15.148:51430) 2024-03-29T04:10:13.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.008+0000 7fb585598640 1 -- 172.21.15.148:0/289832499 learned_addr learned my addr 172.21.15.148:0/289832499 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:13.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.008+0000 7fb585d99640 1 --2- 172.21.15.148:0/289832499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5680051c0 0x7fb5801f1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:13.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.008+0000 7fb58659a640 1 --2- 172.21.15.148:0/289832499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb58015b5b0 0x7fb58015ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:13.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.008+0000 7fb585598640 1 -- 172.21.15.148:0/289832499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb58015b5b0 msgr2=0x7fb58015ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:13.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.008+0000 7fb585598640 1 --2- 172.21.15.148:0/289832499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb58015b5b0 0x7fb58015ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.008+0000 7fb585598640 1 -- 172.21.15.148:0/289832499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5680051c0 msgr2=0x7fb5801f1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:13.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.008+0000 7fb585598640 1 --2- 172.21.15.148:0/289832499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5680051c0 0x7fb5801f1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.008+0000 7fb585598640 1 -- 172.21.15.148:0/289832499 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb58014e800 con 0x7fb58014e010 2024-03-29T04:10:13.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.008+0000 7fb585d99640 1 --2- 172.21.15.148:0/289832499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5680051c0 0x7fb5801f1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:13.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.009+0000 7fb585598640 1 --2- 172.21.15.148:0/289832499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb58014e010 0x7fb5800c6590 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb57400f5f0 tx=0x7fb57400fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:13.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.009+0000 7fb57e7fc640 1 -- 172.21.15.148:0/289832499 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5740052e0 con 0x7fb58014e010 2024-03-29T04:10:13.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.009+0000 7fb57e7fc640 1 -- 172.21.15.148:0/289832499 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb574018070 con 0x7fb58014e010 2024-03-29T04:10:13.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.009+0000 7fb587ffd640 1 -- 172.21.15.148:0/289832499 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5800c6c20 con 0x7fb58014e010 2024-03-29T04:10:13.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.009+0000 7fb57e7fc640 1 -- 172.21.15.148:0/289832499 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb574003810 con 0x7fb58014e010 2024-03-29T04:10:13.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.009+0000 7fb587ffd640 1 -- 172.21.15.148:0/289832499 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5800c71e0 con 0x7fb58014e010 2024-03-29T04:10:13.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.010+0000 7fb587ffd640 1 -- 172.21.15.148:0/289832499 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb580068850 con 0x7fb58014e010 2024-03-29T04:10:13.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.010+0000 7fb57e7fc640 1 -- 172.21.15.148:0/289832499 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb5740039b0 con 0x7fb58014e010 2024-03-29T04:10:13.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.011+0000 7fb57e7fc640 1 --2- 172.21.15.148:0/289832499 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb56c06c560 0x7fb56c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:13.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.011+0000 7fb57e7fc640 1 -- 172.21.15.148:0/289832499 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb57408b730 con 0x7fb58014e010 2024-03-29T04:10:13.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.011+0000 7fb585d99640 1 --2- 172.21.15.148:0/289832499 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb56c06c560 0x7fb56c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:13.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.011+0000 7fb585d99640 1 --2- 172.21.15.148:0/289832499 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb56c06c560 0x7fb56c06ea20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb578007440 tx=0x7fb57800b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:13.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.015+0000 7fb57e7fc640 1 -- 172.21.15.148:0/289832499 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb574059aa0 con 0x7fb58014e010 2024-03-29T04:10:14.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.195+0000 7fb587ffd640 1 -- 172.21.15.148:0/289832499 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb5800d1cb0 con 0x7fb58014e010 2024-03-29T04:10:14.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.196+0000 7fb57e7fc640 1 -- 172.21.15.148:0/289832499 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fb5740593a0 con 0x7fb58014e010 2024-03-29T04:10:14.175 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-29T04:10:14.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.198+0000 7fb587ffd640 1 -- 172.21.15.148:0/289832499 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb56c06c560 msgr2=0x7fb56c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:14.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.198+0000 7fb587ffd640 1 --2- 172.21.15.148:0/289832499 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb56c06c560 0x7fb56c06ea20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb578007440 tx=0x7fb57800b040 comp rx=0 tx=0).stop 2024-03-29T04:10:14.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.198+0000 7fb587ffd640 1 -- 172.21.15.148:0/289832499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb58014e010 msgr2=0x7fb5800c6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:14.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.198+0000 7fb587ffd640 1 --2- 172.21.15.148:0/289832499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb58014e010 0x7fb5800c6590 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb57400f5f0 tx=0x7fb57400fab0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.198+0000 7fb587ffd640 1 -- 172.21.15.148:0/289832499 shutdown_connections 2024-03-29T04:10:14.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.198+0000 7fb587ffd640 1 --2- 172.21.15.148:0/289832499 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb56c06c560 0x7fb56c06ea20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.198+0000 7fb587ffd640 1 --2- 172.21.15.148:0/289832499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb58015b5b0 0x7fb58015ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.198+0000 7fb587ffd640 1 --2- 172.21.15.148:0/289832499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb58014e010 0x7fb5800c6590 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.198+0000 7fb587ffd640 1 --2- 172.21.15.148:0/289832499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5680051c0 0x7fb5801f1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.198+0000 7fb587ffd640 1 -- 172.21.15.148:0/289832499 >> 172.21.15.148:0/289832499 conn(0x7fb5800cead0 msgr2=0x7fb5801586c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:14.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.198+0000 7fb587ffd640 1 -- 172.21.15.148:0/289832499 shutdown_connections 2024-03-29T04:10:14.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.198+0000 7fb587ffd640 1 -- 172.21.15.148:0/289832499 wait complete. 2024-03-29T04:10:14.191 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:10:14.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.316+0000 7fd955159640 1 Processor -- start 2024-03-29T04:10:14.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.316+0000 7fd955159640 1 -- start start 2024-03-29T04:10:14.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.317+0000 7fd955159640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd950161e10 0x7fd9500b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:14.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.317+0000 7fd955159640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd95006b9e0 0x7fd95006be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:14.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.317+0000 7fd955159640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9500b2cb0 con 0x7fd9500cc990 2024-03-29T04:10:14.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.317+0000 7fd955159640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fd95006c380 con 0x7fd950161e10 2024-03-29T04:10:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.317+0000 7fd955159640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd95006e750 con 0x7fd95006b9e0 2024-03-29T04:10:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.317+0000 7fd94ed9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd950161e10 0x7fd9500b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.317+0000 7fd94ed9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd950161e10 0x7fd9500b5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51434/0 (socket says 172.21.15.148:51434) 2024-03-29T04:10:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.317+0000 7fd94ed9d640 1 -- 172.21.15.148:0/3232229144 learned_addr learned my addr 172.21.15.148:0/3232229144 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.317+0000 7fd94659c640 1 --2- 172.21.15.148:0/3232229144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd95006b9e0 0x7fd95006be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.318+0000 7fd94ed9d640 1 -- 172.21.15.148:0/3232229144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd95006b9e0 msgr2=0x7fd95006be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.318+0000 7fd94ed9d640 1 --2- 172.21.15.148:0/3232229144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd95006b9e0 0x7fd95006be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.318+0000 7fd94ed9d640 1 -- 172.21.15.148:0/3232229144 >> v1:172.21.15.157:6789/0 conn(0x7fd9500cc990 legacy=0x7fd9500cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.318+0000 7fd94ed9d640 1 -- 172.21.15.148:0/3232229144 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd95006ea30 con 0x7fd950161e10 2024-03-29T04:10:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.318+0000 7fd94ed9d640 1 --2- 172.21.15.148:0/3232229144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd950161e10 0x7fd9500b5b80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd940009870 tx=0x7fd94002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b063d0f0139913 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.318+0000 7fd94e59c640 1 -- 172.21.15.148:0/3232229144 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94002f840 con 0x7fd950161e10 2024-03-29T04:10:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.319+0000 7fd94e59c640 1 -- 172.21.15.148:0/3232229144 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd94002f9e0 con 0x7fd950161e10 2024-03-29T04:10:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.319+0000 7fd94e59c640 1 -- 172.21.15.148:0/3232229144 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94002fdb0 con 0x7fd950161e10 2024-03-29T04:10:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.319+0000 7fd955159640 1 -- 172.21.15.148:0/3232229144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd950161e10 msgr2=0x7fd9500b5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.319+0000 7fd955159640 1 --2- 172.21.15.148:0/3232229144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd950161e10 0x7fd9500b5b80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd940009870 tx=0x7fd94002ba20 comp rx=0 tx=0).stop 2024-03-29T04:10:14.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.319+0000 7fd955159640 1 -- 172.21.15.148:0/3232229144 shutdown_connections 2024-03-29T04:10:14.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.319+0000 7fd955159640 1 --2- 172.21.15.148:0/3232229144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd95006b9e0 0x7fd95006be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.319+0000 7fd955159640 1 --2- 172.21.15.148:0/3232229144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd950161e10 0x7fd9500b5b80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.319+0000 7fd955159640 1 -- 172.21.15.148:0/3232229144 >> 172.21.15.148:0/3232229144 conn(0x7fd9500b1ef0 msgr2=0x7fd9500b2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:14.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.319+0000 7fd955159640 1 -- 172.21.15.148:0/3232229144 shutdown_connections 2024-03-29T04:10:14.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.319+0000 7fd955159640 1 -- 172.21.15.148:0/3232229144 wait complete. 2024-03-29T04:10:14.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.321+0000 7fd955159640 1 Processor -- start 2024-03-29T04:10:14.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.321+0000 7fd955159640 1 -- start start 2024-03-29T04:10:14.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.323+0000 7fd955159640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd95006b9e0 0x7fd9501f5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:14.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.323+0000 7fd955159640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9500cc990 0x7fd9501594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:14.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.323+0000 7fd955159640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd950161e10 0x7fd9500c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:14.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.323+0000 7fd955159640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd95006f2d0 con 0x7fd950161e10 2024-03-29T04:10:14.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.323+0000 7fd955159640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd95006eff0 con 0x7fd9500cc990 2024-03-29T04:10:14.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.323+0000 7fd955159640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd95006f7f0 con 0x7fd95006b9e0 2024-03-29T04:10:14.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.323+0000 7fd94f59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd950161e10 0x7fd9500c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:14.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.323+0000 7fd94f59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd950161e10 0x7fd9500c7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51448/0 (socket says 172.21.15.148:51448) 2024-03-29T04:10:14.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.323+0000 7fd94f59e640 1 -- 172.21.15.148:0/3015728729 learned_addr learned my addr 172.21.15.148:0/3015728729 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:14.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.323+0000 7fd94ed9d640 1 --2- 172.21.15.148:0/3015728729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd95006b9e0 0x7fd9501f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:14.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.323+0000 7fd94659c640 1 --2- 172.21.15.148:0/3015728729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9500cc990 0x7fd9501594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:14.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.323+0000 7fd94f59e640 1 -- 172.21.15.148:0/3015728729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd95006b9e0 msgr2=0x7fd9501f5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:14.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.323+0000 7fd94f59e640 1 --2- 172.21.15.148:0/3015728729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd95006b9e0 0x7fd9501f5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.323+0000 7fd94f59e640 1 -- 172.21.15.148:0/3015728729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9500cc990 msgr2=0x7fd9501594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:14.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.323+0000 7fd94f59e640 1 --2- 172.21.15.148:0/3015728729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9500cc990 0x7fd9501594f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.323+0000 7fd94f59e640 1 -- 172.21.15.148:0/3015728729 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd95015ee20 con 0x7fd950161e10 2024-03-29T04:10:14.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.323+0000 7fd94f59e640 1 --2- 172.21.15.148:0/3015728729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd950161e10 0x7fd9500c7d70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd94800c7d0 tx=0x7fd94800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:14.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.323+0000 7fd947fff640 1 -- 172.21.15.148:0/3015728729 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd948003b70 con 0x7fd950161e10 2024-03-29T04:10:14.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.323+0000 7fd947fff640 1 -- 172.21.15.148:0/3015728729 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd948003d30 con 0x7fd950161e10 2024-03-29T04:10:14.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.323+0000 7fd955159640 1 -- 172.21.15.148:0/3015728729 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9500ada10 con 0x7fd950161e10 2024-03-29T04:10:14.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.323+0000 7fd947fff640 1 -- 172.21.15.148:0/3015728729 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94800aa90 con 0x7fd950161e10 2024-03-29T04:10:14.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.324+0000 7fd955159640 1 -- 172.21.15.148:0/3015728729 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9500adfd0 con 0x7fd950161e10 2024-03-29T04:10:14.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.324+0000 7fd955159640 1 -- 172.21.15.148:0/3015728729 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd950068850 con 0x7fd950161e10 2024-03-29T04:10:14.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.325+0000 7fd947fff640 1 -- 172.21.15.148:0/3015728729 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd94800ac30 con 0x7fd950161e10 2024-03-29T04:10:14.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.325+0000 7fd947fff640 1 --2- 172.21.15.148:0/3015728729 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd93006c560 0x7fd93006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:14.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.326+0000 7fd94ed9d640 1 --2- 172.21.15.148:0/3015728729 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd93006c560 0x7fd93006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:14.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.326+0000 7fd947fff640 1 -- 172.21.15.148:0/3015728729 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd94808b7b0 con 0x7fd950161e10 2024-03-29T04:10:14.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.326+0000 7fd94ed9d640 1 --2- 172.21.15.148:0/3015728729 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd93006c560 0x7fd93006ea20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd940036000 tx=0x7fd940008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:14.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.330+0000 7fd947fff640 1 -- 172.21.15.148:0/3015728729 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd95015ee20 con 0x7fd950161e10 2024-03-29T04:10:14.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.510+0000 7fd955159640 1 -- 172.21.15.148:0/3015728729 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd950067b50 con 0x7fd950161e10 2024-03-29T04:10:14.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.510+0000 7fd947fff640 1 -- 172.21.15.148:0/3015728729 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7fd948059440 con 0x7fd950161e10 2024-03-29T04:10:14.489 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:10:14.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.512+0000 7fd955159640 1 -- 172.21.15.148:0/3015728729 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd93006c560 msgr2=0x7fd93006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:14.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.512+0000 7fd955159640 1 --2- 172.21.15.148:0/3015728729 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd93006c560 0x7fd93006ea20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd940036000 tx=0x7fd940008030 comp rx=0 tx=0).stop 2024-03-29T04:10:14.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.512+0000 7fd955159640 1 -- 172.21.15.148:0/3015728729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd950161e10 msgr2=0x7fd9500c7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:14.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.512+0000 7fd955159640 1 --2- 172.21.15.148:0/3015728729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd950161e10 0x7fd9500c7d70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd94800c7d0 tx=0x7fd94800cc90 comp rx=0 tx=0).stop 2024-03-29T04:10:14.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.512+0000 7fd955159640 1 -- 172.21.15.148:0/3015728729 shutdown_connections 2024-03-29T04:10:14.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.512+0000 7fd955159640 1 --2- 172.21.15.148:0/3015728729 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd93006c560 0x7fd93006ea20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.512+0000 7fd955159640 1 --2- 172.21.15.148:0/3015728729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd950161e10 0x7fd9500c7d70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.512+0000 7fd955159640 1 --2- 172.21.15.148:0/3015728729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9500cc990 0x7fd9501594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.512+0000 7fd955159640 1 --2- 172.21.15.148:0/3015728729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd95006b9e0 0x7fd9501f5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.512+0000 7fd955159640 1 -- 172.21.15.148:0/3015728729 >> 172.21.15.148:0/3015728729 conn(0x7fd9500b1ef0 msgr2=0x7fd950073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:14.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.512+0000 7fd955159640 1 -- 172.21.15.148:0/3015728729 shutdown_connections 2024-03-29T04:10:14.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.512+0000 7fd955159640 1 -- 172.21.15.148:0/3015728729 wait complete. 2024-03-29T04:10:14.505 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:10:14.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.629+0000 7f87b96a5640 1 Processor -- start 2024-03-29T04:10:14.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.630+0000 7f87b96a5640 1 -- start start 2024-03-29T04:10:14.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.631+0000 7f87b96a5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87b415c5f0 0x7f87b40cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:14.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.631+0000 7f87b96a5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f87b41608b0 0x7f87b415cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:14.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.631+0000 7f87b96a5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87b40d19d0 con 0x7f87b40ceb50 2024-03-29T04:10:14.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.631+0000 7f87b96a5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f87b40cf470 con 0x7f87b41608b0 2024-03-29T04:10:14.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.631+0000 7f87b96a5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f87b40ccde0 con 0x7f87b415c5f0 2024-03-29T04:10:14.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.631+0000 7f87b2ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f87b40ceb50 0x7f87b40cfd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50290/0 (socket says 172.21.15.148:50290) 2024-03-29T04:10:14.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.631+0000 7f87b2ffd640 1 -- 172.21.15.148:0/2990885863 learned_addr learned my addr 172.21.15.148:0/2990885863 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:14.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.631+0000 7f87b27fc640 1 --2- 172.21.15.148:0/2990885863 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87b415c5f0 0x7f87b40cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:14.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.631+0000 7f87b1ffb640 1 -- 172.21.15.148:0/2990885863 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1638620454 0 0) 0x7f87b40d19d0 con 0x7f87b40ceb50 2024-03-29T04:10:14.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.631+0000 7f87b1ffb640 1 -- 172.21.15.148:0/2990885863 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f879c003520 con 0x7f87b40ceb50 2024-03-29T04:10:14.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.631+0000 7f87b37fe640 1 --2- 172.21.15.148:0/2990885863 >> v2:172.21.15.157:3300/0 conn(0x7f87b41608b0 0x7f87b415cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:14.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.631+0000 7f87b1ffb640 1 -- 172.21.15.148:0/2990885863 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2486484531 0 0) 0x7f879c003520 con 0x7f87b40ceb50 2024-03-29T04:10:14.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.631+0000 7f87b1ffb640 1 -- 172.21.15.148:0/2990885863 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87b415c5f0 msgr2=0x7f87b40cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:14.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.631+0000 7f87b1ffb640 1 --2- 172.21.15.148:0/2990885863 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87b415c5f0 0x7f87b40cef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.631+0000 7f87b1ffb640 1 -- 172.21.15.148:0/2990885863 >> v2:172.21.15.157:3300/0 conn(0x7f87b41608b0 msgr2=0x7f87b415cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:14.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.631+0000 7f87b1ffb640 1 --2- 172.21.15.148:0/2990885863 >> v2:172.21.15.157:3300/0 conn(0x7f87b41608b0 0x7f87b415cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.631+0000 7f87b1ffb640 1 -- 172.21.15.148:0/2990885863 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b40ccb00 con 0x7f87b40ceb50 2024-03-29T04:10:14.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.631+0000 7f87b1ffb640 1 -- 172.21.15.148:0/2990885863 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f87a4001690 con 0x7f87b40ceb50 2024-03-29T04:10:14.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.631+0000 7f87b1ffb640 1 -- 172.21.15.148:0/2990885863 >> v1:172.21.15.148:6789/0 conn(0x7f87b40ceb50 legacy=0x7f87b40cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:14.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.631+0000 7f87b1ffb640 1 --2- 172.21.15.148:0/2990885863 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f879c0039b0 0x7f879c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:14.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.632+0000 7f87b1ffb640 1 --2- 172.21.15.148:0/2990885863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f879c004310 0x7f879c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:14.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.632+0000 7f87b1ffb640 1 --2- 172.21.15.148:0/2990885863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f879c004f00 0x7f879c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:14.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.632+0000 7f87b1ffb640 1 -- 172.21.15.148:0/2990885863 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f87b40cf470 con 0x7f879c004f00 2024-03-29T04:10:14.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.632+0000 7f87b1ffb640 1 -- 172.21.15.148:0/2990885863 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f87b40ccde0 con 0x7f879c004310 2024-03-29T04:10:14.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.632+0000 7f87b1ffb640 1 -- 172.21.15.148:0/2990885863 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f879c00bf00 con 0x7f879c0039b0 2024-03-29T04:10:14.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.632+0000 7f87b37fe640 1 --2- 172.21.15.148:0/2990885863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f879c004f00 0x7f879c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:14.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.632+0000 7f87b2ffd640 1 --2- 172.21.15.148:0/2990885863 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f879c0039b0 0x7f879c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:14.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.632+0000 7f87b27fc640 1 --2- 172.21.15.148:0/2990885863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f879c004310 0x7f879c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:14.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.632+0000 7f87b37fe640 1 -- 172.21.15.148:0/2990885863 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f879c0039b0 msgr2=0x7f879c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:14.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.632+0000 7f87b37fe640 1 --2- 172.21.15.148:0/2990885863 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f879c0039b0 0x7f879c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.632+0000 7f87b37fe640 1 -- 172.21.15.148:0/2990885863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f879c004310 msgr2=0x7f879c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:14.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.632+0000 7f87b37fe640 1 --2- 172.21.15.148:0/2990885863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f879c004310 0x7f879c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.632+0000 7f87b37fe640 1 -- 172.21.15.148:0/2990885863 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879c003520 con 0x7f879c004f00 2024-03-29T04:10:14.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.633+0000 7f87b37fe640 1 --2- 172.21.15.148:0/2990885863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f879c004f00 0x7f879c00b9c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f87ac007360 tx=0x7f879c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3dd8b951c7b01e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:14.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.633+0000 7f87b1ffb640 1 -- 172.21.15.148:0/2990885863 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac039070 con 0x7f879c004f00 2024-03-29T04:10:14.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.633+0000 7f87b1ffb640 1 -- 172.21.15.148:0/2990885863 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87ac02ccf0 con 0x7f879c004f00 2024-03-29T04:10:14.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.633+0000 7f87b1ffb640 1 -- 172.21.15.148:0/2990885863 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac009be0 con 0x7f879c004f00 2024-03-29T04:10:14.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.634+0000 7f87b96a5640 1 -- 172.21.15.148:0/2990885863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f879c004f00 msgr2=0x7f879c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:14.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.634+0000 7f87b96a5640 1 --2- 172.21.15.148:0/2990885863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f879c004f00 0x7f879c00b9c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f87ac007360 tx=0x7f879c005830 comp rx=0 tx=0).stop 2024-03-29T04:10:14.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.634+0000 7f87b37fe640 1 -- 172.21.15.148:0/2990885863 reap_dead start 2024-03-29T04:10:14.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.634+0000 7f87b96a5640 1 -- 172.21.15.148:0/2990885863 shutdown_connections 2024-03-29T04:10:14.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.634+0000 7f87b96a5640 1 -- 172.21.15.148:0/2990885863 >> 172.21.15.148:0/2990885863 conn(0x7f87b40b3c70 msgr2=0x7f87b40b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:14.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.634+0000 7f87b96a5640 1 -- 172.21.15.148:0/2990885863 shutdown_connections 2024-03-29T04:10:14.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.634+0000 7f87b96a5640 1 -- 172.21.15.148:0/2990885863 wait complete. 2024-03-29T04:10:14.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.635+0000 7f87b96a5640 1 Processor -- start 2024-03-29T04:10:14.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.635+0000 7f87b96a5640 1 -- start start 2024-03-29T04:10:14.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.636+0000 7f87b96a5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87b41443f0 0x7f87b415cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:14.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.636+0000 7f87b96a5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87b41447d0 0x7f87b40c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:14.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.636+0000 7f87b96a5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87b415d1c0 0x7f87b40c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:14.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.636+0000 7f87b96a5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f87b40ce540 con 0x7f87b41447d0 2024-03-29T04:10:14.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.636+0000 7f87b96a5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f87b40cc790 con 0x7f87b415d1c0 2024-03-29T04:10:14.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.636+0000 7f87b96a5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f87b40ce6c0 con 0x7f87b41443f0 2024-03-29T04:10:14.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.636+0000 7f87b27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87b41447d0 0x7f87b40c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:14.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.636+0000 7f87b27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87b41447d0 0x7f87b40c80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51470/0 (socket says 172.21.15.148:51470) 2024-03-29T04:10:14.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.636+0000 7f87b27fc640 1 -- 172.21.15.148:0/2830274476 learned_addr learned my addr 172.21.15.148:0/2830274476 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:14.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.636+0000 7f87b2ffd640 1 --2- 172.21.15.148:0/2830274476 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87b41443f0 0x7f87b415cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:14.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.636+0000 7f87b37fe640 1 --2- 172.21.15.148:0/2830274476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87b415d1c0 0x7f87b40c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:14.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.636+0000 7f87b27fc640 1 -- 172.21.15.148:0/2830274476 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87b41443f0 msgr2=0x7f87b415cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:14.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.636+0000 7f87b27fc640 1 --2- 172.21.15.148:0/2830274476 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87b41443f0 0x7f87b415cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.636+0000 7f87b27fc640 1 -- 172.21.15.148:0/2830274476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87b415d1c0 msgr2=0x7f87b40c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:14.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.636+0000 7f87b27fc640 1 --2- 172.21.15.148:0/2830274476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87b415d1c0 0x7f87b40c85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.636+0000 7f87b27fc640 1 -- 172.21.15.148:0/2830274476 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b40cad10 con 0x7f87b41447d0 2024-03-29T04:10:14.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.636+0000 7f87b27fc640 1 --2- 172.21.15.148:0/2830274476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87b41447d0 0x7f87b40c80b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f87a80119a0 tx=0x7f87a8011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:14.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.637+0000 7f879b7fe640 1 -- 172.21.15.148:0/2830274476 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a800faf0 con 0x7f87b41447d0 2024-03-29T04:10:14.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.637+0000 7f879b7fe640 1 -- 172.21.15.148:0/2830274476 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87a800fc90 con 0x7f87b41447d0 2024-03-29T04:10:14.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.637+0000 7f87b96a5640 1 -- 172.21.15.148:0/2830274476 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b40caff0 con 0x7f87b41447d0 2024-03-29T04:10:14.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.637+0000 7f879b7fe640 1 -- 172.21.15.148:0/2830274476 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8014440 con 0x7f87b41447d0 2024-03-29T04:10:14.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.637+0000 7f87b96a5640 1 -- 172.21.15.148:0/2830274476 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b40cb5b0 con 0x7f87b41447d0 2024-03-29T04:10:14.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.638+0000 7f87b96a5640 1 -- 172.21.15.148:0/2830274476 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87b40687e0 con 0x7f87b41447d0 2024-03-29T04:10:14.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.638+0000 7f879b7fe640 1 -- 172.21.15.148:0/2830274476 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f87a8012040 con 0x7f87b41447d0 2024-03-29T04:10:14.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.638+0000 7f879b7fe640 1 --2- 172.21.15.148:0/2830274476 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f87a006c530 0x7f87a006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:14.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.638+0000 7f879b7fe640 1 -- 172.21.15.148:0/2830274476 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f87a808c170 con 0x7f87b41447d0 2024-03-29T04:10:14.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.639+0000 7f87b2ffd640 1 --2- 172.21.15.148:0/2830274476 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f87a006c530 0x7f87a006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:14.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.639+0000 7f87b2ffd640 1 --2- 172.21.15.148:0/2830274476 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f87a006c530 0x7f87a006e9f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f87a4007fd0 tx=0x7f87a4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:14.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.643+0000 7f879b7fe640 1 -- 172.21.15.148:0/2830274476 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f87a8019030 con 0x7f87b41447d0 2024-03-29T04:10:14.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.941+0000 7f87b96a5640 1 -- 172.21.15.148:0/2830274476 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f87b40701d0 con 0x7f87b41447d0 2024-03-29T04:10:14.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.941+0000 7f879b7fe640 1 -- 172.21.15.148:0/2830274476 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f87a8059d70 con 0x7f87b41447d0 2024-03-29T04:10:14.920 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-03-29T04:10:14.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.943+0000 7f87b96a5640 1 -- 172.21.15.148:0/2830274476 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f87a006c530 msgr2=0x7f87a006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:14.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.943+0000 7f87b96a5640 1 --2- 172.21.15.148:0/2830274476 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f87a006c530 0x7f87a006e9f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f87a4007fd0 tx=0x7f87a4009040 comp rx=0 tx=0).stop 2024-03-29T04:10:14.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.943+0000 7f87b96a5640 1 -- 172.21.15.148:0/2830274476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87b41447d0 msgr2=0x7f87b40c80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:14.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.943+0000 7f87b96a5640 1 --2- 172.21.15.148:0/2830274476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87b41447d0 0x7f87b40c80b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f87a80119a0 tx=0x7f87a8011e60 comp rx=0 tx=0).stop 2024-03-29T04:10:14.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.943+0000 7f87b96a5640 1 -- 172.21.15.148:0/2830274476 shutdown_connections 2024-03-29T04:10:14.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.943+0000 7f87b96a5640 1 --2- 172.21.15.148:0/2830274476 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f87a006c530 0x7f87a006e9f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.943+0000 7f87b96a5640 1 --2- 172.21.15.148:0/2830274476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87b415d1c0 0x7f87b40c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.943+0000 7f87b96a5640 1 --2- 172.21.15.148:0/2830274476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87b41447d0 0x7f87b40c80b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.943+0000 7f87b96a5640 1 --2- 172.21.15.148:0/2830274476 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87b41443f0 0x7f87b415cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.943+0000 7f87b96a5640 1 -- 172.21.15.148:0/2830274476 >> 172.21.15.148:0/2830274476 conn(0x7f87b40b3c70 msgr2=0x7f87b406f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:14.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.943+0000 7f87b96a5640 1 -- 172.21.15.148:0/2830274476 shutdown_connections 2024-03-29T04:10:14.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:14.943+0000 7f87b96a5640 1 -- 172.21.15.148:0/2830274476 wait complete. 2024-03-29T04:10:14.933 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi148.front.sepia.ceph.com... 2024-03-29T04:10:14.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:14.933 DEBUG:teuthology.orchestra.run.smithi148:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-29T04:10:14.959 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi148.stderr:ceph-fuse[44699]: fuse finished with error 0 and tester_r 0 2024-03-29T04:10:14.961 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:10:21.062 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-03-29T04:10:21.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:21.062 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-29T04:10:21.091 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:21.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.257+0000 7f69709bd640 1 Processor -- start 2024-03-29T04:10:21.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.257+0000 7f69709bd640 1 -- start start 2024-03-29T04:10:21.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.258+0000 7f69709bd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f696c15e380 0x7f696c0afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:21.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.258+0000 7f69709bd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f696c06d6d0 0x7f696c06db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:21.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.258+0000 7f69709bd640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f696c0b6370 con 0x7f696c06b5b0 2024-03-29T04:10:21.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.258+0000 7f69709bd640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f696c0b0260 con 0x7f696c15e380 2024-03-29T04:10:21.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.258+0000 7f69709bd640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f696c15e760 con 0x7f696c06d6d0 2024-03-29T04:10:21.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.258+0000 7f696a59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f696c15e380 0x7f696c0afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:21.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.258+0000 7f696a59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f696c15e380 0x7f696c0afd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46324/0 (socket says 172.21.15.148:46324) 2024-03-29T04:10:21.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.258+0000 7f696a59c640 1 -- 172.21.15.148:0/3157440020 learned_addr learned my addr 172.21.15.148:0/3157440020 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:21.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.259+0000 7f6969d9b640 1 --2- 172.21.15.148:0/3157440020 >> v2:172.21.15.157:3300/0 conn(0x7f696c06d6d0 0x7f696c06db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:21.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.259+0000 7f696a59c640 1 -- 172.21.15.148:0/3157440020 >> v2:172.21.15.157:3300/0 conn(0x7f696c06d6d0 msgr2=0x7f696c06db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:21.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.259+0000 7f696a59c640 1 --2- 172.21.15.148:0/3157440020 >> v2:172.21.15.157:3300/0 conn(0x7f696c06d6d0 0x7f696c06db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.259+0000 7f696a59c640 1 -- 172.21.15.148:0/3157440020 >> v1:172.21.15.157:6789/0 conn(0x7f696c06b5b0 legacy=0x7f696c06b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:21.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.259+0000 7f696a59c640 1 -- 172.21.15.148:0/3157440020 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f696c06f4a0 con 0x7f696c15e380 2024-03-29T04:10:21.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.259+0000 7f6969d9b640 1 --2- 172.21.15.148:0/3157440020 >> v2:172.21.15.157:3300/0 conn(0x7f696c06d6d0 0x7f696c06db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:21.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.259+0000 7f696a59c640 1 --2- 172.21.15.148:0/3157440020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f696c15e380 0x7f696c0afd20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f696c0739b0 tx=0x7f696002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe26eb37e1a17d05 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:21.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.259+0000 7f696959a640 1 -- 172.21.15.148:0/3157440020 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6960035020 con 0x7f696c15e380 2024-03-29T04:10:21.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.260+0000 7f696959a640 1 -- 172.21.15.148:0/3157440020 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6960031870 con 0x7f696c15e380 2024-03-29T04:10:21.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.260+0000 7f696959a640 1 -- 172.21.15.148:0/3157440020 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6960031c40 con 0x7f696c15e380 2024-03-29T04:10:21.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.260+0000 7f69709bd640 1 -- 172.21.15.148:0/3157440020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f696c15e380 msgr2=0x7f696c0afd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:21.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.260+0000 7f69709bd640 1 --2- 172.21.15.148:0/3157440020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f696c15e380 0x7f696c0afd20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f696c0739b0 tx=0x7f696002b910 comp rx=0 tx=0).stop 2024-03-29T04:10:21.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.260+0000 7f69709bd640 1 -- 172.21.15.148:0/3157440020 shutdown_connections 2024-03-29T04:10:21.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.260+0000 7f69709bd640 1 --2- 172.21.15.148:0/3157440020 >> v2:172.21.15.157:3300/0 conn(0x7f696c06d6d0 0x7f696c06db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.260+0000 7f69709bd640 1 --2- 172.21.15.148:0/3157440020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f696c15e380 0x7f696c0afd20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.260+0000 7f69709bd640 1 -- 172.21.15.148:0/3157440020 >> 172.21.15.148:0/3157440020 conn(0x7f696c072450 msgr2=0x7f696c072860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:21.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.260+0000 7f69709bd640 1 -- 172.21.15.148:0/3157440020 shutdown_connections 2024-03-29T04:10:21.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.260+0000 7f69709bd640 1 -- 172.21.15.148:0/3157440020 wait complete. 2024-03-29T04:10:21.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.261+0000 7f69709bd640 1 Processor -- start 2024-03-29T04:10:21.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.262+0000 7f69709bd640 1 -- start start 2024-03-29T04:10:21.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.262+0000 7f69709bd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f696c06b5b0 0x7f696c15e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:21.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.262+0000 7f69709bd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f696c06d6d0 0x7f696c147890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:21.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.262+0000 7f69709bd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f696c15e380 0x7f696c1609a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:21.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.262+0000 7f69709bd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f696c0b4b60 con 0x7f696c06d6d0 2024-03-29T04:10:21.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.262+0000 7f69709bd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f696c0b4880 con 0x7f696c15e380 2024-03-29T04:10:21.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.262+0000 7f69709bd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f696c0b4e40 con 0x7f696c06b5b0 2024-03-29T04:10:21.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.262+0000 7f6969d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f696c06d6d0 0x7f696c147890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:21.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.262+0000 7f6969d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f696c06d6d0 0x7f696c147890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46340/0 (socket says 172.21.15.148:46340) 2024-03-29T04:10:21.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.262+0000 7f6969d9b640 1 -- 172.21.15.148:0/2292522412 learned_addr learned my addr 172.21.15.148:0/2292522412 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:21.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.262+0000 7f696ad9d640 1 --2- 172.21.15.148:0/2292522412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f696c15e380 0x7f696c1609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:21.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.262+0000 7f696a59c640 1 --2- 172.21.15.148:0/2292522412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f696c06b5b0 0x7f696c15e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:21.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.263+0000 7f6969d9b640 1 -- 172.21.15.148:0/2292522412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f696c06b5b0 msgr2=0x7f696c15e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:21.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.263+0000 7f6969d9b640 1 --2- 172.21.15.148:0/2292522412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f696c06b5b0 0x7f696c15e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.263+0000 7f6969d9b640 1 -- 172.21.15.148:0/2292522412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f696c15e380 msgr2=0x7f696c1609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:21.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.263+0000 7f6969d9b640 1 --2- 172.21.15.148:0/2292522412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f696c15e380 0x7f696c1609a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.263+0000 7f6969d9b640 1 -- 172.21.15.148:0/2292522412 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f696c15f150 con 0x7f696c06d6d0 2024-03-29T04:10:21.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.263+0000 7f6969d9b640 1 --2- 172.21.15.148:0/2292522412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f696c06d6d0 0x7f696c147890 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f695c00c820 tx=0x7f695c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:21.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.263+0000 7f695affd640 1 -- 172.21.15.148:0/2292522412 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c015070 con 0x7f696c06d6d0 2024-03-29T04:10:21.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.263+0000 7f695affd640 1 -- 172.21.15.148:0/2292522412 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f695c007a30 con 0x7f696c06d6d0 2024-03-29T04:10:21.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.263+0000 7f695affd640 1 -- 172.21.15.148:0/2292522412 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c007e00 con 0x7f696c06d6d0 2024-03-29T04:10:21.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.263+0000 7f69709bd640 1 -- 172.21.15.148:0/2292522412 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f696c15f430 con 0x7f696c06d6d0 2024-03-29T04:10:21.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.263+0000 7f69709bd640 1 -- 172.21.15.148:0/2292522412 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f696c1601a0 con 0x7f696c06d6d0 2024-03-29T04:10:21.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.264+0000 7f69709bd640 1 -- 172.21.15.148:0/2292522412 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f696c0687e0 con 0x7f696c06d6d0 2024-03-29T04:10:21.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.265+0000 7f695affd640 1 -- 172.21.15.148:0/2292522412 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f695c018070 con 0x7f696c06d6d0 2024-03-29T04:10:21.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.265+0000 7f695affd640 1 --2- 172.21.15.148:0/2292522412 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f694c06c530 0x7f694c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:21.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.265+0000 7f695affd640 1 -- 172.21.15.148:0/2292522412 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f695c08b8b0 con 0x7f696c06d6d0 2024-03-29T04:10:21.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.265+0000 7f696a59c640 1 --2- 172.21.15.148:0/2292522412 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f694c06c530 0x7f694c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:21.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.266+0000 7f696a59c640 1 --2- 172.21.15.148:0/2292522412 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f694c06c530 0x7f694c06e9f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6960036000 tx=0x7f696003c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:21.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.270+0000 7f695affd640 1 -- 172.21.15.148:0/2292522412 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f695c0595d0 con 0x7f696c06d6d0 2024-03-29T04:10:21.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.471+0000 7f69709bd640 1 -- 172.21.15.148:0/2292522412 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f696c15ee40 con 0x7f696c06d6d0 2024-03-29T04:10:21.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.472+0000 7f695affd640 1 -- 172.21.15.148:0/2292522412 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7f695c05d8d0 con 0x7f696c06d6d0 2024-03-29T04:10:21.451 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:21.452 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:05.026481+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:21.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.475+0000 7f69709bd640 1 -- 172.21.15.148:0/2292522412 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f694c06c530 msgr2=0x7f694c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:21.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.475+0000 7f69709bd640 1 --2- 172.21.15.148:0/2292522412 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f694c06c530 0x7f694c06e9f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6960036000 tx=0x7f696003c040 comp rx=0 tx=0).stop 2024-03-29T04:10:21.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.475+0000 7f69709bd640 1 -- 172.21.15.148:0/2292522412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f696c06d6d0 msgr2=0x7f696c147890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:21.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.475+0000 7f69709bd640 1 --2- 172.21.15.148:0/2292522412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f696c06d6d0 0x7f696c147890 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f695c00c820 tx=0x7f695c00cc20 comp rx=0 tx=0).stop 2024-03-29T04:10:21.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.475+0000 7f69709bd640 1 -- 172.21.15.148:0/2292522412 shutdown_connections 2024-03-29T04:10:21.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.475+0000 7f69709bd640 1 --2- 172.21.15.148:0/2292522412 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f694c06c530 0x7f694c06e9f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.475+0000 7f69709bd640 1 --2- 172.21.15.148:0/2292522412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f696c15e380 0x7f696c1609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.475+0000 7f69709bd640 1 --2- 172.21.15.148:0/2292522412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f696c06d6d0 0x7f696c147890 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.475+0000 7f69709bd640 1 --2- 172.21.15.148:0/2292522412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f696c06b5b0 0x7f696c15e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.475+0000 7f69709bd640 1 -- 172.21.15.148:0/2292522412 >> 172.21.15.148:0/2292522412 conn(0x7f696c072450 msgr2=0x7f696c0ad8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:21.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.475+0000 7f69709bd640 1 -- 172.21.15.148:0/2292522412 shutdown_connections 2024-03-29T04:10:21.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.475+0000 7f69709bd640 1 -- 172.21.15.148:0/2292522412 wait complete. 2024-03-29T04:10:21.465 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:21.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.588+0000 7fd353b9d640 1 Processor -- start 2024-03-29T04:10:21.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.588+0000 7fd353b9d640 1 -- start start 2024-03-29T04:10:21.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.589+0000 7fd353b9d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd34c15c5f0 0x7fd34c0cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:21.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.589+0000 7fd353b9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd34c1608b0 0x7fd34c15cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:21.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.589+0000 7fd353b9d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd34c0d19d0 con 0x7fd34c0ceb50 2024-03-29T04:10:21.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.589+0000 7fd353b9d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd34c0cf470 con 0x7fd34c1608b0 2024-03-29T04:10:21.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.589+0000 7fd353b9d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd34c0ccde0 con 0x7fd34c15c5f0 2024-03-29T04:10:21.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.589+0000 7fd351138640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd34c15c5f0 0x7fd34c0cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:21.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.589+0000 7fd351138640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd34c15c5f0 0x7fd34c0cef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.148:39426/0 (socket says 172.21.15.148:39426) 2024-03-29T04:10:21.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.589+0000 7fd351138640 1 -- 172.21.15.148:0/3871536775 learned_addr learned my addr 172.21.15.148:0/3871536775 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:21.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.589+0000 7fd35213a640 1 --2- 172.21.15.148:0/3871536775 >> v2:172.21.15.157:3300/0 conn(0x7fd34c1608b0 0x7fd34c15cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:21.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.590+0000 7fd351138640 1 -- 172.21.15.148:0/3871536775 >> v2:172.21.15.157:3300/0 conn(0x7fd34c1608b0 msgr2=0x7fd34c15cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:21.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.590+0000 7fd351138640 1 --2- 172.21.15.148:0/3871536775 >> v2:172.21.15.157:3300/0 conn(0x7fd34c1608b0 0x7fd34c15cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.590+0000 7fd351138640 1 -- 172.21.15.148:0/3871536775 >> v1:172.21.15.157:6789/0 conn(0x7fd34c0ceb50 legacy=0x7fd34c0cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:21.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.590+0000 7fd351138640 1 -- 172.21.15.148:0/3871536775 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34c0ccb00 con 0x7fd34c15c5f0 2024-03-29T04:10:21.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.591+0000 7fd351138640 1 --2- 172.21.15.148:0/3871536775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd34c15c5f0 0x7fd34c0cef30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd33c009870 tx=0x7fd33c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9159a57ce3d149a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:21.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.591+0000 7fd350937640 1 -- 172.21.15.148:0/3871536775 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c034070 con 0x7fd34c15c5f0 2024-03-29T04:10:21.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.591+0000 7fd350937640 1 -- 172.21.15.148:0/3871536775 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd33c030840 con 0x7fd34c15c5f0 2024-03-29T04:10:21.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.591+0000 7fd350937640 1 -- 172.21.15.148:0/3871536775 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c030c10 con 0x7fd34c15c5f0 2024-03-29T04:10:21.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.591+0000 7fd353b9d640 1 -- 172.21.15.148:0/3871536775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd34c15c5f0 msgr2=0x7fd34c0cef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:21.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.591+0000 7fd353b9d640 1 --2- 172.21.15.148:0/3871536775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd34c15c5f0 0x7fd34c0cef30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd33c009870 tx=0x7fd33c02b980 comp rx=0 tx=0).stop 2024-03-29T04:10:21.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.591+0000 7fd353b9d640 1 -- 172.21.15.148:0/3871536775 shutdown_connections 2024-03-29T04:10:21.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.591+0000 7fd353b9d640 1 --2- 172.21.15.148:0/3871536775 >> v2:172.21.15.157:3300/0 conn(0x7fd34c1608b0 0x7fd34c15cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.591+0000 7fd353b9d640 1 --2- 172.21.15.148:0/3871536775 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd34c15c5f0 0x7fd34c0cef30 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.591+0000 7fd353b9d640 1 -- 172.21.15.148:0/3871536775 >> 172.21.15.148:0/3871536775 conn(0x7fd34c0b3c70 msgr2=0x7fd34c0b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:21.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.591+0000 7fd353b9d640 1 -- 172.21.15.148:0/3871536775 shutdown_connections 2024-03-29T04:10:21.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.591+0000 7fd353b9d640 1 -- 172.21.15.148:0/3871536775 wait complete. 2024-03-29T04:10:21.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.592+0000 7fd353b9d640 1 Processor -- start 2024-03-29T04:10:21.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.593+0000 7fd353b9d640 1 -- start start 2024-03-29T04:10:21.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.593+0000 7fd353b9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd34c0ceb50 0x7fd34c0c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:21.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.593+0000 7fd353b9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd34c1608b0 0x7fd34c15abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:21.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.593+0000 7fd353b9d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd34c0c5c70 0x7fd34c0c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:21.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.593+0000 7fd353b9d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c0ce540 con 0x7fd34c1608b0 2024-03-29T04:10:21.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.593+0000 7fd353b9d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c0cc790 con 0x7fd34c0ceb50 2024-03-29T04:10:21.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.593+0000 7fd353b9d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd34c0ce6c0 con 0x7fd34c0c5c70 2024-03-29T04:10:21.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.593+0000 7fd351138640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd34c1608b0 0x7fd34c15abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:21.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.593+0000 7fd351138640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd34c1608b0 0x7fd34c15abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46350/0 (socket says 172.21.15.148:46350) 2024-03-29T04:10:21.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.593+0000 7fd351138640 1 -- 172.21.15.148:0/2140995879 learned_addr learned my addr 172.21.15.148:0/2140995879 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:21.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.593+0000 7fd35213a640 1 --2- 172.21.15.148:0/2140995879 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd34c0c5c70 0x7fd34c0c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:21.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.593+0000 7fd351939640 1 --2- 172.21.15.148:0/2140995879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd34c0ceb50 0x7fd34c0c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:21.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.594+0000 7fd351138640 1 -- 172.21.15.148:0/2140995879 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd34c0c5c70 msgr2=0x7fd34c0c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:21.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.594+0000 7fd351138640 1 --2- 172.21.15.148:0/2140995879 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd34c0c5c70 0x7fd34c0c6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.594+0000 7fd351138640 1 -- 172.21.15.148:0/2140995879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd34c0ceb50 msgr2=0x7fd34c0c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:21.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.594+0000 7fd351138640 1 --2- 172.21.15.148:0/2140995879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd34c0ceb50 0x7fd34c0c8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.594+0000 7fd351138640 1 -- 172.21.15.148:0/2140995879 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34c15ed60 con 0x7fd34c1608b0 2024-03-29T04:10:21.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.594+0000 7fd351138640 1 --2- 172.21.15.148:0/2140995879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd34c1608b0 0x7fd34c15abb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd33c000c00 tx=0x7fd33c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:21.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.594+0000 7fd3427fc640 1 -- 172.21.15.148:0/2140995879 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c048070 con 0x7fd34c1608b0 2024-03-29T04:10:21.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.594+0000 7fd3427fc640 1 -- 172.21.15.148:0/2140995879 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd33c030640 con 0x7fd34c1608b0 2024-03-29T04:10:21.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.594+0000 7fd353b9d640 1 -- 172.21.15.148:0/2140995879 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34c1e5e70 con 0x7fd34c1608b0 2024-03-29T04:10:21.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.594+0000 7fd3427fc640 1 -- 172.21.15.148:0/2140995879 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c034040 con 0x7fd34c1608b0 2024-03-29T04:10:21.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.594+0000 7fd353b9d640 1 -- 172.21.15.148:0/2140995879 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd34c15b0f0 con 0x7fd34c1608b0 2024-03-29T04:10:21.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.595+0000 7fd353b9d640 1 -- 172.21.15.148:0/2140995879 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd34c0687e0 con 0x7fd34c1608b0 2024-03-29T04:10:21.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.595+0000 7fd3427fc640 1 -- 172.21.15.148:0/2140995879 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd33c04b070 con 0x7fd34c1608b0 2024-03-29T04:10:21.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.596+0000 7fd3427fc640 1 --2- 172.21.15.148:0/2140995879 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd33006c530 0x7fd33006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:21.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.596+0000 7fd3427fc640 1 -- 172.21.15.148:0/2140995879 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd33c0ad830 con 0x7fd34c1608b0 2024-03-29T04:10:21.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.596+0000 7fd351939640 1 --2- 172.21.15.148:0/2140995879 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd33006c530 0x7fd33006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:21.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.596+0000 7fd351939640 1 --2- 172.21.15.148:0/2140995879 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd33006c530 0x7fd33006e9f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd348000f60 tx=0x7fd348009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:21.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.600+0000 7fd3427fc640 1 -- 172.21.15.148:0/2140995879 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd33c07b5e0 con 0x7fd34c1608b0 2024-03-29T04:10:21.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.782+0000 7fd353b9d640 1 -- 172.21.15.148:0/2140995879 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd34c0701d0 con 0x7fd34c1608b0 2024-03-29T04:10:21.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.784+0000 7fd3427fc640 1 -- 172.21.15.148:0/2140995879 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7fd33c07f8e0 con 0x7fd34c1608b0 2024-03-29T04:10:21.763 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:21.763 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:05.026481+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:21.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.786+0000 7fd353b9d640 1 -- 172.21.15.148:0/2140995879 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd33006c530 msgr2=0x7fd33006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:21.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.786+0000 7fd353b9d640 1 --2- 172.21.15.148:0/2140995879 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd33006c530 0x7fd33006e9f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd348000f60 tx=0x7fd348009040 comp rx=0 tx=0).stop 2024-03-29T04:10:21.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.786+0000 7fd353b9d640 1 -- 172.21.15.148:0/2140995879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd34c1608b0 msgr2=0x7fd34c15abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:21.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.786+0000 7fd353b9d640 1 --2- 172.21.15.148:0/2140995879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd34c1608b0 0x7fd34c15abb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd33c000c00 tx=0x7fd33c004730 comp rx=0 tx=0).stop 2024-03-29T04:10:21.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.786+0000 7fd353b9d640 1 -- 172.21.15.148:0/2140995879 shutdown_connections 2024-03-29T04:10:21.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.786+0000 7fd353b9d640 1 --2- 172.21.15.148:0/2140995879 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd33006c530 0x7fd33006e9f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.786+0000 7fd353b9d640 1 --2- 172.21.15.148:0/2140995879 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd34c0c5c70 0x7fd34c0c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.786+0000 7fd353b9d640 1 --2- 172.21.15.148:0/2140995879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd34c1608b0 0x7fd34c15abb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.786+0000 7fd353b9d640 1 --2- 172.21.15.148:0/2140995879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd34c0ceb50 0x7fd34c0c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.786+0000 7fd353b9d640 1 -- 172.21.15.148:0/2140995879 >> 172.21.15.148:0/2140995879 conn(0x7fd34c0b3c70 msgr2=0x7fd34c0b5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:21.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.786+0000 7fd353b9d640 1 -- 172.21.15.148:0/2140995879 shutdown_connections 2024-03-29T04:10:21.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.786+0000 7fd353b9d640 1 -- 172.21.15.148:0/2140995879 wait complete. 2024-03-29T04:10:21.776 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-29T04:10:21.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.902+0000 7ff5cb582640 1 Processor -- start 2024-03-29T04:10:21.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.903+0000 7ff5cb582640 1 -- start start 2024-03-29T04:10:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.904+0000 7ff5cb582640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff5c406efe0 0x7ff5c40700d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.904+0000 7ff5cb582640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff5c40ce310 0x7ff5c40b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.904+0000 7ff5cb582640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5c415d460 con 0x7ff5c415a120 2024-03-29T04:10:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.904+0000 7ff5cb582640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7ff5c40d1b20 con 0x7ff5c406efe0 2024-03-29T04:10:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.904+0000 7ff5cb582640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff5c40d0f60 con 0x7ff5c40ce310 2024-03-29T04:10:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.904+0000 7ff5c8b1d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff5c406efe0 0x7ff5c40700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.904+0000 7ff5c931e640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff5c415a120 0x7ff5c406e8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:32956/0 (socket says 172.21.15.148:32956) 2024-03-29T04:10:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.904+0000 7ff5c931e640 1 -- 172.21.15.148:0/2941444843 learned_addr learned my addr 172.21.15.148:0/2941444843 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.904+0000 7ff5c9b1f640 1 --2- 172.21.15.148:0/2941444843 >> v2:172.21.15.157:3300/0 conn(0x7ff5c40ce310 0x7ff5c40b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.904+0000 7ff5c8b1d640 1 -- 172.21.15.148:0/2941444843 >> v2:172.21.15.157:3300/0 conn(0x7ff5c40ce310 msgr2=0x7ff5c40b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.904+0000 7ff5c8b1d640 1 --2- 172.21.15.148:0/2941444843 >> v2:172.21.15.157:3300/0 conn(0x7ff5c40ce310 0x7ff5c40b6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.904+0000 7ff5c8b1d640 1 -- 172.21.15.148:0/2941444843 >> v1:172.21.15.148:6789/0 conn(0x7ff5c415a120 legacy=0x7ff5c406e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.904+0000 7ff5c8b1d640 1 -- 172.21.15.148:0/2941444843 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c415d700 con 0x7ff5c406efe0 2024-03-29T04:10:21.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.905+0000 7ff5c8b1d640 1 --2- 172.21.15.148:0/2941444843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5c406efe0 0x7ff5c40700d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40cc630 tx=0x7ff5b402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e63c38d55ea5d6a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:21.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.905+0000 7ff5bbfff640 1 -- 172.21.15.148:0/2941444843 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4035020 con 0x7ff5c406efe0 2024-03-29T04:10:21.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.905+0000 7ff5bbfff640 1 -- 172.21.15.148:0/2941444843 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b4031830 con 0x7ff5c406efe0 2024-03-29T04:10:21.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.905+0000 7ff5bbfff640 1 -- 172.21.15.148:0/2941444843 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4031c00 con 0x7ff5c406efe0 2024-03-29T04:10:21.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.906+0000 7ff5cb582640 1 -- 172.21.15.148:0/2941444843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5c406efe0 msgr2=0x7ff5c40700d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:21.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.906+0000 7ff5cb582640 1 --2- 172.21.15.148:0/2941444843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5c406efe0 0x7ff5c40700d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40cc630 tx=0x7ff5b402b8d0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.906+0000 7ff5cb582640 1 -- 172.21.15.148:0/2941444843 shutdown_connections 2024-03-29T04:10:21.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.906+0000 7ff5cb582640 1 --2- 172.21.15.148:0/2941444843 >> v2:172.21.15.157:3300/0 conn(0x7ff5c40ce310 0x7ff5c40b6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.906+0000 7ff5cb582640 1 --2- 172.21.15.148:0/2941444843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5c406efe0 0x7ff5c40700d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.906+0000 7ff5cb582640 1 -- 172.21.15.148:0/2941444843 >> 172.21.15.148:0/2941444843 conn(0x7ff5c40cead0 msgr2=0x7ff5c40ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:21.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.906+0000 7ff5cb582640 1 -- 172.21.15.148:0/2941444843 shutdown_connections 2024-03-29T04:10:21.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.906+0000 7ff5cb582640 1 -- 172.21.15.148:0/2941444843 wait complete. 2024-03-29T04:10:21.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.907+0000 7ff5cb582640 1 Processor -- start 2024-03-29T04:10:21.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.907+0000 7ff5cb582640 1 -- start start 2024-03-29T04:10:21.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.908+0000 7ff5cb582640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5c406efe0 0x7ff5c40cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:21.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.908+0000 7ff5cb582640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5c40ce310 0x7ff5c4153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:21.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.908+0000 7ff5cb582640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5c415a120 0x7ff5c4158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:21.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.908+0000 7ff5cb582640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c406f5c0 con 0x7ff5c415a120 2024-03-29T04:10:21.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.908+0000 7ff5cb582640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c40ce7d0 con 0x7ff5c40ce310 2024-03-29T04:10:21.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.908+0000 7ff5cb582640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c406f740 con 0x7ff5c406efe0 2024-03-29T04:10:21.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.908+0000 7ff5c9b1f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5c415a120 0x7ff5c4158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:21.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.908+0000 7ff5c9b1f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5c415a120 0x7ff5c4158720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46370/0 (socket says 172.21.15.148:46370) 2024-03-29T04:10:21.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.908+0000 7ff5c9b1f640 1 -- 172.21.15.148:0/1675685784 learned_addr learned my addr 172.21.15.148:0/1675685784 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:21.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.908+0000 7ff5c931e640 1 --2- 172.21.15.148:0/1675685784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5c406efe0 0x7ff5c40cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:21.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.908+0000 7ff5c8b1d640 1 --2- 172.21.15.148:0/1675685784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5c40ce310 0x7ff5c4153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:21.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.908+0000 7ff5c9b1f640 1 -- 172.21.15.148:0/1675685784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5c406efe0 msgr2=0x7ff5c40cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:21.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.908+0000 7ff5c9b1f640 1 --2- 172.21.15.148:0/1675685784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5c406efe0 0x7ff5c40cbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.908+0000 7ff5c9b1f640 1 -- 172.21.15.148:0/1675685784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5c40ce310 msgr2=0x7ff5c4153260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:21.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.908+0000 7ff5c9b1f640 1 --2- 172.21.15.148:0/1675685784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5c40ce310 0x7ff5c4153260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.908+0000 7ff5c9b1f640 1 -- 172.21.15.148:0/1675685784 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c40c92c0 con 0x7ff5c415a120 2024-03-29T04:10:21.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.908+0000 7ff5c9b1f640 1 --2- 172.21.15.148:0/1675685784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5c415a120 0x7ff5c4158720 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc009dd0 tx=0x7ff5bc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:21.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.909+0000 7ff5b9ffb640 1 -- 172.21.15.148:0/1675685784 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc018020 con 0x7ff5c415a120 2024-03-29T04:10:21.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.909+0000 7ff5b9ffb640 1 -- 172.21.15.148:0/1675685784 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5bc014480 con 0x7ff5c415a120 2024-03-29T04:10:21.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.909+0000 7ff5cb582640 1 -- 172.21.15.148:0/1675685784 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c40c95a0 con 0x7ff5c415a120 2024-03-29T04:10:21.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.909+0000 7ff5b9ffb640 1 -- 172.21.15.148:0/1675685784 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc014870 con 0x7ff5c415a120 2024-03-29T04:10:21.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.909+0000 7ff5cb582640 1 -- 172.21.15.148:0/1675685784 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c40c9b60 con 0x7ff5c415a120 2024-03-29T04:10:21.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.910+0000 7ff5cb582640 1 -- 172.21.15.148:0/1675685784 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c4068850 con 0x7ff5c415a120 2024-03-29T04:10:21.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.911+0000 7ff5b9ffb640 1 -- 172.21.15.148:0/1675685784 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff5bc005d10 con 0x7ff5c415a120 2024-03-29T04:10:21.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.911+0000 7ff5b9ffb640 1 --2- 172.21.15.148:0/1675685784 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff5a806c530 0x7ff5a806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:21.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.911+0000 7ff5c931e640 1 --2- 172.21.15.148:0/1675685784 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff5a806c530 0x7ff5a806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:21.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.911+0000 7ff5b9ffb640 1 -- 172.21.15.148:0/1675685784 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff5bc08c580 con 0x7ff5c415a120 2024-03-29T04:10:21.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.912+0000 7ff5c931e640 1 --2- 172.21.15.148:0/1675685784 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff5a806c530 0x7ff5a806e9f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0000f60 tx=0x7ff5c0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:21.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:21.916+0000 7ff5b9ffb640 1 -- 172.21.15.148:0/1675685784 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff5bc022070 con 0x7ff5c415a120 2024-03-29T04:10:22.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.135+0000 7ff5cb582640 1 -- 172.21.15.148:0/1675685784 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff5c415f900 con 0x7ff5c415a120 2024-03-29T04:10:22.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.587+0000 7ff5b9ffb640 1 -- 172.21.15.148:0/1675685784 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7ff5bc010040 con 0x7ff5c415a120 2024-03-29T04:10:22.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.589+0000 7ff5cb582640 1 -- 172.21.15.148:0/1675685784 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff5a806c530 msgr2=0x7ff5a806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:22.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.589+0000 7ff5cb582640 1 --2- 172.21.15.148:0/1675685784 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff5a806c530 0x7ff5a806e9f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0000f60 tx=0x7ff5c0009040 comp rx=0 tx=0).stop 2024-03-29T04:10:22.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.589+0000 7ff5cb582640 1 -- 172.21.15.148:0/1675685784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5c415a120 msgr2=0x7ff5c4158720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:22.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.589+0000 7ff5cb582640 1 --2- 172.21.15.148:0/1675685784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5c415a120 0x7ff5c4158720 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc009dd0 tx=0x7ff5bc00c490 comp rx=0 tx=0).stop 2024-03-29T04:10:22.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.589+0000 7ff5cb582640 1 -- 172.21.15.148:0/1675685784 shutdown_connections 2024-03-29T04:10:22.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.589+0000 7ff5cb582640 1 --2- 172.21.15.148:0/1675685784 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff5a806c530 0x7ff5a806e9f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:22.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.589+0000 7ff5cb582640 1 --2- 172.21.15.148:0/1675685784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5c415a120 0x7ff5c4158720 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:22.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.589+0000 7ff5cb582640 1 --2- 172.21.15.148:0/1675685784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5c40ce310 0x7ff5c4153260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:22.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.589+0000 7ff5cb582640 1 --2- 172.21.15.148:0/1675685784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5c406efe0 0x7ff5c40cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:22.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.589+0000 7ff5cb582640 1 -- 172.21.15.148:0/1675685784 >> 172.21.15.148:0/1675685784 conn(0x7ff5c40cead0 msgr2=0x7ff5c415dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:22.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.589+0000 7ff5cb582640 1 -- 172.21.15.148:0/1675685784 shutdown_connections 2024-03-29T04:10:22.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.589+0000 7ff5cb582640 1 -- 172.21.15.148:0/1675685784 wait complete. 2024-03-29T04:10:22.580 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-03-29T04:10:22.580 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-29T04:10:22.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.710+0000 7f5b21fe8640 1 Processor -- start 2024-03-29T04:10:22.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.710+0000 7f5b21fe8640 1 -- start start 2024-03-29T04:10:22.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.711+0000 7f5b21fe8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5b1c15e3c0 0x7f5b1c161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:22.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.711+0000 7f5b21fe8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5b1c0b2f00 0x7f5b1c1624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:22.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.711+0000 7f5b21fe8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b1c0cf330 con 0x7f5b1c0b3430 2024-03-29T04:10:22.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.711+0000 7f5b21fe8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f5b1c0cf960 con 0x7f5b1c15e3c0 2024-03-29T04:10:22.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.711+0000 7f5b21fe8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5b1c162cf0 con 0x7f5b1c0b2f00 2024-03-29T04:10:22.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.711+0000 7f5b1b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5b1c15e3c0 0x7f5b1c161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:22.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.711+0000 7f5b1b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5b1c15e3c0 0x7f5b1c161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46382/0 (socket says 172.21.15.148:46382) 2024-03-29T04:10:22.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.711+0000 7f5b1b7fe640 1 -- 172.21.15.148:0/1410691512 learned_addr learned my addr 172.21.15.148:0/1410691512 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:22.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.711+0000 7f5b1affd640 1 --2- 172.21.15.148:0/1410691512 >> v2:172.21.15.157:3300/0 conn(0x7f5b1c0b2f00 0x7f5b1c1624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:22.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.711+0000 7f5b1b7fe640 1 -- 172.21.15.148:0/1410691512 >> v2:172.21.15.157:3300/0 conn(0x7f5b1c0b2f00 msgr2=0x7f5b1c1624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:22.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.711+0000 7f5b1b7fe640 1 --2- 172.21.15.148:0/1410691512 >> v2:172.21.15.157:3300/0 conn(0x7f5b1c0b2f00 0x7f5b1c1624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:22.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.711+0000 7f5b1b7fe640 1 -- 172.21.15.148:0/1410691512 >> v1:172.21.15.157:6789/0 conn(0x7f5b1c0b3430 legacy=0x7f5b1c0d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:22.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.711+0000 7f5b1b7fe640 1 -- 172.21.15.148:0/1410691512 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b1c0cf5f0 con 0x7f5b1c15e3c0 2024-03-29T04:10:22.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.712+0000 7f5b1b7fe640 1 --2- 172.21.15.148:0/1410691512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b1c15e3c0 0x7f5b1c161f70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c0c7be0 tx=0x7f5b0c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a87b6657d93037b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:22.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.712+0000 7f5b1a7fc640 1 -- 172.21.15.148:0/1410691512 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c035020 con 0x7f5b1c15e3c0 2024-03-29T04:10:22.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.712+0000 7f5b1a7fc640 1 -- 172.21.15.148:0/1410691512 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b0c031870 con 0x7f5b1c15e3c0 2024-03-29T04:10:22.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.712+0000 7f5b1a7fc640 1 -- 172.21.15.148:0/1410691512 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c031c40 con 0x7f5b1c15e3c0 2024-03-29T04:10:22.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.713+0000 7f5b21fe8640 1 -- 172.21.15.148:0/1410691512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b1c15e3c0 msgr2=0x7f5b1c161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:22.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.713+0000 7f5b21fe8640 1 --2- 172.21.15.148:0/1410691512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b1c15e3c0 0x7f5b1c161f70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c0c7be0 tx=0x7f5b0c02b910 comp rx=0 tx=0).stop 2024-03-29T04:10:22.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.713+0000 7f5b21fe8640 1 -- 172.21.15.148:0/1410691512 shutdown_connections 2024-03-29T04:10:22.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.713+0000 7f5b21fe8640 1 --2- 172.21.15.148:0/1410691512 >> v2:172.21.15.157:3300/0 conn(0x7f5b1c0b2f00 0x7f5b1c1624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:22.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.713+0000 7f5b21fe8640 1 --2- 172.21.15.148:0/1410691512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b1c15e3c0 0x7f5b1c161f70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:22.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.713+0000 7f5b21fe8640 1 -- 172.21.15.148:0/1410691512 >> 172.21.15.148:0/1410691512 conn(0x7f5b1c0b6430 msgr2=0x7f5b1c0d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:22.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.713+0000 7f5b21fe8640 1 -- 172.21.15.148:0/1410691512 shutdown_connections 2024-03-29T04:10:22.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.713+0000 7f5b21fe8640 1 -- 172.21.15.148:0/1410691512 wait complete. 2024-03-29T04:10:22.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.714+0000 7f5b21fe8640 1 Processor -- start 2024-03-29T04:10:22.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.714+0000 7f5b21fe8640 1 -- start start 2024-03-29T04:10:22.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.715+0000 7f5b21fe8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1c0b3430 0x7f5b1c1613e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:22.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.715+0000 7f5b21fe8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b1c15e3c0 0x7f5b1c0c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:22.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.715+0000 7f5b21fe8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b1c0ca550 0x7f5b1c15eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:22.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.715+0000 7f5b21fe8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c0b39a0 con 0x7f5b1c0ca550 2024-03-29T04:10:22.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.715+0000 7f5b21fe8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c15e7a0 con 0x7f5b1c0b3430 2024-03-29T04:10:22.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.715+0000 7f5b21fe8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5b1c0b3b20 con 0x7f5b1c15e3c0 2024-03-29T04:10:22.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.715+0000 7f5b1bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b1c0ca550 0x7f5b1c15eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:22.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.715+0000 7f5b1bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b1c0ca550 0x7f5b1c15eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46384/0 (socket says 172.21.15.148:46384) 2024-03-29T04:10:22.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.715+0000 7f5b1bfff640 1 -- 172.21.15.148:0/3622704547 learned_addr learned my addr 172.21.15.148:0/3622704547 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:22.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.715+0000 7f5b1b7fe640 1 --2- 172.21.15.148:0/3622704547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1c0b3430 0x7f5b1c1613e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:22.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.715+0000 7f5b1affd640 1 --2- 172.21.15.148:0/3622704547 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b1c15e3c0 0x7f5b1c0c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:22.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.715+0000 7f5b1bfff640 1 -- 172.21.15.148:0/3622704547 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b1c15e3c0 msgr2=0x7f5b1c0c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:22.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.715+0000 7f5b1bfff640 1 --2- 172.21.15.148:0/3622704547 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b1c15e3c0 0x7f5b1c0c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:22.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.715+0000 7f5b1bfff640 1 -- 172.21.15.148:0/3622704547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1c0b3430 msgr2=0x7f5b1c1613e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:22.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.715+0000 7f5b1bfff640 1 --2- 172.21.15.148:0/3622704547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1c0b3430 0x7f5b1c1613e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:22.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.715+0000 7f5b1bfff640 1 -- 172.21.15.148:0/3622704547 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b1c0c8770 con 0x7f5b1c0ca550 2024-03-29T04:10:22.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.715+0000 7f5b1bfff640 1 --2- 172.21.15.148:0/3622704547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b1c0ca550 0x7f5b1c15eae0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400c7b0 tx=0x7f5b1400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:22.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.716+0000 7f5b03fff640 1 -- 172.21.15.148:0/3622704547 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b14004b60 con 0x7f5b1c0ca550 2024-03-29T04:10:22.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.716+0000 7f5b03fff640 1 -- 172.21.15.148:0/3622704547 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b14004d00 con 0x7f5b1c0ca550 2024-03-29T04:10:22.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.716+0000 7f5b21fe8640 1 -- 172.21.15.148:0/3622704547 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b1c0c89e0 con 0x7f5b1c0ca550 2024-03-29T04:10:22.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.716+0000 7f5b03fff640 1 -- 172.21.15.148:0/3622704547 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1400aa90 con 0x7f5b1c0ca550 2024-03-29T04:10:22.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.716+0000 7f5b21fe8640 1 -- 172.21.15.148:0/3622704547 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b1c0c8fa0 con 0x7f5b1c0ca550 2024-03-29T04:10:22.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.717+0000 7f5b21fe8640 1 -- 172.21.15.148:0/3622704547 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b1c068850 con 0x7f5b1c0ca550 2024-03-29T04:10:22.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.718+0000 7f5b03fff640 1 -- 172.21.15.148:0/3622704547 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5b1400ac30 con 0x7f5b1c0ca550 2024-03-29T04:10:22.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.718+0000 7f5b03fff640 1 --2- 172.21.15.148:0/3622704547 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5b0806c530 0x7f5b0806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:22.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.718+0000 7f5b1b7fe640 1 --2- 172.21.15.148:0/3622704547 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5b0806c530 0x7f5b0806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:22.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.718+0000 7f5b03fff640 1 -- 172.21.15.148:0/3622704547 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5b1408bbf0 con 0x7f5b1c0ca550 2024-03-29T04:10:22.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.718+0000 7f5b1b7fe640 1 --2- 172.21.15.148:0/3622704547 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5b0806c530 0x7f5b0806e9f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c036000 tx=0x7f5b0c03c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:22.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.722+0000 7f5b03fff640 1 -- 172.21.15.148:0/3622704547 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5b14020070 con 0x7f5b1c0ca550 2024-03-29T04:10:22.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:22.956+0000 7f5b21fe8640 1 -- 172.21.15.148:0/3622704547 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f5b1c0cb050 con 0x7f5b1c0ca550 2024-03-29T04:10:23.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.588+0000 7f5b03fff640 1 -- 172.21.15.148:0/3622704547 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f5b14059ed0 con 0x7f5b1c0ca550 2024-03-29T04:10:23.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.590+0000 7f5b21fe8640 1 -- 172.21.15.148:0/3622704547 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5b0806c530 msgr2=0x7f5b0806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:23.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.590+0000 7f5b21fe8640 1 --2- 172.21.15.148:0/3622704547 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5b0806c530 0x7f5b0806e9f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c036000 tx=0x7f5b0c03c040 comp rx=0 tx=0).stop 2024-03-29T04:10:23.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.590+0000 7f5b21fe8640 1 -- 172.21.15.148:0/3622704547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b1c0ca550 msgr2=0x7f5b1c15eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:23.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.590+0000 7f5b21fe8640 1 --2- 172.21.15.148:0/3622704547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b1c0ca550 0x7f5b1c15eae0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400c7b0 tx=0x7f5b1400cc70 comp rx=0 tx=0).stop 2024-03-29T04:10:23.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.591+0000 7f5b21fe8640 1 -- 172.21.15.148:0/3622704547 shutdown_connections 2024-03-29T04:10:23.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.591+0000 7f5b21fe8640 1 --2- 172.21.15.148:0/3622704547 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5b0806c530 0x7f5b0806e9f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.591+0000 7f5b21fe8640 1 --2- 172.21.15.148:0/3622704547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b1c0ca550 0x7f5b1c15eae0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.591+0000 7f5b21fe8640 1 --2- 172.21.15.148:0/3622704547 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b1c15e3c0 0x7f5b1c0c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.591+0000 7f5b21fe8640 1 --2- 172.21.15.148:0/3622704547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1c0b3430 0x7f5b1c1613e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.591+0000 7f5b21fe8640 1 -- 172.21.15.148:0/3622704547 >> 172.21.15.148:0/3622704547 conn(0x7f5b1c0b6430 msgr2=0x7f5b1c0b4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:23.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.591+0000 7f5b21fe8640 1 -- 172.21.15.148:0/3622704547 shutdown_connections 2024-03-29T04:10:23.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.591+0000 7f5b21fe8640 1 -- 172.21.15.148:0/3622704547 wait complete. 2024-03-29T04:10:23.582 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-29T04:10:23.582 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:10:23.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.713+0000 7f32c6688640 1 Processor -- start 2024-03-29T04:10:23.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.713+0000 7f32c6688640 1 -- start start 2024-03-29T04:10:23.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.714+0000 7f32c6688640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f32c00b3480 0x7f32c00b6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:23.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.714+0000 7f32c6688640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32c00b50a0 con 0x7f32c0070e20 2024-03-29T04:10:23.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.714+0000 7f32c6688640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32c00b5380 con 0x7f32c00b2ff0 2024-03-29T04:10:23.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.714+0000 7f32c6688640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f32c00b5660 con 0x7f32c00b3480 2024-03-29T04:10:23.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.714+0000 7f32c4c25640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f32c00b3480 0x7f32c00b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:23.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.714+0000 7f32bffff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f32c0070e20 0x7f32c0071200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:32968/0 (socket says 172.21.15.148:32968) 2024-03-29T04:10:23.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.714+0000 7f32c4c25640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f32c00b3480 0x7f32c00b6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46386/0 (socket says 172.21.15.148:46386) 2024-03-29T04:10:23.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.714+0000 7f32bffff640 1 -- 172.21.15.148:0/325276372 learned_addr learned my addr 172.21.15.148:0/325276372 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:23.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.715+0000 7f32c4c25640 1 -- 172.21.15.148:0/325276372 >> v1:172.21.15.157:6789/0 conn(0x7f32c00b2ff0 legacy=0x7f32c00b6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:23.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.715+0000 7f32c4c25640 1 -- 172.21.15.148:0/325276372 >> v1:172.21.15.148:6789/0 conn(0x7f32c0070e20 legacy=0x7f32c0071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:23.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.715+0000 7f32beffd640 1 -- 172.21.15.148:0/325276372 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 732156385 0 0) 0x7f32c00b50a0 con 0x7f32c0070e20 2024-03-29T04:10:23.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.715+0000 7f32c4c25640 1 -- 172.21.15.148:0/325276372 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c00b5940 con 0x7f32c00b3480 2024-03-29T04:10:23.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.716+0000 7f32c4c25640 1 --2- 172.21.15.148:0/325276372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c00b3480 0x7f32c00b6850 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009d90 tx=0x7f32b802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b85d54350f49f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:23.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.716+0000 7f32beffd640 1 -- 172.21.15.148:0/325276372 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8039070 con 0x7f32c00b3480 2024-03-29T04:10:23.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.716+0000 7f32beffd640 1 -- 172.21.15.148:0/325276372 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32b802cc60 con 0x7f32c00b3480 2024-03-29T04:10:23.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.716+0000 7f32beffd640 1 -- 172.21.15.148:0/325276372 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8034660 con 0x7f32c00b3480 2024-03-29T04:10:23.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.716+0000 7f32c6688640 1 -- 172.21.15.148:0/325276372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c00b3480 msgr2=0x7f32c00b6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:23.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.716+0000 7f32c6688640 1 --2- 172.21.15.148:0/325276372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c00b3480 0x7f32c00b6850 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009d90 tx=0x7f32b802beb0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.716+0000 7f32c6688640 1 -- 172.21.15.148:0/325276372 shutdown_connections 2024-03-29T04:10:23.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.716+0000 7f32c6688640 1 --2- 172.21.15.148:0/325276372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c00b3480 0x7f32c00b6850 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.716+0000 7f32c6688640 1 -- 172.21.15.148:0/325276372 >> 172.21.15.148:0/325276372 conn(0x7f32c00d2100 msgr2=0x7f32c00d2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:23.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.716+0000 7f32c6688640 1 -- 172.21.15.148:0/325276372 shutdown_connections 2024-03-29T04:10:23.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.716+0000 7f32c6688640 1 -- 172.21.15.148:0/325276372 wait complete. 2024-03-29T04:10:23.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.718+0000 7f32c6688640 1 Processor -- start 2024-03-29T04:10:23.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.718+0000 7f32c6688640 1 -- start start 2024-03-29T04:10:23.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.719+0000 7f32c6688640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c0070e20 0x7f32c015ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:23.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.719+0000 7f32c6688640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c00b2ff0 0x7f32c00cff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:23.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.720+0000 7f32c6688640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32c00b3480 0x7f32c01f5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:23.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.720+0000 7f32c6688640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f32c00ce6e0 con 0x7f32c0070e20 2024-03-29T04:10:23.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.720+0000 7f32c6688640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f32c00ce560 con 0x7f32c00b2ff0 2024-03-29T04:10:23.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.720+0000 7f32c6688640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f32c015c510 con 0x7f32c00b3480 2024-03-29T04:10:23.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.720+0000 7f32bffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c0070e20 0x7f32c015ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:23.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.720+0000 7f32bffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c0070e20 0x7f32c015ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46398/0 (socket says 172.21.15.148:46398) 2024-03-29T04:10:23.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.720+0000 7f32bffff640 1 -- 172.21.15.148:0/668808577 learned_addr learned my addr 172.21.15.148:0/668808577 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:23.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.720+0000 7f32bf7fe640 1 --2- 172.21.15.148:0/668808577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c00b2ff0 0x7f32c00cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:23.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.720+0000 7f32c4c25640 1 --2- 172.21.15.148:0/668808577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32c00b3480 0x7f32c01f5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:23.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.720+0000 7f32bffff640 1 -- 172.21.15.148:0/668808577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32c00b3480 msgr2=0x7f32c01f5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:23.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.720+0000 7f32bffff640 1 --2- 172.21.15.148:0/668808577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32c00b3480 0x7f32c01f5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.720+0000 7f32bffff640 1 -- 172.21.15.148:0/668808577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c00b2ff0 msgr2=0x7f32c00cff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:23.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.720+0000 7f32bffff640 1 --2- 172.21.15.148:0/668808577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c00b2ff0 0x7f32c00cff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.720+0000 7f32bffff640 1 -- 172.21.15.148:0/668808577 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c00c6b00 con 0x7f32c0070e20 2024-03-29T04:10:23.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.720+0000 7f32bffff640 1 --2- 172.21.15.148:0/668808577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c0070e20 0x7f32c015ea00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f32b400c4d0 tx=0x7f32b400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:23.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.720+0000 7f32bcff9640 1 -- 172.21.15.148:0/668808577 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b4018070 con 0x7f32c0070e20 2024-03-29T04:10:23.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.720+0000 7f32bcff9640 1 -- 172.21.15.148:0/668808577 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32b40074e0 con 0x7f32c0070e20 2024-03-29T04:10:23.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.720+0000 7f32c6688640 1 -- 172.21.15.148:0/668808577 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c00c6de0 con 0x7f32c0070e20 2024-03-29T04:10:23.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.720+0000 7f32bcff9640 1 -- 172.21.15.148:0/668808577 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b4004a20 con 0x7f32c0070e20 2024-03-29T04:10:23.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.720+0000 7f32c6688640 1 -- 172.21.15.148:0/668808577 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c00c73a0 con 0x7f32c0070e20 2024-03-29T04:10:23.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.722+0000 7f32c6688640 1 -- 172.21.15.148:0/668808577 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32c00c6fb0 con 0x7f32c0070e20 2024-03-29T04:10:23.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.722+0000 7f32bcff9640 1 -- 172.21.15.148:0/668808577 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f32b401b070 con 0x7f32c0070e20 2024-03-29T04:10:23.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.722+0000 7f32bcff9640 1 --2- 172.21.15.148:0/668808577 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f32ac06c530 0x7f32ac06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:23.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.722+0000 7f32bf7fe640 1 --2- 172.21.15.148:0/668808577 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f32ac06c530 0x7f32ac06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:23.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.722+0000 7f32bcff9640 1 -- 172.21.15.148:0/668808577 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f32b408ac40 con 0x7f32c0070e20 2024-03-29T04:10:23.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.723+0000 7f32bf7fe640 1 --2- 172.21.15.148:0/668808577 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f32ac06c530 0x7f32ac06e9f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f32c00d1d20 tx=0x7f32b0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:23.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:23.727+0000 7f32bcff9640 1 -- 172.21.15.148:0/668808577 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f32b4058960 con 0x7f32c0070e20 2024-03-29T04:10:24.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.042+0000 7f32c6688640 1 -- 172.21.15.148:0/668808577 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32c00687e0 con 0x7f32c0070e20 2024-03-29T04:10:24.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.043+0000 7f32bcff9640 1 -- 172.21.15.148:0/668808577 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f32b405cc60 con 0x7f32c0070e20 2024-03-29T04:10:24.022 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:24.022 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/36660496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":36660496},{"type":"v1","addr":"172.21.15.157:6834","nonce":36660496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2674481055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2674481055},{"type":"v1","addr":"172.21.15.157:6835","nonce":2674481055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3356449643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3356449643},{"type":"v1","addr":"172.21.15.148:6836","nonce":3356449643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:09:59.886367+0000","modified":"2024-03-29T04:10:22.957623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/2453181817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2453181817},{"type":"v1","addr":"172.21.15.148:6837","nonce":2453181817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-03-29T04:10:24.023 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 9 2024-03-29T04:10:24.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.045+0000 7f32c6688640 1 -- 172.21.15.148:0/668808577 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f32ac06c530 msgr2=0x7f32ac06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.045+0000 7f32c6688640 1 --2- 172.21.15.148:0/668808577 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f32ac06c530 0x7f32ac06e9f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f32c00d1d20 tx=0x7f32b0009040 comp rx=0 tx=0).stop 2024-03-29T04:10:24.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.045+0000 7f32c6688640 1 -- 172.21.15.148:0/668808577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c0070e20 msgr2=0x7f32c015ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.045+0000 7f32c6688640 1 --2- 172.21.15.148:0/668808577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c0070e20 0x7f32c015ea00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f32b400c4d0 tx=0x7f32b400c990 comp rx=0 tx=0).stop 2024-03-29T04:10:24.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.045+0000 7f32c6688640 1 -- 172.21.15.148:0/668808577 shutdown_connections 2024-03-29T04:10:24.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.045+0000 7f32c6688640 1 --2- 172.21.15.148:0/668808577 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f32ac06c530 0x7f32ac06e9f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.045+0000 7f32c6688640 1 --2- 172.21.15.148:0/668808577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f32c00b3480 0x7f32c01f5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.045+0000 7f32c6688640 1 --2- 172.21.15.148:0/668808577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c00b2ff0 0x7f32c00cff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.045+0000 7f32c6688640 1 --2- 172.21.15.148:0/668808577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c0070e20 0x7f32c015ea00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.045+0000 7f32c6688640 1 -- 172.21.15.148:0/668808577 >> 172.21.15.148:0/668808577 conn(0x7f32c00d2100 msgr2=0x7f32c015b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:24.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.045+0000 7f32c6688640 1 -- 172.21.15.148:0/668808577 shutdown_connections 2024-03-29T04:10:24.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.045+0000 7f32c6688640 1 -- 172.21.15.148:0/668808577 wait complete. 2024-03-29T04:10:24.035 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:24.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.159+0000 7fa748e0d640 1 Processor -- start 2024-03-29T04:10:24.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.159+0000 7fa748e0d640 1 -- start start 2024-03-29T04:10:24.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.160+0000 7fa748e0d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa744147d90 0x7fa7440c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.160+0000 7fa748e0d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7440cc830 con 0x7fa744162650 2024-03-29T04:10:24.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.160+0000 7fa748e0d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7440ccb10 con 0x7fa7441479b0 2024-03-29T04:10:24.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.160+0000 7fa748e0d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa7440cfe60 con 0x7fa744147d90 2024-03-29T04:10:24.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.160+0000 7fa741d9b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa744162650 0x7fa7440d02f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:32970/0 (socket says 172.21.15.148:32970) 2024-03-29T04:10:24.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.160+0000 7fa741d9b640 1 -- 172.21.15.148:0/1279603714 learned_addr learned my addr 172.21.15.148:0/1279603714 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:24.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.160+0000 7fa742d9d640 1 --2- 172.21.15.148:0/1279603714 >> v2:172.21.15.157:3300/0 conn(0x7fa744147d90 0x7fa7440c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.160+0000 7fa74159a640 1 -- 172.21.15.148:0/1279603714 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2354693128 0 0) 0x7fa7440cc830 con 0x7fa744162650 2024-03-29T04:10:24.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.160+0000 7fa74159a640 1 -- 172.21.15.148:0/1279603714 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa72c003520 con 0x7fa744162650 2024-03-29T04:10:24.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.160+0000 7fa74159a640 1 -- 172.21.15.148:0/1279603714 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3504609134 0 0) 0x7fa7440ccb10 con 0x7fa7441479b0 2024-03-29T04:10:24.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.160+0000 7fa74159a640 1 -- 172.21.15.148:0/1279603714 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7440cc830 con 0x7fa7441479b0 2024-03-29T04:10:24.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.160+0000 7fa742d9d640 1 -- 172.21.15.148:0/1279603714 >> v1:172.21.15.157:6789/0 conn(0x7fa7441479b0 legacy=0x7fa744161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.160+0000 7fa74159a640 1 -- 172.21.15.148:0/1279603714 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 222935296 0 0) 0x7fa72c003520 con 0x7fa744162650 2024-03-29T04:10:24.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.160+0000 7fa742d9d640 1 -- 172.21.15.148:0/1279603714 >> v1:172.21.15.148:6789/0 conn(0x7fa744162650 legacy=0x7fa7440d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.160+0000 7fa742d9d640 1 -- 172.21.15.148:0/1279603714 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7440b4f70 con 0x7fa744147d90 2024-03-29T04:10:24.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.161+0000 7fa742d9d640 1 --2- 172.21.15.148:0/1279603714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa744147d90 0x7fa7440c8ab0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa73c009d90 tx=0x7fa73c02be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a7d91fd6a01489a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:24.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.161+0000 7fa74159a640 1 -- 172.21.15.148:0/1279603714 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c039070 con 0x7fa744147d90 2024-03-29T04:10:24.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.161+0000 7fa74159a640 1 -- 172.21.15.148:0/1279603714 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa73c02c940 con 0x7fa744147d90 2024-03-29T04:10:24.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.161+0000 7fa74159a640 1 -- 172.21.15.148:0/1279603714 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa738001670 con 0x7fa744147d90 2024-03-29T04:10:24.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.162+0000 7fa748e0d640 1 -- 172.21.15.148:0/1279603714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa744147d90 msgr2=0x7fa7440c8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.162+0000 7fa748e0d640 1 --2- 172.21.15.148:0/1279603714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa744147d90 0x7fa7440c8ab0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa73c009d90 tx=0x7fa73c02be10 comp rx=0 tx=0).stop 2024-03-29T04:10:24.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.162+0000 7fa748e0d640 1 -- 172.21.15.148:0/1279603714 shutdown_connections 2024-03-29T04:10:24.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.162+0000 7fa748e0d640 1 --2- 172.21.15.148:0/1279603714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa744147d90 0x7fa7440c8ab0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.162+0000 7fa748e0d640 1 -- 172.21.15.148:0/1279603714 >> 172.21.15.148:0/1279603714 conn(0x7fa7440b3c70 msgr2=0x7fa7440b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:24.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.162+0000 7fa748e0d640 1 -- 172.21.15.148:0/1279603714 shutdown_connections 2024-03-29T04:10:24.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.162+0000 7fa748e0d640 1 -- 172.21.15.148:0/1279603714 wait complete. 2024-03-29T04:10:24.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.163+0000 7fa748e0d640 1 Processor -- start 2024-03-29T04:10:24.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.163+0000 7fa748e0d640 1 -- start start 2024-03-29T04:10:24.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.164+0000 7fa748e0d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7441479b0 0x7fa744160710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.164+0000 7fa748e0d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa744147d90 0x7fa744141da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.164+0000 7fa748e0d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa744162650 0x7fa74415f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.164+0000 7fa748e0d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa7440cf1d0 con 0x7fa744162650 2024-03-29T04:10:24.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.164+0000 7fa748e0d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa7440cf050 con 0x7fa744147d90 2024-03-29T04:10:24.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.164+0000 7fa748e0d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa7440d0a00 con 0x7fa7441479b0 2024-03-29T04:10:24.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.164+0000 7fa742d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa744162650 0x7fa74415f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.164+0000 7fa742d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa744162650 0x7fa74415f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46414/0 (socket says 172.21.15.148:46414) 2024-03-29T04:10:24.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.164+0000 7fa742d9d640 1 -- 172.21.15.148:0/2191413960 learned_addr learned my addr 172.21.15.148:0/2191413960 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:24.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.164+0000 7fa74259c640 1 --2- 172.21.15.148:0/2191413960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7441479b0 0x7fa744160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.164+0000 7fa741d9b640 1 --2- 172.21.15.148:0/2191413960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa744147d90 0x7fa744141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.164+0000 7fa742d9d640 1 -- 172.21.15.148:0/2191413960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7441479b0 msgr2=0x7fa744160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.164+0000 7fa742d9d640 1 --2- 172.21.15.148:0/2191413960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7441479b0 0x7fa744160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.164+0000 7fa742d9d640 1 -- 172.21.15.148:0/2191413960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa744147d90 msgr2=0x7fa744141da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.164+0000 7fa742d9d640 1 --2- 172.21.15.148:0/2191413960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa744147d90 0x7fa744141da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.164+0000 7fa742d9d640 1 -- 172.21.15.148:0/2191413960 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa744160e10 con 0x7fa744162650 2024-03-29T04:10:24.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.164+0000 7fa74259c640 1 --2- 172.21.15.148:0/2191413960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7441479b0 0x7fa744160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:24.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.165+0000 7fa742d9d640 1 --2- 172.21.15.148:0/2191413960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa744162650 0x7fa74415f5d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa73c037040 tx=0x7fa73c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:24.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.165+0000 7fa732ffd640 1 -- 172.21.15.148:0/2191413960 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c039050 con 0x7fa744162650 2024-03-29T04:10:24.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.165+0000 7fa732ffd640 1 -- 172.21.15.148:0/2191413960 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa73c045070 con 0x7fa744162650 2024-03-29T04:10:24.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.165+0000 7fa732ffd640 1 -- 172.21.15.148:0/2191413960 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c009450 con 0x7fa744162650 2024-03-29T04:10:24.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.165+0000 7fa748e0d640 1 -- 172.21.15.148:0/2191413960 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7440c9ed0 con 0x7fa744162650 2024-03-29T04:10:24.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.165+0000 7fa748e0d640 1 -- 172.21.15.148:0/2191413960 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7440ca490 con 0x7fa744162650 2024-03-29T04:10:24.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.166+0000 7fa748e0d640 1 -- 172.21.15.148:0/2191413960 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7440687e0 con 0x7fa744162650 2024-03-29T04:10:24.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.166+0000 7fa732ffd640 1 -- 172.21.15.148:0/2191413960 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa73c040690 con 0x7fa744162650 2024-03-29T04:10:24.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.167+0000 7fa732ffd640 1 --2- 172.21.15.148:0/2191413960 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa72406c530 0x7fa72406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.167+0000 7fa74259c640 1 --2- 172.21.15.148:0/2191413960 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa72406c530 0x7fa72406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.167+0000 7fa732ffd640 1 -- 172.21.15.148:0/2191413960 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa73c0ae110 con 0x7fa744162650 2024-03-29T04:10:24.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.167+0000 7fa74259c640 1 --2- 172.21.15.148:0/2191413960 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa72406c530 0x7fa72406e9f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa7340011e0 tx=0x7fa734009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:24.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.171+0000 7fa732ffd640 1 -- 172.21.15.148:0/2191413960 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa73c04f020 con 0x7fa744162650 2024-03-29T04:10:24.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.353+0000 7fa748e0d640 1 -- 172.21.15.148:0/2191413960 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7440c87b0 con 0x7fa744162650 2024-03-29T04:10:24.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.355+0000 7fa732ffd640 1 -- 172.21.15.148:0/2191413960 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7fa73c07c3f0 con 0x7fa744162650 2024-03-29T04:10:24.334 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:24.335 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:05.026481+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:24.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.357+0000 7fa748e0d640 1 -- 172.21.15.148:0/2191413960 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa72406c530 msgr2=0x7fa72406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.357+0000 7fa748e0d640 1 --2- 172.21.15.148:0/2191413960 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa72406c530 0x7fa72406e9f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa7340011e0 tx=0x7fa734009040 comp rx=0 tx=0).stop 2024-03-29T04:10:24.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.357+0000 7fa748e0d640 1 -- 172.21.15.148:0/2191413960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa744162650 msgr2=0x7fa74415f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.357+0000 7fa748e0d640 1 --2- 172.21.15.148:0/2191413960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa744162650 0x7fa74415f5d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa73c037040 tx=0x7fa73c004a40 comp rx=0 tx=0).stop 2024-03-29T04:10:24.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.357+0000 7fa748e0d640 1 -- 172.21.15.148:0/2191413960 shutdown_connections 2024-03-29T04:10:24.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.357+0000 7fa748e0d640 1 --2- 172.21.15.148:0/2191413960 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa72406c530 0x7fa72406e9f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.357+0000 7fa748e0d640 1 --2- 172.21.15.148:0/2191413960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa744162650 0x7fa74415f5d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.357+0000 7fa748e0d640 1 --2- 172.21.15.148:0/2191413960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa744147d90 0x7fa744141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.357+0000 7fa748e0d640 1 --2- 172.21.15.148:0/2191413960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7441479b0 0x7fa744160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.357+0000 7fa748e0d640 1 -- 172.21.15.148:0/2191413960 >> 172.21.15.148:0/2191413960 conn(0x7fa7440b3c70 msgr2=0x7fa744142cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:24.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.357+0000 7fa748e0d640 1 -- 172.21.15.148:0/2191413960 shutdown_connections 2024-03-29T04:10:24.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.357+0000 7fa748e0d640 1 -- 172.21.15.148:0/2191413960 wait complete. 2024-03-29T04:10:24.347 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-29T04:10:24.348 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:10:24.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.470+0000 7f2752f9d640 1 Processor -- start 2024-03-29T04:10:24.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.470+0000 7f2752f9d640 1 -- start start 2024-03-29T04:10:24.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.471+0000 7f2752f9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f274c070700 0x7f274c1607b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.471+0000 7f2752f9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f274c06f930 0x7f274c06fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.471+0000 7f2752f9d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f274c0cf7e0 con 0x7f274c15c2f0 2024-03-29T04:10:24.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.471+0000 7f2752f9d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f274c15dfd0 con 0x7f274c06f930 2024-03-29T04:10:24.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.471+0000 7f2752f9d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f274c0702c0 con 0x7f274c070700 2024-03-29T04:10:24.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.471+0000 7f275153a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f274c06f930 0x7f274c06fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.471+0000 7f275153a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f274c06f930 0x7f274c06fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46426/0 (socket says 172.21.15.148:46426) 2024-03-29T04:10:24.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.471+0000 7f275153a640 1 -- 172.21.15.148:0/3879356703 learned_addr learned my addr 172.21.15.148:0/3879356703 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:24.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.471+0000 7f274bfff640 1 --2- 172.21.15.148:0/3879356703 >> v2:172.21.15.157:3300/0 conn(0x7f274c070700 0x7f274c1607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.471+0000 7f275153a640 1 -- 172.21.15.148:0/3879356703 >> v2:172.21.15.157:3300/0 conn(0x7f274c070700 msgr2=0x7f274c1607b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:24.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.471+0000 7f275153a640 1 --2- 172.21.15.148:0/3879356703 >> v2:172.21.15.157:3300/0 conn(0x7f274c070700 0x7f274c1607b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.471+0000 7f275153a640 1 -- 172.21.15.148:0/3879356703 >> v1:172.21.15.157:6789/0 conn(0x7f274c15c2f0 legacy=0x7f274c15d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:24.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.471+0000 7f275153a640 1 -- 172.21.15.148:0/3879356703 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274c0cdb00 con 0x7f274c06f930 2024-03-29T04:10:24.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.472+0000 7f275153a640 1 --2- 172.21.15.148:0/3879356703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f274c06f930 0x7f274c06fd80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f274c0b4010 tx=0x7f274402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634b5c97442bb6be server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:24.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.472+0000 7f274b7fe640 1 -- 172.21.15.148:0/3879356703 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274402ca40 con 0x7f274c06f930 2024-03-29T04:10:24.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.472+0000 7f274b7fe640 1 -- 172.21.15.148:0/3879356703 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f274402cbe0 con 0x7f274c06f930 2024-03-29T04:10:24.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.472+0000 7f274b7fe640 1 -- 172.21.15.148:0/3879356703 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27440343f0 con 0x7f274c06f930 2024-03-29T04:10:24.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.473+0000 7f2752f9d640 1 -- 172.21.15.148:0/3879356703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f274c06f930 msgr2=0x7f274c06fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.473+0000 7f2752f9d640 1 --2- 172.21.15.148:0/3879356703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f274c06f930 0x7f274c06fd80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f274c0b4010 tx=0x7f274402bea0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.473+0000 7f2752f9d640 1 -- 172.21.15.148:0/3879356703 shutdown_connections 2024-03-29T04:10:24.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.473+0000 7f2752f9d640 1 --2- 172.21.15.148:0/3879356703 >> v2:172.21.15.157:3300/0 conn(0x7f274c070700 0x7f274c1607b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.473+0000 7f2752f9d640 1 --2- 172.21.15.148:0/3879356703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f274c06f930 0x7f274c06fd80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.473+0000 7f2752f9d640 1 -- 172.21.15.148:0/3879356703 >> 172.21.15.148:0/3879356703 conn(0x7f274c06e360 msgr2=0x7f274c06e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:24.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.473+0000 7f2752f9d640 1 -- 172.21.15.148:0/3879356703 shutdown_connections 2024-03-29T04:10:24.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.473+0000 7f2752f9d640 1 -- 172.21.15.148:0/3879356703 wait complete. 2024-03-29T04:10:24.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.474+0000 7f2752f9d640 1 Processor -- start 2024-03-29T04:10:24.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.474+0000 7f2752f9d640 1 -- start start 2024-03-29T04:10:24.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.475+0000 7f2752f9d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f274c06f930 0x7f274c150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.475+0000 7f2752f9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f274c070700 0x7f274c13c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.475+0000 7f2752f9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f274c15c2f0 0x7f274c0b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.475+0000 7f2752f9d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f274c0ce0c0 con 0x7f274c070700 2024-03-29T04:10:24.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.475+0000 7f2752f9d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f274c0cfb10 con 0x7f274c15c2f0 2024-03-29T04:10:24.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.475+0000 7f2752f9d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f274c0ce240 con 0x7f274c06f930 2024-03-29T04:10:24.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.475+0000 7f274bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f274c070700 0x7f274c13c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.475+0000 7f274bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f274c070700 0x7f274c13c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46434/0 (socket says 172.21.15.148:46434) 2024-03-29T04:10:24.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.475+0000 7f274bfff640 1 -- 172.21.15.148:0/982163546 learned_addr learned my addr 172.21.15.148:0/982163546 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:24.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.475+0000 7f275153a640 1 --2- 172.21.15.148:0/982163546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f274c15c2f0 0x7f274c0b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.475+0000 7f2750d39640 1 --2- 172.21.15.148:0/982163546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f274c06f930 0x7f274c150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.476+0000 7f274bfff640 1 -- 172.21.15.148:0/982163546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f274c06f930 msgr2=0x7f274c150ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.476+0000 7f274bfff640 1 --2- 172.21.15.148:0/982163546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f274c06f930 0x7f274c150ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.476+0000 7f274bfff640 1 -- 172.21.15.148:0/982163546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f274c15c2f0 msgr2=0x7f274c0b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.476+0000 7f274bfff640 1 --2- 172.21.15.148:0/982163546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f274c15c2f0 0x7f274c0b1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.476+0000 7f274bfff640 1 -- 172.21.15.148:0/982163546 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274c14fde0 con 0x7f274c070700 2024-03-29T04:10:24.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.476+0000 7f2750d39640 1 --2- 172.21.15.148:0/982163546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f274c06f930 0x7f274c150ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:24.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.476+0000 7f274bfff640 1 --2- 172.21.15.148:0/982163546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f274c070700 0x7f274c13c4c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f273800d5f0 tx=0x7f273800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:24.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.476+0000 7f27497fa640 1 -- 172.21.15.148:0/982163546 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27380125f0 con 0x7f274c070700 2024-03-29T04:10:24.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.476+0000 7f27497fa640 1 -- 172.21.15.148:0/982163546 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2738019070 con 0x7f274c070700 2024-03-29T04:10:24.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.476+0000 7f2752f9d640 1 -- 172.21.15.148:0/982163546 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274c1500c0 con 0x7f274c070700 2024-03-29T04:10:24.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.476+0000 7f27497fa640 1 -- 172.21.15.148:0/982163546 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27380129d0 con 0x7f274c070700 2024-03-29T04:10:24.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.476+0000 7f2752f9d640 1 -- 172.21.15.148:0/982163546 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f274c14fa90 con 0x7f274c070700 2024-03-29T04:10:24.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.477+0000 7f2752f9d640 1 -- 172.21.15.148:0/982163546 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f274c0687e0 con 0x7f274c070700 2024-03-29T04:10:24.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.477+0000 7f27497fa640 1 -- 172.21.15.148:0/982163546 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2738012b70 con 0x7f274c070700 2024-03-29T04:10:24.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.478+0000 7f27497fa640 1 --2- 172.21.15.148:0/982163546 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f273006c530 0x7f273006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.478+0000 7f27497fa640 1 -- 172.21.15.148:0/982163546 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f273808bd70 con 0x7f274c070700 2024-03-29T04:10:24.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.478+0000 7f2750d39640 1 --2- 172.21.15.148:0/982163546 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f273006c530 0x7f273006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.478+0000 7f2750d39640 1 --2- 172.21.15.148:0/982163546 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f273006c530 0x7f273006e9f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f274c1584e0 tx=0x7f2740009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:24.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.482+0000 7f27497fa640 1 -- 172.21.15.148:0/982163546 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f273805a050 con 0x7f274c070700 2024-03-29T04:10:24.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.669+0000 7f2752f9d640 1 -- 172.21.15.148:0/982163546 --> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f274c0b1aa0 con 0x7f273006c530 2024-03-29T04:10:24.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.677+0000 7f27497fa640 1 -- 172.21.15.148:0/982163546 <== mgr.4113 v2:172.21.15.148:6832/3271445075 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f274c0b1aa0 con 0x7f273006c530 2024-03-29T04:10:24.660 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:24.663 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-03-29T04:10:24.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.681+0000 7f2752f9d640 1 -- 172.21.15.148:0/982163546 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f273006c530 msgr2=0x7f273006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.682+0000 7f2752f9d640 1 --2- 172.21.15.148:0/982163546 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f273006c530 0x7f273006e9f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f274c1584e0 tx=0x7f2740009040 comp rx=0 tx=0).stop 2024-03-29T04:10:24.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.682+0000 7f2752f9d640 1 -- 172.21.15.148:0/982163546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f274c070700 msgr2=0x7f274c13c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.682+0000 7f2752f9d640 1 --2- 172.21.15.148:0/982163546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f274c070700 0x7f274c13c4c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f273800d5f0 tx=0x7f273800dab0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.682+0000 7f2752f9d640 1 -- 172.21.15.148:0/982163546 shutdown_connections 2024-03-29T04:10:24.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.682+0000 7f2752f9d640 1 --2- 172.21.15.148:0/982163546 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f273006c530 0x7f273006e9f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.682+0000 7f2752f9d640 1 --2- 172.21.15.148:0/982163546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f274c15c2f0 0x7f274c0b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.682+0000 7f2752f9d640 1 --2- 172.21.15.148:0/982163546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f274c070700 0x7f274c13c4c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.682+0000 7f2752f9d640 1 --2- 172.21.15.148:0/982163546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f274c06f930 0x7f274c150ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.682+0000 7f2752f9d640 1 -- 172.21.15.148:0/982163546 >> 172.21.15.148:0/982163546 conn(0x7f274c06e360 msgr2=0x7f274c0b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:24.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.682+0000 7f2752f9d640 1 -- 172.21.15.148:0/982163546 shutdown_connections 2024-03-29T04:10:24.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.682+0000 7f2752f9d640 1 -- 172.21.15.148:0/982163546 wait complete. 2024-03-29T04:10:24.673 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-03-29T04:10:23.887357+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236032,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738688,"statfs":{"total":773094113280,"available":772852416512,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002287"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911869+0000","last_change":"2024-03-29T04:09:59.541493+0000","last_active":"2024-03-29T04:10:05.911869+0000","last_peered":"2024-03-29T04:10:05.911869+0000","last_clean":"2024-03-29T04:10:05.911869+0000","last_became_active":"2024-03-29T04:09:59.541208+0000","last_became_peered":"2024-03-29T04:09:59.541208+0000","last_unstale":"2024-03-29T04:10:05.911869+0000","last_undegraded":"2024-03-29T04:10:05.911869+0000","last_fullsized":"2024-03-29T04:10:05.911869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:55:08.672579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910799+0000","last_change":"2024-03-29T04:10:00.548950+0000","last_active":"2024-03-29T04:10:05.910799+0000","last_peered":"2024-03-29T04:10:05.910799+0000","last_clean":"2024-03-29T04:10:05.910799+0000","last_became_active":"2024-03-29T04:10:00.548670+0000","last_became_peered":"2024-03-29T04:10:00.548670+0000","last_unstale":"2024-03-29T04:10:05.910799+0000","last_undegraded":"2024-03-29T04:10:05.910799+0000","last_fullsized":"2024-03-29T04:10:05.910799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:16:02.331457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911034+0000","last_change":"2024-03-29T04:10:04.649988+0000","last_active":"2024-03-29T04:10:05.911034+0000","last_peered":"2024-03-29T04:10:05.911034+0000","last_clean":"2024-03-29T04:10:05.911034+0000","last_became_active":"2024-03-29T04:10:04.649557+0000","last_became_peered":"2024-03-29T04:10:04.649557+0000","last_unstale":"2024-03-29T04:10:05.911034+0000","last_undegraded":"2024-03-29T04:10:05.911034+0000","last_fullsized":"2024-03-29T04:10:05.911034+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:07:56.606322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911032+0000","last_change":"2024-03-29T04:09:59.542509+0000","last_active":"2024-03-29T04:10:05.911032+0000","last_peered":"2024-03-29T04:10:05.911032+0000","last_clean":"2024-03-29T04:10:05.911032+0000","last_became_active":"2024-03-29T04:09:59.542221+0000","last_became_peered":"2024-03-29T04:09:59.542221+0000","last_unstale":"2024-03-29T04:10:05.911032+0000","last_undegraded":"2024-03-29T04:10:05.911032+0000","last_fullsized":"2024-03-29T04:10:05.911032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:58:10.121554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910298+0000","last_change":"2024-03-29T04:10:00.536505+0000","last_active":"2024-03-29T04:10:05.910298+0000","last_peered":"2024-03-29T04:10:05.910298+0000","last_clean":"2024-03-29T04:10:05.910298+0000","last_became_active":"2024-03-29T04:10:00.536200+0000","last_became_peered":"2024-03-29T04:10:00.536200+0000","last_unstale":"2024-03-29T04:10:05.910298+0000","last_undegraded":"2024-03-29T04:10:05.910298+0000","last_fullsized":"2024-03-29T04:10:05.910298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:29.233534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557208+0000","last_change":"2024-03-29T04:10:04.576319+0000","last_active":"2024-03-29T04:10:05.557208+0000","last_peered":"2024-03-29T04:10:05.557208+0000","last_clean":"2024-03-29T04:10:05.557208+0000","last_became_active":"2024-03-29T04:10:04.575508+0000","last_became_peered":"2024-03-29T04:10:04.575508+0000","last_unstale":"2024-03-29T04:10:05.557208+0000","last_undegraded":"2024-03-29T04:10:05.557208+0000","last_fullsized":"2024-03-29T04:10:05.557208+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:34.129098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911504+0000","last_change":"2024-03-29T04:09:59.545281+0000","last_active":"2024-03-29T04:10:05.911504+0000","last_peered":"2024-03-29T04:10:05.911504+0000","last_clean":"2024-03-29T04:10:05.911504+0000","last_became_active":"2024-03-29T04:09:59.544991+0000","last_became_peered":"2024-03-29T04:09:59.544991+0000","last_unstale":"2024-03-29T04:10:05.911504+0000","last_undegraded":"2024-03-29T04:10:05.911504+0000","last_fullsized":"2024-03-29T04:10:05.911504+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:35:01.484589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556861+0000","last_change":"2024-03-29T04:10:00.542510+0000","last_active":"2024-03-29T04:10:05.556861+0000","last_peered":"2024-03-29T04:10:05.556861+0000","last_clean":"2024-03-29T04:10:05.556861+0000","last_became_active":"2024-03-29T04:10:00.542014+0000","last_became_peered":"2024-03-29T04:10:00.542014+0000","last_unstale":"2024-03-29T04:10:05.556861+0000","last_undegraded":"2024-03-29T04:10:05.556861+0000","last_fullsized":"2024-03-29T04:10:05.556861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:30.096622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556709+0000","last_change":"2024-03-29T04:10:04.574735+0000","last_active":"2024-03-29T04:10:05.556709+0000","last_peered":"2024-03-29T04:10:05.556709+0000","last_clean":"2024-03-29T04:10:05.556709+0000","last_became_active":"2024-03-29T04:10:04.573995+0000","last_became_peered":"2024-03-29T04:10:04.573995+0000","last_unstale":"2024-03-29T04:10:05.556709+0000","last_undegraded":"2024-03-29T04:10:05.556709+0000","last_fullsized":"2024-03-29T04:10:05.556709+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:52.246321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911346+0000","last_change":"2024-03-29T04:09:59.542326+0000","last_active":"2024-03-29T04:10:05.911346+0000","last_peered":"2024-03-29T04:10:05.911346+0000","last_clean":"2024-03-29T04:10:05.911346+0000","last_became_active":"2024-03-29T04:09:59.540873+0000","last_became_peered":"2024-03-29T04:09:59.540873+0000","last_unstale":"2024-03-29T04:10:05.911346+0000","last_undegraded":"2024-03-29T04:10:05.911346+0000","last_fullsized":"2024-03-29T04:10:05.911346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:02.879344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557486+0000","last_change":"2024-03-29T04:10:00.543297+0000","last_active":"2024-03-29T04:10:05.557486+0000","last_peered":"2024-03-29T04:10:05.557486+0000","last_clean":"2024-03-29T04:10:05.557486+0000","last_became_active":"2024-03-29T04:10:00.541793+0000","last_became_peered":"2024-03-29T04:10:00.541793+0000","last_unstale":"2024-03-29T04:10:05.557486+0000","last_undegraded":"2024-03-29T04:10:05.557486+0000","last_fullsized":"2024-03-29T04:10:05.557486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:13.002495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557550+0000","last_change":"2024-03-29T04:10:04.572366+0000","last_active":"2024-03-29T04:10:05.557550+0000","last_peered":"2024-03-29T04:10:05.557550+0000","last_clean":"2024-03-29T04:10:05.557550+0000","last_became_active":"2024-03-29T04:10:04.572066+0000","last_became_peered":"2024-03-29T04:10:04.572066+0000","last_unstale":"2024-03-29T04:10:05.557550+0000","last_undegraded":"2024-03-29T04:10:05.557550+0000","last_fullsized":"2024-03-29T04:10:05.557550+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:00:55.353300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910448+0000","last_change":"2024-03-29T04:10:00.547621+0000","last_active":"2024-03-29T04:10:05.910448+0000","last_peered":"2024-03-29T04:10:05.910448+0000","last_clean":"2024-03-29T04:10:05.910448+0000","last_became_active":"2024-03-29T04:10:00.547282+0000","last_became_peered":"2024-03-29T04:10:00.547282+0000","last_unstale":"2024-03-29T04:10:05.910448+0000","last_undegraded":"2024-03-29T04:10:05.910448+0000","last_fullsized":"2024-03-29T04:10:05.910448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:41.517875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911279+0000","last_change":"2024-03-29T04:09:59.539362+0000","last_active":"2024-03-29T04:10:05.911279+0000","last_peered":"2024-03-29T04:10:05.911279+0000","last_clean":"2024-03-29T04:10:05.911279+0000","last_became_active":"2024-03-29T04:09:59.539062+0000","last_became_peered":"2024-03-29T04:09:59.539062+0000","last_unstale":"2024-03-29T04:10:05.911279+0000","last_undegraded":"2024-03-29T04:10:05.911279+0000","last_fullsized":"2024-03-29T04:10:05.911279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:33:50.904931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.558475+0000","last_change":"2024-03-29T04:10:04.579133+0000","last_active":"2024-03-29T04:10:05.558475+0000","last_peered":"2024-03-29T04:10:05.558475+0000","last_clean":"2024-03-29T04:10:05.558475+0000","last_became_active":"2024-03-29T04:10:04.578748+0000","last_became_peered":"2024-03-29T04:10:04.578748+0000","last_unstale":"2024-03-29T04:10:05.558475+0000","last_undegraded":"2024-03-29T04:10:05.558475+0000","last_fullsized":"2024-03-29T04:10:05.558475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:06.998903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910015+0000","last_change":"2024-03-29T04:10:00.547887+0000","last_active":"2024-03-29T04:10:05.910015+0000","last_peered":"2024-03-29T04:10:05.910015+0000","last_clean":"2024-03-29T04:10:05.910015+0000","last_became_active":"2024-03-29T04:10:00.547419+0000","last_became_peered":"2024-03-29T04:10:00.547419+0000","last_unstale":"2024-03-29T04:10:05.910015+0000","last_undegraded":"2024-03-29T04:10:05.910015+0000","last_fullsized":"2024-03-29T04:10:05.910015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:46.707085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912399+0000","last_change":"2024-03-29T04:09:59.544306+0000","last_active":"2024-03-29T04:10:05.912399+0000","last_peered":"2024-03-29T04:10:05.912399+0000","last_clean":"2024-03-29T04:10:05.912399+0000","last_became_active":"2024-03-29T04:09:59.544028+0000","last_became_peered":"2024-03-29T04:09:59.544028+0000","last_unstale":"2024-03-29T04:10:05.912399+0000","last_undegraded":"2024-03-29T04:10:05.912399+0000","last_fullsized":"2024-03-29T04:10:05.912399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:33:50.904931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.558045+0000","last_change":"2024-03-29T04:10:04.577178+0000","last_active":"2024-03-29T04:10:05.558045+0000","last_peered":"2024-03-29T04:10:05.558045+0000","last_clean":"2024-03-29T04:10:05.558045+0000","last_became_active":"2024-03-29T04:10:04.576788+0000","last_became_peered":"2024-03-29T04:10:04.576788+0000","last_unstale":"2024-03-29T04:10:05.558045+0000","last_undegraded":"2024-03-29T04:10:05.558045+0000","last_fullsized":"2024-03-29T04:10:05.558045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:26:14.412794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910704+0000","last_change":"2024-03-29T04:10:00.543623+0000","last_active":"2024-03-29T04:10:05.910704+0000","last_peered":"2024-03-29T04:10:05.910704+0000","last_clean":"2024-03-29T04:10:05.910704+0000","last_became_active":"2024-03-29T04:10:00.542670+0000","last_became_peered":"2024-03-29T04:10:00.542670+0000","last_unstale":"2024-03-29T04:10:05.910704+0000","last_undegraded":"2024-03-29T04:10:05.910704+0000","last_fullsized":"2024-03-29T04:10:05.910704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:48.213845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910045+0000","last_change":"2024-03-29T04:09:59.540613+0000","last_active":"2024-03-29T04:10:05.910045+0000","last_peered":"2024-03-29T04:10:05.910045+0000","last_clean":"2024-03-29T04:10:05.910045+0000","last_became_active":"2024-03-29T04:09:59.540340+0000","last_became_peered":"2024-03-29T04:09:59.540340+0000","last_unstale":"2024-03-29T04:10:05.910045+0000","last_undegraded":"2024-03-29T04:10:05.910045+0000","last_fullsized":"2024-03-29T04:10:05.910045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:05.307537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911330+0000","last_change":"2024-03-29T04:10:04.646383+0000","last_active":"2024-03-29T04:10:05.911330+0000","last_peered":"2024-03-29T04:10:05.911330+0000","last_clean":"2024-03-29T04:10:05.911330+0000","last_became_active":"2024-03-29T04:10:04.646046+0000","last_became_peered":"2024-03-29T04:10:04.646046+0000","last_unstale":"2024-03-29T04:10:05.911330+0000","last_undegraded":"2024-03-29T04:10:05.911330+0000","last_fullsized":"2024-03-29T04:10:05.911330+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:17.034970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557229+0000","last_change":"2024-03-29T04:10:00.537777+0000","last_active":"2024-03-29T04:10:05.557229+0000","last_peered":"2024-03-29T04:10:05.557229+0000","last_clean":"2024-03-29T04:10:05.557229+0000","last_became_active":"2024-03-29T04:10:00.537423+0000","last_became_peered":"2024-03-29T04:10:00.537423+0000","last_unstale":"2024-03-29T04:10:05.557229+0000","last_undegraded":"2024-03-29T04:10:05.557229+0000","last_fullsized":"2024-03-29T04:10:05.557229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:33:51.909341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910053+0000","last_change":"2024-03-29T04:09:59.543268+0000","last_active":"2024-03-29T04:10:05.910053+0000","last_peered":"2024-03-29T04:10:05.910053+0000","last_clean":"2024-03-29T04:10:05.910053+0000","last_became_active":"2024-03-29T04:09:59.542982+0000","last_became_peered":"2024-03-29T04:09:59.542982+0000","last_unstale":"2024-03-29T04:10:05.910053+0000","last_undegraded":"2024-03-29T04:10:05.910053+0000","last_fullsized":"2024-03-29T04:10:05.910053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:01.925223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557817+0000","last_change":"2024-03-29T04:10:04.577232+0000","last_active":"2024-03-29T04:10:05.557817+0000","last_peered":"2024-03-29T04:10:05.557817+0000","last_clean":"2024-03-29T04:10:05.557817+0000","last_became_active":"2024-03-29T04:10:04.576758+0000","last_became_peered":"2024-03-29T04:10:04.576758+0000","last_unstale":"2024-03-29T04:10:05.557817+0000","last_undegraded":"2024-03-29T04:10:05.557817+0000","last_fullsized":"2024-03-29T04:10:05.557817+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:26:08.662188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912635+0000","last_change":"2024-03-29T04:10:00.546235+0000","last_active":"2024-03-29T04:10:05.912635+0000","last_peered":"2024-03-29T04:10:05.912635+0000","last_clean":"2024-03-29T04:10:05.912635+0000","last_became_active":"2024-03-29T04:10:00.545908+0000","last_became_peered":"2024-03-29T04:10:00.545908+0000","last_unstale":"2024-03-29T04:10:05.912635+0000","last_undegraded":"2024-03-29T04:10:05.912635+0000","last_fullsized":"2024-03-29T04:10:05.912635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:16:02.331457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911074+0000","last_change":"2024-03-29T04:09:59.541657+0000","last_active":"2024-03-29T04:10:05.911074+0000","last_peered":"2024-03-29T04:10:05.911074+0000","last_clean":"2024-03-29T04:10:05.911074+0000","last_became_active":"2024-03-29T04:09:59.541305+0000","last_became_peered":"2024-03-29T04:09:59.541305+0000","last_unstale":"2024-03-29T04:10:05.911074+0000","last_undegraded":"2024-03-29T04:10:05.911074+0000","last_fullsized":"2024-03-29T04:10:05.911074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:25:06.123311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556981+0000","last_change":"2024-03-29T04:10:04.646174+0000","last_active":"2024-03-29T04:10:05.556981+0000","last_peered":"2024-03-29T04:10:05.556981+0000","last_clean":"2024-03-29T04:10:05.556981+0000","last_became_active":"2024-03-29T04:10:04.645802+0000","last_became_peered":"2024-03-29T04:10:04.645802+0000","last_unstale":"2024-03-29T04:10:05.556981+0000","last_undegraded":"2024-03-29T04:10:05.556981+0000","last_fullsized":"2024-03-29T04:10:05.556981+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:35:04.485257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911614+0000","last_change":"2024-03-29T04:10:00.543254+0000","last_active":"2024-03-29T04:10:05.911614+0000","last_peered":"2024-03-29T04:10:05.911614+0000","last_clean":"2024-03-29T04:10:05.911614+0000","last_became_active":"2024-03-29T04:10:00.542958+0000","last_became_peered":"2024-03-29T04:10:00.542958+0000","last_unstale":"2024-03-29T04:10:05.911614+0000","last_undegraded":"2024-03-29T04:10:05.911614+0000","last_fullsized":"2024-03-29T04:10:05.911614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:00:51.320824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911924+0000","last_change":"2024-03-29T04:09:59.544320+0000","last_active":"2024-03-29T04:10:05.911924+0000","last_peered":"2024-03-29T04:10:05.911924+0000","last_clean":"2024-03-29T04:10:05.911924+0000","last_became_active":"2024-03-29T04:09:59.543928+0000","last_became_peered":"2024-03-29T04:09:59.543928+0000","last_unstale":"2024-03-29T04:10:05.911924+0000","last_undegraded":"2024-03-29T04:10:05.911924+0000","last_fullsized":"2024-03-29T04:10:05.911924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:02.879344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910274+0000","last_change":"2024-03-29T04:10:04.646905+0000","last_active":"2024-03-29T04:10:05.910274+0000","last_peered":"2024-03-29T04:10:05.910274+0000","last_clean":"2024-03-29T04:10:05.910274+0000","last_became_active":"2024-03-29T04:10:04.646482+0000","last_became_peered":"2024-03-29T04:10:04.646482+0000","last_unstale":"2024-03-29T04:10:05.910274+0000","last_undegraded":"2024-03-29T04:10:05.910274+0000","last_fullsized":"2024-03-29T04:10:05.910274+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:38:53.191487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910879+0000","last_change":"2024-03-29T04:10:00.541465+0000","last_active":"2024-03-29T04:10:05.910879+0000","last_peered":"2024-03-29T04:10:05.910879+0000","last_clean":"2024-03-29T04:10:05.910879+0000","last_became_active":"2024-03-29T04:10:00.541197+0000","last_became_peered":"2024-03-29T04:10:00.541197+0000","last_unstale":"2024-03-29T04:10:05.910879+0000","last_undegraded":"2024-03-29T04:10:05.910879+0000","last_fullsized":"2024-03-29T04:10:05.910879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:52:12.957170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911673+0000","last_change":"2024-03-29T04:09:59.545759+0000","last_active":"2024-03-29T04:10:05.911673+0000","last_peered":"2024-03-29T04:10:05.911673+0000","last_clean":"2024-03-29T04:10:05.911673+0000","last_became_active":"2024-03-29T04:09:59.545373+0000","last_became_peered":"2024-03-29T04:09:59.545373+0000","last_unstale":"2024-03-29T04:10:05.911673+0000","last_undegraded":"2024-03-29T04:10:05.911673+0000","last_fullsized":"2024-03-29T04:10:05.911673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:58:10.121554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911095+0000","last_change":"2024-03-29T04:10:04.644123+0000","last_active":"2024-03-29T04:10:05.911095+0000","last_peered":"2024-03-29T04:10:05.911095+0000","last_clean":"2024-03-29T04:10:05.911095+0000","last_became_active":"2024-03-29T04:10:04.643701+0000","last_became_peered":"2024-03-29T04:10:04.643701+0000","last_unstale":"2024-03-29T04:10:05.911095+0000","last_undegraded":"2024-03-29T04:10:05.911095+0000","last_fullsized":"2024-03-29T04:10:05.911095+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:26:14.412794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557781+0000","last_change":"2024-03-29T04:10:00.543518+0000","last_active":"2024-03-29T04:10:05.557781+0000","last_peered":"2024-03-29T04:10:05.557781+0000","last_clean":"2024-03-29T04:10:05.557781+0000","last_became_active":"2024-03-29T04:10:00.542373+0000","last_became_peered":"2024-03-29T04:10:00.542373+0000","last_unstale":"2024-03-29T04:10:05.557781+0000","last_undegraded":"2024-03-29T04:10:05.557781+0000","last_fullsized":"2024-03-29T04:10:05.557781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:02.966428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910966+0000","last_change":"2024-03-29T04:09:59.542356+0000","last_active":"2024-03-29T04:10:05.910966+0000","last_peered":"2024-03-29T04:10:05.910966+0000","last_clean":"2024-03-29T04:10:05.910966+0000","last_became_active":"2024-03-29T04:09:59.541917+0000","last_became_peered":"2024-03-29T04:09:59.541917+0000","last_unstale":"2024-03-29T04:10:05.910966+0000","last_undegraded":"2024-03-29T04:10:05.910966+0000","last_fullsized":"2024-03-29T04:10:05.910966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:01.925223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912224+0000","last_change":"2024-03-29T04:10:04.578218+0000","last_active":"2024-03-29T04:10:05.912224+0000","last_peered":"2024-03-29T04:10:05.912224+0000","last_clean":"2024-03-29T04:10:05.912224+0000","last_became_active":"2024-03-29T04:10:04.577221+0000","last_became_peered":"2024-03-29T04:10:04.577221+0000","last_unstale":"2024-03-29T04:10:05.912224+0000","last_undegraded":"2024-03-29T04:10:05.912224+0000","last_fullsized":"2024-03-29T04:10:05.912224+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:44:17.161094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.558130+0000","last_change":"2024-03-29T04:10:00.536176+0000","last_active":"2024-03-29T04:10:05.558130+0000","last_peered":"2024-03-29T04:10:05.558130+0000","last_clean":"2024-03-29T04:10:05.558130+0000","last_became_active":"2024-03-29T04:10:00.535854+0000","last_became_peered":"2024-03-29T04:10:00.535854+0000","last_unstale":"2024-03-29T04:10:05.558130+0000","last_undegraded":"2024-03-29T04:10:05.558130+0000","last_fullsized":"2024-03-29T04:10:05.558130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:25:07.127721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557572+0000","last_change":"2024-03-29T04:09:59.542230+0000","last_active":"2024-03-29T04:10:05.557572+0000","last_peered":"2024-03-29T04:10:05.557572+0000","last_clean":"2024-03-29T04:10:05.557572+0000","last_became_active":"2024-03-29T04:09:59.541972+0000","last_became_peered":"2024-03-29T04:09:59.541972+0000","last_unstale":"2024-03-29T04:10:05.557572+0000","last_undegraded":"2024-03-29T04:10:05.557572+0000","last_fullsized":"2024-03-29T04:10:05.557572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:01.925223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910829+0000","last_change":"2024-03-29T04:10:04.574263+0000","last_active":"2024-03-29T04:10:05.910829+0000","last_peered":"2024-03-29T04:10:05.910829+0000","last_clean":"2024-03-29T04:10:05.910829+0000","last_became_active":"2024-03-29T04:10:04.573843+0000","last_became_peered":"2024-03-29T04:10:04.573843+0000","last_unstale":"2024-03-29T04:10:05.910829+0000","last_undegraded":"2024-03-29T04:10:05.910829+0000","last_fullsized":"2024-03-29T04:10:05.910829+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:06.998903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912194+0000","last_change":"2024-03-29T04:10:00.540824+0000","last_active":"2024-03-29T04:10:05.912194+0000","last_peered":"2024-03-29T04:10:05.912194+0000","last_clean":"2024-03-29T04:10:05.912194+0000","last_became_active":"2024-03-29T04:10:00.540505+0000","last_became_peered":"2024-03-29T04:10:00.540505+0000","last_unstale":"2024-03-29T04:10:05.912194+0000","last_undegraded":"2024-03-29T04:10:05.912194+0000","last_fullsized":"2024-03-29T04:10:05.912194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:41.517875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.559855+0000","last_change":"2024-03-29T04:09:59.540415+0000","last_active":"2024-03-29T04:10:05.559855+0000","last_peered":"2024-03-29T04:10:05.559855+0000","last_clean":"2024-03-29T04:10:05.559855+0000","last_became_active":"2024-03-29T04:09:59.540130+0000","last_became_peered":"2024-03-29T04:09:59.540130+0000","last_unstale":"2024-03-29T04:10:05.559855+0000","last_undegraded":"2024-03-29T04:10:05.559855+0000","last_fullsized":"2024-03-29T04:10:05.559855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:05.307537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911492+0000","last_change":"2024-03-29T04:10:04.575156+0000","last_active":"2024-03-29T04:10:05.911492+0000","last_peered":"2024-03-29T04:10:05.911492+0000","last_clean":"2024-03-29T04:10:05.911492+0000","last_became_active":"2024-03-29T04:10:04.573816+0000","last_became_peered":"2024-03-29T04:10:04.573816+0000","last_unstale":"2024-03-29T04:10:05.911492+0000","last_undegraded":"2024-03-29T04:10:05.911492+0000","last_fullsized":"2024-03-29T04:10:05.911492+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:23:51.029457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.909950+0000","last_change":"2024-03-29T04:10:00.531739+0000","last_active":"2024-03-29T04:10:05.909950+0000","last_peered":"2024-03-29T04:10:05.909950+0000","last_clean":"2024-03-29T04:10:05.909950+0000","last_became_active":"2024-03-29T04:10:00.531395+0000","last_became_peered":"2024-03-29T04:10:00.531395+0000","last_unstale":"2024-03-29T04:10:05.909950+0000","last_undegraded":"2024-03-29T04:10:05.909950+0000","last_fullsized":"2024-03-29T04:10:05.909950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:02:32.544769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910076+0000","last_change":"2024-03-29T04:09:59.541400+0000","last_active":"2024-03-29T04:10:05.910076+0000","last_peered":"2024-03-29T04:10:05.910076+0000","last_clean":"2024-03-29T04:10:05.910076+0000","last_became_active":"2024-03-29T04:09:59.541114+0000","last_became_peered":"2024-03-29T04:09:59.541114+0000","last_unstale":"2024-03-29T04:10:05.910076+0000","last_undegraded":"2024-03-29T04:10:05.910076+0000","last_fullsized":"2024-03-29T04:10:05.910076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:25:06.123311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912614+0000","last_change":"2024-03-29T04:10:04.645661+0000","last_active":"2024-03-29T04:10:05.912614+0000","last_peered":"2024-03-29T04:10:05.912614+0000","last_clean":"2024-03-29T04:10:05.912614+0000","last_became_active":"2024-03-29T04:10:04.645258+0000","last_became_peered":"2024-03-29T04:10:04.645258+0000","last_unstale":"2024-03-29T04:10:05.912614+0000","last_undegraded":"2024-03-29T04:10:05.912614+0000","last_fullsized":"2024-03-29T04:10:05.912614+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:44:17.161094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910699+0000","last_change":"2024-03-29T04:10:00.537081+0000","last_active":"2024-03-29T04:10:05.910699+0000","last_peered":"2024-03-29T04:10:05.910699+0000","last_clean":"2024-03-29T04:10:05.910699+0000","last_became_active":"2024-03-29T04:10:00.536267+0000","last_became_peered":"2024-03-29T04:10:00.536267+0000","last_unstale":"2024-03-29T04:10:05.910699+0000","last_undegraded":"2024-03-29T04:10:05.910699+0000","last_fullsized":"2024-03-29T04:10:05.910699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:41.517875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557767+0000","last_change":"2024-03-29T04:09:59.535631+0000","last_active":"2024-03-29T04:10:05.557767+0000","last_peered":"2024-03-29T04:10:05.557767+0000","last_clean":"2024-03-29T04:10:05.557767+0000","last_became_active":"2024-03-29T04:09:59.535323+0000","last_became_peered":"2024-03-29T04:09:59.535323+0000","last_unstale":"2024-03-29T04:10:05.557767+0000","last_undegraded":"2024-03-29T04:10:05.557767+0000","last_fullsized":"2024-03-29T04:10:05.557767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:05.307537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910183+0000","last_change":"2024-03-29T04:09:49.002570+0000","last_active":"2024-03-29T04:10:05.910183+0000","last_peered":"2024-03-29T04:10:05.910183+0000","last_clean":"2024-03-29T04:10:05.910183+0000","last_became_active":"2024-03-29T04:09:49.002235+0000","last_became_peered":"2024-03-29T04:09:49.002235+0000","last_unstale":"2024-03-29T04:10:05.910183+0000","last_undegraded":"2024-03-29T04:10:05.910183+0000","last_fullsized":"2024-03-29T04:10:05.910183+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:47.900733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:47.900733+0000","last_clean_scrub_stamp":"2024-03-29T04:09:47.900733+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:44:04.608265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911029+0000","last_change":"2024-03-29T04:10:04.646698+0000","last_active":"2024-03-29T04:10:05.911029+0000","last_peered":"2024-03-29T04:10:05.911029+0000","last_clean":"2024-03-29T04:10:05.911029+0000","last_became_active":"2024-03-29T04:10:04.646221+0000","last_became_peered":"2024-03-29T04:10:04.646221+0000","last_unstale":"2024-03-29T04:10:05.911029+0000","last_undegraded":"2024-03-29T04:10:05.911029+0000","last_fullsized":"2024-03-29T04:10:05.911029+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:17.034970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911371+0000","last_change":"2024-03-29T04:10:00.540055+0000","last_active":"2024-03-29T04:10:05.911371+0000","last_peered":"2024-03-29T04:10:05.911371+0000","last_clean":"2024-03-29T04:10:05.911371+0000","last_became_active":"2024-03-29T04:10:00.539766+0000","last_became_peered":"2024-03-29T04:10:00.539766+0000","last_unstale":"2024-03-29T04:10:05.911371+0000","last_undegraded":"2024-03-29T04:10:05.911371+0000","last_fullsized":"2024-03-29T04:10:05.911371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:46.804154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.560157+0000","last_change":"2024-03-29T04:09:59.536860+0000","last_active":"2024-03-29T04:10:05.560157+0000","last_peered":"2024-03-29T04:10:05.560157+0000","last_clean":"2024-03-29T04:10:05.560157+0000","last_became_active":"2024-03-29T04:09:59.536504+0000","last_became_peered":"2024-03-29T04:09:59.536504+0000","last_unstale":"2024-03-29T04:10:05.560157+0000","last_undegraded":"2024-03-29T04:10:05.560157+0000","last_fullsized":"2024-03-29T04:10:05.560157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:02.879344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910369+0000","last_change":"2024-03-29T04:10:04.644446+0000","last_active":"2024-03-29T04:10:05.910369+0000","last_peered":"2024-03-29T04:10:05.910369+0000","last_clean":"2024-03-29T04:10:05.910369+0000","last_became_active":"2024-03-29T04:10:04.643982+0000","last_became_peered":"2024-03-29T04:10:04.643982+0000","last_unstale":"2024-03-29T04:10:05.910369+0000","last_undegraded":"2024-03-29T04:10:05.910369+0000","last_fullsized":"2024-03-29T04:10:05.910369+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:26:14.412794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556955+0000","last_change":"2024-03-29T04:10:00.544801+0000","last_active":"2024-03-29T04:10:05.556955+0000","last_peered":"2024-03-29T04:10:05.556955+0000","last_clean":"2024-03-29T04:10:05.556955+0000","last_became_active":"2024-03-29T04:10:00.544481+0000","last_became_peered":"2024-03-29T04:10:00.544481+0000","last_unstale":"2024-03-29T04:10:05.556955+0000","last_undegraded":"2024-03-29T04:10:05.556955+0000","last_fullsized":"2024-03-29T04:10:05.556955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:00:51.320824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911910+0000","last_change":"2024-03-29T04:09:59.541406+0000","last_active":"2024-03-29T04:10:05.911910+0000","last_peered":"2024-03-29T04:10:05.911910+0000","last_clean":"2024-03-29T04:10:05.911910+0000","last_became_active":"2024-03-29T04:09:59.541099+0000","last_became_peered":"2024-03-29T04:09:59.541099+0000","last_unstale":"2024-03-29T04:10:05.911910+0000","last_undegraded":"2024-03-29T04:10:05.911910+0000","last_fullsized":"2024-03-29T04:10:05.911910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:25:06.123311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911325+0000","last_change":"2024-03-29T04:10:04.579423+0000","last_active":"2024-03-29T04:10:05.911325+0000","last_peered":"2024-03-29T04:10:05.911325+0000","last_clean":"2024-03-29T04:10:05.911325+0000","last_became_active":"2024-03-29T04:10:04.579062+0000","last_became_peered":"2024-03-29T04:10:04.579062+0000","last_unstale":"2024-03-29T04:10:05.911325+0000","last_undegraded":"2024-03-29T04:10:05.911325+0000","last_fullsized":"2024-03-29T04:10:05.911325+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:06.998903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911863+0000","last_change":"2024-03-29T04:10:00.547282+0000","last_active":"2024-03-29T04:10:05.911863+0000","last_peered":"2024-03-29T04:10:05.911863+0000","last_clean":"2024-03-29T04:10:05.911863+0000","last_became_active":"2024-03-29T04:10:00.547026+0000","last_became_peered":"2024-03-29T04:10:00.547026+0000","last_unstale":"2024-03-29T04:10:05.911863+0000","last_undegraded":"2024-03-29T04:10:05.911863+0000","last_fullsized":"2024-03-29T04:10:05.911863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:34:06.367730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.560063+0000","last_change":"2024-03-29T04:09:59.540515+0000","last_active":"2024-03-29T04:10:05.560063+0000","last_peered":"2024-03-29T04:10:05.560063+0000","last_clean":"2024-03-29T04:10:05.560063+0000","last_became_active":"2024-03-29T04:09:59.540198+0000","last_became_peered":"2024-03-29T04:09:59.540198+0000","last_unstale":"2024-03-29T04:10:05.560063+0000","last_undegraded":"2024-03-29T04:10:05.560063+0000","last_fullsized":"2024-03-29T04:10:05.560063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:01.925223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556468+0000","last_change":"2024-03-29T04:10:04.644258+0000","last_active":"2024-03-29T04:10:05.556468+0000","last_peered":"2024-03-29T04:10:05.556468+0000","last_clean":"2024-03-29T04:10:05.556468+0000","last_became_active":"2024-03-29T04:10:04.643890+0000","last_became_peered":"2024-03-29T04:10:04.643890+0000","last_unstale":"2024-03-29T04:10:05.556468+0000","last_undegraded":"2024-03-29T04:10:05.556468+0000","last_fullsized":"2024-03-29T04:10:05.556468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:46:14.769141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911624+0000","last_change":"2024-03-29T04:10:00.549814+0000","last_active":"2024-03-29T04:10:05.911624+0000","last_peered":"2024-03-29T04:10:05.911624+0000","last_clean":"2024-03-29T04:10:05.911624+0000","last_became_active":"2024-03-29T04:10:00.549571+0000","last_became_peered":"2024-03-29T04:10:00.549571+0000","last_unstale":"2024-03-29T04:10:05.911624+0000","last_undegraded":"2024-03-29T04:10:05.911624+0000","last_fullsized":"2024-03-29T04:10:05.911624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:00:51.320824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557978+0000","last_change":"2024-03-29T04:09:59.538979+0000","last_active":"2024-03-29T04:10:05.557978+0000","last_peered":"2024-03-29T04:10:05.557978+0000","last_clean":"2024-03-29T04:10:05.557978+0000","last_became_active":"2024-03-29T04:09:59.538707+0000","last_became_peered":"2024-03-29T04:09:59.538707+0000","last_unstale":"2024-03-29T04:10:05.557978+0000","last_undegraded":"2024-03-29T04:10:05.557978+0000","last_fullsized":"2024-03-29T04:10:05.557978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:07:35.127071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556613+0000","last_change":"2024-03-29T04:10:04.574569+0000","last_active":"2024-03-29T04:10:05.556613+0000","last_peered":"2024-03-29T04:10:05.556613+0000","last_clean":"2024-03-29T04:10:05.556613+0000","last_became_active":"2024-03-29T04:10:04.574048+0000","last_became_peered":"2024-03-29T04:10:04.574048+0000","last_unstale":"2024-03-29T04:10:05.556613+0000","last_undegraded":"2024-03-29T04:10:05.556613+0000","last_fullsized":"2024-03-29T04:10:05.556613+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:33.266010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910658+0000","last_change":"2024-03-29T04:10:00.549205+0000","last_active":"2024-03-29T04:10:05.910658+0000","last_peered":"2024-03-29T04:10:05.910658+0000","last_clean":"2024-03-29T04:10:05.910658+0000","last_became_active":"2024-03-29T04:10:00.548863+0000","last_became_peered":"2024-03-29T04:10:00.548863+0000","last_unstale":"2024-03-29T04:10:05.910658+0000","last_undegraded":"2024-03-29T04:10:05.910658+0000","last_fullsized":"2024-03-29T04:10:05.910658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:30.096622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.559751+0000","last_change":"2024-03-29T04:09:59.538016+0000","last_active":"2024-03-29T04:10:05.559751+0000","last_peered":"2024-03-29T04:10:05.559751+0000","last_clean":"2024-03-29T04:10:05.559751+0000","last_became_active":"2024-03-29T04:09:59.537671+0000","last_became_peered":"2024-03-29T04:09:59.537671+0000","last_unstale":"2024-03-29T04:10:05.559751+0000","last_undegraded":"2024-03-29T04:10:05.559751+0000","last_fullsized":"2024-03-29T04:10:05.559751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:25:00.672418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912466+0000","last_change":"2024-03-29T04:10:04.574244+0000","last_active":"2024-03-29T04:10:05.912466+0000","last_peered":"2024-03-29T04:10:05.912466+0000","last_clean":"2024-03-29T04:10:05.912466+0000","last_became_active":"2024-03-29T04:10:04.573865+0000","last_became_peered":"2024-03-29T04:10:04.573865+0000","last_unstale":"2024-03-29T04:10:05.912466+0000","last_undegraded":"2024-03-29T04:10:05.912466+0000","last_fullsized":"2024-03-29T04:10:05.912466+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:23:51.029457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910228+0000","last_change":"2024-03-29T04:10:00.547804+0000","last_active":"2024-03-29T04:10:05.910228+0000","last_peered":"2024-03-29T04:10:05.910228+0000","last_clean":"2024-03-29T04:10:05.910228+0000","last_became_active":"2024-03-29T04:10:00.547500+0000","last_became_peered":"2024-03-29T04:10:00.547500+0000","last_unstale":"2024-03-29T04:10:05.910228+0000","last_undegraded":"2024-03-29T04:10:05.910228+0000","last_fullsized":"2024-03-29T04:10:05.910228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:52:12.957170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910669+0000","last_change":"2024-03-29T04:09:59.538286+0000","last_active":"2024-03-29T04:10:05.910669+0000","last_peered":"2024-03-29T04:10:05.910669+0000","last_clean":"2024-03-29T04:10:05.910669+0000","last_became_active":"2024-03-29T04:09:59.537970+0000","last_became_peered":"2024-03-29T04:09:59.537970+0000","last_unstale":"2024-03-29T04:10:05.910669+0000","last_undegraded":"2024-03-29T04:10:05.910669+0000","last_fullsized":"2024-03-29T04:10:05.910669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:05.307537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911473+0000","last_change":"2024-03-29T04:10:04.647055+0000","last_active":"2024-03-29T04:10:05.911473+0000","last_peered":"2024-03-29T04:10:05.911473+0000","last_clean":"2024-03-29T04:10:05.911473+0000","last_became_active":"2024-03-29T04:10:04.646604+0000","last_became_peered":"2024-03-29T04:10:04.646604+0000","last_unstale":"2024-03-29T04:10:05.911473+0000","last_undegraded":"2024-03-29T04:10:05.911473+0000","last_fullsized":"2024-03-29T04:10:05.911473+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:32:59.580503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911224+0000","last_change":"2024-03-29T04:10:00.548354+0000","last_active":"2024-03-29T04:10:05.911224+0000","last_peered":"2024-03-29T04:10:05.911224+0000","last_clean":"2024-03-29T04:10:05.911224+0000","last_became_active":"2024-03-29T04:10:00.548084+0000","last_became_peered":"2024-03-29T04:10:00.548084+0000","last_unstale":"2024-03-29T04:10:05.911224+0000","last_undegraded":"2024-03-29T04:10:05.911224+0000","last_fullsized":"2024-03-29T04:10:05.911224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:02.966428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557470+0000","last_change":"2024-03-29T04:09:59.538823+0000","last_active":"2024-03-29T04:10:05.557470+0000","last_peered":"2024-03-29T04:10:05.557470+0000","last_clean":"2024-03-29T04:10:05.557470+0000","last_became_active":"2024-03-29T04:09:59.538570+0000","last_became_peered":"2024-03-29T04:09:59.538570+0000","last_unstale":"2024-03-29T04:10:05.557470+0000","last_undegraded":"2024-03-29T04:10:05.557470+0000","last_fullsized":"2024-03-29T04:10:05.557470+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:02.879344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911403+0000","last_change":"2024-03-29T04:10:04.644206+0000","last_active":"2024-03-29T04:10:05.911403+0000","last_peered":"2024-03-29T04:10:05.911403+0000","last_clean":"2024-03-29T04:10:05.911403+0000","last_became_active":"2024-03-29T04:10:04.643690+0000","last_became_peered":"2024-03-29T04:10:04.643690+0000","last_unstale":"2024-03-29T04:10:05.911403+0000","last_undegraded":"2024-03-29T04:10:05.911403+0000","last_fullsized":"2024-03-29T04:10:05.911403+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:10:07.163684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910609+0000","last_change":"2024-03-29T04:10:00.541166+0000","last_active":"2024-03-29T04:10:05.910609+0000","last_peered":"2024-03-29T04:10:05.910609+0000","last_clean":"2024-03-29T04:10:05.910609+0000","last_became_active":"2024-03-29T04:10:00.540895+0000","last_became_peered":"2024-03-29T04:10:00.540895+0000","last_unstale":"2024-03-29T04:10:05.910609+0000","last_undegraded":"2024-03-29T04:10:05.910609+0000","last_fullsized":"2024-03-29T04:10:05.910609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:21:11.329145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911540+0000","last_change":"2024-03-29T04:09:59.538433+0000","last_active":"2024-03-29T04:10:05.911540+0000","last_peered":"2024-03-29T04:10:05.911540+0000","last_clean":"2024-03-29T04:10:05.911540+0000","last_became_active":"2024-03-29T04:09:59.538039+0000","last_became_peered":"2024-03-29T04:09:59.538039+0000","last_unstale":"2024-03-29T04:10:05.911540+0000","last_undegraded":"2024-03-29T04:10:05.911540+0000","last_fullsized":"2024-03-29T04:10:05.911540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:45:51.875624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557426+0000","last_change":"2024-03-29T04:10:04.647019+0000","last_active":"2024-03-29T04:10:05.557426+0000","last_peered":"2024-03-29T04:10:05.557426+0000","last_clean":"2024-03-29T04:10:05.557426+0000","last_became_active":"2024-03-29T04:10:04.646642+0000","last_became_peered":"2024-03-29T04:10:04.646642+0000","last_unstale":"2024-03-29T04:10:05.557426+0000","last_undegraded":"2024-03-29T04:10:05.557426+0000","last_fullsized":"2024-03-29T04:10:05.557426+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:07:56.606322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911280+0000","last_change":"2024-03-29T04:10:00.544134+0000","last_active":"2024-03-29T04:10:05.911280+0000","last_peered":"2024-03-29T04:10:05.911280+0000","last_clean":"2024-03-29T04:10:05.911280+0000","last_became_active":"2024-03-29T04:10:00.543534+0000","last_became_peered":"2024-03-29T04:10:00.543534+0000","last_unstale":"2024-03-29T04:10:05.911280+0000","last_undegraded":"2024-03-29T04:10:05.911280+0000","last_fullsized":"2024-03-29T04:10:05.911280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:26:04.629712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:10.817289+0000","last_change":"2024-03-29T04:09:59.544458+0000","last_active":"2024-03-29T04:10:10.817289+0000","last_peered":"2024-03-29T04:10:10.817289+0000","last_clean":"2024-03-29T04:10:10.817289+0000","last_became_active":"2024-03-29T04:09:59.544137+0000","last_became_peered":"2024-03-29T04:09:59.544137+0000","last_unstale":"2024-03-29T04:10:10.817289+0000","last_undegraded":"2024-03-29T04:10:10.817289+0000","last_fullsized":"2024-03-29T04:10:10.817289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:02.879344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912395+0000","last_change":"2024-03-29T04:10:04.576867+0000","last_active":"2024-03-29T04:10:05.912395+0000","last_peered":"2024-03-29T04:10:05.912395+0000","last_clean":"2024-03-29T04:10:05.912395+0000","last_became_active":"2024-03-29T04:10:04.576416+0000","last_became_peered":"2024-03-29T04:10:04.576416+0000","last_unstale":"2024-03-29T04:10:05.912395+0000","last_undegraded":"2024-03-29T04:10:05.912395+0000","last_fullsized":"2024-03-29T04:10:05.912395+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:34:10.400206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.558230+0000","last_change":"2024-03-29T04:10:00.547277+0000","last_active":"2024-03-29T04:10:05.558230+0000","last_peered":"2024-03-29T04:10:05.558230+0000","last_clean":"2024-03-29T04:10:05.558230+0000","last_became_active":"2024-03-29T04:10:00.547024+0000","last_became_peered":"2024-03-29T04:10:00.547024+0000","last_unstale":"2024-03-29T04:10:05.558230+0000","last_undegraded":"2024-03-29T04:10:05.558230+0000","last_fullsized":"2024-03-29T04:10:05.558230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:58:11.125964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911813+0000","last_change":"2024-03-29T04:09:59.541483+0000","last_active":"2024-03-29T04:10:05.911813+0000","last_peered":"2024-03-29T04:10:05.911813+0000","last_clean":"2024-03-29T04:10:05.911813+0000","last_became_active":"2024-03-29T04:09:59.541196+0000","last_became_peered":"2024-03-29T04:09:59.541196+0000","last_unstale":"2024-03-29T04:10:05.911813+0000","last_undegraded":"2024-03-29T04:10:05.911813+0000","last_fullsized":"2024-03-29T04:10:05.911813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:35:01.484589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557918+0000","last_change":"2024-03-29T04:10:04.574437+0000","last_active":"2024-03-29T04:10:05.557918+0000","last_peered":"2024-03-29T04:10:05.557918+0000","last_clean":"2024-03-29T04:10:05.557918+0000","last_became_active":"2024-03-29T04:10:04.573898+0000","last_became_peered":"2024-03-29T04:10:04.573898+0000","last_unstale":"2024-03-29T04:10:05.557918+0000","last_undegraded":"2024-03-29T04:10:05.557918+0000","last_fullsized":"2024-03-29T04:10:05.557918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:50.836630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911733+0000","last_change":"2024-03-29T04:10:00.536536+0000","last_active":"2024-03-29T04:10:05.911733+0000","last_peered":"2024-03-29T04:10:05.911733+0000","last_clean":"2024-03-29T04:10:05.911733+0000","last_became_active":"2024-03-29T04:10:00.536254+0000","last_became_peered":"2024-03-29T04:10:00.536254+0000","last_unstale":"2024-03-29T04:10:05.911733+0000","last_undegraded":"2024-03-29T04:10:05.911733+0000","last_fullsized":"2024-03-29T04:10:05.911733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:48.213845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557997+0000","last_change":"2024-03-29T04:09:59.541406+0000","last_active":"2024-03-29T04:10:05.557997+0000","last_peered":"2024-03-29T04:10:05.557997+0000","last_clean":"2024-03-29T04:10:05.557997+0000","last_became_active":"2024-03-29T04:09:59.541101+0000","last_became_peered":"2024-03-29T04:09:59.541101+0000","last_unstale":"2024-03-29T04:10:05.557997+0000","last_undegraded":"2024-03-29T04:10:05.557997+0000","last_fullsized":"2024-03-29T04:10:05.557997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:25:06.123311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910592+0000","last_change":"2024-03-29T04:10:04.577680+0000","last_active":"2024-03-29T04:10:05.910592+0000","last_peered":"2024-03-29T04:10:05.910592+0000","last_clean":"2024-03-29T04:10:05.910592+0000","last_became_active":"2024-03-29T04:10:04.577291+0000","last_became_peered":"2024-03-29T04:10:04.577291+0000","last_unstale":"2024-03-29T04:10:05.910592+0000","last_undegraded":"2024-03-29T04:10:05.910592+0000","last_fullsized":"2024-03-29T04:10:05.910592+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:46:14.769141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557898+0000","last_change":"2024-03-29T04:10:00.542773+0000","last_active":"2024-03-29T04:10:05.557898+0000","last_peered":"2024-03-29T04:10:05.557898+0000","last_clean":"2024-03-29T04:10:05.557898+0000","last_became_active":"2024-03-29T04:10:00.542081+0000","last_became_peered":"2024-03-29T04:10:00.542081+0000","last_unstale":"2024-03-29T04:10:05.557898+0000","last_undegraded":"2024-03-29T04:10:05.557898+0000","last_fullsized":"2024-03-29T04:10:05.557898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:26:04.629712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910800+0000","last_change":"2024-03-29T04:09:59.543623+0000","last_active":"2024-03-29T04:10:05.910800+0000","last_peered":"2024-03-29T04:10:05.910800+0000","last_clean":"2024-03-29T04:10:05.910800+0000","last_became_active":"2024-03-29T04:09:59.542122+0000","last_became_peered":"2024-03-29T04:09:59.542122+0000","last_unstale":"2024-03-29T04:10:05.910800+0000","last_undegraded":"2024-03-29T04:10:05.910800+0000","last_fullsized":"2024-03-29T04:10:05.910800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:01.925223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911195+0000","last_change":"2024-03-29T04:10:04.648725+0000","last_active":"2024-03-29T04:10:05.911195+0000","last_peered":"2024-03-29T04:10:05.911195+0000","last_clean":"2024-03-29T04:10:05.911195+0000","last_became_active":"2024-03-29T04:10:04.648125+0000","last_became_peered":"2024-03-29T04:10:04.648125+0000","last_unstale":"2024-03-29T04:10:05.911195+0000","last_undegraded":"2024-03-29T04:10:05.911195+0000","last_fullsized":"2024-03-29T04:10:05.911195+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:16:31.571145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912466+0000","last_change":"2024-03-29T04:10:04.646516+0000","last_active":"2024-03-29T04:10:05.912466+0000","last_peered":"2024-03-29T04:10:05.912466+0000","last_clean":"2024-03-29T04:10:05.912466+0000","last_became_active":"2024-03-29T04:10:04.646118+0000","last_became_peered":"2024-03-29T04:10:04.646118+0000","last_unstale":"2024-03-29T04:10:05.912466+0000","last_undegraded":"2024-03-29T04:10:05.912466+0000","last_fullsized":"2024-03-29T04:10:05.912466+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:17.034970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.909860+0000","last_change":"2024-03-29T04:10:00.542208+0000","last_active":"2024-03-29T04:10:05.909860+0000","last_peered":"2024-03-29T04:10:05.909860+0000","last_clean":"2024-03-29T04:10:05.909860+0000","last_became_active":"2024-03-29T04:10:00.541927+0000","last_became_peered":"2024-03-29T04:10:00.541927+0000","last_unstale":"2024-03-29T04:10:05.909860+0000","last_undegraded":"2024-03-29T04:10:05.909860+0000","last_fullsized":"2024-03-29T04:10:05.909860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:16:02.331457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556546+0000","last_change":"2024-03-29T04:09:59.539257+0000","last_active":"2024-03-29T04:10:05.556546+0000","last_peered":"2024-03-29T04:10:05.556546+0000","last_clean":"2024-03-29T04:10:05.556546+0000","last_became_active":"2024-03-29T04:09:59.538298+0000","last_became_peered":"2024-03-29T04:09:59.538298+0000","last_unstale":"2024-03-29T04:10:05.556546+0000","last_undegraded":"2024-03-29T04:10:05.556546+0000","last_fullsized":"2024-03-29T04:10:05.556546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:33:50.904931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911024+0000","last_change":"2024-03-29T04:10:04.575619+0000","last_active":"2024-03-29T04:10:05.911024+0000","last_peered":"2024-03-29T04:10:05.911024+0000","last_clean":"2024-03-29T04:10:05.911024+0000","last_became_active":"2024-03-29T04:10:04.575138+0000","last_became_peered":"2024-03-29T04:10:04.575138+0000","last_unstale":"2024-03-29T04:10:05.911024+0000","last_undegraded":"2024-03-29T04:10:05.911024+0000","last_fullsized":"2024-03-29T04:10:05.911024+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:06.998903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910644+0000","last_change":"2024-03-29T04:10:00.543853+0000","last_active":"2024-03-29T04:10:05.910644+0000","last_peered":"2024-03-29T04:10:05.910644+0000","last_clean":"2024-03-29T04:10:05.910644+0000","last_became_active":"2024-03-29T04:10:00.543030+0000","last_became_peered":"2024-03-29T04:10:00.543030+0000","last_unstale":"2024-03-29T04:10:05.910644+0000","last_undegraded":"2024-03-29T04:10:05.910644+0000","last_fullsized":"2024-03-29T04:10:05.910644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:29.233534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911422+0000","last_change":"2024-03-29T04:09:59.537737+0000","last_active":"2024-03-29T04:10:05.911422+0000","last_peered":"2024-03-29T04:10:05.911422+0000","last_clean":"2024-03-29T04:10:05.911422+0000","last_became_active":"2024-03-29T04:09:59.537453+0000","last_became_peered":"2024-03-29T04:09:59.537453+0000","last_unstale":"2024-03-29T04:10:05.911422+0000","last_undegraded":"2024-03-29T04:10:05.911422+0000","last_fullsized":"2024-03-29T04:10:05.911422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:05.307537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910688+0000","last_change":"2024-03-29T04:09:59.538269+0000","last_active":"2024-03-29T04:10:05.910688+0000","last_peered":"2024-03-29T04:10:05.910688+0000","last_clean":"2024-03-29T04:10:05.910688+0000","last_became_active":"2024-03-29T04:09:59.537365+0000","last_became_peered":"2024-03-29T04:09:59.537365+0000","last_unstale":"2024-03-29T04:10:05.910688+0000","last_undegraded":"2024-03-29T04:10:05.910688+0000","last_fullsized":"2024-03-29T04:10:05.910688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:33:50.904931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911456+0000","last_change":"2024-03-29T04:10:00.532121+0000","last_active":"2024-03-29T04:10:05.911456+0000","last_peered":"2024-03-29T04:10:05.911456+0000","last_clean":"2024-03-29T04:10:05.911456+0000","last_became_active":"2024-03-29T04:10:00.531664+0000","last_became_peered":"2024-03-29T04:10:00.531664+0000","last_unstale":"2024-03-29T04:10:05.911456+0000","last_undegraded":"2024-03-29T04:10:05.911456+0000","last_fullsized":"2024-03-29T04:10:05.911456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:13:32.378332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.560130+0000","last_change":"2024-03-29T04:10:04.574558+0000","last_active":"2024-03-29T04:10:05.560130+0000","last_peered":"2024-03-29T04:10:05.560130+0000","last_clean":"2024-03-29T04:10:05.560130+0000","last_became_active":"2024-03-29T04:10:04.574156+0000","last_became_peered":"2024-03-29T04:10:04.574156+0000","last_unstale":"2024-03-29T04:10:05.560130+0000","last_undegraded":"2024-03-29T04:10:05.560130+0000","last_fullsized":"2024-03-29T04:10:05.560130+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:02:36.577244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910098+0000","last_change":"2024-03-29T04:09:59.536444+0000","last_active":"2024-03-29T04:10:05.910098+0000","last_peered":"2024-03-29T04:10:05.910098+0000","last_clean":"2024-03-29T04:10:05.910098+0000","last_became_active":"2024-03-29T04:09:59.536165+0000","last_became_peered":"2024-03-29T04:09:59.536165+0000","last_unstale":"2024-03-29T04:10:05.910098+0000","last_undegraded":"2024-03-29T04:10:05.910098+0000","last_fullsized":"2024-03-29T04:10:05.910098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:37:46.716233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.558124+0000","last_change":"2024-03-29T04:10:00.544569+0000","last_active":"2024-03-29T04:10:05.558124+0000","last_peered":"2024-03-29T04:10:05.558124+0000","last_clean":"2024-03-29T04:10:05.558124+0000","last_became_active":"2024-03-29T04:10:00.544290+0000","last_became_peered":"2024-03-29T04:10:00.544290+0000","last_unstale":"2024-03-29T04:10:05.558124+0000","last_undegraded":"2024-03-29T04:10:05.558124+0000","last_fullsized":"2024-03-29T04:10:05.558124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:02.929633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910585+0000","last_change":"2024-03-29T04:10:04.580538+0000","last_active":"2024-03-29T04:10:05.910585+0000","last_peered":"2024-03-29T04:10:05.910585+0000","last_clean":"2024-03-29T04:10:05.910585+0000","last_became_active":"2024-03-29T04:10:04.580221+0000","last_became_peered":"2024-03-29T04:10:04.580221+0000","last_unstale":"2024-03-29T04:10:05.910585+0000","last_undegraded":"2024-03-29T04:10:05.910585+0000","last_fullsized":"2024-03-29T04:10:05.910585+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:01:23.958242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910087+0000","last_change":"2024-03-29T04:10:00.546325+0000","last_active":"2024-03-29T04:10:05.910087+0000","last_peered":"2024-03-29T04:10:05.910087+0000","last_clean":"2024-03-29T04:10:05.910087+0000","last_became_active":"2024-03-29T04:10:00.545988+0000","last_became_peered":"2024-03-29T04:10:00.545988+0000","last_unstale":"2024-03-29T04:10:05.910087+0000","last_undegraded":"2024-03-29T04:10:05.910087+0000","last_fullsized":"2024-03-29T04:10:05.910087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:16:02.331457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.559836+0000","last_change":"2024-03-29T04:10:00.541221+0000","last_active":"2024-03-29T04:10:05.559836+0000","last_peered":"2024-03-29T04:10:05.559836+0000","last_clean":"2024-03-29T04:10:05.559836+0000","last_became_active":"2024-03-29T04:10:00.540946+0000","last_became_peered":"2024-03-29T04:10:00.540946+0000","last_unstale":"2024-03-29T04:10:05.559836+0000","last_undegraded":"2024-03-29T04:10:05.559836+0000","last_fullsized":"2024-03-29T04:10:05.559836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:35:02.488999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557358+0000","last_change":"2024-03-29T04:10:00.546304+0000","last_active":"2024-03-29T04:10:05.557358+0000","last_peered":"2024-03-29T04:10:05.557358+0000","last_clean":"2024-03-29T04:10:05.557358+0000","last_became_active":"2024-03-29T04:10:00.545529+0000","last_became_peered":"2024-03-29T04:10:00.545529+0000","last_unstale":"2024-03-29T04:10:05.557358+0000","last_undegraded":"2024-03-29T04:10:05.557358+0000","last_fullsized":"2024-03-29T04:10:05.557358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:44:13.128618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911371+0000","last_change":"2024-03-29T04:10:00.542033+0000","last_active":"2024-03-29T04:10:05.911371+0000","last_peered":"2024-03-29T04:10:05.911371+0000","last_clean":"2024-03-29T04:10:05.911371+0000","last_became_active":"2024-03-29T04:10:00.541764+0000","last_became_peered":"2024-03-29T04:10:00.541764+0000","last_unstale":"2024-03-29T04:10:05.911371+0000","last_undegraded":"2024-03-29T04:10:05.911371+0000","last_fullsized":"2024-03-29T04:10:05.911371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:46.707085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911616+0000","last_change":"2024-03-29T04:10:00.543634+0000","last_active":"2024-03-29T04:10:05.911616+0000","last_peered":"2024-03-29T04:10:05.911616+0000","last_clean":"2024-03-29T04:10:05.911616+0000","last_became_active":"2024-03-29T04:10:00.543343+0000","last_became_peered":"2024-03-29T04:10:00.543343+0000","last_unstale":"2024-03-29T04:10:05.911616+0000","last_undegraded":"2024-03-29T04:10:05.911616+0000","last_fullsized":"2024-03-29T04:10:05.911616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:35:00.452782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911632+0000","last_change":"2024-03-29T04:10:00.554863+0000","last_active":"2024-03-29T04:10:05.911632+0000","last_peered":"2024-03-29T04:10:05.911632+0000","last_clean":"2024-03-29T04:10:05.911632+0000","last_became_active":"2024-03-29T04:10:00.554560+0000","last_became_peered":"2024-03-29T04:10:00.554560+0000","last_unstale":"2024-03-29T04:10:05.911632+0000","last_undegraded":"2024-03-29T04:10:05.911632+0000","last_fullsized":"2024-03-29T04:10:05.911632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:29.233534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910973+0000","last_change":"2024-03-29T04:10:00.541999+0000","last_active":"2024-03-29T04:10:05.910973+0000","last_peered":"2024-03-29T04:10:05.910973+0000","last_clean":"2024-03-29T04:10:05.910973+0000","last_became_active":"2024-03-29T04:10:00.541699+0000","last_became_peered":"2024-03-29T04:10:00.541699+0000","last_unstale":"2024-03-29T04:10:05.910973+0000","last_undegraded":"2024-03-29T04:10:05.910973+0000","last_fullsized":"2024-03-29T04:10:05.910973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:48.213845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911525+0000","last_change":"2024-03-29T04:10:00.537506+0000","last_active":"2024-03-29T04:10:05.911525+0000","last_peered":"2024-03-29T04:10:05.911525+0000","last_clean":"2024-03-29T04:10:05.911525+0000","last_became_active":"2024-03-29T04:10:00.537239+0000","last_became_peered":"2024-03-29T04:10:00.537239+0000","last_unstale":"2024-03-29T04:10:05.911525+0000","last_undegraded":"2024-03-29T04:10:05.911525+0000","last_fullsized":"2024-03-29T04:10:05.911525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:30.096622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556535+0000","last_change":"2024-03-29T04:10:00.544443+0000","last_active":"2024-03-29T04:10:05.556535+0000","last_peered":"2024-03-29T04:10:05.556535+0000","last_clean":"2024-03-29T04:10:05.556535+0000","last_became_active":"2024-03-29T04:10:00.543427+0000","last_became_peered":"2024-03-29T04:10:00.543427+0000","last_unstale":"2024-03-29T04:10:05.556535+0000","last_undegraded":"2024-03-29T04:10:05.556535+0000","last_fullsized":"2024-03-29T04:10:05.556535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:48.213845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911292+0000","last_change":"2024-03-29T04:10:00.538050+0000","last_active":"2024-03-29T04:10:05.911292+0000","last_peered":"2024-03-29T04:10:05.911292+0000","last_clean":"2024-03-29T04:10:05.911292+0000","last_became_active":"2024-03-29T04:10:00.537771+0000","last_became_peered":"2024-03-29T04:10:00.537771+0000","last_unstale":"2024-03-29T04:10:05.911292+0000","last_undegraded":"2024-03-29T04:10:05.911292+0000","last_fullsized":"2024-03-29T04:10:05.911292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:21:11.329145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557260+0000","last_change":"2024-03-29T04:10:00.546534+0000","last_active":"2024-03-29T04:10:05.557260+0000","last_peered":"2024-03-29T04:10:05.557260+0000","last_clean":"2024-03-29T04:10:05.557260+0000","last_became_active":"2024-03-29T04:10:00.546028+0000","last_became_peered":"2024-03-29T04:10:00.546028+0000","last_unstale":"2024-03-29T04:10:05.557260+0000","last_undegraded":"2024-03-29T04:10:05.557260+0000","last_fullsized":"2024-03-29T04:10:05.557260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:23:46.996981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911142+0000","last_change":"2024-03-29T04:10:00.539866+0000","last_active":"2024-03-29T04:10:05.911142+0000","last_peered":"2024-03-29T04:10:05.911142+0000","last_clean":"2024-03-29T04:10:05.911142+0000","last_became_active":"2024-03-29T04:10:00.539562+0000","last_became_peered":"2024-03-29T04:10:00.539562+0000","last_unstale":"2024-03-29T04:10:05.911142+0000","last_undegraded":"2024-03-29T04:10:05.911142+0000","last_fullsized":"2024-03-29T04:10:05.911142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:52:12.957170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910850+0000","last_change":"2024-03-29T04:10:00.542361+0000","last_active":"2024-03-29T04:10:05.910850+0000","last_peered":"2024-03-29T04:10:05.910850+0000","last_clean":"2024-03-29T04:10:05.910850+0000","last_became_active":"2024-03-29T04:10:00.541625+0000","last_became_peered":"2024-03-29T04:10:00.541625+0000","last_unstale":"2024-03-29T04:10:05.910850+0000","last_undegraded":"2024-03-29T04:10:05.910850+0000","last_fullsized":"2024-03-29T04:10:05.910850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:46.707085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911557+0000","last_change":"2024-03-29T04:10:00.539117+0000","last_active":"2024-03-29T04:10:05.911557+0000","last_peered":"2024-03-29T04:10:05.911557+0000","last_clean":"2024-03-29T04:10:05.911557+0000","last_became_active":"2024-03-29T04:10:00.538833+0000","last_became_peered":"2024-03-29T04:10:00.538833+0000","last_unstale":"2024-03-29T04:10:05.911557+0000","last_undegraded":"2024-03-29T04:10:05.911557+0000","last_fullsized":"2024-03-29T04:10:05.911557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:13.002495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.560038+0000","last_change":"2024-03-29T04:10:00.551402+0000","last_active":"2024-03-29T04:10:05.560038+0000","last_peered":"2024-03-29T04:10:05.560038+0000","last_clean":"2024-03-29T04:10:05.560038+0000","last_became_active":"2024-03-29T04:10:00.551092+0000","last_became_peered":"2024-03-29T04:10:00.551092+0000","last_unstale":"2024-03-29T04:10:05.560038+0000","last_undegraded":"2024-03-29T04:10:05.560038+0000","last_fullsized":"2024-03-29T04:10:05.560038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:58:11.125964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.558449+0000","last_change":"2024-03-29T04:10:00.542777+0000","last_active":"2024-03-29T04:10:05.558449+0000","last_peered":"2024-03-29T04:10:05.558449+0000","last_clean":"2024-03-29T04:10:05.558449+0000","last_became_active":"2024-03-29T04:10:00.542241+0000","last_became_peered":"2024-03-29T04:10:00.542241+0000","last_unstale":"2024-03-29T04:10:05.558449+0000","last_undegraded":"2024-03-29T04:10:05.558449+0000","last_fullsized":"2024-03-29T04:10:05.558449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:46.804154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556402+0000","last_change":"2024-03-29T04:10:00.545811+0000","last_active":"2024-03-29T04:10:05.556402+0000","last_peered":"2024-03-29T04:10:05.556402+0000","last_clean":"2024-03-29T04:10:05.556402+0000","last_became_active":"2024-03-29T04:10:00.545501+0000","last_became_peered":"2024-03-29T04:10:00.545501+0000","last_unstale":"2024-03-29T04:10:05.556402+0000","last_undegraded":"2024-03-29T04:10:05.556402+0000","last_fullsized":"2024-03-29T04:10:05.556402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:29.233534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.909960+0000","last_change":"2024-03-29T04:10:00.545471+0000","last_active":"2024-03-29T04:10:05.909960+0000","last_peered":"2024-03-29T04:10:05.909960+0000","last_clean":"2024-03-29T04:10:05.909960+0000","last_became_active":"2024-03-29T04:10:00.545173+0000","last_became_peered":"2024-03-29T04:10:00.545173+0000","last_unstale":"2024-03-29T04:10:05.909960+0000","last_undegraded":"2024-03-29T04:10:05.909960+0000","last_fullsized":"2024-03-29T04:10:05.909960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:13:32.378332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910520+0000","last_change":"2024-03-29T04:10:00.545485+0000","last_active":"2024-03-29T04:10:05.910520+0000","last_peered":"2024-03-29T04:10:05.910520+0000","last_clean":"2024-03-29T04:10:05.910520+0000","last_became_active":"2024-03-29T04:10:00.545139+0000","last_became_peered":"2024-03-29T04:10:00.545139+0000","last_unstale":"2024-03-29T04:10:05.910520+0000","last_undegraded":"2024-03-29T04:10:05.910520+0000","last_fullsized":"2024-03-29T04:10:05.910520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:14:59.937144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.909924+0000","last_change":"2024-03-29T04:10:00.550441+0000","last_active":"2024-03-29T04:10:05.909924+0000","last_peered":"2024-03-29T04:10:05.909924+0000","last_clean":"2024-03-29T04:10:05.909924+0000","last_became_active":"2024-03-29T04:10:00.550150+0000","last_became_peered":"2024-03-29T04:10:00.550150+0000","last_unstale":"2024-03-29T04:10:05.909924+0000","last_undegraded":"2024-03-29T04:10:05.909924+0000","last_fullsized":"2024-03-29T04:10:05.909924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:52:12.957170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.558164+0000","last_change":"2024-03-29T04:10:00.540368+0000","last_active":"2024-03-29T04:10:05.558164+0000","last_peered":"2024-03-29T04:10:05.558164+0000","last_clean":"2024-03-29T04:10:05.558164+0000","last_became_active":"2024-03-29T04:10:00.538360+0000","last_became_peered":"2024-03-29T04:10:00.538360+0000","last_unstale":"2024-03-29T04:10:05.558164+0000","last_undegraded":"2024-03-29T04:10:05.558164+0000","last_fullsized":"2024-03-29T04:10:05.558164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:35:02.488999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911011+0000","last_change":"2024-03-29T04:10:00.550531+0000","last_active":"2024-03-29T04:10:05.911011+0000","last_peered":"2024-03-29T04:10:05.911011+0000","last_clean":"2024-03-29T04:10:05.911011+0000","last_became_active":"2024-03-29T04:10:00.550079+0000","last_became_peered":"2024-03-29T04:10:00.550079+0000","last_unstale":"2024-03-29T04:10:05.911011+0000","last_undegraded":"2024-03-29T04:10:05.911011+0000","last_fullsized":"2024-03-29T04:10:05.911011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:13:32.378332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910949+0000","last_change":"2024-03-29T04:10:00.550387+0000","last_active":"2024-03-29T04:10:05.910949+0000","last_peered":"2024-03-29T04:10:05.910949+0000","last_clean":"2024-03-29T04:10:05.910949+0000","last_became_active":"2024-03-29T04:10:00.550063+0000","last_became_peered":"2024-03-29T04:10:00.550063+0000","last_unstale":"2024-03-29T04:10:05.910949+0000","last_undegraded":"2024-03-29T04:10:05.910949+0000","last_fullsized":"2024-03-29T04:10:05.910949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:29.233534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911804+0000","last_change":"2024-03-29T04:10:00.541680+0000","last_active":"2024-03-29T04:10:05.911804+0000","last_peered":"2024-03-29T04:10:05.911804+0000","last_clean":"2024-03-29T04:10:05.911804+0000","last_became_active":"2024-03-29T04:10:00.541354+0000","last_became_peered":"2024-03-29T04:10:00.541354+0000","last_unstale":"2024-03-29T04:10:05.911804+0000","last_undegraded":"2024-03-29T04:10:05.911804+0000","last_fullsized":"2024-03-29T04:10:05.911804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:44:13.128618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910384+0000","last_change":"2024-03-29T04:10:00.544081+0000","last_active":"2024-03-29T04:10:05.910384+0000","last_peered":"2024-03-29T04:10:05.910384+0000","last_clean":"2024-03-29T04:10:05.910384+0000","last_became_active":"2024-03-29T04:10:00.543353+0000","last_became_peered":"2024-03-29T04:10:00.543353+0000","last_unstale":"2024-03-29T04:10:05.910384+0000","last_undegraded":"2024-03-29T04:10:05.910384+0000","last_fullsized":"2024-03-29T04:10:05.910384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:13:32.378332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910548+0000","last_change":"2024-03-29T04:10:00.543517+0000","last_active":"2024-03-29T04:10:05.910548+0000","last_peered":"2024-03-29T04:10:05.910548+0000","last_clean":"2024-03-29T04:10:05.910548+0000","last_became_active":"2024-03-29T04:10:00.542854+0000","last_became_peered":"2024-03-29T04:10:00.542854+0000","last_unstale":"2024-03-29T04:10:05.910548+0000","last_undegraded":"2024-03-29T04:10:05.910548+0000","last_fullsized":"2024-03-29T04:10:05.910548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:35:00.452782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557167+0000","last_change":"2024-03-29T04:10:00.546548+0000","last_active":"2024-03-29T04:10:05.557167+0000","last_peered":"2024-03-29T04:10:05.557167+0000","last_clean":"2024-03-29T04:10:05.557167+0000","last_became_active":"2024-03-29T04:10:00.546267+0000","last_became_peered":"2024-03-29T04:10:00.546267+0000","last_unstale":"2024-03-29T04:10:05.557167+0000","last_undegraded":"2024-03-29T04:10:05.557167+0000","last_fullsized":"2024-03-29T04:10:05.557167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:26:10.380318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911101+0000","last_change":"2024-03-29T04:10:00.539796+0000","last_active":"2024-03-29T04:10:05.911101+0000","last_peered":"2024-03-29T04:10:05.911101+0000","last_clean":"2024-03-29T04:10:05.911101+0000","last_became_active":"2024-03-29T04:10:00.539532+0000","last_became_peered":"2024-03-29T04:10:00.539532+0000","last_unstale":"2024-03-29T04:10:05.911101+0000","last_undegraded":"2024-03-29T04:10:05.911101+0000","last_fullsized":"2024-03-29T04:10:05.911101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:26:10.380318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556840+0000","last_change":"2024-03-29T04:10:00.549077+0000","last_active":"2024-03-29T04:10:05.556840+0000","last_peered":"2024-03-29T04:10:05.556840+0000","last_clean":"2024-03-29T04:10:05.556840+0000","last_became_active":"2024-03-29T04:10:00.548266+0000","last_became_peered":"2024-03-29T04:10:00.548266+0000","last_unstale":"2024-03-29T04:10:05.556840+0000","last_undegraded":"2024-03-29T04:10:05.556840+0000","last_fullsized":"2024-03-29T04:10:05.556840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:35:00.452782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910792+0000","last_change":"2024-03-29T04:10:00.535635+0000","last_active":"2024-03-29T04:10:05.910792+0000","last_peered":"2024-03-29T04:10:05.910792+0000","last_clean":"2024-03-29T04:10:05.910792+0000","last_became_active":"2024-03-29T04:10:00.535357+0000","last_became_peered":"2024-03-29T04:10:00.535357+0000","last_unstale":"2024-03-29T04:10:05.910792+0000","last_undegraded":"2024-03-29T04:10:05.910792+0000","last_fullsized":"2024-03-29T04:10:05.910792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:41.517875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911835+0000","last_change":"2024-03-29T04:10:00.547651+0000","last_active":"2024-03-29T04:10:05.911835+0000","last_peered":"2024-03-29T04:10:05.911835+0000","last_clean":"2024-03-29T04:10:05.911835+0000","last_became_active":"2024-03-29T04:10:00.547369+0000","last_became_peered":"2024-03-29T04:10:00.547369+0000","last_unstale":"2024-03-29T04:10:05.911835+0000","last_undegraded":"2024-03-29T04:10:05.911835+0000","last_fullsized":"2024-03-29T04:10:05.911835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:46.707085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911744+0000","last_change":"2024-03-29T04:10:00.545663+0000","last_active":"2024-03-29T04:10:05.911744+0000","last_peered":"2024-03-29T04:10:05.911744+0000","last_clean":"2024-03-29T04:10:05.911744+0000","last_became_active":"2024-03-29T04:10:00.545394+0000","last_became_peered":"2024-03-29T04:10:00.545394+0000","last_unstale":"2024-03-29T04:10:05.911744+0000","last_undegraded":"2024-03-29T04:10:05.911744+0000","last_fullsized":"2024-03-29T04:10:05.911744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:23:46.996981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:10:24.679 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:10:24.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.837+0000 7fbfe4382640 1 Processor -- start 2024-03-29T04:10:24.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.837+0000 7fbfe4382640 1 -- start start 2024-03-29T04:10:24.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.838+0000 7fbfe4382640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbfdc0ceb50 0x7fbfdc0cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.838+0000 7fbfe4382640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbfdc1608b0 0x7fbfdc15cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.838+0000 7fbfe4382640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfdc0d0250 con 0x7fbfdc15c5f0 2024-03-29T04:10:24.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.838+0000 7fbfe4382640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fbfdc0d19d0 con 0x7fbfdc0ceb50 2024-03-29T04:10:24.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.838+0000 7fbfe4382640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fbfdc0ccde0 con 0x7fbfdc1608b0 2024-03-29T04:10:24.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.838+0000 7fbfe211e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbfdc0ceb50 0x7fbfdc0cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.838+0000 7fbfe191d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbfdc15c5f0 0x7fbfdc0cef30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:32980/0 (socket says 172.21.15.148:32980) 2024-03-29T04:10:24.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.838+0000 7fbfe191d640 1 -- 172.21.15.148:0/2170357051 learned_addr learned my addr 172.21.15.148:0/2170357051 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:24.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.838+0000 7fbfe291f640 1 --2- 172.21.15.148:0/2170357051 >> v2:172.21.15.157:3300/0 conn(0x7fbfdc1608b0 0x7fbfdc15cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.838+0000 7fbfe211e640 1 -- 172.21.15.148:0/2170357051 >> v2:172.21.15.157:3300/0 conn(0x7fbfdc1608b0 msgr2=0x7fbfdc15cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:24.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.838+0000 7fbfe211e640 1 --2- 172.21.15.148:0/2170357051 >> v2:172.21.15.157:3300/0 conn(0x7fbfdc1608b0 0x7fbfdc15cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.838+0000 7fbfe211e640 1 -- 172.21.15.148:0/2170357051 >> v1:172.21.15.148:6789/0 conn(0x7fbfdc15c5f0 legacy=0x7fbfdc0cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.838+0000 7fbfe211e640 1 -- 172.21.15.148:0/2170357051 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfdc0ccb00 con 0x7fbfdc0ceb50 2024-03-29T04:10:24.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.838+0000 7fbfe111c640 1 -- 172.21.15.148:0/2170357051 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 193754473 0 0) 0x7fbfdc0d0250 con 0x7fbfdc15c5f0 2024-03-29T04:10:24.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.839+0000 7fbfe211e640 1 --2- 172.21.15.148:0/2170357051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfdc0ceb50 0x7fbfdc0cfd10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc0cc690 tx=0x7fbfd802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8abe10455d469e73 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:24.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.839+0000 7fbfe111c640 1 -- 172.21.15.148:0/2170357051 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8035020 con 0x7fbfdc0ceb50 2024-03-29T04:10:24.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.839+0000 7fbfe111c640 1 -- 172.21.15.148:0/2170357051 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfd8031830 con 0x7fbfdc0ceb50 2024-03-29T04:10:24.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.839+0000 7fbfe111c640 1 -- 172.21.15.148:0/2170357051 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8031c00 con 0x7fbfdc0ceb50 2024-03-29T04:10:24.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.840+0000 7fbfe4382640 1 -- 172.21.15.148:0/2170357051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfdc0ceb50 msgr2=0x7fbfdc0cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.840+0000 7fbfe4382640 1 --2- 172.21.15.148:0/2170357051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfdc0ceb50 0x7fbfdc0cfd10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc0cc690 tx=0x7fbfd802b8d0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.840+0000 7fbfe4382640 1 -- 172.21.15.148:0/2170357051 shutdown_connections 2024-03-29T04:10:24.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.840+0000 7fbfe4382640 1 --2- 172.21.15.148:0/2170357051 >> v2:172.21.15.157:3300/0 conn(0x7fbfdc1608b0 0x7fbfdc15cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.840+0000 7fbfe4382640 1 --2- 172.21.15.148:0/2170357051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfdc0ceb50 0x7fbfdc0cfd10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.840+0000 7fbfe4382640 1 -- 172.21.15.148:0/2170357051 >> 172.21.15.148:0/2170357051 conn(0x7fbfdc0b3c70 msgr2=0x7fbfdc0b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:24.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.840+0000 7fbfe4382640 1 -- 172.21.15.148:0/2170357051 shutdown_connections 2024-03-29T04:10:24.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.840+0000 7fbfe4382640 1 -- 172.21.15.148:0/2170357051 wait complete. 2024-03-29T04:10:24.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.841+0000 7fbfe4382640 1 Processor -- start 2024-03-29T04:10:24.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.841+0000 7fbfe4382640 1 -- start start 2024-03-29T04:10:24.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.842+0000 7fbfe4382640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfdc0ceb50 0x7fbfdc14c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.842+0000 7fbfe4382640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfdc1608b0 0x7fbfdc0c9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.842+0000 7fbfe4382640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfdc1f6da0 0x7fbfdc15a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.842+0000 7fbfe4382640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbfdc0ce540 con 0x7fbfdc0ceb50 2024-03-29T04:10:24.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.842+0000 7fbfe4382640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbfdc0cc790 con 0x7fbfdc1608b0 2024-03-29T04:10:24.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.842+0000 7fbfe4382640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbfdc0ce6c0 con 0x7fbfdc1f6da0 2024-03-29T04:10:24.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.842+0000 7fbfe211e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfdc0ceb50 0x7fbfdc14c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.842+0000 7fbfe211e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfdc0ceb50 0x7fbfdc14c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46440/0 (socket says 172.21.15.148:46440) 2024-03-29T04:10:24.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.842+0000 7fbfe211e640 1 -- 172.21.15.148:0/2621310139 learned_addr learned my addr 172.21.15.148:0/2621310139 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:24.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.842+0000 7fbfe191d640 1 --2- 172.21.15.148:0/2621310139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfdc1608b0 0x7fbfdc0c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.842+0000 7fbfe291f640 1 --2- 172.21.15.148:0/2621310139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfdc1f6da0 0x7fbfdc15a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.842+0000 7fbfe211e640 1 -- 172.21.15.148:0/2621310139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfdc1f6da0 msgr2=0x7fbfdc15a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.842+0000 7fbfe211e640 1 --2- 172.21.15.148:0/2621310139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfdc1f6da0 0x7fbfdc15a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.842+0000 7fbfe211e640 1 -- 172.21.15.148:0/2621310139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfdc1608b0 msgr2=0x7fbfdc0c9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.842+0000 7fbfe211e640 1 --2- 172.21.15.148:0/2621310139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfdc1608b0 0x7fbfdc0c9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.842+0000 7fbfe211e640 1 -- 172.21.15.148:0/2621310139 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfdc06dfd0 con 0x7fbfdc0ceb50 2024-03-29T04:10:24.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.842+0000 7fbfe291f640 1 --2- 172.21.15.148:0/2621310139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfdc1f6da0 0x7fbfdc15a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:24.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.842+0000 7fbfe191d640 1 --2- 172.21.15.148:0/2621310139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfdc1608b0 0x7fbfdc0c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:24.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.842+0000 7fbfe211e640 1 --2- 172.21.15.148:0/2621310139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfdc0ceb50 0x7fbfdc14c060 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009890 tx=0x7fbfd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:24.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.842+0000 7fbfd2ffd640 1 -- 172.21.15.148:0/2621310139 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8041070 con 0x7fbfdc0ceb50 2024-03-29T04:10:24.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.842+0000 7fbfd2ffd640 1 -- 172.21.15.148:0/2621310139 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfd803a950 con 0x7fbfdc0ceb50 2024-03-29T04:10:24.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.842+0000 7fbfd2ffd640 1 -- 172.21.15.148:0/2621310139 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8035040 con 0x7fbfdc0ceb50 2024-03-29T04:10:24.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.842+0000 7fbfe4382640 1 -- 172.21.15.148:0/2621310139 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfdc06e2b0 con 0x7fbfdc0ceb50 2024-03-29T04:10:24.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.843+0000 7fbfe4382640 1 -- 172.21.15.148:0/2621310139 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfdc149470 con 0x7fbfdc0ceb50 2024-03-29T04:10:24.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.844+0000 7fbfe4382640 1 -- 172.21.15.148:0/2621310139 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfdc0687e0 con 0x7fbfdc0ceb50 2024-03-29T04:10:24.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.844+0000 7fbfd2ffd640 1 -- 172.21.15.148:0/2621310139 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbfd8038020 con 0x7fbfdc0ceb50 2024-03-29T04:10:24.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.844+0000 7fbfd2ffd640 1 --2- 172.21.15.148:0/2621310139 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbfc006c530 0x7fbfc006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.844+0000 7fbfd2ffd640 1 -- 172.21.15.148:0/2621310139 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fbfd80ad3c0 con 0x7fbfdc0ceb50 2024-03-29T04:10:24.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.844+0000 7fbfe191d640 1 --2- 172.21.15.148:0/2621310139 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbfc006c530 0x7fbfc006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.845+0000 7fbfe191d640 1 --2- 172.21.15.148:0/2621310139 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbfc006c530 0x7fbfc006e9f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc06e8a0 tx=0x7fbfcc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:24.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:24.849+0000 7fbfd2ffd640 1 -- 172.21.15.148:0/2621310139 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbfd807b730 con 0x7fbfdc0ceb50 2024-03-29T04:10:25.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.021+0000 7fbfe4382640 1 -- 172.21.15.148:0/2621310139 --> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbfdc0c9980 con 0x7fbfc006c530 2024-03-29T04:10:25.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.036+0000 7fbfd2ffd640 1 -- 172.21.15.148:0/2621310139 <== mgr.4113 v2:172.21.15.148:6832/3271445075 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7fbfdc0c9980 con 0x7fbfc006c530 2024-03-29T04:10:25.019 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:25.021 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-03-29T04:10:25.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.040+0000 7fbfe4382640 1 -- 172.21.15.148:0/2621310139 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbfc006c530 msgr2=0x7fbfc006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.040+0000 7fbfe4382640 1 --2- 172.21.15.148:0/2621310139 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbfc006c530 0x7fbfc006e9f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc06e8a0 tx=0x7fbfcc009040 comp rx=0 tx=0).stop 2024-03-29T04:10:25.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.040+0000 7fbfe4382640 1 -- 172.21.15.148:0/2621310139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfdc0ceb50 msgr2=0x7fbfdc14c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.040+0000 7fbfe4382640 1 --2- 172.21.15.148:0/2621310139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfdc0ceb50 0x7fbfdc14c060 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009890 tx=0x7fbfd80046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.040+0000 7fbfe4382640 1 -- 172.21.15.148:0/2621310139 shutdown_connections 2024-03-29T04:10:25.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.040+0000 7fbfe4382640 1 --2- 172.21.15.148:0/2621310139 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbfc006c530 0x7fbfc006e9f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.040+0000 7fbfe4382640 1 --2- 172.21.15.148:0/2621310139 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfdc1f6da0 0x7fbfdc15a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.040+0000 7fbfe4382640 1 --2- 172.21.15.148:0/2621310139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfdc1608b0 0x7fbfdc0c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.040+0000 7fbfe4382640 1 --2- 172.21.15.148:0/2621310139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfdc0ceb50 0x7fbfdc14c060 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.040+0000 7fbfe4382640 1 -- 172.21.15.148:0/2621310139 >> 172.21.15.148:0/2621310139 conn(0x7fbfdc0b3c70 msgr2=0x7fbfdc0b5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:25.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.040+0000 7fbfe4382640 1 -- 172.21.15.148:0/2621310139 shutdown_connections 2024-03-29T04:10:25.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.040+0000 7fbfe4382640 1 -- 172.21.15.148:0/2621310139 wait complete. 2024-03-29T04:10:25.030 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-03-29T04:10:23.887357+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236032,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738688,"statfs":{"total":773094113280,"available":772852416512,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002287"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911869+0000","last_change":"2024-03-29T04:09:59.541493+0000","last_active":"2024-03-29T04:10:05.911869+0000","last_peered":"2024-03-29T04:10:05.911869+0000","last_clean":"2024-03-29T04:10:05.911869+0000","last_became_active":"2024-03-29T04:09:59.541208+0000","last_became_peered":"2024-03-29T04:09:59.541208+0000","last_unstale":"2024-03-29T04:10:05.911869+0000","last_undegraded":"2024-03-29T04:10:05.911869+0000","last_fullsized":"2024-03-29T04:10:05.911869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:55:08.672579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910799+0000","last_change":"2024-03-29T04:10:00.548950+0000","last_active":"2024-03-29T04:10:05.910799+0000","last_peered":"2024-03-29T04:10:05.910799+0000","last_clean":"2024-03-29T04:10:05.910799+0000","last_became_active":"2024-03-29T04:10:00.548670+0000","last_became_peered":"2024-03-29T04:10:00.548670+0000","last_unstale":"2024-03-29T04:10:05.910799+0000","last_undegraded":"2024-03-29T04:10:05.910799+0000","last_fullsized":"2024-03-29T04:10:05.910799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:16:02.331457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911034+0000","last_change":"2024-03-29T04:10:04.649988+0000","last_active":"2024-03-29T04:10:05.911034+0000","last_peered":"2024-03-29T04:10:05.911034+0000","last_clean":"2024-03-29T04:10:05.911034+0000","last_became_active":"2024-03-29T04:10:04.649557+0000","last_became_peered":"2024-03-29T04:10:04.649557+0000","last_unstale":"2024-03-29T04:10:05.911034+0000","last_undegraded":"2024-03-29T04:10:05.911034+0000","last_fullsized":"2024-03-29T04:10:05.911034+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:07:56.606322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911032+0000","last_change":"2024-03-29T04:09:59.542509+0000","last_active":"2024-03-29T04:10:05.911032+0000","last_peered":"2024-03-29T04:10:05.911032+0000","last_clean":"2024-03-29T04:10:05.911032+0000","last_became_active":"2024-03-29T04:09:59.542221+0000","last_became_peered":"2024-03-29T04:09:59.542221+0000","last_unstale":"2024-03-29T04:10:05.911032+0000","last_undegraded":"2024-03-29T04:10:05.911032+0000","last_fullsized":"2024-03-29T04:10:05.911032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:58:10.121554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910298+0000","last_change":"2024-03-29T04:10:00.536505+0000","last_active":"2024-03-29T04:10:05.910298+0000","last_peered":"2024-03-29T04:10:05.910298+0000","last_clean":"2024-03-29T04:10:05.910298+0000","last_became_active":"2024-03-29T04:10:00.536200+0000","last_became_peered":"2024-03-29T04:10:00.536200+0000","last_unstale":"2024-03-29T04:10:05.910298+0000","last_undegraded":"2024-03-29T04:10:05.910298+0000","last_fullsized":"2024-03-29T04:10:05.910298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:29.233534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557208+0000","last_change":"2024-03-29T04:10:04.576319+0000","last_active":"2024-03-29T04:10:05.557208+0000","last_peered":"2024-03-29T04:10:05.557208+0000","last_clean":"2024-03-29T04:10:05.557208+0000","last_became_active":"2024-03-29T04:10:04.575508+0000","last_became_peered":"2024-03-29T04:10:04.575508+0000","last_unstale":"2024-03-29T04:10:05.557208+0000","last_undegraded":"2024-03-29T04:10:05.557208+0000","last_fullsized":"2024-03-29T04:10:05.557208+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:34.129098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911504+0000","last_change":"2024-03-29T04:09:59.545281+0000","last_active":"2024-03-29T04:10:05.911504+0000","last_peered":"2024-03-29T04:10:05.911504+0000","last_clean":"2024-03-29T04:10:05.911504+0000","last_became_active":"2024-03-29T04:09:59.544991+0000","last_became_peered":"2024-03-29T04:09:59.544991+0000","last_unstale":"2024-03-29T04:10:05.911504+0000","last_undegraded":"2024-03-29T04:10:05.911504+0000","last_fullsized":"2024-03-29T04:10:05.911504+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:35:01.484589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556861+0000","last_change":"2024-03-29T04:10:00.542510+0000","last_active":"2024-03-29T04:10:05.556861+0000","last_peered":"2024-03-29T04:10:05.556861+0000","last_clean":"2024-03-29T04:10:05.556861+0000","last_became_active":"2024-03-29T04:10:00.542014+0000","last_became_peered":"2024-03-29T04:10:00.542014+0000","last_unstale":"2024-03-29T04:10:05.556861+0000","last_undegraded":"2024-03-29T04:10:05.556861+0000","last_fullsized":"2024-03-29T04:10:05.556861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:30.096622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556709+0000","last_change":"2024-03-29T04:10:04.574735+0000","last_active":"2024-03-29T04:10:05.556709+0000","last_peered":"2024-03-29T04:10:05.556709+0000","last_clean":"2024-03-29T04:10:05.556709+0000","last_became_active":"2024-03-29T04:10:04.573995+0000","last_became_peered":"2024-03-29T04:10:04.573995+0000","last_unstale":"2024-03-29T04:10:05.556709+0000","last_undegraded":"2024-03-29T04:10:05.556709+0000","last_fullsized":"2024-03-29T04:10:05.556709+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:52.246321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911346+0000","last_change":"2024-03-29T04:09:59.542326+0000","last_active":"2024-03-29T04:10:05.911346+0000","last_peered":"2024-03-29T04:10:05.911346+0000","last_clean":"2024-03-29T04:10:05.911346+0000","last_became_active":"2024-03-29T04:09:59.540873+0000","last_became_peered":"2024-03-29T04:09:59.540873+0000","last_unstale":"2024-03-29T04:10:05.911346+0000","last_undegraded":"2024-03-29T04:10:05.911346+0000","last_fullsized":"2024-03-29T04:10:05.911346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:02.879344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557486+0000","last_change":"2024-03-29T04:10:00.543297+0000","last_active":"2024-03-29T04:10:05.557486+0000","last_peered":"2024-03-29T04:10:05.557486+0000","last_clean":"2024-03-29T04:10:05.557486+0000","last_became_active":"2024-03-29T04:10:00.541793+0000","last_became_peered":"2024-03-29T04:10:00.541793+0000","last_unstale":"2024-03-29T04:10:05.557486+0000","last_undegraded":"2024-03-29T04:10:05.557486+0000","last_fullsized":"2024-03-29T04:10:05.557486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:13.002495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557550+0000","last_change":"2024-03-29T04:10:04.572366+0000","last_active":"2024-03-29T04:10:05.557550+0000","last_peered":"2024-03-29T04:10:05.557550+0000","last_clean":"2024-03-29T04:10:05.557550+0000","last_became_active":"2024-03-29T04:10:04.572066+0000","last_became_peered":"2024-03-29T04:10:04.572066+0000","last_unstale":"2024-03-29T04:10:05.557550+0000","last_undegraded":"2024-03-29T04:10:05.557550+0000","last_fullsized":"2024-03-29T04:10:05.557550+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:00:55.353300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910448+0000","last_change":"2024-03-29T04:10:00.547621+0000","last_active":"2024-03-29T04:10:05.910448+0000","last_peered":"2024-03-29T04:10:05.910448+0000","last_clean":"2024-03-29T04:10:05.910448+0000","last_became_active":"2024-03-29T04:10:00.547282+0000","last_became_peered":"2024-03-29T04:10:00.547282+0000","last_unstale":"2024-03-29T04:10:05.910448+0000","last_undegraded":"2024-03-29T04:10:05.910448+0000","last_fullsized":"2024-03-29T04:10:05.910448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:41.517875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911279+0000","last_change":"2024-03-29T04:09:59.539362+0000","last_active":"2024-03-29T04:10:05.911279+0000","last_peered":"2024-03-29T04:10:05.911279+0000","last_clean":"2024-03-29T04:10:05.911279+0000","last_became_active":"2024-03-29T04:09:59.539062+0000","last_became_peered":"2024-03-29T04:09:59.539062+0000","last_unstale":"2024-03-29T04:10:05.911279+0000","last_undegraded":"2024-03-29T04:10:05.911279+0000","last_fullsized":"2024-03-29T04:10:05.911279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:33:50.904931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.558475+0000","last_change":"2024-03-29T04:10:04.579133+0000","last_active":"2024-03-29T04:10:05.558475+0000","last_peered":"2024-03-29T04:10:05.558475+0000","last_clean":"2024-03-29T04:10:05.558475+0000","last_became_active":"2024-03-29T04:10:04.578748+0000","last_became_peered":"2024-03-29T04:10:04.578748+0000","last_unstale":"2024-03-29T04:10:05.558475+0000","last_undegraded":"2024-03-29T04:10:05.558475+0000","last_fullsized":"2024-03-29T04:10:05.558475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:06.998903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910015+0000","last_change":"2024-03-29T04:10:00.547887+0000","last_active":"2024-03-29T04:10:05.910015+0000","last_peered":"2024-03-29T04:10:05.910015+0000","last_clean":"2024-03-29T04:10:05.910015+0000","last_became_active":"2024-03-29T04:10:00.547419+0000","last_became_peered":"2024-03-29T04:10:00.547419+0000","last_unstale":"2024-03-29T04:10:05.910015+0000","last_undegraded":"2024-03-29T04:10:05.910015+0000","last_fullsized":"2024-03-29T04:10:05.910015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:46.707085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912399+0000","last_change":"2024-03-29T04:09:59.544306+0000","last_active":"2024-03-29T04:10:05.912399+0000","last_peered":"2024-03-29T04:10:05.912399+0000","last_clean":"2024-03-29T04:10:05.912399+0000","last_became_active":"2024-03-29T04:09:59.544028+0000","last_became_peered":"2024-03-29T04:09:59.544028+0000","last_unstale":"2024-03-29T04:10:05.912399+0000","last_undegraded":"2024-03-29T04:10:05.912399+0000","last_fullsized":"2024-03-29T04:10:05.912399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:33:50.904931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.558045+0000","last_change":"2024-03-29T04:10:04.577178+0000","last_active":"2024-03-29T04:10:05.558045+0000","last_peered":"2024-03-29T04:10:05.558045+0000","last_clean":"2024-03-29T04:10:05.558045+0000","last_became_active":"2024-03-29T04:10:04.576788+0000","last_became_peered":"2024-03-29T04:10:04.576788+0000","last_unstale":"2024-03-29T04:10:05.558045+0000","last_undegraded":"2024-03-29T04:10:05.558045+0000","last_fullsized":"2024-03-29T04:10:05.558045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:26:14.412794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910704+0000","last_change":"2024-03-29T04:10:00.543623+0000","last_active":"2024-03-29T04:10:05.910704+0000","last_peered":"2024-03-29T04:10:05.910704+0000","last_clean":"2024-03-29T04:10:05.910704+0000","last_became_active":"2024-03-29T04:10:00.542670+0000","last_became_peered":"2024-03-29T04:10:00.542670+0000","last_unstale":"2024-03-29T04:10:05.910704+0000","last_undegraded":"2024-03-29T04:10:05.910704+0000","last_fullsized":"2024-03-29T04:10:05.910704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:48.213845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910045+0000","last_change":"2024-03-29T04:09:59.540613+0000","last_active":"2024-03-29T04:10:05.910045+0000","last_peered":"2024-03-29T04:10:05.910045+0000","last_clean":"2024-03-29T04:10:05.910045+0000","last_became_active":"2024-03-29T04:09:59.540340+0000","last_became_peered":"2024-03-29T04:09:59.540340+0000","last_unstale":"2024-03-29T04:10:05.910045+0000","last_undegraded":"2024-03-29T04:10:05.910045+0000","last_fullsized":"2024-03-29T04:10:05.910045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:05.307537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911330+0000","last_change":"2024-03-29T04:10:04.646383+0000","last_active":"2024-03-29T04:10:05.911330+0000","last_peered":"2024-03-29T04:10:05.911330+0000","last_clean":"2024-03-29T04:10:05.911330+0000","last_became_active":"2024-03-29T04:10:04.646046+0000","last_became_peered":"2024-03-29T04:10:04.646046+0000","last_unstale":"2024-03-29T04:10:05.911330+0000","last_undegraded":"2024-03-29T04:10:05.911330+0000","last_fullsized":"2024-03-29T04:10:05.911330+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:17.034970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557229+0000","last_change":"2024-03-29T04:10:00.537777+0000","last_active":"2024-03-29T04:10:05.557229+0000","last_peered":"2024-03-29T04:10:05.557229+0000","last_clean":"2024-03-29T04:10:05.557229+0000","last_became_active":"2024-03-29T04:10:00.537423+0000","last_became_peered":"2024-03-29T04:10:00.537423+0000","last_unstale":"2024-03-29T04:10:05.557229+0000","last_undegraded":"2024-03-29T04:10:05.557229+0000","last_fullsized":"2024-03-29T04:10:05.557229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:33:51.909341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910053+0000","last_change":"2024-03-29T04:09:59.543268+0000","last_active":"2024-03-29T04:10:05.910053+0000","last_peered":"2024-03-29T04:10:05.910053+0000","last_clean":"2024-03-29T04:10:05.910053+0000","last_became_active":"2024-03-29T04:09:59.542982+0000","last_became_peered":"2024-03-29T04:09:59.542982+0000","last_unstale":"2024-03-29T04:10:05.910053+0000","last_undegraded":"2024-03-29T04:10:05.910053+0000","last_fullsized":"2024-03-29T04:10:05.910053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:01.925223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557817+0000","last_change":"2024-03-29T04:10:04.577232+0000","last_active":"2024-03-29T04:10:05.557817+0000","last_peered":"2024-03-29T04:10:05.557817+0000","last_clean":"2024-03-29T04:10:05.557817+0000","last_became_active":"2024-03-29T04:10:04.576758+0000","last_became_peered":"2024-03-29T04:10:04.576758+0000","last_unstale":"2024-03-29T04:10:05.557817+0000","last_undegraded":"2024-03-29T04:10:05.557817+0000","last_fullsized":"2024-03-29T04:10:05.557817+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:26:08.662188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912635+0000","last_change":"2024-03-29T04:10:00.546235+0000","last_active":"2024-03-29T04:10:05.912635+0000","last_peered":"2024-03-29T04:10:05.912635+0000","last_clean":"2024-03-29T04:10:05.912635+0000","last_became_active":"2024-03-29T04:10:00.545908+0000","last_became_peered":"2024-03-29T04:10:00.545908+0000","last_unstale":"2024-03-29T04:10:05.912635+0000","last_undegraded":"2024-03-29T04:10:05.912635+0000","last_fullsized":"2024-03-29T04:10:05.912635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:16:02.331457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911074+0000","last_change":"2024-03-29T04:09:59.541657+0000","last_active":"2024-03-29T04:10:05.911074+0000","last_peered":"2024-03-29T04:10:05.911074+0000","last_clean":"2024-03-29T04:10:05.911074+0000","last_became_active":"2024-03-29T04:09:59.541305+0000","last_became_peered":"2024-03-29T04:09:59.541305+0000","last_unstale":"2024-03-29T04:10:05.911074+0000","last_undegraded":"2024-03-29T04:10:05.911074+0000","last_fullsized":"2024-03-29T04:10:05.911074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:25:06.123311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556981+0000","last_change":"2024-03-29T04:10:04.646174+0000","last_active":"2024-03-29T04:10:05.556981+0000","last_peered":"2024-03-29T04:10:05.556981+0000","last_clean":"2024-03-29T04:10:05.556981+0000","last_became_active":"2024-03-29T04:10:04.645802+0000","last_became_peered":"2024-03-29T04:10:04.645802+0000","last_unstale":"2024-03-29T04:10:05.556981+0000","last_undegraded":"2024-03-29T04:10:05.556981+0000","last_fullsized":"2024-03-29T04:10:05.556981+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:35:04.485257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911614+0000","last_change":"2024-03-29T04:10:00.543254+0000","last_active":"2024-03-29T04:10:05.911614+0000","last_peered":"2024-03-29T04:10:05.911614+0000","last_clean":"2024-03-29T04:10:05.911614+0000","last_became_active":"2024-03-29T04:10:00.542958+0000","last_became_peered":"2024-03-29T04:10:00.542958+0000","last_unstale":"2024-03-29T04:10:05.911614+0000","last_undegraded":"2024-03-29T04:10:05.911614+0000","last_fullsized":"2024-03-29T04:10:05.911614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:00:51.320824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911924+0000","last_change":"2024-03-29T04:09:59.544320+0000","last_active":"2024-03-29T04:10:05.911924+0000","last_peered":"2024-03-29T04:10:05.911924+0000","last_clean":"2024-03-29T04:10:05.911924+0000","last_became_active":"2024-03-29T04:09:59.543928+0000","last_became_peered":"2024-03-29T04:09:59.543928+0000","last_unstale":"2024-03-29T04:10:05.911924+0000","last_undegraded":"2024-03-29T04:10:05.911924+0000","last_fullsized":"2024-03-29T04:10:05.911924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:02.879344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910274+0000","last_change":"2024-03-29T04:10:04.646905+0000","last_active":"2024-03-29T04:10:05.910274+0000","last_peered":"2024-03-29T04:10:05.910274+0000","last_clean":"2024-03-29T04:10:05.910274+0000","last_became_active":"2024-03-29T04:10:04.646482+0000","last_became_peered":"2024-03-29T04:10:04.646482+0000","last_unstale":"2024-03-29T04:10:05.910274+0000","last_undegraded":"2024-03-29T04:10:05.910274+0000","last_fullsized":"2024-03-29T04:10:05.910274+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:38:53.191487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910879+0000","last_change":"2024-03-29T04:10:00.541465+0000","last_active":"2024-03-29T04:10:05.910879+0000","last_peered":"2024-03-29T04:10:05.910879+0000","last_clean":"2024-03-29T04:10:05.910879+0000","last_became_active":"2024-03-29T04:10:00.541197+0000","last_became_peered":"2024-03-29T04:10:00.541197+0000","last_unstale":"2024-03-29T04:10:05.910879+0000","last_undegraded":"2024-03-29T04:10:05.910879+0000","last_fullsized":"2024-03-29T04:10:05.910879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:52:12.957170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911673+0000","last_change":"2024-03-29T04:09:59.545759+0000","last_active":"2024-03-29T04:10:05.911673+0000","last_peered":"2024-03-29T04:10:05.911673+0000","last_clean":"2024-03-29T04:10:05.911673+0000","last_became_active":"2024-03-29T04:09:59.545373+0000","last_became_peered":"2024-03-29T04:09:59.545373+0000","last_unstale":"2024-03-29T04:10:05.911673+0000","last_undegraded":"2024-03-29T04:10:05.911673+0000","last_fullsized":"2024-03-29T04:10:05.911673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:58:10.121554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911095+0000","last_change":"2024-03-29T04:10:04.644123+0000","last_active":"2024-03-29T04:10:05.911095+0000","last_peered":"2024-03-29T04:10:05.911095+0000","last_clean":"2024-03-29T04:10:05.911095+0000","last_became_active":"2024-03-29T04:10:04.643701+0000","last_became_peered":"2024-03-29T04:10:04.643701+0000","last_unstale":"2024-03-29T04:10:05.911095+0000","last_undegraded":"2024-03-29T04:10:05.911095+0000","last_fullsized":"2024-03-29T04:10:05.911095+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:26:14.412794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557781+0000","last_change":"2024-03-29T04:10:00.543518+0000","last_active":"2024-03-29T04:10:05.557781+0000","last_peered":"2024-03-29T04:10:05.557781+0000","last_clean":"2024-03-29T04:10:05.557781+0000","last_became_active":"2024-03-29T04:10:00.542373+0000","last_became_peered":"2024-03-29T04:10:00.542373+0000","last_unstale":"2024-03-29T04:10:05.557781+0000","last_undegraded":"2024-03-29T04:10:05.557781+0000","last_fullsized":"2024-03-29T04:10:05.557781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:02.966428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910966+0000","last_change":"2024-03-29T04:09:59.542356+0000","last_active":"2024-03-29T04:10:05.910966+0000","last_peered":"2024-03-29T04:10:05.910966+0000","last_clean":"2024-03-29T04:10:05.910966+0000","last_became_active":"2024-03-29T04:09:59.541917+0000","last_became_peered":"2024-03-29T04:09:59.541917+0000","last_unstale":"2024-03-29T04:10:05.910966+0000","last_undegraded":"2024-03-29T04:10:05.910966+0000","last_fullsized":"2024-03-29T04:10:05.910966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:01.925223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912224+0000","last_change":"2024-03-29T04:10:04.578218+0000","last_active":"2024-03-29T04:10:05.912224+0000","last_peered":"2024-03-29T04:10:05.912224+0000","last_clean":"2024-03-29T04:10:05.912224+0000","last_became_active":"2024-03-29T04:10:04.577221+0000","last_became_peered":"2024-03-29T04:10:04.577221+0000","last_unstale":"2024-03-29T04:10:05.912224+0000","last_undegraded":"2024-03-29T04:10:05.912224+0000","last_fullsized":"2024-03-29T04:10:05.912224+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:44:17.161094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.558130+0000","last_change":"2024-03-29T04:10:00.536176+0000","last_active":"2024-03-29T04:10:05.558130+0000","last_peered":"2024-03-29T04:10:05.558130+0000","last_clean":"2024-03-29T04:10:05.558130+0000","last_became_active":"2024-03-29T04:10:00.535854+0000","last_became_peered":"2024-03-29T04:10:00.535854+0000","last_unstale":"2024-03-29T04:10:05.558130+0000","last_undegraded":"2024-03-29T04:10:05.558130+0000","last_fullsized":"2024-03-29T04:10:05.558130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:25:07.127721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557572+0000","last_change":"2024-03-29T04:09:59.542230+0000","last_active":"2024-03-29T04:10:05.557572+0000","last_peered":"2024-03-29T04:10:05.557572+0000","last_clean":"2024-03-29T04:10:05.557572+0000","last_became_active":"2024-03-29T04:09:59.541972+0000","last_became_peered":"2024-03-29T04:09:59.541972+0000","last_unstale":"2024-03-29T04:10:05.557572+0000","last_undegraded":"2024-03-29T04:10:05.557572+0000","last_fullsized":"2024-03-29T04:10:05.557572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:01.925223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910829+0000","last_change":"2024-03-29T04:10:04.574263+0000","last_active":"2024-03-29T04:10:05.910829+0000","last_peered":"2024-03-29T04:10:05.910829+0000","last_clean":"2024-03-29T04:10:05.910829+0000","last_became_active":"2024-03-29T04:10:04.573843+0000","last_became_peered":"2024-03-29T04:10:04.573843+0000","last_unstale":"2024-03-29T04:10:05.910829+0000","last_undegraded":"2024-03-29T04:10:05.910829+0000","last_fullsized":"2024-03-29T04:10:05.910829+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:06.998903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912194+0000","last_change":"2024-03-29T04:10:00.540824+0000","last_active":"2024-03-29T04:10:05.912194+0000","last_peered":"2024-03-29T04:10:05.912194+0000","last_clean":"2024-03-29T04:10:05.912194+0000","last_became_active":"2024-03-29T04:10:00.540505+0000","last_became_peered":"2024-03-29T04:10:00.540505+0000","last_unstale":"2024-03-29T04:10:05.912194+0000","last_undegraded":"2024-03-29T04:10:05.912194+0000","last_fullsized":"2024-03-29T04:10:05.912194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:41.517875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.559855+0000","last_change":"2024-03-29T04:09:59.540415+0000","last_active":"2024-03-29T04:10:05.559855+0000","last_peered":"2024-03-29T04:10:05.559855+0000","last_clean":"2024-03-29T04:10:05.559855+0000","last_became_active":"2024-03-29T04:09:59.540130+0000","last_became_peered":"2024-03-29T04:09:59.540130+0000","last_unstale":"2024-03-29T04:10:05.559855+0000","last_undegraded":"2024-03-29T04:10:05.559855+0000","last_fullsized":"2024-03-29T04:10:05.559855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:05.307537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911492+0000","last_change":"2024-03-29T04:10:04.575156+0000","last_active":"2024-03-29T04:10:05.911492+0000","last_peered":"2024-03-29T04:10:05.911492+0000","last_clean":"2024-03-29T04:10:05.911492+0000","last_became_active":"2024-03-29T04:10:04.573816+0000","last_became_peered":"2024-03-29T04:10:04.573816+0000","last_unstale":"2024-03-29T04:10:05.911492+0000","last_undegraded":"2024-03-29T04:10:05.911492+0000","last_fullsized":"2024-03-29T04:10:05.911492+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:23:51.029457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.909950+0000","last_change":"2024-03-29T04:10:00.531739+0000","last_active":"2024-03-29T04:10:05.909950+0000","last_peered":"2024-03-29T04:10:05.909950+0000","last_clean":"2024-03-29T04:10:05.909950+0000","last_became_active":"2024-03-29T04:10:00.531395+0000","last_became_peered":"2024-03-29T04:10:00.531395+0000","last_unstale":"2024-03-29T04:10:05.909950+0000","last_undegraded":"2024-03-29T04:10:05.909950+0000","last_fullsized":"2024-03-29T04:10:05.909950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:02:32.544769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910076+0000","last_change":"2024-03-29T04:09:59.541400+0000","last_active":"2024-03-29T04:10:05.910076+0000","last_peered":"2024-03-29T04:10:05.910076+0000","last_clean":"2024-03-29T04:10:05.910076+0000","last_became_active":"2024-03-29T04:09:59.541114+0000","last_became_peered":"2024-03-29T04:09:59.541114+0000","last_unstale":"2024-03-29T04:10:05.910076+0000","last_undegraded":"2024-03-29T04:10:05.910076+0000","last_fullsized":"2024-03-29T04:10:05.910076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:25:06.123311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912614+0000","last_change":"2024-03-29T04:10:04.645661+0000","last_active":"2024-03-29T04:10:05.912614+0000","last_peered":"2024-03-29T04:10:05.912614+0000","last_clean":"2024-03-29T04:10:05.912614+0000","last_became_active":"2024-03-29T04:10:04.645258+0000","last_became_peered":"2024-03-29T04:10:04.645258+0000","last_unstale":"2024-03-29T04:10:05.912614+0000","last_undegraded":"2024-03-29T04:10:05.912614+0000","last_fullsized":"2024-03-29T04:10:05.912614+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:44:17.161094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910699+0000","last_change":"2024-03-29T04:10:00.537081+0000","last_active":"2024-03-29T04:10:05.910699+0000","last_peered":"2024-03-29T04:10:05.910699+0000","last_clean":"2024-03-29T04:10:05.910699+0000","last_became_active":"2024-03-29T04:10:00.536267+0000","last_became_peered":"2024-03-29T04:10:00.536267+0000","last_unstale":"2024-03-29T04:10:05.910699+0000","last_undegraded":"2024-03-29T04:10:05.910699+0000","last_fullsized":"2024-03-29T04:10:05.910699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:41.517875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557767+0000","last_change":"2024-03-29T04:09:59.535631+0000","last_active":"2024-03-29T04:10:05.557767+0000","last_peered":"2024-03-29T04:10:05.557767+0000","last_clean":"2024-03-29T04:10:05.557767+0000","last_became_active":"2024-03-29T04:09:59.535323+0000","last_became_peered":"2024-03-29T04:09:59.535323+0000","last_unstale":"2024-03-29T04:10:05.557767+0000","last_undegraded":"2024-03-29T04:10:05.557767+0000","last_fullsized":"2024-03-29T04:10:05.557767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:05.307537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910183+0000","last_change":"2024-03-29T04:09:49.002570+0000","last_active":"2024-03-29T04:10:05.910183+0000","last_peered":"2024-03-29T04:10:05.910183+0000","last_clean":"2024-03-29T04:10:05.910183+0000","last_became_active":"2024-03-29T04:09:49.002235+0000","last_became_peered":"2024-03-29T04:09:49.002235+0000","last_unstale":"2024-03-29T04:10:05.910183+0000","last_undegraded":"2024-03-29T04:10:05.910183+0000","last_fullsized":"2024-03-29T04:10:05.910183+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:47.900733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:47.900733+0000","last_clean_scrub_stamp":"2024-03-29T04:09:47.900733+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:44:04.608265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911029+0000","last_change":"2024-03-29T04:10:04.646698+0000","last_active":"2024-03-29T04:10:05.911029+0000","last_peered":"2024-03-29T04:10:05.911029+0000","last_clean":"2024-03-29T04:10:05.911029+0000","last_became_active":"2024-03-29T04:10:04.646221+0000","last_became_peered":"2024-03-29T04:10:04.646221+0000","last_unstale":"2024-03-29T04:10:05.911029+0000","last_undegraded":"2024-03-29T04:10:05.911029+0000","last_fullsized":"2024-03-29T04:10:05.911029+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:17.034970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911371+0000","last_change":"2024-03-29T04:10:00.540055+0000","last_active":"2024-03-29T04:10:05.911371+0000","last_peered":"2024-03-29T04:10:05.911371+0000","last_clean":"2024-03-29T04:10:05.911371+0000","last_became_active":"2024-03-29T04:10:00.539766+0000","last_became_peered":"2024-03-29T04:10:00.539766+0000","last_unstale":"2024-03-29T04:10:05.911371+0000","last_undegraded":"2024-03-29T04:10:05.911371+0000","last_fullsized":"2024-03-29T04:10:05.911371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:46.804154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.560157+0000","last_change":"2024-03-29T04:09:59.536860+0000","last_active":"2024-03-29T04:10:05.560157+0000","last_peered":"2024-03-29T04:10:05.560157+0000","last_clean":"2024-03-29T04:10:05.560157+0000","last_became_active":"2024-03-29T04:09:59.536504+0000","last_became_peered":"2024-03-29T04:09:59.536504+0000","last_unstale":"2024-03-29T04:10:05.560157+0000","last_undegraded":"2024-03-29T04:10:05.560157+0000","last_fullsized":"2024-03-29T04:10:05.560157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:02.879344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910369+0000","last_change":"2024-03-29T04:10:04.644446+0000","last_active":"2024-03-29T04:10:05.910369+0000","last_peered":"2024-03-29T04:10:05.910369+0000","last_clean":"2024-03-29T04:10:05.910369+0000","last_became_active":"2024-03-29T04:10:04.643982+0000","last_became_peered":"2024-03-29T04:10:04.643982+0000","last_unstale":"2024-03-29T04:10:05.910369+0000","last_undegraded":"2024-03-29T04:10:05.910369+0000","last_fullsized":"2024-03-29T04:10:05.910369+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:26:14.412794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556955+0000","last_change":"2024-03-29T04:10:00.544801+0000","last_active":"2024-03-29T04:10:05.556955+0000","last_peered":"2024-03-29T04:10:05.556955+0000","last_clean":"2024-03-29T04:10:05.556955+0000","last_became_active":"2024-03-29T04:10:00.544481+0000","last_became_peered":"2024-03-29T04:10:00.544481+0000","last_unstale":"2024-03-29T04:10:05.556955+0000","last_undegraded":"2024-03-29T04:10:05.556955+0000","last_fullsized":"2024-03-29T04:10:05.556955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:00:51.320824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911910+0000","last_change":"2024-03-29T04:09:59.541406+0000","last_active":"2024-03-29T04:10:05.911910+0000","last_peered":"2024-03-29T04:10:05.911910+0000","last_clean":"2024-03-29T04:10:05.911910+0000","last_became_active":"2024-03-29T04:09:59.541099+0000","last_became_peered":"2024-03-29T04:09:59.541099+0000","last_unstale":"2024-03-29T04:10:05.911910+0000","last_undegraded":"2024-03-29T04:10:05.911910+0000","last_fullsized":"2024-03-29T04:10:05.911910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:25:06.123311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911325+0000","last_change":"2024-03-29T04:10:04.579423+0000","last_active":"2024-03-29T04:10:05.911325+0000","last_peered":"2024-03-29T04:10:05.911325+0000","last_clean":"2024-03-29T04:10:05.911325+0000","last_became_active":"2024-03-29T04:10:04.579062+0000","last_became_peered":"2024-03-29T04:10:04.579062+0000","last_unstale":"2024-03-29T04:10:05.911325+0000","last_undegraded":"2024-03-29T04:10:05.911325+0000","last_fullsized":"2024-03-29T04:10:05.911325+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:06.998903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911863+0000","last_change":"2024-03-29T04:10:00.547282+0000","last_active":"2024-03-29T04:10:05.911863+0000","last_peered":"2024-03-29T04:10:05.911863+0000","last_clean":"2024-03-29T04:10:05.911863+0000","last_became_active":"2024-03-29T04:10:00.547026+0000","last_became_peered":"2024-03-29T04:10:00.547026+0000","last_unstale":"2024-03-29T04:10:05.911863+0000","last_undegraded":"2024-03-29T04:10:05.911863+0000","last_fullsized":"2024-03-29T04:10:05.911863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:34:06.367730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.560063+0000","last_change":"2024-03-29T04:09:59.540515+0000","last_active":"2024-03-29T04:10:05.560063+0000","last_peered":"2024-03-29T04:10:05.560063+0000","last_clean":"2024-03-29T04:10:05.560063+0000","last_became_active":"2024-03-29T04:09:59.540198+0000","last_became_peered":"2024-03-29T04:09:59.540198+0000","last_unstale":"2024-03-29T04:10:05.560063+0000","last_undegraded":"2024-03-29T04:10:05.560063+0000","last_fullsized":"2024-03-29T04:10:05.560063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:01.925223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556468+0000","last_change":"2024-03-29T04:10:04.644258+0000","last_active":"2024-03-29T04:10:05.556468+0000","last_peered":"2024-03-29T04:10:05.556468+0000","last_clean":"2024-03-29T04:10:05.556468+0000","last_became_active":"2024-03-29T04:10:04.643890+0000","last_became_peered":"2024-03-29T04:10:04.643890+0000","last_unstale":"2024-03-29T04:10:05.556468+0000","last_undegraded":"2024-03-29T04:10:05.556468+0000","last_fullsized":"2024-03-29T04:10:05.556468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:46:14.769141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911624+0000","last_change":"2024-03-29T04:10:00.549814+0000","last_active":"2024-03-29T04:10:05.911624+0000","last_peered":"2024-03-29T04:10:05.911624+0000","last_clean":"2024-03-29T04:10:05.911624+0000","last_became_active":"2024-03-29T04:10:00.549571+0000","last_became_peered":"2024-03-29T04:10:00.549571+0000","last_unstale":"2024-03-29T04:10:05.911624+0000","last_undegraded":"2024-03-29T04:10:05.911624+0000","last_fullsized":"2024-03-29T04:10:05.911624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:00:51.320824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557978+0000","last_change":"2024-03-29T04:09:59.538979+0000","last_active":"2024-03-29T04:10:05.557978+0000","last_peered":"2024-03-29T04:10:05.557978+0000","last_clean":"2024-03-29T04:10:05.557978+0000","last_became_active":"2024-03-29T04:09:59.538707+0000","last_became_peered":"2024-03-29T04:09:59.538707+0000","last_unstale":"2024-03-29T04:10:05.557978+0000","last_undegraded":"2024-03-29T04:10:05.557978+0000","last_fullsized":"2024-03-29T04:10:05.557978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:07:35.127071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556613+0000","last_change":"2024-03-29T04:10:04.574569+0000","last_active":"2024-03-29T04:10:05.556613+0000","last_peered":"2024-03-29T04:10:05.556613+0000","last_clean":"2024-03-29T04:10:05.556613+0000","last_became_active":"2024-03-29T04:10:04.574048+0000","last_became_peered":"2024-03-29T04:10:04.574048+0000","last_unstale":"2024-03-29T04:10:05.556613+0000","last_undegraded":"2024-03-29T04:10:05.556613+0000","last_fullsized":"2024-03-29T04:10:05.556613+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:33.266010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910658+0000","last_change":"2024-03-29T04:10:00.549205+0000","last_active":"2024-03-29T04:10:05.910658+0000","last_peered":"2024-03-29T04:10:05.910658+0000","last_clean":"2024-03-29T04:10:05.910658+0000","last_became_active":"2024-03-29T04:10:00.548863+0000","last_became_peered":"2024-03-29T04:10:00.548863+0000","last_unstale":"2024-03-29T04:10:05.910658+0000","last_undegraded":"2024-03-29T04:10:05.910658+0000","last_fullsized":"2024-03-29T04:10:05.910658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:30.096622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.559751+0000","last_change":"2024-03-29T04:09:59.538016+0000","last_active":"2024-03-29T04:10:05.559751+0000","last_peered":"2024-03-29T04:10:05.559751+0000","last_clean":"2024-03-29T04:10:05.559751+0000","last_became_active":"2024-03-29T04:09:59.537671+0000","last_became_peered":"2024-03-29T04:09:59.537671+0000","last_unstale":"2024-03-29T04:10:05.559751+0000","last_undegraded":"2024-03-29T04:10:05.559751+0000","last_fullsized":"2024-03-29T04:10:05.559751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:25:00.672418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912466+0000","last_change":"2024-03-29T04:10:04.574244+0000","last_active":"2024-03-29T04:10:05.912466+0000","last_peered":"2024-03-29T04:10:05.912466+0000","last_clean":"2024-03-29T04:10:05.912466+0000","last_became_active":"2024-03-29T04:10:04.573865+0000","last_became_peered":"2024-03-29T04:10:04.573865+0000","last_unstale":"2024-03-29T04:10:05.912466+0000","last_undegraded":"2024-03-29T04:10:05.912466+0000","last_fullsized":"2024-03-29T04:10:05.912466+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:23:51.029457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910228+0000","last_change":"2024-03-29T04:10:00.547804+0000","last_active":"2024-03-29T04:10:05.910228+0000","last_peered":"2024-03-29T04:10:05.910228+0000","last_clean":"2024-03-29T04:10:05.910228+0000","last_became_active":"2024-03-29T04:10:00.547500+0000","last_became_peered":"2024-03-29T04:10:00.547500+0000","last_unstale":"2024-03-29T04:10:05.910228+0000","last_undegraded":"2024-03-29T04:10:05.910228+0000","last_fullsized":"2024-03-29T04:10:05.910228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:52:12.957170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910669+0000","last_change":"2024-03-29T04:09:59.538286+0000","last_active":"2024-03-29T04:10:05.910669+0000","last_peered":"2024-03-29T04:10:05.910669+0000","last_clean":"2024-03-29T04:10:05.910669+0000","last_became_active":"2024-03-29T04:09:59.537970+0000","last_became_peered":"2024-03-29T04:09:59.537970+0000","last_unstale":"2024-03-29T04:10:05.910669+0000","last_undegraded":"2024-03-29T04:10:05.910669+0000","last_fullsized":"2024-03-29T04:10:05.910669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:05.307537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911473+0000","last_change":"2024-03-29T04:10:04.647055+0000","last_active":"2024-03-29T04:10:05.911473+0000","last_peered":"2024-03-29T04:10:05.911473+0000","last_clean":"2024-03-29T04:10:05.911473+0000","last_became_active":"2024-03-29T04:10:04.646604+0000","last_became_peered":"2024-03-29T04:10:04.646604+0000","last_unstale":"2024-03-29T04:10:05.911473+0000","last_undegraded":"2024-03-29T04:10:05.911473+0000","last_fullsized":"2024-03-29T04:10:05.911473+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:32:59.580503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911224+0000","last_change":"2024-03-29T04:10:00.548354+0000","last_active":"2024-03-29T04:10:05.911224+0000","last_peered":"2024-03-29T04:10:05.911224+0000","last_clean":"2024-03-29T04:10:05.911224+0000","last_became_active":"2024-03-29T04:10:00.548084+0000","last_became_peered":"2024-03-29T04:10:00.548084+0000","last_unstale":"2024-03-29T04:10:05.911224+0000","last_undegraded":"2024-03-29T04:10:05.911224+0000","last_fullsized":"2024-03-29T04:10:05.911224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:02.966428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557470+0000","last_change":"2024-03-29T04:09:59.538823+0000","last_active":"2024-03-29T04:10:05.557470+0000","last_peered":"2024-03-29T04:10:05.557470+0000","last_clean":"2024-03-29T04:10:05.557470+0000","last_became_active":"2024-03-29T04:09:59.538570+0000","last_became_peered":"2024-03-29T04:09:59.538570+0000","last_unstale":"2024-03-29T04:10:05.557470+0000","last_undegraded":"2024-03-29T04:10:05.557470+0000","last_fullsized":"2024-03-29T04:10:05.557470+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:02.879344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911403+0000","last_change":"2024-03-29T04:10:04.644206+0000","last_active":"2024-03-29T04:10:05.911403+0000","last_peered":"2024-03-29T04:10:05.911403+0000","last_clean":"2024-03-29T04:10:05.911403+0000","last_became_active":"2024-03-29T04:10:04.643690+0000","last_became_peered":"2024-03-29T04:10:04.643690+0000","last_unstale":"2024-03-29T04:10:05.911403+0000","last_undegraded":"2024-03-29T04:10:05.911403+0000","last_fullsized":"2024-03-29T04:10:05.911403+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:10:07.163684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910609+0000","last_change":"2024-03-29T04:10:00.541166+0000","last_active":"2024-03-29T04:10:05.910609+0000","last_peered":"2024-03-29T04:10:05.910609+0000","last_clean":"2024-03-29T04:10:05.910609+0000","last_became_active":"2024-03-29T04:10:00.540895+0000","last_became_peered":"2024-03-29T04:10:00.540895+0000","last_unstale":"2024-03-29T04:10:05.910609+0000","last_undegraded":"2024-03-29T04:10:05.910609+0000","last_fullsized":"2024-03-29T04:10:05.910609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:21:11.329145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911540+0000","last_change":"2024-03-29T04:09:59.538433+0000","last_active":"2024-03-29T04:10:05.911540+0000","last_peered":"2024-03-29T04:10:05.911540+0000","last_clean":"2024-03-29T04:10:05.911540+0000","last_became_active":"2024-03-29T04:09:59.538039+0000","last_became_peered":"2024-03-29T04:09:59.538039+0000","last_unstale":"2024-03-29T04:10:05.911540+0000","last_undegraded":"2024-03-29T04:10:05.911540+0000","last_fullsized":"2024-03-29T04:10:05.911540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:45:51.875624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557426+0000","last_change":"2024-03-29T04:10:04.647019+0000","last_active":"2024-03-29T04:10:05.557426+0000","last_peered":"2024-03-29T04:10:05.557426+0000","last_clean":"2024-03-29T04:10:05.557426+0000","last_became_active":"2024-03-29T04:10:04.646642+0000","last_became_peered":"2024-03-29T04:10:04.646642+0000","last_unstale":"2024-03-29T04:10:05.557426+0000","last_undegraded":"2024-03-29T04:10:05.557426+0000","last_fullsized":"2024-03-29T04:10:05.557426+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:07:56.606322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911280+0000","last_change":"2024-03-29T04:10:00.544134+0000","last_active":"2024-03-29T04:10:05.911280+0000","last_peered":"2024-03-29T04:10:05.911280+0000","last_clean":"2024-03-29T04:10:05.911280+0000","last_became_active":"2024-03-29T04:10:00.543534+0000","last_became_peered":"2024-03-29T04:10:00.543534+0000","last_unstale":"2024-03-29T04:10:05.911280+0000","last_undegraded":"2024-03-29T04:10:05.911280+0000","last_fullsized":"2024-03-29T04:10:05.911280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:26:04.629712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:10.817289+0000","last_change":"2024-03-29T04:09:59.544458+0000","last_active":"2024-03-29T04:10:10.817289+0000","last_peered":"2024-03-29T04:10:10.817289+0000","last_clean":"2024-03-29T04:10:10.817289+0000","last_became_active":"2024-03-29T04:09:59.544137+0000","last_became_peered":"2024-03-29T04:09:59.544137+0000","last_unstale":"2024-03-29T04:10:10.817289+0000","last_undegraded":"2024-03-29T04:10:10.817289+0000","last_fullsized":"2024-03-29T04:10:10.817289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:02.879344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912395+0000","last_change":"2024-03-29T04:10:04.576867+0000","last_active":"2024-03-29T04:10:05.912395+0000","last_peered":"2024-03-29T04:10:05.912395+0000","last_clean":"2024-03-29T04:10:05.912395+0000","last_became_active":"2024-03-29T04:10:04.576416+0000","last_became_peered":"2024-03-29T04:10:04.576416+0000","last_unstale":"2024-03-29T04:10:05.912395+0000","last_undegraded":"2024-03-29T04:10:05.912395+0000","last_fullsized":"2024-03-29T04:10:05.912395+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:34:10.400206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.558230+0000","last_change":"2024-03-29T04:10:00.547277+0000","last_active":"2024-03-29T04:10:05.558230+0000","last_peered":"2024-03-29T04:10:05.558230+0000","last_clean":"2024-03-29T04:10:05.558230+0000","last_became_active":"2024-03-29T04:10:00.547024+0000","last_became_peered":"2024-03-29T04:10:00.547024+0000","last_unstale":"2024-03-29T04:10:05.558230+0000","last_undegraded":"2024-03-29T04:10:05.558230+0000","last_fullsized":"2024-03-29T04:10:05.558230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:58:11.125964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911813+0000","last_change":"2024-03-29T04:09:59.541483+0000","last_active":"2024-03-29T04:10:05.911813+0000","last_peered":"2024-03-29T04:10:05.911813+0000","last_clean":"2024-03-29T04:10:05.911813+0000","last_became_active":"2024-03-29T04:09:59.541196+0000","last_became_peered":"2024-03-29T04:09:59.541196+0000","last_unstale":"2024-03-29T04:10:05.911813+0000","last_undegraded":"2024-03-29T04:10:05.911813+0000","last_fullsized":"2024-03-29T04:10:05.911813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:35:01.484589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557918+0000","last_change":"2024-03-29T04:10:04.574437+0000","last_active":"2024-03-29T04:10:05.557918+0000","last_peered":"2024-03-29T04:10:05.557918+0000","last_clean":"2024-03-29T04:10:05.557918+0000","last_became_active":"2024-03-29T04:10:04.573898+0000","last_became_peered":"2024-03-29T04:10:04.573898+0000","last_unstale":"2024-03-29T04:10:05.557918+0000","last_undegraded":"2024-03-29T04:10:05.557918+0000","last_fullsized":"2024-03-29T04:10:05.557918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:50.836630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911733+0000","last_change":"2024-03-29T04:10:00.536536+0000","last_active":"2024-03-29T04:10:05.911733+0000","last_peered":"2024-03-29T04:10:05.911733+0000","last_clean":"2024-03-29T04:10:05.911733+0000","last_became_active":"2024-03-29T04:10:00.536254+0000","last_became_peered":"2024-03-29T04:10:00.536254+0000","last_unstale":"2024-03-29T04:10:05.911733+0000","last_undegraded":"2024-03-29T04:10:05.911733+0000","last_fullsized":"2024-03-29T04:10:05.911733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:48.213845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557997+0000","last_change":"2024-03-29T04:09:59.541406+0000","last_active":"2024-03-29T04:10:05.557997+0000","last_peered":"2024-03-29T04:10:05.557997+0000","last_clean":"2024-03-29T04:10:05.557997+0000","last_became_active":"2024-03-29T04:09:59.541101+0000","last_became_peered":"2024-03-29T04:09:59.541101+0000","last_unstale":"2024-03-29T04:10:05.557997+0000","last_undegraded":"2024-03-29T04:10:05.557997+0000","last_fullsized":"2024-03-29T04:10:05.557997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:25:06.123311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910592+0000","last_change":"2024-03-29T04:10:04.577680+0000","last_active":"2024-03-29T04:10:05.910592+0000","last_peered":"2024-03-29T04:10:05.910592+0000","last_clean":"2024-03-29T04:10:05.910592+0000","last_became_active":"2024-03-29T04:10:04.577291+0000","last_became_peered":"2024-03-29T04:10:04.577291+0000","last_unstale":"2024-03-29T04:10:05.910592+0000","last_undegraded":"2024-03-29T04:10:05.910592+0000","last_fullsized":"2024-03-29T04:10:05.910592+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:46:14.769141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557898+0000","last_change":"2024-03-29T04:10:00.542773+0000","last_active":"2024-03-29T04:10:05.557898+0000","last_peered":"2024-03-29T04:10:05.557898+0000","last_clean":"2024-03-29T04:10:05.557898+0000","last_became_active":"2024-03-29T04:10:00.542081+0000","last_became_peered":"2024-03-29T04:10:00.542081+0000","last_unstale":"2024-03-29T04:10:05.557898+0000","last_undegraded":"2024-03-29T04:10:05.557898+0000","last_fullsized":"2024-03-29T04:10:05.557898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:26:04.629712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910800+0000","last_change":"2024-03-29T04:09:59.543623+0000","last_active":"2024-03-29T04:10:05.910800+0000","last_peered":"2024-03-29T04:10:05.910800+0000","last_clean":"2024-03-29T04:10:05.910800+0000","last_became_active":"2024-03-29T04:09:59.542122+0000","last_became_peered":"2024-03-29T04:09:59.542122+0000","last_unstale":"2024-03-29T04:10:05.910800+0000","last_undegraded":"2024-03-29T04:10:05.910800+0000","last_fullsized":"2024-03-29T04:10:05.910800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:01.925223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911195+0000","last_change":"2024-03-29T04:10:04.648725+0000","last_active":"2024-03-29T04:10:05.911195+0000","last_peered":"2024-03-29T04:10:05.911195+0000","last_clean":"2024-03-29T04:10:05.911195+0000","last_became_active":"2024-03-29T04:10:04.648125+0000","last_became_peered":"2024-03-29T04:10:04.648125+0000","last_unstale":"2024-03-29T04:10:05.911195+0000","last_undegraded":"2024-03-29T04:10:05.911195+0000","last_fullsized":"2024-03-29T04:10:05.911195+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:16:31.571145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912466+0000","last_change":"2024-03-29T04:10:04.646516+0000","last_active":"2024-03-29T04:10:05.912466+0000","last_peered":"2024-03-29T04:10:05.912466+0000","last_clean":"2024-03-29T04:10:05.912466+0000","last_became_active":"2024-03-29T04:10:04.646118+0000","last_became_peered":"2024-03-29T04:10:04.646118+0000","last_unstale":"2024-03-29T04:10:05.912466+0000","last_undegraded":"2024-03-29T04:10:05.912466+0000","last_fullsized":"2024-03-29T04:10:05.912466+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:17.034970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.909860+0000","last_change":"2024-03-29T04:10:00.542208+0000","last_active":"2024-03-29T04:10:05.909860+0000","last_peered":"2024-03-29T04:10:05.909860+0000","last_clean":"2024-03-29T04:10:05.909860+0000","last_became_active":"2024-03-29T04:10:00.541927+0000","last_became_peered":"2024-03-29T04:10:00.541927+0000","last_unstale":"2024-03-29T04:10:05.909860+0000","last_undegraded":"2024-03-29T04:10:05.909860+0000","last_fullsized":"2024-03-29T04:10:05.909860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:16:02.331457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556546+0000","last_change":"2024-03-29T04:09:59.539257+0000","last_active":"2024-03-29T04:10:05.556546+0000","last_peered":"2024-03-29T04:10:05.556546+0000","last_clean":"2024-03-29T04:10:05.556546+0000","last_became_active":"2024-03-29T04:09:59.538298+0000","last_became_peered":"2024-03-29T04:09:59.538298+0000","last_unstale":"2024-03-29T04:10:05.556546+0000","last_undegraded":"2024-03-29T04:10:05.556546+0000","last_fullsized":"2024-03-29T04:10:05.556546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:33:50.904931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911024+0000","last_change":"2024-03-29T04:10:04.575619+0000","last_active":"2024-03-29T04:10:05.911024+0000","last_peered":"2024-03-29T04:10:05.911024+0000","last_clean":"2024-03-29T04:10:05.911024+0000","last_became_active":"2024-03-29T04:10:04.575138+0000","last_became_peered":"2024-03-29T04:10:04.575138+0000","last_unstale":"2024-03-29T04:10:05.911024+0000","last_undegraded":"2024-03-29T04:10:05.911024+0000","last_fullsized":"2024-03-29T04:10:05.911024+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:06.998903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910644+0000","last_change":"2024-03-29T04:10:00.543853+0000","last_active":"2024-03-29T04:10:05.910644+0000","last_peered":"2024-03-29T04:10:05.910644+0000","last_clean":"2024-03-29T04:10:05.910644+0000","last_became_active":"2024-03-29T04:10:00.543030+0000","last_became_peered":"2024-03-29T04:10:00.543030+0000","last_unstale":"2024-03-29T04:10:05.910644+0000","last_undegraded":"2024-03-29T04:10:05.910644+0000","last_fullsized":"2024-03-29T04:10:05.910644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:29.233534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911422+0000","last_change":"2024-03-29T04:09:59.537737+0000","last_active":"2024-03-29T04:10:05.911422+0000","last_peered":"2024-03-29T04:10:05.911422+0000","last_clean":"2024-03-29T04:10:05.911422+0000","last_became_active":"2024-03-29T04:09:59.537453+0000","last_became_peered":"2024-03-29T04:09:59.537453+0000","last_unstale":"2024-03-29T04:10:05.911422+0000","last_undegraded":"2024-03-29T04:10:05.911422+0000","last_fullsized":"2024-03-29T04:10:05.911422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:05.307537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910688+0000","last_change":"2024-03-29T04:09:59.538269+0000","last_active":"2024-03-29T04:10:05.910688+0000","last_peered":"2024-03-29T04:10:05.910688+0000","last_clean":"2024-03-29T04:10:05.910688+0000","last_became_active":"2024-03-29T04:09:59.537365+0000","last_became_peered":"2024-03-29T04:09:59.537365+0000","last_unstale":"2024-03-29T04:10:05.910688+0000","last_undegraded":"2024-03-29T04:10:05.910688+0000","last_fullsized":"2024-03-29T04:10:05.910688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:33:50.904931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911456+0000","last_change":"2024-03-29T04:10:00.532121+0000","last_active":"2024-03-29T04:10:05.911456+0000","last_peered":"2024-03-29T04:10:05.911456+0000","last_clean":"2024-03-29T04:10:05.911456+0000","last_became_active":"2024-03-29T04:10:00.531664+0000","last_became_peered":"2024-03-29T04:10:00.531664+0000","last_unstale":"2024-03-29T04:10:05.911456+0000","last_undegraded":"2024-03-29T04:10:05.911456+0000","last_fullsized":"2024-03-29T04:10:05.911456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:13:32.378332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.560130+0000","last_change":"2024-03-29T04:10:04.574558+0000","last_active":"2024-03-29T04:10:05.560130+0000","last_peered":"2024-03-29T04:10:05.560130+0000","last_clean":"2024-03-29T04:10:05.560130+0000","last_became_active":"2024-03-29T04:10:04.574156+0000","last_became_peered":"2024-03-29T04:10:04.574156+0000","last_unstale":"2024-03-29T04:10:05.560130+0000","last_undegraded":"2024-03-29T04:10:05.560130+0000","last_fullsized":"2024-03-29T04:10:05.560130+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:02:36.577244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910098+0000","last_change":"2024-03-29T04:09:59.536444+0000","last_active":"2024-03-29T04:10:05.910098+0000","last_peered":"2024-03-29T04:10:05.910098+0000","last_clean":"2024-03-29T04:10:05.910098+0000","last_became_active":"2024-03-29T04:09:59.536165+0000","last_became_peered":"2024-03-29T04:09:59.536165+0000","last_unstale":"2024-03-29T04:10:05.910098+0000","last_undegraded":"2024-03-29T04:10:05.910098+0000","last_fullsized":"2024-03-29T04:10:05.910098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:37:46.716233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.558124+0000","last_change":"2024-03-29T04:10:00.544569+0000","last_active":"2024-03-29T04:10:05.558124+0000","last_peered":"2024-03-29T04:10:05.558124+0000","last_clean":"2024-03-29T04:10:05.558124+0000","last_became_active":"2024-03-29T04:10:00.544290+0000","last_became_peered":"2024-03-29T04:10:00.544290+0000","last_unstale":"2024-03-29T04:10:05.558124+0000","last_undegraded":"2024-03-29T04:10:05.558124+0000","last_fullsized":"2024-03-29T04:10:05.558124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:02.929633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910585+0000","last_change":"2024-03-29T04:10:04.580538+0000","last_active":"2024-03-29T04:10:05.910585+0000","last_peered":"2024-03-29T04:10:05.910585+0000","last_clean":"2024-03-29T04:10:05.910585+0000","last_became_active":"2024-03-29T04:10:04.580221+0000","last_became_peered":"2024-03-29T04:10:04.580221+0000","last_unstale":"2024-03-29T04:10:05.910585+0000","last_undegraded":"2024-03-29T04:10:05.910585+0000","last_fullsized":"2024-03-29T04:10:05.910585+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:01:23.958242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910087+0000","last_change":"2024-03-29T04:10:00.546325+0000","last_active":"2024-03-29T04:10:05.910087+0000","last_peered":"2024-03-29T04:10:05.910087+0000","last_clean":"2024-03-29T04:10:05.910087+0000","last_became_active":"2024-03-29T04:10:00.545988+0000","last_became_peered":"2024-03-29T04:10:00.545988+0000","last_unstale":"2024-03-29T04:10:05.910087+0000","last_undegraded":"2024-03-29T04:10:05.910087+0000","last_fullsized":"2024-03-29T04:10:05.910087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:16:02.331457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.559836+0000","last_change":"2024-03-29T04:10:00.541221+0000","last_active":"2024-03-29T04:10:05.559836+0000","last_peered":"2024-03-29T04:10:05.559836+0000","last_clean":"2024-03-29T04:10:05.559836+0000","last_became_active":"2024-03-29T04:10:00.540946+0000","last_became_peered":"2024-03-29T04:10:00.540946+0000","last_unstale":"2024-03-29T04:10:05.559836+0000","last_undegraded":"2024-03-29T04:10:05.559836+0000","last_fullsized":"2024-03-29T04:10:05.559836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:35:02.488999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557358+0000","last_change":"2024-03-29T04:10:00.546304+0000","last_active":"2024-03-29T04:10:05.557358+0000","last_peered":"2024-03-29T04:10:05.557358+0000","last_clean":"2024-03-29T04:10:05.557358+0000","last_became_active":"2024-03-29T04:10:00.545529+0000","last_became_peered":"2024-03-29T04:10:00.545529+0000","last_unstale":"2024-03-29T04:10:05.557358+0000","last_undegraded":"2024-03-29T04:10:05.557358+0000","last_fullsized":"2024-03-29T04:10:05.557358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:44:13.128618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911371+0000","last_change":"2024-03-29T04:10:00.542033+0000","last_active":"2024-03-29T04:10:05.911371+0000","last_peered":"2024-03-29T04:10:05.911371+0000","last_clean":"2024-03-29T04:10:05.911371+0000","last_became_active":"2024-03-29T04:10:00.541764+0000","last_became_peered":"2024-03-29T04:10:00.541764+0000","last_unstale":"2024-03-29T04:10:05.911371+0000","last_undegraded":"2024-03-29T04:10:05.911371+0000","last_fullsized":"2024-03-29T04:10:05.911371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:46.707085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911616+0000","last_change":"2024-03-29T04:10:00.543634+0000","last_active":"2024-03-29T04:10:05.911616+0000","last_peered":"2024-03-29T04:10:05.911616+0000","last_clean":"2024-03-29T04:10:05.911616+0000","last_became_active":"2024-03-29T04:10:00.543343+0000","last_became_peered":"2024-03-29T04:10:00.543343+0000","last_unstale":"2024-03-29T04:10:05.911616+0000","last_undegraded":"2024-03-29T04:10:05.911616+0000","last_fullsized":"2024-03-29T04:10:05.911616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:35:00.452782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911632+0000","last_change":"2024-03-29T04:10:00.554863+0000","last_active":"2024-03-29T04:10:05.911632+0000","last_peered":"2024-03-29T04:10:05.911632+0000","last_clean":"2024-03-29T04:10:05.911632+0000","last_became_active":"2024-03-29T04:10:00.554560+0000","last_became_peered":"2024-03-29T04:10:00.554560+0000","last_unstale":"2024-03-29T04:10:05.911632+0000","last_undegraded":"2024-03-29T04:10:05.911632+0000","last_fullsized":"2024-03-29T04:10:05.911632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:29.233534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910973+0000","last_change":"2024-03-29T04:10:00.541999+0000","last_active":"2024-03-29T04:10:05.910973+0000","last_peered":"2024-03-29T04:10:05.910973+0000","last_clean":"2024-03-29T04:10:05.910973+0000","last_became_active":"2024-03-29T04:10:00.541699+0000","last_became_peered":"2024-03-29T04:10:00.541699+0000","last_unstale":"2024-03-29T04:10:05.910973+0000","last_undegraded":"2024-03-29T04:10:05.910973+0000","last_fullsized":"2024-03-29T04:10:05.910973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:48.213845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911525+0000","last_change":"2024-03-29T04:10:00.537506+0000","last_active":"2024-03-29T04:10:05.911525+0000","last_peered":"2024-03-29T04:10:05.911525+0000","last_clean":"2024-03-29T04:10:05.911525+0000","last_became_active":"2024-03-29T04:10:00.537239+0000","last_became_peered":"2024-03-29T04:10:00.537239+0000","last_unstale":"2024-03-29T04:10:05.911525+0000","last_undegraded":"2024-03-29T04:10:05.911525+0000","last_fullsized":"2024-03-29T04:10:05.911525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:30.096622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556535+0000","last_change":"2024-03-29T04:10:00.544443+0000","last_active":"2024-03-29T04:10:05.556535+0000","last_peered":"2024-03-29T04:10:05.556535+0000","last_clean":"2024-03-29T04:10:05.556535+0000","last_became_active":"2024-03-29T04:10:00.543427+0000","last_became_peered":"2024-03-29T04:10:00.543427+0000","last_unstale":"2024-03-29T04:10:05.556535+0000","last_undegraded":"2024-03-29T04:10:05.556535+0000","last_fullsized":"2024-03-29T04:10:05.556535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:48.213845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911292+0000","last_change":"2024-03-29T04:10:00.538050+0000","last_active":"2024-03-29T04:10:05.911292+0000","last_peered":"2024-03-29T04:10:05.911292+0000","last_clean":"2024-03-29T04:10:05.911292+0000","last_became_active":"2024-03-29T04:10:00.537771+0000","last_became_peered":"2024-03-29T04:10:00.537771+0000","last_unstale":"2024-03-29T04:10:05.911292+0000","last_undegraded":"2024-03-29T04:10:05.911292+0000","last_fullsized":"2024-03-29T04:10:05.911292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:21:11.329145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557260+0000","last_change":"2024-03-29T04:10:00.546534+0000","last_active":"2024-03-29T04:10:05.557260+0000","last_peered":"2024-03-29T04:10:05.557260+0000","last_clean":"2024-03-29T04:10:05.557260+0000","last_became_active":"2024-03-29T04:10:00.546028+0000","last_became_peered":"2024-03-29T04:10:00.546028+0000","last_unstale":"2024-03-29T04:10:05.557260+0000","last_undegraded":"2024-03-29T04:10:05.557260+0000","last_fullsized":"2024-03-29T04:10:05.557260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:23:46.996981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911142+0000","last_change":"2024-03-29T04:10:00.539866+0000","last_active":"2024-03-29T04:10:05.911142+0000","last_peered":"2024-03-29T04:10:05.911142+0000","last_clean":"2024-03-29T04:10:05.911142+0000","last_became_active":"2024-03-29T04:10:00.539562+0000","last_became_peered":"2024-03-29T04:10:00.539562+0000","last_unstale":"2024-03-29T04:10:05.911142+0000","last_undegraded":"2024-03-29T04:10:05.911142+0000","last_fullsized":"2024-03-29T04:10:05.911142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:52:12.957170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910850+0000","last_change":"2024-03-29T04:10:00.542361+0000","last_active":"2024-03-29T04:10:05.910850+0000","last_peered":"2024-03-29T04:10:05.910850+0000","last_clean":"2024-03-29T04:10:05.910850+0000","last_became_active":"2024-03-29T04:10:00.541625+0000","last_became_peered":"2024-03-29T04:10:00.541625+0000","last_unstale":"2024-03-29T04:10:05.910850+0000","last_undegraded":"2024-03-29T04:10:05.910850+0000","last_fullsized":"2024-03-29T04:10:05.910850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:46.707085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911557+0000","last_change":"2024-03-29T04:10:00.539117+0000","last_active":"2024-03-29T04:10:05.911557+0000","last_peered":"2024-03-29T04:10:05.911557+0000","last_clean":"2024-03-29T04:10:05.911557+0000","last_became_active":"2024-03-29T04:10:00.538833+0000","last_became_peered":"2024-03-29T04:10:00.538833+0000","last_unstale":"2024-03-29T04:10:05.911557+0000","last_undegraded":"2024-03-29T04:10:05.911557+0000","last_fullsized":"2024-03-29T04:10:05.911557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:13.002495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.560038+0000","last_change":"2024-03-29T04:10:00.551402+0000","last_active":"2024-03-29T04:10:05.560038+0000","last_peered":"2024-03-29T04:10:05.560038+0000","last_clean":"2024-03-29T04:10:05.560038+0000","last_became_active":"2024-03-29T04:10:00.551092+0000","last_became_peered":"2024-03-29T04:10:00.551092+0000","last_unstale":"2024-03-29T04:10:05.560038+0000","last_undegraded":"2024-03-29T04:10:05.560038+0000","last_fullsized":"2024-03-29T04:10:05.560038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:58:11.125964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.558449+0000","last_change":"2024-03-29T04:10:00.542777+0000","last_active":"2024-03-29T04:10:05.558449+0000","last_peered":"2024-03-29T04:10:05.558449+0000","last_clean":"2024-03-29T04:10:05.558449+0000","last_became_active":"2024-03-29T04:10:00.542241+0000","last_became_peered":"2024-03-29T04:10:00.542241+0000","last_unstale":"2024-03-29T04:10:05.558449+0000","last_undegraded":"2024-03-29T04:10:05.558449+0000","last_fullsized":"2024-03-29T04:10:05.558449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:46.804154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556402+0000","last_change":"2024-03-29T04:10:00.545811+0000","last_active":"2024-03-29T04:10:05.556402+0000","last_peered":"2024-03-29T04:10:05.556402+0000","last_clean":"2024-03-29T04:10:05.556402+0000","last_became_active":"2024-03-29T04:10:00.545501+0000","last_became_peered":"2024-03-29T04:10:00.545501+0000","last_unstale":"2024-03-29T04:10:05.556402+0000","last_undegraded":"2024-03-29T04:10:05.556402+0000","last_fullsized":"2024-03-29T04:10:05.556402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:29.233534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.909960+0000","last_change":"2024-03-29T04:10:00.545471+0000","last_active":"2024-03-29T04:10:05.909960+0000","last_peered":"2024-03-29T04:10:05.909960+0000","last_clean":"2024-03-29T04:10:05.909960+0000","last_became_active":"2024-03-29T04:10:00.545173+0000","last_became_peered":"2024-03-29T04:10:00.545173+0000","last_unstale":"2024-03-29T04:10:05.909960+0000","last_undegraded":"2024-03-29T04:10:05.909960+0000","last_fullsized":"2024-03-29T04:10:05.909960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:13:32.378332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910520+0000","last_change":"2024-03-29T04:10:00.545485+0000","last_active":"2024-03-29T04:10:05.910520+0000","last_peered":"2024-03-29T04:10:05.910520+0000","last_clean":"2024-03-29T04:10:05.910520+0000","last_became_active":"2024-03-29T04:10:00.545139+0000","last_became_peered":"2024-03-29T04:10:00.545139+0000","last_unstale":"2024-03-29T04:10:05.910520+0000","last_undegraded":"2024-03-29T04:10:05.910520+0000","last_fullsized":"2024-03-29T04:10:05.910520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:14:59.937144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.909924+0000","last_change":"2024-03-29T04:10:00.550441+0000","last_active":"2024-03-29T04:10:05.909924+0000","last_peered":"2024-03-29T04:10:05.909924+0000","last_clean":"2024-03-29T04:10:05.909924+0000","last_became_active":"2024-03-29T04:10:00.550150+0000","last_became_peered":"2024-03-29T04:10:00.550150+0000","last_unstale":"2024-03-29T04:10:05.909924+0000","last_undegraded":"2024-03-29T04:10:05.909924+0000","last_fullsized":"2024-03-29T04:10:05.909924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:52:12.957170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.558164+0000","last_change":"2024-03-29T04:10:00.540368+0000","last_active":"2024-03-29T04:10:05.558164+0000","last_peered":"2024-03-29T04:10:05.558164+0000","last_clean":"2024-03-29T04:10:05.558164+0000","last_became_active":"2024-03-29T04:10:00.538360+0000","last_became_peered":"2024-03-29T04:10:00.538360+0000","last_unstale":"2024-03-29T04:10:05.558164+0000","last_undegraded":"2024-03-29T04:10:05.558164+0000","last_fullsized":"2024-03-29T04:10:05.558164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:35:02.488999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911011+0000","last_change":"2024-03-29T04:10:00.550531+0000","last_active":"2024-03-29T04:10:05.911011+0000","last_peered":"2024-03-29T04:10:05.911011+0000","last_clean":"2024-03-29T04:10:05.911011+0000","last_became_active":"2024-03-29T04:10:00.550079+0000","last_became_peered":"2024-03-29T04:10:00.550079+0000","last_unstale":"2024-03-29T04:10:05.911011+0000","last_undegraded":"2024-03-29T04:10:05.911011+0000","last_fullsized":"2024-03-29T04:10:05.911011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:13:32.378332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910949+0000","last_change":"2024-03-29T04:10:00.550387+0000","last_active":"2024-03-29T04:10:05.910949+0000","last_peered":"2024-03-29T04:10:05.910949+0000","last_clean":"2024-03-29T04:10:05.910949+0000","last_became_active":"2024-03-29T04:10:00.550063+0000","last_became_peered":"2024-03-29T04:10:00.550063+0000","last_unstale":"2024-03-29T04:10:05.910949+0000","last_undegraded":"2024-03-29T04:10:05.910949+0000","last_fullsized":"2024-03-29T04:10:05.910949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:29.233534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911804+0000","last_change":"2024-03-29T04:10:00.541680+0000","last_active":"2024-03-29T04:10:05.911804+0000","last_peered":"2024-03-29T04:10:05.911804+0000","last_clean":"2024-03-29T04:10:05.911804+0000","last_became_active":"2024-03-29T04:10:00.541354+0000","last_became_peered":"2024-03-29T04:10:00.541354+0000","last_unstale":"2024-03-29T04:10:05.911804+0000","last_undegraded":"2024-03-29T04:10:05.911804+0000","last_fullsized":"2024-03-29T04:10:05.911804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:44:13.128618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910384+0000","last_change":"2024-03-29T04:10:00.544081+0000","last_active":"2024-03-29T04:10:05.910384+0000","last_peered":"2024-03-29T04:10:05.910384+0000","last_clean":"2024-03-29T04:10:05.910384+0000","last_became_active":"2024-03-29T04:10:00.543353+0000","last_became_peered":"2024-03-29T04:10:00.543353+0000","last_unstale":"2024-03-29T04:10:05.910384+0000","last_undegraded":"2024-03-29T04:10:05.910384+0000","last_fullsized":"2024-03-29T04:10:05.910384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:13:32.378332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910548+0000","last_change":"2024-03-29T04:10:00.543517+0000","last_active":"2024-03-29T04:10:05.910548+0000","last_peered":"2024-03-29T04:10:05.910548+0000","last_clean":"2024-03-29T04:10:05.910548+0000","last_became_active":"2024-03-29T04:10:00.542854+0000","last_became_peered":"2024-03-29T04:10:00.542854+0000","last_unstale":"2024-03-29T04:10:05.910548+0000","last_undegraded":"2024-03-29T04:10:05.910548+0000","last_fullsized":"2024-03-29T04:10:05.910548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:35:00.452782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557167+0000","last_change":"2024-03-29T04:10:00.546548+0000","last_active":"2024-03-29T04:10:05.557167+0000","last_peered":"2024-03-29T04:10:05.557167+0000","last_clean":"2024-03-29T04:10:05.557167+0000","last_became_active":"2024-03-29T04:10:00.546267+0000","last_became_peered":"2024-03-29T04:10:00.546267+0000","last_unstale":"2024-03-29T04:10:05.557167+0000","last_undegraded":"2024-03-29T04:10:05.557167+0000","last_fullsized":"2024-03-29T04:10:05.557167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:26:10.380318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911101+0000","last_change":"2024-03-29T04:10:00.539796+0000","last_active":"2024-03-29T04:10:05.911101+0000","last_peered":"2024-03-29T04:10:05.911101+0000","last_clean":"2024-03-29T04:10:05.911101+0000","last_became_active":"2024-03-29T04:10:00.539532+0000","last_became_peered":"2024-03-29T04:10:00.539532+0000","last_unstale":"2024-03-29T04:10:05.911101+0000","last_undegraded":"2024-03-29T04:10:05.911101+0000","last_fullsized":"2024-03-29T04:10:05.911101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:26:10.380318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556840+0000","last_change":"2024-03-29T04:10:00.549077+0000","last_active":"2024-03-29T04:10:05.556840+0000","last_peered":"2024-03-29T04:10:05.556840+0000","last_clean":"2024-03-29T04:10:05.556840+0000","last_became_active":"2024-03-29T04:10:00.548266+0000","last_became_peered":"2024-03-29T04:10:00.548266+0000","last_unstale":"2024-03-29T04:10:05.556840+0000","last_undegraded":"2024-03-29T04:10:05.556840+0000","last_fullsized":"2024-03-29T04:10:05.556840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:35:00.452782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910792+0000","last_change":"2024-03-29T04:10:00.535635+0000","last_active":"2024-03-29T04:10:05.910792+0000","last_peered":"2024-03-29T04:10:05.910792+0000","last_clean":"2024-03-29T04:10:05.910792+0000","last_became_active":"2024-03-29T04:10:00.535357+0000","last_became_peered":"2024-03-29T04:10:00.535357+0000","last_unstale":"2024-03-29T04:10:05.910792+0000","last_undegraded":"2024-03-29T04:10:05.910792+0000","last_fullsized":"2024-03-29T04:10:05.910792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:41.517875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911835+0000","last_change":"2024-03-29T04:10:00.547651+0000","last_active":"2024-03-29T04:10:05.911835+0000","last_peered":"2024-03-29T04:10:05.911835+0000","last_clean":"2024-03-29T04:10:05.911835+0000","last_became_active":"2024-03-29T04:10:00.547369+0000","last_became_peered":"2024-03-29T04:10:00.547369+0000","last_unstale":"2024-03-29T04:10:05.911835+0000","last_undegraded":"2024-03-29T04:10:05.911835+0000","last_fullsized":"2024-03-29T04:10:05.911835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:46.707085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911744+0000","last_change":"2024-03-29T04:10:00.545663+0000","last_active":"2024-03-29T04:10:05.911744+0000","last_peered":"2024-03-29T04:10:05.911744+0000","last_clean":"2024-03-29T04:10:05.911744+0000","last_became_active":"2024-03-29T04:10:00.545394+0000","last_became_peered":"2024-03-29T04:10:00.545394+0000","last_unstale":"2024-03-29T04:10:05.911744+0000","last_undegraded":"2024-03-29T04:10:05.911744+0000","last_fullsized":"2024-03-29T04:10:05.911744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:23:46.996981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:10:25.035 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-29T04:10:25.035 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-29T04:10:25.035 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-29T04:10:25.036 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-29T04:10:25.036 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-29T04:10:25.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.221+0000 7fef730c4640 1 Processor -- start 2024-03-29T04:10:25.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.222+0000 7fef730c4640 1 -- start start 2024-03-29T04:10:25.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.222+0000 7fef730c4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fef6c0b5e90 0x7fef6c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.222+0000 7fef730c4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef6c06b930 con 0x7fef6c15e380 2024-03-29T04:10:25.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.222+0000 7fef730c4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef6c06bc10 con 0x7fef6c0b05c0 2024-03-29T04:10:25.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.222+0000 7fef730c4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fef6c06bef0 con 0x7fef6c0b5e90 2024-03-29T04:10:25.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.223+0000 7fef71661640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fef6c0b5e90 0x7fef6c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.223+0000 7fef70e60640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fef6c15e380 0x7fef6c0afeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:32986/0 (socket says 172.21.15.148:32986) 2024-03-29T04:10:25.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.223+0000 7fef70e60640 1 -- 172.21.15.148:0/358101256 learned_addr learned my addr 172.21.15.148:0/358101256 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:25.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.223+0000 7fef71661640 1 -- 172.21.15.148:0/358101256 >> v1:172.21.15.157:6789/0 conn(0x7fef6c0b05c0 legacy=0x7fef6c06e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.223+0000 7fef71661640 1 -- 172.21.15.148:0/358101256 >> v1:172.21.15.148:6789/0 conn(0x7fef6c15e380 legacy=0x7fef6c0afeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.223+0000 7fef71661640 1 -- 172.21.15.148:0/358101256 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6c06c1d0 con 0x7fef6c0b5e90 2024-03-29T04:10:25.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.223+0000 7fef637fe640 1 -- 172.21.15.148:0/358101256 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3727562818 0 0) 0x7fef6c06b930 con 0x7fef6c15e380 2024-03-29T04:10:25.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.224+0000 7fef71661640 1 --2- 172.21.15.148:0/358101256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef6c0b5e90 0x7fef6c06ede0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fef640097a0 tx=0x7fef6402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=661f99bab0ce9068 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:25.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.224+0000 7fef637fe640 1 -- 172.21.15.148:0/358101256 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef64034070 con 0x7fef6c0b5e90 2024-03-29T04:10:25.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.224+0000 7fef637fe640 1 -- 172.21.15.148:0/358101256 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef64030610 con 0x7fef6c0b5e90 2024-03-29T04:10:25.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.224+0000 7fef637fe640 1 -- 172.21.15.148:0/358101256 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef640309e0 con 0x7fef6c0b5e90 2024-03-29T04:10:25.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.224+0000 7fef730c4640 1 -- 172.21.15.148:0/358101256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef6c0b5e90 msgr2=0x7fef6c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.224+0000 7fef730c4640 1 --2- 172.21.15.148:0/358101256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef6c0b5e90 0x7fef6c06ede0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fef640097a0 tx=0x7fef6402b960 comp rx=0 tx=0).stop 2024-03-29T04:10:25.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.224+0000 7fef730c4640 1 -- 172.21.15.148:0/358101256 shutdown_connections 2024-03-29T04:10:25.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.224+0000 7fef730c4640 1 --2- 172.21.15.148:0/358101256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef6c0b5e90 0x7fef6c06ede0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.224+0000 7fef730c4640 1 -- 172.21.15.148:0/358101256 >> 172.21.15.148:0/358101256 conn(0x7fef6c0b2c60 msgr2=0x7fef6c0b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:25.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.224+0000 7fef730c4640 1 -- 172.21.15.148:0/358101256 shutdown_connections 2024-03-29T04:10:25.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.224+0000 7fef730c4640 1 -- 172.21.15.148:0/358101256 wait complete. 2024-03-29T04:10:25.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.226+0000 7fef730c4640 1 Processor -- start 2024-03-29T04:10:25.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.226+0000 7fef730c4640 1 -- start start 2024-03-29T04:10:25.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.228+0000 7fef730c4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef6c0b05c0 0x7fef6c1f5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.228+0000 7fef730c4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef6c0b5e90 0x7fef6c151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.228+0000 7fef730c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef6c15e380 0x7fef6c159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.228+0000 7fef730c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c06f320 con 0x7fef6c15e380 2024-03-29T04:10:25.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.228+0000 7fef730c4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c06b5b0 con 0x7fef6c0b5e90 2024-03-29T04:10:25.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.228+0000 7fef730c4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fef6c06f4a0 con 0x7fef6c0b05c0 2024-03-29T04:10:25.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.228+0000 7fef71661640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef6c15e380 0x7fef6c159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.228+0000 7fef71661640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef6c15e380 0x7fef6c159630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46444/0 (socket says 172.21.15.148:46444) 2024-03-29T04:10:25.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.228+0000 7fef71661640 1 -- 172.21.15.148:0/729825692 learned_addr learned my addr 172.21.15.148:0/729825692 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:25.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.228+0000 7fef70e60640 1 --2- 172.21.15.148:0/729825692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef6c0b05c0 0x7fef6c1f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.228+0000 7fef63fff640 1 --2- 172.21.15.148:0/729825692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef6c0b5e90 0x7fef6c151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.228+0000 7fef71661640 1 -- 172.21.15.148:0/729825692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef6c0b05c0 msgr2=0x7fef6c1f5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.228+0000 7fef71661640 1 --2- 172.21.15.148:0/729825692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef6c0b05c0 0x7fef6c1f5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.228+0000 7fef71661640 1 -- 172.21.15.148:0/729825692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef6c0b5e90 msgr2=0x7fef6c151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.228+0000 7fef71661640 1 --2- 172.21.15.148:0/729825692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef6c0b5e90 0x7fef6c151040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.228+0000 7fef71661640 1 -- 172.21.15.148:0/729825692 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6c1516d0 con 0x7fef6c15e380 2024-03-29T04:10:25.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.228+0000 7fef70e60640 1 --2- 172.21.15.148:0/729825692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef6c0b05c0 0x7fef6c1f5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:25.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.228+0000 7fef63fff640 1 --2- 172.21.15.148:0/729825692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef6c0b5e90 0x7fef6c151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:25.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.228+0000 7fef71661640 1 --2- 172.21.15.148:0/729825692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef6c15e380 0x7fef6c159630 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fef640098c0 tx=0x7fef640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:25.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.228+0000 7fef617fa640 1 -- 172.21.15.148:0/729825692 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef64034040 con 0x7fef6c15e380 2024-03-29T04:10:25.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.228+0000 7fef617fa640 1 -- 172.21.15.148:0/729825692 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef64039990 con 0x7fef6c15e380 2024-03-29T04:10:25.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.228+0000 7fef730c4640 1 -- 172.21.15.148:0/729825692 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef6c1519b0 con 0x7fef6c15e380 2024-03-29T04:10:25.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.228+0000 7fef617fa640 1 -- 172.21.15.148:0/729825692 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef64039d80 con 0x7fef6c15e380 2024-03-29T04:10:25.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.228+0000 7fef730c4640 1 -- 172.21.15.148:0/729825692 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef6c1f4990 con 0x7fef6c15e380 2024-03-29T04:10:25.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.230+0000 7fef730c4640 1 -- 172.21.15.148:0/729825692 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef6c0687e0 con 0x7fef6c15e380 2024-03-29T04:10:25.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.230+0000 7fef617fa640 1 -- 172.21.15.148:0/729825692 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fef64037070 con 0x7fef6c15e380 2024-03-29T04:10:25.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.230+0000 7fef617fa640 1 --2- 172.21.15.148:0/729825692 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fef5806c530 0x7fef5806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.230+0000 7fef617fa640 1 -- 172.21.15.148:0/729825692 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fef640adaa0 con 0x7fef6c15e380 2024-03-29T04:10:25.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.230+0000 7fef70e60640 1 --2- 172.21.15.148:0/729825692 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fef5806c530 0x7fef5806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.231+0000 7fef70e60640 1 --2- 172.21.15.148:0/729825692 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fef5806c530 0x7fef5806e9f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fef680012e0 tx=0x7fef68005c30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:25.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.235+0000 7fef617fa640 1 -- 172.21.15.148:0/729825692 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fef6407be10 con 0x7fef6c15e380 2024-03-29T04:10:25.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.466+0000 7fef730c4640 1 -- 172.21.15.148:0/729825692 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fef6c0b4830 con 0x7fef6c15e380 2024-03-29T04:10:25.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.469+0000 7fef617fa640 1 -- 172.21.15.148:0/729825692 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+173740 (secure 0 0 0) 0x7fef6407b710 con 0x7fef6c15e380 2024-03-29T04:10:25.450 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:25.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.471+0000 7fef730c4640 1 -- 172.21.15.148:0/729825692 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fef5806c530 msgr2=0x7fef5806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.471+0000 7fef730c4640 1 --2- 172.21.15.148:0/729825692 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fef5806c530 0x7fef5806e9f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fef680012e0 tx=0x7fef68005c30 comp rx=0 tx=0).stop 2024-03-29T04:10:25.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.471+0000 7fef730c4640 1 -- 172.21.15.148:0/729825692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef6c15e380 msgr2=0x7fef6c159630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.471+0000 7fef730c4640 1 --2- 172.21.15.148:0/729825692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef6c15e380 0x7fef6c159630 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fef640098c0 tx=0x7fef640046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.471+0000 7fef730c4640 1 -- 172.21.15.148:0/729825692 shutdown_connections 2024-03-29T04:10:25.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.471+0000 7fef730c4640 1 --2- 172.21.15.148:0/729825692 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fef5806c530 0x7fef5806e9f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.471+0000 7fef730c4640 1 --2- 172.21.15.148:0/729825692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef6c15e380 0x7fef6c159630 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.471+0000 7fef730c4640 1 --2- 172.21.15.148:0/729825692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef6c0b5e90 0x7fef6c151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.471+0000 7fef730c4640 1 --2- 172.21.15.148:0/729825692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef6c0b05c0 0x7fef6c1f5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.471+0000 7fef730c4640 1 -- 172.21.15.148:0/729825692 >> 172.21.15.148:0/729825692 conn(0x7fef6c0b2c60 msgr2=0x7fef6c06f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:25.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.471+0000 7fef730c4640 1 -- 172.21.15.148:0/729825692 shutdown_connections 2024-03-29T04:10:25.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.471+0000 7fef730c4640 1 -- 172.21.15.148:0/729825692 wait complete. 2024-03-29T04:10:25.462 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":5,"flags":0,"active_gid":4113,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3271445075},{"type":"v1","addr":"172.21.15.148:6833","nonce":3271445075}]},"active_addr":"172.21.15.148:6833/3271445075","active_change":"2024-03-29T04:09:39.874775+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4102,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":529513299}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":176439294}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":3465400959}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":3288979645}]}]} 2024-03-29T04:10:25.465 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-29T04:10:25.465 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-29T04:10:25.465 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:25.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.587+0000 7fa75f22a640 1 Processor -- start 2024-03-29T04:10:25.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.588+0000 7fa75f22a640 1 -- start start 2024-03-29T04:10:25.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.588+0000 7fa75f22a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa7580afce0 0x7fa7580cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.588+0000 7fa75f22a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa75815d3e0 0x7fa7580cd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.588+0000 7fa75f22a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7580cfa20 con 0x7fa7580cd650 2024-03-29T04:10:25.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.588+0000 7fa75f22a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa75806b5b0 con 0x7fa7580afce0 2024-03-29T04:10:25.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.588+0000 7fa75f22a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa7580cfd00 con 0x7fa75815d3e0 2024-03-29T04:10:25.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.589+0000 7fa757fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa75815d3e0 0x7fa7580cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.589+0000 7fa75cfc6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa7580afce0 0x7fa7580cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.589+0000 7fa757fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa75815d3e0 0x7fa7580cd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.148:44302/0 (socket says 172.21.15.148:44302) 2024-03-29T04:10:25.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.589+0000 7fa757fff640 1 -- 172.21.15.148:0/1209125892 learned_addr learned my addr 172.21.15.148:0/1209125892 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:25.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.589+0000 7fa75cfc6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa7580afce0 0x7fa7580cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:39248/0 (socket says 172.21.15.148:39248) 2024-03-29T04:10:25.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.589+0000 7fa75d7c7640 1 --1- 172.21.15.148:0/1209125892 >> v1:172.21.15.157:6789/0 conn(0x7fa7580cd650 0x7fa75815fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.148:38670/0 (socket says 172.21.15.148:38670) 2024-03-29T04:10:25.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.589+0000 7fa757fff640 1 -- 172.21.15.148:0/1209125892 >> v2:172.21.15.157:3300/0 conn(0x7fa7580afce0 msgr2=0x7fa7580cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:25.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.589+0000 7fa757fff640 1 --2- 172.21.15.148:0/1209125892 >> v2:172.21.15.157:3300/0 conn(0x7fa7580afce0 0x7fa7580cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.589+0000 7fa757fff640 1 -- 172.21.15.148:0/1209125892 >> v1:172.21.15.157:6789/0 conn(0x7fa7580cd650 legacy=0x7fa75815fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.589+0000 7fa757fff640 1 -- 172.21.15.148:0/1209125892 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7580cffe0 con 0x7fa75815d3e0 2024-03-29T04:10:25.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.589+0000 7fa75cfc6640 1 --2- 172.21.15.148:0/1209125892 >> v2:172.21.15.157:3300/0 conn(0x7fa7580afce0 0x7fa7580cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:25.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.590+0000 7fa757fff640 1 --2- 172.21.15.148:0/1209125892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa75815d3e0 0x7fa7580cd110 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa740009870 tx=0x7fa74002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7de92f26b8c5193 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:25.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.590+0000 7fa7577fe640 1 -- 172.21.15.148:0/1209125892 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa740034070 con 0x7fa75815d3e0 2024-03-29T04:10:25.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.590+0000 7fa7577fe640 1 -- 172.21.15.148:0/1209125892 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa740030840 con 0x7fa75815d3e0 2024-03-29T04:10:25.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.590+0000 7fa7577fe640 1 -- 172.21.15.148:0/1209125892 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa740030c10 con 0x7fa75815d3e0 2024-03-29T04:10:25.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.590+0000 7fa75f22a640 1 -- 172.21.15.148:0/1209125892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa75815d3e0 msgr2=0x7fa7580cd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.590+0000 7fa75f22a640 1 --2- 172.21.15.148:0/1209125892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa75815d3e0 0x7fa7580cd110 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa740009870 tx=0x7fa74002b980 comp rx=0 tx=0).stop 2024-03-29T04:10:25.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.591+0000 7fa75f22a640 1 -- 172.21.15.148:0/1209125892 shutdown_connections 2024-03-29T04:10:25.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.591+0000 7fa75f22a640 1 --2- 172.21.15.148:0/1209125892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa75815d3e0 0x7fa7580cd110 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.591+0000 7fa75f22a640 1 --2- 172.21.15.148:0/1209125892 >> v2:172.21.15.157:3300/0 conn(0x7fa7580afce0 0x7fa7580cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.591+0000 7fa75f22a640 1 -- 172.21.15.148:0/1209125892 >> 172.21.15.148:0/1209125892 conn(0x7fa7580d0700 msgr2=0x7fa7580d0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:25.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.591+0000 7fa75f22a640 1 -- 172.21.15.148:0/1209125892 shutdown_connections 2024-03-29T04:10:25.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.591+0000 7fa75f22a640 1 -- 172.21.15.148:0/1209125892 wait complete. 2024-03-29T04:10:25.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.592+0000 7fa75f22a640 1 Processor -- start 2024-03-29T04:10:25.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.592+0000 7fa75f22a640 1 -- start start 2024-03-29T04:10:25.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.593+0000 7fa75f22a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7580afce0 0x7fa758156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.593+0000 7fa75f22a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7580cd650 0x7fa75815b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.593+0000 7fa75f22a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa75815d3e0 0x7fa7581559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.593+0000 7fa75f22a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa7580cf5f0 con 0x7fa7580afce0 2024-03-29T04:10:25.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.593+0000 7fa75f22a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa7580b00c0 con 0x7fa7580cd650 2024-03-29T04:10:25.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.593+0000 7fa75f22a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa7580cf770 con 0x7fa75815d3e0 2024-03-29T04:10:25.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.593+0000 7fa75cfc6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7580afce0 0x7fa758156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.593+0000 7fa75cfc6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7580afce0 0x7fa758156db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48442/0 (socket says 172.21.15.148:48442) 2024-03-29T04:10:25.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.593+0000 7fa75cfc6640 1 -- 172.21.15.148:0/2001886371 learned_addr learned my addr 172.21.15.148:0/2001886371 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:25.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.593+0000 7fa757fff640 1 --2- 172.21.15.148:0/2001886371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7580cd650 0x7fa75815b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.593+0000 7fa75d7c7640 1 --2- 172.21.15.148:0/2001886371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa75815d3e0 0x7fa7581559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.593+0000 7fa75cfc6640 1 -- 172.21.15.148:0/2001886371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa75815d3e0 msgr2=0x7fa7581559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.593+0000 7fa75cfc6640 1 --2- 172.21.15.148:0/2001886371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa75815d3e0 0x7fa7581559d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.593+0000 7fa75cfc6640 1 -- 172.21.15.148:0/2001886371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7580cd650 msgr2=0x7fa75815b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.593+0000 7fa75cfc6640 1 --2- 172.21.15.148:0/2001886371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7580cd650 0x7fa75815b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.593+0000 7fa75cfc6640 1 -- 172.21.15.148:0/2001886371 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa758157b90 con 0x7fa7580afce0 2024-03-29T04:10:25.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.593+0000 7fa75cfc6640 1 --2- 172.21.15.148:0/2001886371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7580afce0 0x7fa758156db0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa75000efc0 tx=0x7fa75000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:25.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.594+0000 7fa7557fa640 1 -- 172.21.15.148:0/2001886371 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa750018020 con 0x7fa7580afce0 2024-03-29T04:10:25.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.594+0000 7fa7557fa640 1 -- 172.21.15.148:0/2001886371 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa750014680 con 0x7fa7580afce0 2024-03-29T04:10:25.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.594+0000 7fa75f22a640 1 -- 172.21.15.148:0/2001886371 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7581f56a0 con 0x7fa7580afce0 2024-03-29T04:10:25.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.594+0000 7fa7557fa640 1 -- 172.21.15.148:0/2001886371 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa750014a50 con 0x7fa7580afce0 2024-03-29T04:10:25.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.594+0000 7fa75f22a640 1 -- 172.21.15.148:0/2001886371 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa758136690 con 0x7fa7580afce0 2024-03-29T04:10:25.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.595+0000 7fa75f22a640 1 -- 172.21.15.148:0/2001886371 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7580744a0 con 0x7fa7580afce0 2024-03-29T04:10:25.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.595+0000 7fa7557fa640 1 -- 172.21.15.148:0/2001886371 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa750005d10 con 0x7fa7580afce0 2024-03-29T04:10:25.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.596+0000 7fa7557fa640 1 --2- 172.21.15.148:0/2001886371 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa73806c530 0x7fa73806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.596+0000 7fa7557fa640 1 -- 172.21.15.148:0/2001886371 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa75008c7b0 con 0x7fa7580afce0 2024-03-29T04:10:25.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.596+0000 7fa757fff640 1 --2- 172.21.15.148:0/2001886371 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa73806c530 0x7fa73806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.596+0000 7fa757fff640 1 --2- 172.21.15.148:0/2001886371 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa73806c530 0x7fa73806e9f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa740036000 tx=0x7fa74003b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:25.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.600+0000 7fa7557fa640 1 -- 172.21.15.148:0/2001886371 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa750022030 con 0x7fa7580afce0 2024-03-29T04:10:25.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.784+0000 7fa75f22a640 1 -- 172.21.15.148:0/2001886371 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7581f5490 con 0x7fa7580afce0 2024-03-29T04:10:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.786+0000 7fa7557fa640 1 -- 172.21.15.148:0/2001886371 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7fa750010040 con 0x7fa7580afce0 2024-03-29T04:10:25.765 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:25.765 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:05.026481+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:25.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.788+0000 7fa75f22a640 1 -- 172.21.15.148:0/2001886371 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa73806c530 msgr2=0x7fa73806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.788+0000 7fa75f22a640 1 --2- 172.21.15.148:0/2001886371 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa73806c530 0x7fa73806e9f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa740036000 tx=0x7fa74003b040 comp rx=0 tx=0).stop 2024-03-29T04:10:25.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.788+0000 7fa75f22a640 1 -- 172.21.15.148:0/2001886371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7580afce0 msgr2=0x7fa758156db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.788+0000 7fa75f22a640 1 --2- 172.21.15.148:0/2001886371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7580afce0 0x7fa758156db0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa75000efc0 tx=0x7fa75000c490 comp rx=0 tx=0).stop 2024-03-29T04:10:25.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.788+0000 7fa75f22a640 1 -- 172.21.15.148:0/2001886371 shutdown_connections 2024-03-29T04:10:25.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.788+0000 7fa75f22a640 1 --2- 172.21.15.148:0/2001886371 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa73806c530 0x7fa73806e9f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.788+0000 7fa75f22a640 1 --2- 172.21.15.148:0/2001886371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa75815d3e0 0x7fa7581559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.788+0000 7fa75f22a640 1 --2- 172.21.15.148:0/2001886371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7580cd650 0x7fa75815b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.788+0000 7fa75f22a640 1 --2- 172.21.15.148:0/2001886371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7580afce0 0x7fa758156db0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.788+0000 7fa75f22a640 1 -- 172.21.15.148:0/2001886371 >> 172.21.15.148:0/2001886371 conn(0x7fa7580d0700 msgr2=0x7fa7581441b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:25.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.788+0000 7fa75f22a640 1 -- 172.21.15.148:0/2001886371 shutdown_connections 2024-03-29T04:10:25.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.788+0000 7fa75f22a640 1 -- 172.21.15.148:0/2001886371 wait complete. 2024-03-29T04:10:25.778 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-29T04:10:25.778 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:25.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.901+0000 7f1648a1b640 1 Processor -- start 2024-03-29T04:10:25.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.901+0000 7f1648a1b640 1 -- start start 2024-03-29T04:10:25.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.902+0000 7f1648a1b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f164415e380 0x7f16440afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.902+0000 7f1648a1b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16440b5e90 0x7f16440b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.902+0000 7f1648a1b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f16440b4fa0 0x7f1644161f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.902+0000 7f1648a1b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f164406b930 con 0x7f16440b4fa0 2024-03-29T04:10:25.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.902+0000 7f1648a1b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f164406bc10 con 0x7f164415e380 2024-03-29T04:10:25.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.902+0000 7f1648a1b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f164406bef0 con 0x7f16440b5e90 2024-03-29T04:10:25.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.902+0000 7f1642d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f16440b4fa0 0x7f1644161f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.902+0000 7f1642d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f16440b4fa0 0x7f1644161f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48450/0 (socket says 172.21.15.148:48450) 2024-03-29T04:10:25.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.902+0000 7f1642d9d640 1 -- 172.21.15.148:0/3991429932 learned_addr learned my addr 172.21.15.148:0/3991429932 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:25.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.903+0000 7f1641d9b640 1 --2- 172.21.15.148:0/3991429932 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16440b5e90 0x7f16440b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.903+0000 7f164259c640 1 --2- 172.21.15.148:0/3991429932 >> v2:172.21.15.157:3300/0 conn(0x7f164415e380 0x7f16440afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.903+0000 7f1642d9d640 1 -- 172.21.15.148:0/3991429932 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16440b5e90 msgr2=0x7f16440b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:25.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.903+0000 7f1642d9d640 1 --2- 172.21.15.148:0/3991429932 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16440b5e90 0x7f16440b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.903+0000 7f1642d9d640 1 -- 172.21.15.148:0/3991429932 >> v2:172.21.15.157:3300/0 conn(0x7f164415e380 msgr2=0x7f16440afeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:25.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.903+0000 7f1642d9d640 1 --2- 172.21.15.148:0/3991429932 >> v2:172.21.15.157:3300/0 conn(0x7f164415e380 0x7f16440afeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.903+0000 7f1642d9d640 1 -- 172.21.15.148:0/3991429932 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f164406c1d0 con 0x7f16440b4fa0 2024-03-29T04:10:25.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.903+0000 7f164259c640 1 --2- 172.21.15.148:0/3991429932 >> v2:172.21.15.157:3300/0 conn(0x7f164415e380 0x7f16440afeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:25.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.903+0000 7f1641d9b640 1 --2- 172.21.15.148:0/3991429932 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16440b5e90 0x7f16440b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:25.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.903+0000 7f1642d9d640 1 --2- 172.21.15.148:0/3991429932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16440b4fa0 0x7f1644161f30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f16440738a0 tx=0x7f163c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745f4c6167a3b84c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:25.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.904+0000 7f164159a640 1 -- 172.21.15.148:0/3991429932 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c02ca70 con 0x7f16440b4fa0 2024-03-29T04:10:25.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.904+0000 7f164159a640 1 -- 172.21.15.148:0/3991429932 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f163c02cc10 con 0x7f16440b4fa0 2024-03-29T04:10:25.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.904+0000 7f164159a640 1 -- 172.21.15.148:0/3991429932 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c0343f0 con 0x7f16440b4fa0 2024-03-29T04:10:25.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.904+0000 7f1648a1b640 1 -- 172.21.15.148:0/3991429932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16440b4fa0 msgr2=0x7f1644161f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.904+0000 7f1648a1b640 1 --2- 172.21.15.148:0/3991429932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16440b4fa0 0x7f1644161f30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f16440738a0 tx=0x7f163c02bed0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.904+0000 7f1648a1b640 1 -- 172.21.15.148:0/3991429932 shutdown_connections 2024-03-29T04:10:25.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.904+0000 7f1648a1b640 1 --2- 172.21.15.148:0/3991429932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16440b4fa0 0x7f1644161f30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.904+0000 7f1648a1b640 1 --2- 172.21.15.148:0/3991429932 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16440b5e90 0x7f16440b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.904+0000 7f1648a1b640 1 --2- 172.21.15.148:0/3991429932 >> v2:172.21.15.157:3300/0 conn(0x7f164415e380 0x7f16440afeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.905+0000 7f1648a1b640 1 -- 172.21.15.148:0/3991429932 >> 172.21.15.148:0/3991429932 conn(0x7f16440b2c60 msgr2=0x7f16440b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:25.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.905+0000 7f1648a1b640 1 -- 172.21.15.148:0/3991429932 shutdown_connections 2024-03-29T04:10:25.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.905+0000 7f1648a1b640 1 -- 172.21.15.148:0/3991429932 wait complete. 2024-03-29T04:10:25.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.906+0000 7f1648a1b640 1 Processor -- start 2024-03-29T04:10:25.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.907+0000 7f1648a1b640 1 -- start start 2024-03-29T04:10:25.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.908+0000 7f1648a1b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16440b4fa0 0x7f16441571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.908+0000 7f1648a1b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16440b5e90 0x7f16440cf060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.908+0000 7f1648a1b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f164415e380 0x7f16440cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.908+0000 7f1648a1b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f164406c9d0 con 0x7f16440b4fa0 2024-03-29T04:10:25.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.908+0000 7f1648a1b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f164406b5b0 con 0x7f16440b5e90 2024-03-29T04:10:25.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.908+0000 7f1648a1b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f164406cb50 con 0x7f164415e380 2024-03-29T04:10:25.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.908+0000 7f164259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16440b4fa0 0x7f16441571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.908+0000 7f164259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16440b4fa0 0x7f16441571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48460/0 (socket says 172.21.15.148:48460) 2024-03-29T04:10:25.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.908+0000 7f164259c640 1 -- 172.21.15.148:0/3823512347 learned_addr learned my addr 172.21.15.148:0/3823512347 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:25.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.908+0000 7f1642d9d640 1 --2- 172.21.15.148:0/3823512347 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f164415e380 0x7f16440cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.908+0000 7f1641d9b640 1 --2- 172.21.15.148:0/3823512347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16440b5e90 0x7f16440cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.908+0000 7f164259c640 1 -- 172.21.15.148:0/3823512347 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f164415e380 msgr2=0x7f16440cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.908+0000 7f164259c640 1 --2- 172.21.15.148:0/3823512347 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f164415e380 0x7f16440cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.908+0000 7f164259c640 1 -- 172.21.15.148:0/3823512347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16440b5e90 msgr2=0x7f16440cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.908+0000 7f164259c640 1 --2- 172.21.15.148:0/3823512347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16440b5e90 0x7f16440cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.908+0000 7f164259c640 1 -- 172.21.15.148:0/3823512347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16441576e0 con 0x7f16440b4fa0 2024-03-29T04:10:25.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.908+0000 7f1642d9d640 1 --2- 172.21.15.148:0/3823512347 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f164415e380 0x7f16440cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:25.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.908+0000 7f1641d9b640 1 --2- 172.21.15.148:0/3823512347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16440b5e90 0x7f16440cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:25.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.908+0000 7f164259c640 1 --2- 172.21.15.148:0/3823512347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16440b4fa0 0x7f16441571a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f163400d7d0 tx=0x7f163400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:25.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.908+0000 7f1632ffd640 1 -- 172.21.15.148:0/3823512347 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634004030 con 0x7f16440b4fa0 2024-03-29T04:10:25.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.908+0000 7f1632ffd640 1 -- 172.21.15.148:0/3823512347 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16340041d0 con 0x7f16440b4fa0 2024-03-29T04:10:25.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.908+0000 7f1648a1b640 1 -- 172.21.15.148:0/3823512347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1644157980 con 0x7f16440b4fa0 2024-03-29T04:10:25.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.908+0000 7f1632ffd640 1 -- 172.21.15.148:0/3823512347 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16340096a0 con 0x7f16440b4fa0 2024-03-29T04:10:25.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.909+0000 7f1648a1b640 1 -- 172.21.15.148:0/3823512347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1644157f40 con 0x7f16440b4fa0 2024-03-29T04:10:25.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.909+0000 7f1648a1b640 1 -- 172.21.15.148:0/3823512347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16440687e0 con 0x7f16440b4fa0 2024-03-29T04:10:25.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.910+0000 7f1632ffd640 1 -- 172.21.15.148:0/3823512347 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1634009840 con 0x7f16440b4fa0 2024-03-29T04:10:25.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.911+0000 7f1632ffd640 1 --2- 172.21.15.148:0/3823512347 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f162406c530 0x7f162406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.911+0000 7f1632ffd640 1 -- 172.21.15.148:0/3823512347 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f163408bc50 con 0x7f16440b4fa0 2024-03-29T04:10:25.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.911+0000 7f1641d9b640 1 --2- 172.21.15.148:0/3823512347 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f162406c530 0x7f162406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.911+0000 7f1641d9b640 1 --2- 172.21.15.148:0/3823512347 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f162406c530 0x7f162406e9f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1638006fd0 tx=0x7f1638008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:25.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:25.915+0000 7f1632ffd640 1 -- 172.21.15.148:0/3823512347 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1634091020 con 0x7f16440b4fa0 2024-03-29T04:10:26.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.098+0000 7f1648a1b640 1 -- 172.21.15.148:0/3823512347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16441596c0 con 0x7f16440b4fa0 2024-03-29T04:10:26.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.099+0000 7f1632ffd640 1 -- 172.21.15.148:0/3823512347 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7f1634059fc0 con 0x7f16440b4fa0 2024-03-29T04:10:26.079 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:26.079 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:05.026481+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:26.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.101+0000 7f1648a1b640 1 -- 172.21.15.148:0/3823512347 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f162406c530 msgr2=0x7f162406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.101+0000 7f1648a1b640 1 --2- 172.21.15.148:0/3823512347 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f162406c530 0x7f162406e9f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1638006fd0 tx=0x7f1638008040 comp rx=0 tx=0).stop 2024-03-29T04:10:26.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.101+0000 7f1648a1b640 1 -- 172.21.15.148:0/3823512347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16440b4fa0 msgr2=0x7f16441571a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.101+0000 7f1648a1b640 1 --2- 172.21.15.148:0/3823512347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16440b4fa0 0x7f16441571a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f163400d7d0 tx=0x7f163400dc90 comp rx=0 tx=0).stop 2024-03-29T04:10:26.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.102+0000 7f1648a1b640 1 -- 172.21.15.148:0/3823512347 shutdown_connections 2024-03-29T04:10:26.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.102+0000 7f1648a1b640 1 --2- 172.21.15.148:0/3823512347 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f162406c530 0x7f162406e9f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.102+0000 7f1648a1b640 1 --2- 172.21.15.148:0/3823512347 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f164415e380 0x7f16440cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.102+0000 7f1648a1b640 1 --2- 172.21.15.148:0/3823512347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16440b5e90 0x7f16440cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.102+0000 7f1648a1b640 1 --2- 172.21.15.148:0/3823512347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16440b4fa0 0x7f16441571a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.102+0000 7f1648a1b640 1 -- 172.21.15.148:0/3823512347 >> 172.21.15.148:0/3823512347 conn(0x7f16440b2c60 msgr2=0x7f164406ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.102+0000 7f1648a1b640 1 -- 172.21.15.148:0/3823512347 shutdown_connections 2024-03-29T04:10:26.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.102+0000 7f1648a1b640 1 -- 172.21.15.148:0/3823512347 wait complete. 2024-03-29T04:10:26.096 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-29T04:10:26.096 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-29T04:10:26.097 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-29T04:10:26.097 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-29T04:10:26.097 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-29T04:10:26.098 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-29T04:10:26.098 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-29T04:10:26.098 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-29T04:10:26.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.292+0000 7fb4de396640 1 Processor -- start 2024-03-29T04:10:26.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.292+0000 7fb4de396640 1 -- start start 2024-03-29T04:10:26.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.293+0000 7fb53311c640 1 Processor -- start 2024-03-29T04:10:26.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.293+0000 7f4a91590640 1 Processor -- start 2024-03-29T04:10:26.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.293+0000 7fb53311c640 1 -- start start 2024-03-29T04:10:26.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.293+0000 7f4a91590640 1 -- start start 2024-03-29T04:10:26.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.293+0000 7fb53311c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb52c1639e0 0x7fb52c162e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.294+0000 7fb53311c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb52c06c8d0 0x7fb52c0c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.294+0000 7fb53311c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb52c06d120 con 0x7fb52c0c75e0 2024-03-29T04:10:26.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.294+0000 7fb53311c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fb52c06dda0 con 0x7fb52c1639e0 2024-03-29T04:10:26.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.294+0000 7fb53311c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb52c0c7a30 con 0x7fb52c06c8d0 2024-03-29T04:10:26.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.294+0000 7fb530eb8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb52c1639e0 0x7fb52c162e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.294+0000 7fb530eb8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb52c1639e0 0x7fb52c162e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48464/0 (socket says 172.21.15.148:48464) 2024-03-29T04:10:26.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.294+0000 7fb530eb8640 1 -- 172.21.15.148:0/3646868772 learned_addr learned my addr 172.21.15.148:0/3646868772 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:26.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.294+0000 7fb52bfff640 1 --2- 172.21.15.148:0/3646868772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb52c06c8d0 0x7fb52c0c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.294+0000 7fb530eb8640 1 -- 172.21.15.148:0/3646868772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb52c06c8d0 msgr2=0x7fb52c0c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:26.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.294+0000 7fb530eb8640 1 --2- 172.21.15.148:0/3646868772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb52c06c8d0 0x7fb52c0c70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.294+0000 7fb530eb8640 1 -- 172.21.15.148:0/3646868772 >> v1:172.21.15.157:6789/0 conn(0x7fb52c0c75e0 legacy=0x7fb52c1664e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.294+0000 7fb530eb8640 1 -- 172.21.15.148:0/3646868772 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb52c06d400 con 0x7fb52c1639e0 2024-03-29T04:10:26.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.295+0000 7fb52b7fe640 1 -- 172.21.15.148:0/3646868772 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 830229983 0 0) 0x7fb52c06d120 con 0x7fb52c0c75e0 2024-03-29T04:10:26.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.295+0000 7f4a91590640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4a8c0b3810 0x7f4a8c0ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.296+0000 7fb530eb8640 1 --2- 172.21.15.148:0/3646868772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb52c1639e0 0x7fb52c162e70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb524009e50 tx=0x7fb52402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85c3ec3fecb3af4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.295+0000 7f4a91590640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4a8c0d2530 0x7f4a8c0cee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.295+0000 7f4a91590640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a8c163010 con 0x7f4a8c162c30 2024-03-29T04:10:26.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.295+0000 7f4a91590640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f4a8c0b48b0 con 0x7f4a8c0b3810 2024-03-29T04:10:26.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.295+0000 7f4a91590640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f4a8c0b3ca0 con 0x7f4a8c0d2530 2024-03-29T04:10:26.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.295+0000 7f4a8b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4a8c0d2530 0x7f4a8c0cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.296+0000 7fb52b7fe640 1 -- 172.21.15.148:0/3646868772 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52402c9e0 con 0x7fb52c1639e0 2024-03-29T04:10:26.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.295+0000 7f4a8b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4a8c0d2530 0x7f4a8c0cee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:39268/0 (socket says 172.21.15.148:39268) 2024-03-29T04:10:26.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.295+0000 7f4a8b7fe640 1 -- 172.21.15.148:0/960490326 learned_addr learned my addr 172.21.15.148:0/960490326 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:26.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.296+0000 7fb52b7fe640 1 -- 172.21.15.148:0/3646868772 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb52402cb80 con 0x7fb52c1639e0 2024-03-29T04:10:26.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.295+0000 7f4a8b7fe640 1 -- 172.21.15.148:0/960490326 >> v2:172.21.15.148:3300/0 conn(0x7f4a8c0b3810 msgr2=0x7f4a8c0ce950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:10:26.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.296+0000 7f4a89ffb640 1 -- 172.21.15.148:0/960490326 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3035849754 0 0) 0x7f4a84001490 con 0x7f4a8c162c30 2024-03-29T04:10:26.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.296+0000 7f4a8a7fc640 1 --2- 172.21.15.148:0/960490326 >> v2:172.21.15.148:3300/0 conn(0x7f4a8c0b3810 0x7f4a8c0ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.297+0000 7fb53311c640 1 -- 172.21.15.148:0/3646868772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb52c1639e0 msgr2=0x7fb52c162e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.297+0000 7fb53311c640 1 --2- 172.21.15.148:0/3646868772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb52c1639e0 0x7fb52c162e70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb524009e50 tx=0x7fb52402be40 comp rx=0 tx=0).stop 2024-03-29T04:10:26.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.297+0000 7fb53311c640 1 -- 172.21.15.148:0/3646868772 shutdown_connections 2024-03-29T04:10:26.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.297+0000 7fb53311c640 1 --2- 172.21.15.148:0/3646868772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb52c06c8d0 0x7fb52c0c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.297+0000 7fb53311c640 1 --2- 172.21.15.148:0/3646868772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb52c1639e0 0x7fb52c162e70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.297+0000 7fb53311c640 1 -- 172.21.15.148:0/3646868772 >> 172.21.15.148:0/3646868772 conn(0x7fb52c0cec80 msgr2=0x7fb52c0cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.296+0000 7f4a8b7fe640 1 --2- 172.21.15.148:0/960490326 >> v2:172.21.15.148:3300/0 conn(0x7f4a8c0b3810 0x7f4a8c0ce950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.296+0000 7f4a8b7fe640 1 -- 172.21.15.148:0/960490326 >> v1:172.21.15.157:6789/0 conn(0x7f4a8c162c30 legacy=0x7f4a8c0c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.298+0000 7fb53311c640 1 -- 172.21.15.148:0/3646868772 shutdown_connections 2024-03-29T04:10:26.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.296+0000 7f4a8b7fe640 1 -- 172.21.15.148:0/960490326 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8c0b3f80 con 0x7f4a8c0d2530 2024-03-29T04:10:26.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.297+0000 7f4a8a7fc640 1 --2- 172.21.15.148:0/960490326 >> v2:172.21.15.148:3300/0 conn(0x7f4a8c0b3810 0x7f4a8c0ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:26.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.298+0000 7f4a8b7fe640 1 --2- 172.21.15.148:0/960490326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a8c0d2530 0x7f4a8c0cee90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0d29c0 tx=0x7f4a8002b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a285fa0d89b1e44d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.298+0000 7fb53311c640 1 -- 172.21.15.148:0/3646868772 wait complete. 2024-03-29T04:10:26.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.298+0000 7f4a89ffb640 1 -- 172.21.15.148:0/960490326 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a80034070 con 0x7f4a8c0d2530 2024-03-29T04:10:26.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.298+0000 7f4a89ffb640 1 -- 172.21.15.148:0/960490326 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a80030890 con 0x7f4a8c0d2530 2024-03-29T04:10:26.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.299+0000 7fb53311c640 1 Processor -- start 2024-03-29T04:10:26.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.298+0000 7f4a89ffb640 1 -- 172.21.15.148:0/960490326 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a80030c60 con 0x7f4a8c0d2530 2024-03-29T04:10:26.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.299+0000 7fb53311c640 1 -- start start 2024-03-29T04:10:26.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.299+0000 7f4a91590640 1 -- 172.21.15.148:0/960490326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a8c0d2530 msgr2=0x7f4a8c0cee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.299+0000 7f4a91590640 1 --2- 172.21.15.148:0/960490326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a8c0d2530 0x7f4a8c0cee90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0d29c0 tx=0x7f4a8002b9d0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.299+0000 7f4a91590640 1 -- 172.21.15.148:0/960490326 shutdown_connections 2024-03-29T04:10:26.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.299+0000 7f4a91590640 1 --2- 172.21.15.148:0/960490326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a8c0d2530 0x7f4a8c0cee90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.299+0000 7f4a91590640 1 --2- 172.21.15.148:0/960490326 >> v2:172.21.15.148:3300/0 conn(0x7f4a8c0b3810 0x7f4a8c0ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.299+0000 7f4a91590640 1 -- 172.21.15.148:0/960490326 >> 172.21.15.148:0/960490326 conn(0x7f4a8c0d4300 msgr2=0x7f4a8c0d4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.300+0000 7f4a91590640 1 -- 172.21.15.148:0/960490326 shutdown_connections 2024-03-29T04:10:26.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.300+0000 7f4a91590640 1 -- 172.21.15.148:0/960490326 wait complete. 2024-03-29T04:10:26.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.300+0000 7f85901f4640 1 Processor -- start 2024-03-29T04:10:26.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.301+0000 7fb53311c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb52c06c8d0 0x7fb52c14c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.301+0000 7fb53311c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb52c0c75e0 0x7fb52c0c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.301+0000 7fb53311c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb52c1639e0 0x7fb52c0cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.301+0000 7fb53311c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c163dc0 con 0x7fb52c0c75e0 2024-03-29T04:10:26.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.301+0000 7fb53311c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c06d9c0 con 0x7fb52c06c8d0 2024-03-29T04:10:26.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.301+0000 7fb53311c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb52c06cd30 con 0x7fb52c1639e0 2024-03-29T04:10:26.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.301+0000 7fb52bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb52c0c75e0 0x7fb52c0c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.301+0000 7fb52bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb52c0c75e0 0x7fb52c0c92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48494/0 (socket says 172.21.15.148:48494) 2024-03-29T04:10:26.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.301+0000 7fb52bfff640 1 -- 172.21.15.148:0/388324995 learned_addr learned my addr 172.21.15.148:0/388324995 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:26.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.301+0000 7fb530eb8640 1 --2- 172.21.15.148:0/388324995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb52c06c8d0 0x7fb52c14c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.301+0000 7fb5316b9640 1 --2- 172.21.15.148:0/388324995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb52c1639e0 0x7fb52c0cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.302+0000 7fb52bfff640 1 -- 172.21.15.148:0/388324995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb52c1639e0 msgr2=0x7fb52c0cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.302+0000 7fb52bfff640 1 --2- 172.21.15.148:0/388324995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb52c1639e0 0x7fb52c0cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.302+0000 7fb52bfff640 1 -- 172.21.15.148:0/388324995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb52c06c8d0 msgr2=0x7fb52c14c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.302+0000 7fb52bfff640 1 --2- 172.21.15.148:0/388324995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb52c06c8d0 0x7fb52c14c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.302+0000 7f50974c8640 1 Processor -- start 2024-03-29T04:10:26.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.302+0000 7f4a91590640 1 Processor -- start 2024-03-29T04:10:26.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.302+0000 7fb52bfff640 1 -- 172.21.15.148:0/388324995 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb52c0c99d0 con 0x7fb52c0c75e0 2024-03-29T04:10:26.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.302+0000 7fb52bfff640 1 --2- 172.21.15.148:0/388324995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb52c0c75e0 0x7fb52c0c92d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb52000e9b0 tx=0x7fb52000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.302+0000 7fb530eb8640 1 --2- 172.21.15.148:0/388324995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb52c06c8d0 0x7fb52c14c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:26.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.302+0000 7f4a91590640 1 -- start start 2024-03-29T04:10:26.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.302+0000 7fb5297fa640 1 -- 172.21.15.148:0/388324995 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb520017070 con 0x7fb52c0c75e0 2024-03-29T04:10:26.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.302+0000 7fb5297fa640 1 -- 172.21.15.148:0/388324995 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb52000c7f0 con 0x7fb52c0c75e0 2024-03-29T04:10:26.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.302+0000 7fb5297fa640 1 -- 172.21.15.148:0/388324995 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52000cbe0 con 0x7fb52c0c75e0 2024-03-29T04:10:26.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.303+0000 7f4a91590640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a8c0d2530 0x7f4a8c0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.303+0000 7f4a91590640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a8c162c30 0x7f4a8c159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.303+0000 7f4a91590640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a8c164d60 0x7f4a8c1571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.303+0000 7f4a91590640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c0caff0 con 0x7f4a8c164d60 2024-03-29T04:10:26.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.303+0000 7f4a91590640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c0b4540 con 0x7f4a8c162c30 2024-03-29T04:10:26.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.303+0000 7f4a91590640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4a8c0cb170 con 0x7f4a8c0d2530 2024-03-29T04:10:26.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.302+0000 7fb53311c640 1 -- 172.21.15.148:0/388324995 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb52c0c9cb0 con 0x7fb52c0c75e0 2024-03-29T04:10:26.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.302+0000 7fb53311c640 1 -- 172.21.15.148:0/388324995 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb52c0ca270 con 0x7fb52c0c75e0 2024-03-29T04:10:26.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.302+0000 7f50974c8640 1 -- start start 2024-03-29T04:10:26.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.303+0000 7f50974c8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f50900cef80 0x7f50900b7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.303+0000 7f50974c8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f50900b4c40 0x7f50900b5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.303+0000 7f50974c8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50900b7c50 con 0x7f50900d3430 2024-03-29T04:10:26.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.303+0000 7f50974c8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f509015ec20 con 0x7f50900b4c40 2024-03-29T04:10:26.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.303+0000 7f50974c8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f50900b47b0 con 0x7f50900cef80 2024-03-29T04:10:26.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.304+0000 7fb4de396640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb4d80cd600 0x7fb4d806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.304+0000 7f4a8affd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a8c0d2530 0x7f4a8c0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.304+0000 7f4a8b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a8c164d60 0x7f4a8c1571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.304+0000 7f4a8affd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a8c0d2530 0x7f4a8c0ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.148:44372/0 (socket says 172.21.15.148:44372) 2024-03-29T04:10:26.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.304+0000 7f4a8affd640 1 -- 172.21.15.148:0/2778737557 learned_addr learned my addr 172.21.15.148:0/2778737557 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:26.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.304+0000 7f5094a63640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f50900d3430 0x7f50900cf360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47220/0 (socket says 172.21.15.148:47220) 2024-03-29T04:10:26.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.304+0000 7f4a8a7fc640 1 --2- 172.21.15.148:0/2778737557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a8c162c30 0x7f4a8c159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.304+0000 7fb4de396640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4d80cd9e0 con 0x7fb4d806c6c0 2024-03-29T04:10:26.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.304+0000 7fb4d77fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb4d806c6c0 0x7fb4d815c140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47232/0 (socket says 172.21.15.148:47232) 2024-03-29T04:10:26.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.304+0000 7f5094a63640 1 -- 172.21.15.148:0/4238295984 learned_addr learned my addr 172.21.15.148:0/4238295984 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:26.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.304+0000 7fb4d77fe640 1 -- 172.21.15.148:0/3448880872 learned_addr learned my addr 172.21.15.148:0/3448880872 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:26.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.304+0000 7f4a8affd640 1 -- 172.21.15.148:0/2778737557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a8c162c30 msgr2=0x7f4a8c159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.304+0000 7f4a8affd640 1 --2- 172.21.15.148:0/2778737557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a8c162c30 0x7f4a8c159a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.304+0000 7f5095264640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f50900cef80 0x7f50900b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.304+0000 7f4a8affd640 1 -- 172.21.15.148:0/2778737557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a8c164d60 msgr2=0x7f4a8c1571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.304+0000 7f4a8affd640 1 --2- 172.21.15.148:0/2778737557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a8c164d60 0x7f4a8c1571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.304+0000 7fb4d7fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb4d80cd600 0x7fb4d806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.305+0000 7f4a8affd640 1 -- 172.21.15.148:0/2778737557 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8c1fc400 con 0x7f4a8c0d2530 2024-03-29T04:10:26.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.305+0000 7fb4de396640 1 -- 172.21.15.148:0/3448880872 --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4d806cbf0 con 0x7fb4d80b5280 2024-03-29T04:10:26.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.305+0000 7f4a8affd640 1 --2- 172.21.15.148:0/2778737557 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a8c0d2530 0x7f4a8c0ca660 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400c7b0 tx=0x7f4a8400cc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.305+0000 7fb4de396640 1 -- 172.21.15.148:0/3448880872 --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb4d806d800 con 0x7fb4d80cd600 2024-03-29T04:10:26.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.305+0000 7fb4d6ffd640 1 -- 172.21.15.148:0/3448880872 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4076544528 0 0) 0x7fb4d806cbf0 con 0x7fb4d80b5280 2024-03-29T04:10:26.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.305+0000 7fb4d6ffd640 1 -- 172.21.15.148:0/3448880872 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4c0003520 con 0x7fb4d80b5280 2024-03-29T04:10:26.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.305+0000 7fb4d6ffd640 1 -- 172.21.15.148:0/3448880872 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3976404080 0 0) 0x7fb4d80cd9e0 con 0x7fb4d806c6c0 2024-03-29T04:10:26.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.305+0000 7f5095a65640 1 --2- 172.21.15.148:0/4238295984 >> v2:172.21.15.148:3300/0 conn(0x7f50900b4c40 0x7f50900b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.305+0000 7f5087fff640 1 -- 172.21.15.148:0/4238295984 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1775077150 0 0) 0x7f50900b7c50 con 0x7f50900d3430 2024-03-29T04:10:26.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.305+0000 7f5087fff640 1 -- 172.21.15.148:0/4238295984 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f507c003520 con 0x7f50900d3430 2024-03-29T04:10:26.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.305+0000 7fb4d6ffd640 1 -- 172.21.15.148:0/3448880872 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4d806cbf0 con 0x7fb4d806c6c0 2024-03-29T04:10:26.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.306+0000 7fb5297fa640 1 -- 172.21.15.148:0/388324995 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb52001a070 con 0x7fb52c0c75e0 2024-03-29T04:10:26.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.305+0000 7fb4d6ffd640 1 -- 172.21.15.148:0/3448880872 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1713171070 0 0) 0x7fb4c0003520 con 0x7fb4d80b5280 2024-03-29T04:10:26.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.306+0000 7fb4d6ffd640 1 -- 172.21.15.148:0/3448880872 >> v2:172.21.15.157:3300/0 conn(0x7fb4d80cd600 msgr2=0x7fb4d806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:26.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.306+0000 7fb4d6ffd640 1 --2- 172.21.15.148:0/3448880872 >> v2:172.21.15.157:3300/0 conn(0x7fb4d80cd600 0x7fb4d806dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.306+0000 7fb4d6ffd640 1 -- 172.21.15.148:0/3448880872 >> v1:172.21.15.148:6789/0 conn(0x7fb4d806c6c0 legacy=0x7fb4d815c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.306+0000 7fb4d6ffd640 1 -- 172.21.15.148:0/3448880872 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d806ced0 con 0x7fb4d80b5280 2024-03-29T04:10:26.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.306+0000 7fb4d6ffd640 1 -- 172.21.15.148:0/3448880872 <== mon.1 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7fb4d00019f0 con 0x7fb4d80b5280 2024-03-29T04:10:26.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.306+0000 7fb4d6ffd640 1 -- 172.21.15.148:0/3448880872 >> v1:172.21.15.157:6789/0 conn(0x7fb4d80b5280 legacy=0x7fb4d80b5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.306+0000 7fb4d6ffd640 1 --2- 172.21.15.148:0/3448880872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4c0003b40 0x7fb4c0003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.306+0000 7fb4d6ffd640 1 --2- 172.21.15.148:0/3448880872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4c0004800 0x7fb4c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.306+0000 7fb4d6ffd640 1 --2- 172.21.15.148:0/3448880872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4c00051c0 0x7fb4c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.306+0000 7fb4d6ffd640 1 -- 172.21.15.148:0/3448880872 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d806d800 con 0x7fb4c0003b40 2024-03-29T04:10:26.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.306+0000 7fb4d6ffd640 1 -- 172.21.15.148:0/3448880872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c000c010 con 0x7fb4c0004800 2024-03-29T04:10:26.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.306+0000 7fb4d7fff640 1 --2- 172.21.15.148:0/3448880872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4c0003b40 0x7fb4c0003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.306+0000 7fb4d6ffd640 1 -- 172.21.15.148:0/3448880872 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb4c000c210 con 0x7fb4c00051c0 2024-03-29T04:10:26.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.306+0000 7fb4dc933640 1 --2- 172.21.15.148:0/3448880872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4c00051c0 0x7fb4c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.305+0000 7f5095264640 1 -- 172.21.15.148:0/4238295984 >> v2:172.21.15.148:3300/0 conn(0x7f50900b4c40 msgr2=0x7f50900b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:26.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.305+0000 7f5095264640 1 --2- 172.21.15.148:0/4238295984 >> v2:172.21.15.148:3300/0 conn(0x7f50900b4c40 0x7f50900b5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.306+0000 7fb5297fa640 1 --2- 172.21.15.148:0/388324995 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb51806c580 0x7fb51806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.307+0000 7fb5297fa640 1 -- 172.21.15.148:0/388324995 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb52008bb10 con 0x7fb52c0c75e0 2024-03-29T04:10:26.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.307+0000 7fb530eb8640 1 --2- 172.21.15.148:0/388324995 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb51806c580 0x7fb51806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.308+0000 7fb530eb8640 1 --2- 172.21.15.148:0/388324995 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb51806c580 0x7fb51806ea40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb524035000 tx=0x7fb524030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.307+0000 7fb4dc933640 1 -- 172.21.15.148:0/3448880872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4c0004800 msgr2=0x7fb4c0004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-29T04:10:26.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.307+0000 7fb4dc933640 1 --2- 172.21.15.148:0/3448880872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4c0004800 0x7fb4c0004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.307+0000 7fb4dc933640 1 -- 172.21.15.148:0/3448880872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4c0003b40 msgr2=0x7fb4c0003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:26.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.307+0000 7fb4dc933640 1 --2- 172.21.15.148:0/3448880872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4c0003b40 0x7fb4c0003f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.307+0000 7fb4dc933640 1 -- 172.21.15.148:0/3448880872 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c0003520 con 0x7fb4c00051c0 2024-03-29T04:10:26.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.308+0000 7fb4dc933640 1 -- 172.21.15.148:0/3448880872 reap_dead start 2024-03-29T04:10:26.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.306+0000 7f4a737fe640 1 -- 172.21.15.148:0/2778737557 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a84003da0 con 0x7f4a8c0d2530 2024-03-29T04:10:26.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.307+0000 7f4a737fe640 1 -- 172.21.15.148:0/2778737557 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a84005070 con 0x7f4a8c0d2530 2024-03-29T04:10:26.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.307+0000 7f4a737fe640 1 -- 172.21.15.148:0/2778737557 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a84005420 con 0x7f4a8c0d2530 2024-03-29T04:10:26.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.307+0000 7f4a91590640 1 -- 172.21.15.148:0/2778737557 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a8c1fc6e0 con 0x7f4a8c0d2530 2024-03-29T04:10:26.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.307+0000 7f4a91590640 1 -- 172.21.15.148:0/2778737557 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a8c1fcca0 con 0x7f4a8c0d2530 2024-03-29T04:10:26.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.305+0000 7f5095264640 1 -- 172.21.15.148:0/4238295984 >> v1:172.21.15.148:6789/0 conn(0x7f50900d3430 legacy=0x7f50900cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.305+0000 7f5095264640 1 -- 172.21.15.148:0/4238295984 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50900b3ba0 con 0x7f50900cef80 2024-03-29T04:10:26.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.307+0000 7f5095264640 1 --2- 172.21.15.148:0/4238295984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50900cef80 0x7f50900b7710 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f50900b6e00 tx=0x7f508c02b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7c8dbbf8885e146 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.308+0000 7f5087fff640 1 -- 172.21.15.148:0/4238295984 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c035020 con 0x7f50900cef80 2024-03-29T04:10:26.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.308+0000 7f5087fff640 1 -- 172.21.15.148:0/4238295984 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f508c031870 con 0x7f50900cef80 2024-03-29T04:10:26.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.308+0000 7f5087fff640 1 -- 172.21.15.148:0/4238295984 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c031c80 con 0x7f50900cef80 2024-03-29T04:10:26.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.308+0000 7fb4dc933640 1 --2- 172.21.15.148:0/3448880872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4c00051c0 0x7fb4c000b9c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0004360 tx=0x7fb4c0005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd1f1252d15f384c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.307+0000 7f85901f4640 1 -- start start 2024-03-29T04:10:26.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.308+0000 7f85901f4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f858815aab0 0x7f85880b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.308+0000 7fb4d6ffd640 1 -- 172.21.15.148:0/3448880872 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0003f30 con 0x7fb4c00051c0 2024-03-29T04:10:26.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.309+0000 7fb4d6ffd640 1 -- 172.21.15.148:0/3448880872 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4d0004be0 con 0x7fb4c00051c0 2024-03-29T04:10:26.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.309+0000 7fb4d6ffd640 1 -- 172.21.15.148:0/3448880872 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0006070 con 0x7fb4c00051c0 2024-03-29T04:10:26.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.309+0000 7f85901f4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f858806d860 con 0x7f85880b40b0 2024-03-29T04:10:26.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.309+0000 7f858e791640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f8588072a00 0x7f8588164060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.148:38710/0 (socket says 172.21.15.148:38710) 2024-03-29T04:10:26.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.309+0000 7f858e791640 1 -- 172.21.15.148:0/2636459707 learned_addr learned my addr 172.21.15.148:0/2636459707 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:26.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.309+0000 7f50974c8640 1 -- 172.21.15.148:0/4238295984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50900cef80 msgr2=0x7f50900b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.309+0000 7f50974c8640 1 --2- 172.21.15.148:0/4238295984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50900cef80 0x7f50900b7710 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f50900b6e00 tx=0x7f508c02b910 comp rx=0 tx=0).stop 2024-03-29T04:10:26.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.309+0000 7f85901f4640 1 -- 172.21.15.148:0/2636459707 --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85880b46e0 con 0x7f8588072a00 2024-03-29T04:10:26.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.309+0000 7f50974c8640 1 -- 172.21.15.148:0/4238295984 shutdown_connections 2024-03-29T04:10:26.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.309+0000 7f85901f4640 1 -- 172.21.15.148:0/2636459707 --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f85880b52f0 con 0x7f858815aab0 2024-03-29T04:10:26.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.309+0000 7f50974c8640 1 --2- 172.21.15.148:0/4238295984 >> v2:172.21.15.148:3300/0 conn(0x7f50900b4c40 0x7f50900b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.309+0000 7f858df90640 1 --2- 172.21.15.148:0/2636459707 >> v2:172.21.15.148:3300/0 conn(0x7f858815aab0 0x7f85880b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.309+0000 7f50974c8640 1 --2- 172.21.15.148:0/4238295984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50900cef80 0x7f50900b7710 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.309+0000 7f50974c8640 1 -- 172.21.15.148:0/4238295984 >> 172.21.15.148:0/4238295984 conn(0x7f50900d4300 msgr2=0x7f50900d4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.309+0000 7f50974c8640 1 -- 172.21.15.148:0/4238295984 shutdown_connections 2024-03-29T04:10:26.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7fb4de396640 1 -- 172.21.15.148:0/3448880872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4c00051c0 msgr2=0x7fb4c000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7fb4de396640 1 --2- 172.21.15.148:0/3448880872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4c00051c0 0x7fb4c000b9c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0004360 tx=0x7fb4c0005830 comp rx=0 tx=0).stop 2024-03-29T04:10:26.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7fb53311c640 1 --2- 172.21.15.148:0/388324995 >> [v2:172.21.15.148:6800/686342976,v1:172.21.15.148:6801/686342976] conn(0x7fb52c149010 0x7fb52c1493f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7f858cf8e640 1 -- 172.21.15.148:0/2636459707 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1555210171 0 0) 0x7f85880b46e0 con 0x7f8588072a00 2024-03-29T04:10:26.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7f858cf8e640 1 -- 172.21.15.148:0/2636459707 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8574003520 con 0x7f8588072a00 2024-03-29T04:10:26.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7fb53311c640 1 -- 172.21.15.148:0/388324995 --> [v2:172.21.15.148:6800/686342976,v1:172.21.15.148:6801/686342976] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb52c0ca830 con 0x7fb52c149010 2024-03-29T04:10:26.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7fb4de396640 1 -- 172.21.15.148:0/3448880872 shutdown_connections 2024-03-29T04:10:26.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7f858cf8e640 1 -- 172.21.15.148:0/2636459707 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 950890511 0 0) 0x7f858806d860 con 0x7f85880b40b0 2024-03-29T04:10:26.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7f858cf8e640 1 -- 172.21.15.148:0/2636459707 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85880b46e0 con 0x7f85880b40b0 2024-03-29T04:10:26.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7fb4de396640 1 --2- 172.21.15.148:0/3448880872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4c00051c0 0x7fb4c000b9c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7fb4de396640 1 -- 172.21.15.148:0/3448880872 >> 172.21.15.148:0/3448880872 conn(0x7fb4d80b2c60 msgr2=0x7fb4d80b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7f4a737fe640 1 -- 172.21.15.148:0/2778737557 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4a840055c0 con 0x7f4a8c0d2530 2024-03-29T04:10:26.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7f50974c8640 1 -- 172.21.15.148:0/4238295984 wait complete. 2024-03-29T04:10:26.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7fb4de396640 1 -- 172.21.15.148:0/3448880872 shutdown_connections 2024-03-29T04:10:26.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7fb4de396640 1 -- 172.21.15.148:0/3448880872 wait complete. 2024-03-29T04:10:26.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.311+0000 7f4a737fe640 1 --2- 172.21.15.148:0/2778737557 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4a7806c600 0x7f4a7806eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.311+0000 7f4a737fe640 1 -- 172.21.15.148:0/2778737557 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4a8408b870 con 0x7f4a8c0d2530 2024-03-29T04:10:26.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7f858cf8e640 1 -- 172.21.15.148:0/2636459707 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1540801193 0 0) 0x7f8574003520 con 0x7f8588072a00 2024-03-29T04:10:26.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7f858cf8e640 1 -- 172.21.15.148:0/2636459707 >> v2:172.21.15.148:3300/0 conn(0x7f858815aab0 msgr2=0x7f85880b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:26.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7f858cf8e640 1 --2- 172.21.15.148:0/2636459707 >> v2:172.21.15.148:3300/0 conn(0x7f858815aab0 0x7f85880b3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7f858cf8e640 1 -- 172.21.15.148:0/2636459707 >> v1:172.21.15.148:6789/0 conn(0x7f85880b40b0 legacy=0x7f858815ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7f858cf8e640 1 -- 172.21.15.148:0/2636459707 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85880b49c0 con 0x7f8588072a00 2024-03-29T04:10:26.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7f858cf8e640 1 -- 172.21.15.148:0/2636459707 <== mon.1 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f8584001670 con 0x7f8588072a00 2024-03-29T04:10:26.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7f858cf8e640 1 -- 172.21.15.148:0/2636459707 >> v1:172.21.15.157:6789/0 conn(0x7f8588072a00 legacy=0x7f8588164060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7f858cf8e640 1 --2- 172.21.15.148:0/2636459707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8574003b90 0x7f8574003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7f858cf8e640 1 --2- 172.21.15.148:0/2636459707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85740079a0 0x7f8574007e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7f858cf8e640 1 --2- 172.21.15.148:0/2636459707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8574008360 0x7f8574008810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7f858cf8e640 1 -- 172.21.15.148:0/2636459707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f85880b52f0 con 0x7f85740079a0 2024-03-29T04:10:26.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7f858cf8e640 1 -- 172.21.15.148:0/2636459707 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f85740090f0 con 0x7f8574008360 2024-03-29T04:10:26.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7f858cf8e640 1 -- 172.21.15.148:0/2636459707 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f85740092f0 con 0x7f8574003b90 2024-03-29T04:10:26.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7f858e791640 1 --2- 172.21.15.148:0/2636459707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8574008360 0x7f8574008810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.310+0000 7f858df90640 1 --2- 172.21.15.148:0/2636459707 >> v2:172.21.15.148:3300/0 conn(0x7f858815aab0 0x7f85880b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:26.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.311+0000 7f858e791640 1 -- 172.21.15.148:0/2636459707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8574003b90 msgr2=0x7f8574003f90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-29T04:10:26.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.311+0000 7f858df90640 1 --2- 172.21.15.148:0/2636459707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8574003b90 0x7f8574003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.311+0000 7f858e791640 1 --2- 172.21.15.148:0/2636459707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8574003b90 0x7f8574003f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.311+0000 7f858e791640 1 -- 172.21.15.148:0/2636459707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85740079a0 msgr2=0x7f8574007e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:10:26.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.311+0000 7f858e791640 1 --2- 172.21.15.148:0/2636459707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85740079a0 0x7f8574007e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.311+0000 7f858e791640 1 -- 172.21.15.148:0/2636459707 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85880b49c0 con 0x7f8574008360 2024-03-29T04:10:26.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.312+0000 7f50974c8640 1 Processor -- start 2024-03-29T04:10:26.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.312+0000 7f858e791640 1 -- 172.21.15.148:0/2636459707 reap_dead start 2024-03-29T04:10:26.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.312+0000 7f4a8a7fc640 1 --2- 172.21.15.148:0/2778737557 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4a7806c600 0x7f4a7806eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.312+0000 7fb4de396640 1 Processor -- start 2024-03-29T04:10:26.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.313+0000 7fb4de396640 1 -- start start 2024-03-29T04:10:26.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.312+0000 7f50974c8640 1 -- start start 2024-03-29T04:10:26.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.313+0000 7fb5316b9640 1 --2- 172.21.15.148:0/388324995 >> [v2:172.21.15.148:6800/686342976,v1:172.21.15.148:6801/686342976] conn(0x7fb52c149010 0x7fb52c1493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.313+0000 7f50974c8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50900b4c40 0x7f5090161070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.313+0000 7f50974c8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50900cef80 0x7f50900cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.313+0000 7f50974c8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50900d3430 0x7f5090152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.313+0000 7f4a8a7fc640 1 --2- 172.21.15.148:0/2778737557 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4a7806c600 0x7f4a7806eac0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c15a760 tx=0x7f4a7c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.313+0000 7f50974c8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f50900b4440 con 0x7f50900d3430 2024-03-29T04:10:26.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.313+0000 7f50974c8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f50900b4160 con 0x7f50900b4c40 2024-03-29T04:10:26.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.313+0000 7f50974c8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f50900cfa70 con 0x7f50900cef80 2024-03-29T04:10:26.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.313+0000 7f858e791640 1 --2- 172.21.15.148:0/2636459707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8574008360 0x7f8574008810 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f858401d290 tx=0x7f8574005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a28a4b64c090ec77 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.314+0000 7fb5316b9640 1 --2- 172.21.15.148:0/388324995 >> [v2:172.21.15.148:6800/686342976,v1:172.21.15.148:6801/686342976] conn(0x7fb52c149010 0x7fb52c1493f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.314+0000 7fb4de396640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4c00051c0 0x7fb4d815df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.314+0000 7fb4de396640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4d8154c20 0x7fb4d81550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.315+0000 7fb4d7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4c00051c0 0x7fb4d815df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.315+0000 7fb4d77fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4d8154c20 0x7fb4d81550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.315+0000 7fb4d77fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4d8154c20 0x7fb4d81550a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.148:44398/0 (socket says 172.21.15.148:44398) 2024-03-29T04:10:26.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.315+0000 7fb4d7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4c00051c0 0x7fb4d815df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48564/0 (socket says 172.21.15.148:48564) 2024-03-29T04:10:26.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.315+0000 7fb4de396640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4d81efc10 0x7fb4d81f0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.315+0000 7fb5297fa640 1 -- 172.21.15.148:0/388324995 <== osd.1 v2:172.21.15.148:6800/686342976 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb52c0ca830 con 0x7fb52c149010 2024-03-29T04:10:26.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.315+0000 7fb4d77fe640 1 -- 172.21.15.148:0/2551615609 learned_addr learned my addr 172.21.15.148:0/2551615609 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:26.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.315+0000 7fb4de396640 1 -- 172.21.15.148:0/2551615609 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d806dc90 con 0x7fb4c00051c0 2024-03-29T04:10:26.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.315+0000 7fb4de396640 1 -- 172.21.15.148:0/2551615609 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d806d490 con 0x7fb4d81efc10 2024-03-29T04:10:26.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.315+0000 7fb4dc933640 1 --2- 172.21.15.148:0/2551615609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4d81efc10 0x7fb4d81f0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.315+0000 7f4a91590640 1 --2- 172.21.15.148:0/2778737557 >> [v2:172.21.15.157:6816/1608291780,v1:172.21.15.157:6817/1608291780] conn(0x7f4a8c0b3810 0x7f4a8c0d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.315+0000 7fb4de396640 1 -- 172.21.15.148:0/2551615609 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb4d806de10 con 0x7fb4d8154c20 2024-03-29T04:10:26.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.315+0000 7f4a91590640 1 -- 172.21.15.148:0/2778737557 --> [v2:172.21.15.157:6816/1608291780,v1:172.21.15.157:6817/1608291780] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a8c157bf0 con 0x7f4a8c0b3810 2024-03-29T04:10:26.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.315+0000 7f4a8b7fe640 1 --2- 172.21.15.148:0/2778737557 >> [v2:172.21.15.157:6816/1608291780,v1:172.21.15.157:6817/1608291780] conn(0x7f4a8c0b3810 0x7f4a8c0d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.316+0000 7fb4d7fff640 1 -- 172.21.15.148:0/2551615609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4d8154c20 msgr2=0x7fb4d81550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.316+0000 7fb4d7fff640 1 --2- 172.21.15.148:0/2551615609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4d8154c20 0x7fb4d81550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.316+0000 7fb4d7fff640 1 -- 172.21.15.148:0/2551615609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4d81efc10 msgr2=0x7fb4d81f0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.316+0000 7fb4d7fff640 1 --2- 172.21.15.148:0/2551615609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4d81efc10 0x7fb4d81f0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.316+0000 7fb4d7fff640 1 -- 172.21.15.148:0/2551615609 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d81f05b0 con 0x7fb4c00051c0 2024-03-29T04:10:26.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.316+0000 7fb4d7fff640 1 --2- 172.21.15.148:0/2551615609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4c00051c0 0x7fb4d815df00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc011990 tx=0x7fb4cc011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.314+0000 7f5095a65640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50900d3430 0x7f5090152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.314+0000 7f5095a65640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50900d3430 0x7f5090152780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48562/0 (socket says 172.21.15.148:48562) 2024-03-29T04:10:26.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.314+0000 7f5095a65640 1 -- 172.21.15.148:0/508044913 learned_addr learned my addr 172.21.15.148:0/508044913 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:26.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.314+0000 7f5094a63640 1 --2- 172.21.15.148:0/508044913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50900cef80 0x7f50900cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.314+0000 7f5095a65640 1 -- 172.21.15.148:0/508044913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50900cef80 msgr2=0x7f50900cb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.314+0000 7f5095a65640 1 --2- 172.21.15.148:0/508044913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50900cef80 0x7f50900cb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.314+0000 7f5095a65640 1 -- 172.21.15.148:0/508044913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50900b4c40 msgr2=0x7f5090161070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.314+0000 7f5095a65640 1 --2- 172.21.15.148:0/508044913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50900b4c40 0x7f5090161070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.314+0000 7f5095a65640 1 -- 172.21.15.148:0/508044913 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50900cbc00 con 0x7f50900d3430 2024-03-29T04:10:26.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.315+0000 7f5095a65640 1 --2- 172.21.15.148:0/508044913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50900d3430 0x7f5090152780 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f508000d830 tx=0x7f508000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.315+0000 7f5085ffb640 1 -- 172.21.15.148:0/508044913 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508000b6c0 con 0x7f50900d3430 2024-03-29T04:10:26.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.315+0000 7f5085ffb640 1 -- 172.21.15.148:0/508044913 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5080019070 con 0x7f50900d3430 2024-03-29T04:10:26.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.315+0000 7f5085ffb640 1 -- 172.21.15.148:0/508044913 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508000baa0 con 0x7f50900d3430 2024-03-29T04:10:26.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.315+0000 7f858cf8e640 1 -- 172.21.15.148:0/2636459707 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858404a070 con 0x7f8574008360 2024-03-29T04:10:26.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.315+0000 7f858cf8e640 1 -- 172.21.15.148:0/2636459707 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f858403dae0 con 0x7f8574008360 2024-03-29T04:10:26.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.315+0000 7f858cf8e640 1 -- 172.21.15.148:0/2636459707 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8584018d40 con 0x7f8574008360 2024-03-29T04:10:26.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.316+0000 7f4a8b7fe640 1 --2- 172.21.15.148:0/2778737557 >> [v2:172.21.15.157:6816/1608291780,v1:172.21.15.157:6817/1608291780] conn(0x7f4a8c0b3810 0x7f4a8c0d2a10 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.316+0000 7fb4d4ff9640 1 -- 172.21.15.148:0/2551615609 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc00fa10 con 0x7fb4c00051c0 2024-03-29T04:10:26.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.317+0000 7fb4d4ff9640 1 -- 172.21.15.148:0/2551615609 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4cc00fbb0 con 0x7fb4c00051c0 2024-03-29T04:10:26.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.317+0000 7fb4d4ff9640 1 -- 172.21.15.148:0/2551615609 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc004020 con 0x7fb4c00051c0 2024-03-29T04:10:26.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.317+0000 7f50974c8640 1 -- 172.21.15.148:0/508044913 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50900cbee0 con 0x7f50900d3430 2024-03-29T04:10:26.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.317+0000 7f85901f4640 1 -- 172.21.15.148:0/2636459707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8574008360 msgr2=0x7f8574008810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.317+0000 7f85901f4640 1 --2- 172.21.15.148:0/2636459707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8574008360 0x7f8574008810 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f858401d290 tx=0x7f8574005830 comp rx=0 tx=0).stop 2024-03-29T04:10:26.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.317+0000 7f85901f4640 1 -- 172.21.15.148:0/2636459707 shutdown_connections 2024-03-29T04:10:26.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.317+0000 7f85901f4640 1 --2- 172.21.15.148:0/2636459707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8574008360 0x7f8574008810 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.317+0000 7f85901f4640 1 -- 172.21.15.148:0/2636459707 >> 172.21.15.148:0/2636459707 conn(0x7f858806b5b0 msgr2=0x7f858806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.317+0000 7fb4de396640 1 -- 172.21.15.148:0/2551615609 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d8158ce0 con 0x7fb4c00051c0 2024-03-29T04:10:26.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.317+0000 7fb4de396640 1 -- 172.21.15.148:0/2551615609 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d81592a0 con 0x7fb4c00051c0 2024-03-29T04:10:26.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.317+0000 7f50974c8640 1 -- 172.21.15.148:0/508044913 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50900cc4a0 con 0x7f50900d3430 2024-03-29T04:10:26.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.318+0000 7f4a737fe640 1 -- 172.21.15.148:0/2778737557 <== osd.5 v2:172.21.15.157:6816/1608291780 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a80030960 con 0x7f4a8c0b3810 2024-03-29T04:10:26.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.318+0000 7f50974c8640 1 -- 172.21.15.148:0/508044913 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f50900687e0 con 0x7f50900d3430 2024-03-29T04:10:26.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.319+0000 7f5085ffb640 1 -- 172.21.15.148:0/508044913 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f508001e460 con 0x7f50900d3430 2024-03-29T04:10:26.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.320+0000 7f5085ffb640 1 --2- 172.21.15.148:0/508044913 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f507406c600 0x7f507406eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.320+0000 7fc2a89d1640 1 Processor -- start 2024-03-29T04:10:26.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.320+0000 7f5085ffb640 1 -- 172.21.15.148:0/508044913 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f508008cb30 con 0x7f50900d3430 2024-03-29T04:10:26.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.320+0000 7f5085ffb640 1 --2- 172.21.15.148:0/508044913 >> [v2:172.21.15.157:6808/3576931649,v1:172.21.15.157:6809/3576931649] conn(0x7f5074075d80 0x7f50740781c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.320+0000 7f5085ffb640 1 -- 172.21.15.148:0/508044913 --> [v2:172.21.15.157:6808/3576931649,v1:172.21.15.157:6809/3576931649] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5074078890 con 0x7f5074075d80 2024-03-29T04:10:26.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.320+0000 7f5085ffb640 1 -- 172.21.15.148:0/508044913 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f508008cf20 con 0x7f50900d3430 2024-03-29T04:10:26.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.320+0000 7f5095264640 1 --2- 172.21.15.148:0/508044913 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f507406c600 0x7f507406eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.319+0000 7f85901f4640 1 -- 172.21.15.148:0/2636459707 shutdown_connections 2024-03-29T04:10:26.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.320+0000 7f85901f4640 1 -- 172.21.15.148:0/2636459707 wait complete. 2024-03-29T04:10:26.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.321+0000 7fb4de396640 1 -- 172.21.15.148:0/2551615609 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb4d80687e0 con 0x7fb4c00051c0 2024-03-29T04:10:26.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.321+0000 7f5094a63640 1 --2- 172.21.15.148:0/508044913 >> [v2:172.21.15.157:6808/3576931649,v1:172.21.15.157:6809/3576931649] conn(0x7f5074075d80 0x7f50740781c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.321+0000 7f5095264640 1 --2- 172.21.15.148:0/508044913 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f507406c600 0x7f507406eac0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f508c036000 tx=0x7f508c030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.320+0000 7fc2a89d1640 1 -- start start 2024-03-29T04:10:26.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.321+0000 7fc2a89d1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc29c0e75d0 0x7fc29c03bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.321+0000 7fc2a89d1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc29c053c00 0x7fc29c006ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.321+0000 7fc2a2ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc29c0554e0 0x7fc29c0534f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47236/0 (socket says 172.21.15.148:47236) 2024-03-29T04:10:26.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.322+0000 7fc2a89d1640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc29c057cc0 con 0x7fc29c0554e0 2024-03-29T04:10:26.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.322+0000 7fc2a89d1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fc29c058da0 con 0x7fc29c0e75d0 2024-03-29T04:10:26.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.322+0000 7fc2a37fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc29c0e75d0 0x7fc29c03bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.322+0000 7fc2a37fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc29c0e75d0 0x7fc29c03bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48576/0 (socket says 172.21.15.148:48576) 2024-03-29T04:10:26.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.322+0000 7fc2a37fe640 1 -- 172.21.15.148:0/4283181099 learned_addr learned my addr 172.21.15.148:0/4283181099 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:26.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.322+0000 7fc2a3fff640 1 --2- 172.21.15.148:0/4283181099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc29c053c00 0x7fc29c006ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.322+0000 7f85901f4640 1 Processor -- start 2024-03-29T04:10:26.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.322+0000 7f85901f4640 1 -- start start 2024-03-29T04:10:26.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.321+0000 7fb4d4ff9640 1 -- 172.21.15.148:0/2551615609 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb4cc012040 con 0x7fb4c00051c0 2024-03-29T04:10:26.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.321+0000 7fb4d4ff9640 1 --2- 172.21.15.148:0/2551615609 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb4c406c5b0 0x7fb4c406ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.322+0000 7fb4d4ff9640 1 -- 172.21.15.148:0/2551615609 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb4cc0578a0 con 0x7fb4c00051c0 2024-03-29T04:10:26.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.322+0000 7fb4d77fe640 1 --2- 172.21.15.148:0/2551615609 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb4c406c5b0 0x7fb4c406ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.322+0000 7fb4d4ff9640 1 --2- 172.21.15.148:0/2551615609 >> [v2:172.21.15.148:6816/466300755,v1:172.21.15.148:6817/466300755] conn(0x7fb4c4076980 0x7fb4c4078e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.322+0000 7fb4d4ff9640 1 -- 172.21.15.148:0/2551615609 --> [v2:172.21.15.148:6816/466300755,v1:172.21.15.148:6817/466300755] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb4c40794f0 con 0x7fb4c4076980 2024-03-29T04:10:26.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.322+0000 7fb4d4ff9640 1 -- 172.21.15.148:0/2551615609 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb4cc056230 con 0x7fb4c00051c0 2024-03-29T04:10:26.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.322+0000 7fb4dc933640 1 --2- 172.21.15.148:0/2551615609 >> [v2:172.21.15.148:6816/466300755,v1:172.21.15.148:6817/466300755] conn(0x7fb4c4076980 0x7fb4c4078e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.322+0000 7fc2a27fc640 1 -- 172.21.15.148:0/4283181099 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2584675499 0 0) 0x7fc29c057cc0 con 0x7fc29c0554e0 2024-03-29T04:10:26.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.322+0000 7fc2a89d1640 1 -- 172.21.15.148:0/4283181099 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc29c006850 con 0x7fc29c053c00 2024-03-29T04:10:26.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.323+0000 7fefa72d9640 1 Processor -- start 2024-03-29T04:10:26.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.323+0000 7fefa72d9640 1 -- start start 2024-03-29T04:10:26.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.323+0000 7fb4dc933640 1 --2- 172.21.15.148:0/2551615609 >> [v2:172.21.15.148:6816/466300755,v1:172.21.15.148:6817/466300755] conn(0x7fb4c4076980 0x7fb4c4078e20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.323+0000 7fb4d77fe640 1 --2- 172.21.15.148:0/2551615609 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb4c406c5b0 0x7fb4c406ea70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb4d815e8a0 tx=0x7fb4c8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.323+0000 7fc2a27fc640 1 -- 172.21.15.148:0/4283181099 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc294003520 con 0x7fc29c0554e0 2024-03-29T04:10:26.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.323+0000 7f85901f4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8574008360 0x7f85881f00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.323+0000 7f85901f4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f858813de40 0x7f8588140af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.324+0000 7f85901f4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f858813e2c0 0x7f8588141030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.324+0000 7f85901f4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8588143f40 con 0x7f858813e2c0 2024-03-29T04:10:26.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.324+0000 7f85901f4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f85880b4f80 con 0x7f858813de40 2024-03-29T04:10:26.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.324+0000 7f85901f4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f85881440c0 con 0x7f8574008360 2024-03-29T04:10:26.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.324+0000 7f5094a63640 1 --2- 172.21.15.148:0/508044913 >> [v2:172.21.15.157:6808/3576931649,v1:172.21.15.157:6809/3576931649] conn(0x7f5074075d80 0x7f50740781c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.324+0000 7fefa72d9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fefa015c370 0x7fefa00cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.324+0000 7fb4d4ff9640 1 -- 172.21.15.148:0/2551615609 <== osd.3 v2:172.21.15.148:6816/466300755 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb4c40794f0 con 0x7fb4c4076980 2024-03-29T04:10:26.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.325+0000 7fefa4874640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fefa00d1bc0 0x7fefa00d0290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47252/0 (socket says 172.21.15.148:47252) 2024-03-29T04:10:26.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.325+0000 7fefa4874640 1 -- 172.21.15.148:0/3917390129 learned_addr learned my addr 172.21.15.148:0/3917390129 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:26.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.324+0000 7fc2a37fe640 1 -- 172.21.15.148:0/4283181099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc29c053c00 msgr2=0x7fc29c006ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:26.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.324+0000 7fc2a37fe640 1 --2- 172.21.15.148:0/4283181099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc29c053c00 0x7fc29c006ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.324+0000 7fc2a37fe640 1 -- 172.21.15.148:0/4283181099 >> v1:172.21.15.148:6789/0 conn(0x7fc29c0554e0 legacy=0x7fc29c0534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.324+0000 7fc2a37fe640 1 -- 172.21.15.148:0/4283181099 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29c005f20 con 0x7fc29c0e75d0 2024-03-29T04:10:26.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.325+0000 7fefa72d9640 1 --2- 172.21.15.148:0/3917390129 >> v2:172.21.15.148:3300/0 conn(0x7fefa00ce620 0x7fefa00cead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.325+0000 7fefa72d9640 1 -- 172.21.15.148:0/3917390129 --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefa00b3b00 con 0x7fefa00d1bc0 2024-03-29T04:10:26.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.325+0000 7fefa72d9640 1 -- 172.21.15.148:0/3917390129 --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fefa00ccd90 con 0x7fefa00ce620 2024-03-29T04:10:26.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.325+0000 7fefa5075640 1 --2- 172.21.15.148:0/3917390129 >> v2:172.21.15.157:3300/0 conn(0x7fefa015c370 0x7fefa00cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.325+0000 7fefa72d9640 1 -- 172.21.15.148:0/3917390129 --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fefa00cc7d0 con 0x7fefa015c370 2024-03-29T04:10:26.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.325+0000 7fefa5876640 1 --2- 172.21.15.148:0/3917390129 >> v2:172.21.15.148:3300/0 conn(0x7fefa00ce620 0x7fefa00cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.325+0000 7fef97fff640 1 -- 172.21.15.148:0/3917390129 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 542793314 0 0) 0x7fefa00b3b00 con 0x7fefa00d1bc0 2024-03-29T04:10:26.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.325+0000 7fef97fff640 1 -- 172.21.15.148:0/3917390129 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef8c003520 con 0x7fefa00d1bc0 2024-03-29T04:10:26.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.325+0000 7f858df90640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8574008360 0x7f85881f00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.326+0000 7f858df90640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8574008360 0x7f85881f00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.148:44416/0 (socket says 172.21.15.148:44416) 2024-03-29T04:10:26.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.326+0000 7f858df90640 1 -- 172.21.15.148:0/3878150777 learned_addr learned my addr 172.21.15.148:0/3878150777 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:26.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.326+0000 7fefa5876640 1 -- 172.21.15.148:0/3917390129 >> v2:172.21.15.157:3300/0 conn(0x7fefa015c370 msgr2=0x7fefa00cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:26.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.326+0000 7fefa5876640 1 --2- 172.21.15.148:0/3917390129 >> v2:172.21.15.157:3300/0 conn(0x7fefa015c370 0x7fefa00cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.326+0000 7fefa5876640 1 -- 172.21.15.148:0/3917390129 >> v1:172.21.15.148:6789/0 conn(0x7fefa00d1bc0 legacy=0x7fefa00d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.326+0000 7fef97fff640 1 -- 172.21.15.148:0/3917390129 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 115919709 0 0) 0x7fef8c003520 con 0x7fefa00d1bc0 2024-03-29T04:10:26.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.326+0000 7fefa5876640 1 -- 172.21.15.148:0/3917390129 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa00b2730 con 0x7fefa00ce620 2024-03-29T04:10:26.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.326+0000 7fc2a37fe640 1 --2- 172.21.15.148:0/4283181099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc29c0e75d0 0x7fc29c03bd80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc298009870 tx=0x7fc29802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a20485ef25dab777 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.326+0000 7f5085ffb640 1 -- 172.21.15.148:0/508044913 <== osd.6 v2:172.21.15.157:6808/3576931649 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5074078890 con 0x7f5074075d80 2024-03-29T04:10:26.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.326+0000 7f858e791640 1 --2- 172.21.15.148:0/3878150777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f858813e2c0 0x7f8588141030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.326+0000 7fc2a27fc640 1 -- 172.21.15.148:0/4283181099 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc298035020 con 0x7fc29c0e75d0 2024-03-29T04:10:26.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.326+0000 7fc2a27fc640 1 -- 172.21.15.148:0/4283181099 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc298031640 con 0x7fc29c0e75d0 2024-03-29T04:10:26.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.326+0000 7fc2a27fc640 1 -- 172.21.15.148:0/4283181099 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29802d040 con 0x7fc29c0e75d0 2024-03-29T04:10:26.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.326+0000 7f858d78f640 1 --2- 172.21.15.148:0/3878150777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f858813de40 0x7f8588140af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.327+0000 7fefa5876640 1 --2- 172.21.15.148:0/3917390129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefa00ce620 0x7fefa00cead0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fefa00cd880 tx=0x7fef9802bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e5790f2f4b01b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.327+0000 7f858e791640 1 -- 172.21.15.148:0/3878150777 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8574008360 msgr2=0x7f85881f00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.327+0000 7f858e791640 1 --2- 172.21.15.148:0/3878150777 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8574008360 0x7f85881f00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.327+0000 7f858e791640 1 -- 172.21.15.148:0/3878150777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f858813de40 msgr2=0x7f8588140af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.327+0000 7f858e791640 1 --2- 172.21.15.148:0/3878150777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f858813de40 0x7f8588140af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.327+0000 7f858e791640 1 -- 172.21.15.148:0/3878150777 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f858813aeb0 con 0x7f858813e2c0 2024-03-29T04:10:26.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.327+0000 7f858d78f640 1 --2- 172.21.15.148:0/3878150777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f858813de40 0x7f8588140af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:26.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.327+0000 7f858e791640 1 --2- 172.21.15.148:0/3878150777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f858813e2c0 0x7f8588141030 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f85880b55b0 tx=0x7f8584001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.327+0000 7f857e7fc640 1 -- 172.21.15.148:0/3878150777 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8584052070 con 0x7f858813e2c0 2024-03-29T04:10:26.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.327+0000 7f857e7fc640 1 -- 172.21.15.148:0/3878150777 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8584004e50 con 0x7f858813e2c0 2024-03-29T04:10:26.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.327+0000 7f857e7fc640 1 -- 172.21.15.148:0/3878150777 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858404a040 con 0x7f858813e2c0 2024-03-29T04:10:26.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.328+0000 7fc2a89d1640 1 -- 172.21.15.148:0/4283181099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc29c0e75d0 msgr2=0x7fc29c03bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.328+0000 7fc2a89d1640 1 --2- 172.21.15.148:0/4283181099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc29c0e75d0 0x7fc29c03bd80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc298009870 tx=0x7fc29802b8f0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.327+0000 7f85901f4640 1 -- 172.21.15.148:0/3878150777 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8588157ce0 con 0x7f858813e2c0 2024-03-29T04:10:26.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.327+0000 7f85901f4640 1 -- 172.21.15.148:0/3878150777 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f858813b430 con 0x7f858813e2c0 2024-03-29T04:10:26.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.329+0000 7f856bfff640 1 -- 172.21.15.148:0/3878150777 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8564001030 con 0x7f858813e2c0 2024-03-29T04:10:26.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.330+0000 7fef97fff640 1 -- 172.21.15.148:0/3917390129 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98039070 con 0x7fefa00ce620 2024-03-29T04:10:26.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.330+0000 7fef97fff640 1 -- 172.21.15.148:0/3917390129 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef9802c830 con 0x7fefa00ce620 2024-03-29T04:10:26.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.330+0000 7fef97fff640 1 -- 172.21.15.148:0/3917390129 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef90001670 con 0x7fefa00ce620 2024-03-29T04:10:26.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.331+0000 7fefa72d9640 1 -- 172.21.15.148:0/3917390129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefa00ce620 msgr2=0x7fefa00cead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.331+0000 7fefa72d9640 1 --2- 172.21.15.148:0/3917390129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefa00ce620 0x7fefa00cead0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fefa00cd880 tx=0x7fef9802bd00 comp rx=0 tx=0).stop 2024-03-29T04:10:26.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.329+0000 7fc2a89d1640 1 -- 172.21.15.148:0/4283181099 shutdown_connections 2024-03-29T04:10:26.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.329+0000 7fc2a89d1640 1 --2- 172.21.15.148:0/4283181099 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc29c053c00 0x7fc29c006ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.329+0000 7fc2a89d1640 1 --2- 172.21.15.148:0/4283181099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc29c0e75d0 0x7fc29c03bd80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.329+0000 7fc2a89d1640 1 -- 172.21.15.148:0/4283181099 >> 172.21.15.148:0/4283181099 conn(0x7fc29c03aa20 msgr2=0x7fc29c03ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.332+0000 7fc2a89d1640 1 -- 172.21.15.148:0/4283181099 shutdown_connections 2024-03-29T04:10:26.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.332+0000 7fefa72d9640 1 -- 172.21.15.148:0/3917390129 shutdown_connections 2024-03-29T04:10:26.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.332+0000 7fefa72d9640 1 --2- 172.21.15.148:0/3917390129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefa00ce620 0x7fefa00cead0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.332+0000 7fefa72d9640 1 --2- 172.21.15.148:0/3917390129 >> v2:172.21.15.157:3300/0 conn(0x7fefa015c370 0x7fefa00cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.332+0000 7fefa72d9640 1 -- 172.21.15.148:0/3917390129 >> 172.21.15.148:0/3917390129 conn(0x7fefa00b3d50 msgr2=0x7fefa00b4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.332+0000 7fefa72d9640 1 -- 172.21.15.148:0/3917390129 shutdown_connections 2024-03-29T04:10:26.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.332+0000 7fefa72d9640 1 -- 172.21.15.148:0/3917390129 wait complete. 2024-03-29T04:10:26.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.332+0000 7fc2a89d1640 1 -- 172.21.15.148:0/4283181099 wait complete. 2024-03-29T04:10:26.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.334+0000 7fc2a89d1640 1 Processor -- start 2024-03-29T04:10:26.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.334+0000 7fefa72d9640 1 Processor -- start 2024-03-29T04:10:26.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.334+0000 7fefa72d9640 1 -- start start 2024-03-29T04:10:26.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.335+0000 7fefa72d9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefa00ce620 0x7fefa014e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.335+0000 7fefa72d9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefa00d1bc0 0x7fefa015a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.335+0000 7fefa72d9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefa015c370 0x7fefa015ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.335+0000 7fefa72d9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fefa00b5050 con 0x7fefa00d1bc0 2024-03-29T04:10:26.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.335+0000 7fefa72d9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fefa00cca20 con 0x7fefa00ce620 2024-03-29T04:10:26.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.335+0000 7fc2a89d1640 1 -- start start 2024-03-29T04:10:26.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.336+0000 7fc2a89d1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc29c053c00 0x7fc29c0e0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.336+0000 7fc2a89d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc29c0554e0 0x7fc29c051010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.336+0000 7fc2a89d1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc29c0e75d0 0x7fc29c051550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.336+0000 7fc2a89d1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c057fa0 con 0x7fc29c0554e0 2024-03-29T04:10:26.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.336+0000 7fc2a89d1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c0064e0 con 0x7fc29c0e75d0 2024-03-29T04:10:26.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.336+0000 7fc2a89d1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc29c0e79b0 con 0x7fc29c053c00 2024-03-29T04:10:26.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.335+0000 7fefa72d9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fefa00b51d0 con 0x7fefa015c370 2024-03-29T04:10:26.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.336+0000 7fefa4874640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefa00d1bc0 0x7fefa015a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.336+0000 7fefa4874640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefa00d1bc0 0x7fefa015a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48598/0 (socket says 172.21.15.148:48598) 2024-03-29T04:10:26.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.337+0000 7fefa4874640 1 -- 172.21.15.148:0/3262938009 learned_addr learned my addr 172.21.15.148:0/3262938009 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:26.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.337+0000 7fc2a3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc29c0e75d0 0x7fc29c051550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.337+0000 7fefa5075640 1 --2- 172.21.15.148:0/3262938009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefa00ce620 0x7fefa014e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.337+0000 7fc2a3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc29c0e75d0 0x7fc29c051550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:39398/0 (socket says 172.21.15.148:39398) 2024-03-29T04:10:26.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.337+0000 7fc2a3fff640 1 -- 172.21.15.148:0/2807083736 learned_addr learned my addr 172.21.15.148:0/2807083736 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:26.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.337+0000 7fefa4874640 1 -- 172.21.15.148:0/3262938009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefa015c370 msgr2=0x7fefa015ac50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-29T04:10:26.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.337+0000 7fefa4874640 1 --2- 172.21.15.148:0/3262938009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fefa015c370 0x7fefa015ac50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.337+0000 7f857e7fc640 1 -- 172.21.15.148:0/3878150777 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f85840065a0 con 0x7f858813e2c0 2024-03-29T04:10:26.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.337+0000 7fefa4874640 1 -- 172.21.15.148:0/3262938009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefa00ce620 msgr2=0x7fefa014e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.337+0000 7fefa4874640 1 --2- 172.21.15.148:0/3262938009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefa00ce620 0x7fefa014e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.337+0000 7fefa4874640 1 -- 172.21.15.148:0/3262938009 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa00c6200 con 0x7fefa00d1bc0 2024-03-29T04:10:26.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.338+0000 7fc2a3fff640 1 -- 172.21.15.148:0/2807083736 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc29c053c00 msgr2=0x7fc29c0e0bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T04:10:26.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.338+0000 7fc2a3fff640 1 --2- 172.21.15.148:0/2807083736 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc29c053c00 0x7fc29c0e0bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.338+0000 7fc2a3fff640 1 -- 172.21.15.148:0/2807083736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc29c0554e0 msgr2=0x7fc29c051010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.338+0000 7fc2a3fff640 1 --2- 172.21.15.148:0/2807083736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc29c0554e0 0x7fc29c051010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.338+0000 7fc2a3fff640 1 -- 172.21.15.148:0/2807083736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29c04ff70 con 0x7fc29c0e75d0 2024-03-29T04:10:26.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.338+0000 7fefa4874640 1 --2- 172.21.15.148:0/3262938009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefa00d1bc0 0x7fefa015a710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fef9000c750 tx=0x7fef9000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.338+0000 7f857e7fc640 1 --2- 172.21.15.148:0/3878150777 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f856c06c560 0x7f856c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.338+0000 7f857e7fc640 1 -- 172.21.15.148:0/3878150777 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f85840bed60 con 0x7f858813e2c0 2024-03-29T04:10:26.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.338+0000 7f857e7fc640 1 --2- 172.21.15.148:0/3878150777 >> [v2:172.21.15.157:6800/1932472006,v1:172.21.15.157:6801/1932472006] conn(0x7f856c075c50 0x7f856c078090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.338+0000 7f857e7fc640 1 -- 172.21.15.148:0/3878150777 --> [v2:172.21.15.157:6800/1932472006,v1:172.21.15.157:6801/1932472006] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f858405ce50 con 0x7f856c075c50 2024-03-29T04:10:26.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.338+0000 7f857e7fc640 1 -- 172.21.15.148:0/3878150777 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f85840bf1f0 con 0x7f858813e2c0 2024-03-29T04:10:26.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.338+0000 7f858df90640 1 --2- 172.21.15.148:0/3878150777 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f856c06c560 0x7f856c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.339+0000 7fef95ffb640 1 -- 172.21.15.148:0/3262938009 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef90017070 con 0x7fefa00d1bc0 2024-03-29T04:10:26.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.339+0000 7fefa5075640 1 --2- 172.21.15.148:0/3262938009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefa00ce620 0x7fefa014e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:26.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.339+0000 7fc2a3fff640 1 --2- 172.21.15.148:0/2807083736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc29c0e75d0 0x7fc29c051550 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc2a402d940 tx=0x7fc2a4069280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.339+0000 7fef95ffb640 1 -- 172.21.15.148:0/3262938009 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef90005ad0 con 0x7fefa00d1bc0 2024-03-29T04:10:26.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.339+0000 7fef95ffb640 1 -- 172.21.15.148:0/3262938009 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef90017070 con 0x7fefa00d1bc0 2024-03-29T04:10:26.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.339+0000 7f858df90640 1 --2- 172.21.15.148:0/3878150777 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f856c06c560 0x7f856c06ea20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8580009fd0 tx=0x7f858000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.339+0000 7fc28ffff640 1 -- 172.21.15.148:0/2807083736 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a40743f0 con 0x7fc29c0e75d0 2024-03-29T04:10:26.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.339+0000 7f858d78f640 1 --2- 172.21.15.148:0/3878150777 >> [v2:172.21.15.157:6800/1932472006,v1:172.21.15.157:6801/1932472006] conn(0x7f856c075c50 0x7f856c078090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.339+0000 7fefa72d9640 1 -- 172.21.15.148:0/3262938009 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa00c64e0 con 0x7fefa00d1bc0 2024-03-29T04:10:26.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.339+0000 7fefa72d9640 1 -- 172.21.15.148:0/3262938009 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa00c6aa0 con 0x7fefa00d1bc0 2024-03-29T04:10:26.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.339+0000 7fc2a89d1640 1 -- 172.21.15.148:0/2807083736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc29c050250 con 0x7fc29c0e75d0 2024-03-29T04:10:26.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.339+0000 7fc2a89d1640 1 -- 172.21.15.148:0/2807083736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc29c050810 con 0x7fc29c0e75d0 2024-03-29T04:10:26.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.340+0000 7fc28ffff640 1 -- 172.21.15.148:0/2807083736 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2a4074a20 con 0x7fc29c0e75d0 2024-03-29T04:10:26.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.340+0000 7fc28ffff640 1 -- 172.21.15.148:0/2807083736 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4074e10 con 0x7fc29c0e75d0 2024-03-29T04:10:26.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.340+0000 7fefa72d9640 1 -- 172.21.15.148:0/3262938009 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fefa00687e0 con 0x7fefa00d1bc0 2024-03-29T04:10:26.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.341+0000 7f858d78f640 1 --2- 172.21.15.148:0/3878150777 >> [v2:172.21.15.157:6800/1932472006,v1:172.21.15.157:6801/1932472006] conn(0x7f856c075c50 0x7f856c078090 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.342+0000 7f857e7fc640 1 -- 172.21.15.148:0/3878150777 <== osd.4 v2:172.21.15.157:6800/1932472006 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f858405ce50 con 0x7f856c075c50 2024-03-29T04:10:26.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.342+0000 7fc28ffff640 1 -- 172.21.15.148:0/2807083736 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc2a4094420 con 0x7fc29c0e75d0 2024-03-29T04:10:26.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.342+0000 7fc2a89d1640 1 -- 172.21.15.148:0/2807083736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc284001030 con 0x7fc29c0e75d0 2024-03-29T04:10:26.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.343+0000 7fc28ffff640 1 --2- 172.21.15.148:0/2807083736 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc28806c600 0x7fc28806eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.342+0000 7fef95ffb640 1 -- 172.21.15.148:0/3262938009 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fef9001a070 con 0x7fefa00d1bc0 2024-03-29T04:10:26.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.343+0000 7fef95ffb640 1 --2- 172.21.15.148:0/3262938009 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fef8406c930 0x7fef8406edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.343+0000 7fefa5075640 1 --2- 172.21.15.148:0/3262938009 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fef8406c930 0x7fef8406edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.343+0000 7fef95ffb640 1 -- 172.21.15.148:0/3262938009 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fef900058d0 con 0x7fefa00d1bc0 2024-03-29T04:10:26.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.343+0000 7fef95ffb640 1 --2- 172.21.15.148:0/3262938009 >> [v2:172.21.15.148:6824/1406601759,v1:172.21.15.148:6825/1406601759] conn(0x7fef84076c00 0x7fef840790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.343+0000 7fef95ffb640 1 -- 172.21.15.148:0/3262938009 --> [v2:172.21.15.148:6824/1406601759,v1:172.21.15.148:6825/1406601759] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fef84079770 con 0x7fef84076c00 2024-03-29T04:10:26.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.343+0000 7fef95ffb640 1 -- 172.21.15.148:0/3262938009 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fef90055a60 con 0x7fefa00d1bc0 2024-03-29T04:10:26.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.344+0000 7fefa5876640 1 --2- 172.21.15.148:0/3262938009 >> [v2:172.21.15.148:6824/1406601759,v1:172.21.15.148:6825/1406601759] conn(0x7fef84076c00 0x7fef840790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.344+0000 7fefa5075640 1 --2- 172.21.15.148:0/3262938009 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fef8406c930 0x7fef8406edf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fefa014ed40 tx=0x7fef9c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.344+0000 7fcc9b462640 1 Processor -- start 2024-03-29T04:10:26.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.344+0000 7fc2a37fe640 1 --2- 172.21.15.148:0/2807083736 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc28806c600 0x7fc28806eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.344+0000 7fcc9b462640 1 -- start start 2024-03-29T04:10:26.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.345+0000 7fc2a37fe640 1 --2- 172.21.15.148:0/2807083736 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc28806c600 0x7fc28806eac0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc298036000 tx=0x7fc298008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.345+0000 7fc28ffff640 1 -- 172.21.15.148:0/2807083736 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc2a4106340 con 0x7fc29c0e75d0 2024-03-29T04:10:26.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.345+0000 7fc28ffff640 1 --2- 172.21.15.148:0/2807083736 >> [v2:172.21.15.148:6808/90026237,v1:172.21.15.148:6809/90026237] conn(0x7fc288075d00 0x7fc288078140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.345+0000 7fc28ffff640 1 -- 172.21.15.148:0/2807083736 --> [v2:172.21.15.148:6808/90026237,v1:172.21.15.148:6809/90026237] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc288078810 con 0x7fc288075d00 2024-03-29T04:10:26.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.345+0000 7fc28ffff640 1 -- 172.21.15.148:0/2807083736 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc2a410d050 con 0x7fc29c0e75d0 2024-03-29T04:10:26.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.346+0000 7fcc9b462640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcc8c0dfef0 0x7fcc8c0085d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.346+0000 7fcc9b462640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc8c056590 con 0x7fcc8c0e02d0 2024-03-29T04:10:26.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.346+0000 7fcc9b462640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc8c056870 con 0x7fcc8c0e42d0 2024-03-29T04:10:26.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.346+0000 7fcc9b462640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fcc8c0094b0 con 0x7fcc8c0dfef0 2024-03-29T04:10:26.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.345+0000 7fefa5876640 1 --2- 172.21.15.148:0/3262938009 >> [v2:172.21.15.148:6824/1406601759,v1:172.21.15.148:6825/1406601759] conn(0x7fef84076c00 0x7fef840790a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.346+0000 7fcc9ac61640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fcc8c0e42d0 0x7fcc8c0539f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.148:38718/0 (socket says 172.21.15.148:38718) 2024-03-29T04:10:26.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.346+0000 7fef95ffb640 1 -- 172.21.15.148:0/3262938009 <== osd.0 v2:172.21.15.148:6824/1406601759 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fef84079770 con 0x7fef84076c00 2024-03-29T04:10:26.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.346+0000 7fc2a2ffd640 1 --2- 172.21.15.148:0/2807083736 >> [v2:172.21.15.148:6808/90026237,v1:172.21.15.148:6809/90026237] conn(0x7fc288075d00 0x7fc288078140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.346+0000 7fcc9ac61640 1 -- 172.21.15.148:0/4040683491 learned_addr learned my addr 172.21.15.148:0/4040683491 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:26.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.346+0000 7f4a91590640 1 -- 172.21.15.148:0/2778737557 --> [v2:172.21.15.157:6816/1608291780,v1:172.21.15.157:6817/1608291780] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4a8c0687e0 con 0x7f4a8c0b3810 2024-03-29T04:10:26.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.347+0000 7fcc9945e640 1 -- 172.21.15.148:0/4040683491 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3423908447 0 0) 0x7fcc8c056870 con 0x7fcc8c0e42d0 2024-03-29T04:10:26.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.347+0000 7fcc9945e640 1 -- 172.21.15.148:0/4040683491 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc80003520 con 0x7fcc8c0e42d0 2024-03-29T04:10:26.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.348+0000 7fb53311c640 1 -- 172.21.15.148:0/388324995 --> [v2:172.21.15.148:6800/686342976,v1:172.21.15.148:6801/686342976] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb52c0687e0 con 0x7fb52c149010 2024-03-29T04:10:26.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.348+0000 7fb5297fa640 1 -- 172.21.15.148:0/388324995 <== osd.1 v2:172.21.15.148:6800/686342976 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb52c0687e0 con 0x7fb52c149010 2024-03-29T04:10:26.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.347+0000 7fc2a2ffd640 1 --2- 172.21.15.148:0/2807083736 >> [v2:172.21.15.148:6808/90026237,v1:172.21.15.148:6809/90026237] conn(0x7fc288075d00 0x7fc288078140 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.347+0000 7f4a737fe640 1 -- 172.21.15.148:0/2778737557 <== osd.5 v2:172.21.15.157:6816/1608291780 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4a8c0687e0 con 0x7f4a8c0b3810 2024-03-29T04:10:26.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.347+0000 7fcc9945e640 1 -- 172.21.15.148:0/4040683491 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1972883570 0 0) 0x7fcc8c056590 con 0x7fcc8c0e02d0 2024-03-29T04:10:26.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.347+0000 7fcc9945e640 1 -- 172.21.15.148:0/4040683491 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc8c056870 con 0x7fcc8c0e02d0 2024-03-29T04:10:26.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.347+0000 7fcc9945e640 1 -- 172.21.15.148:0/4040683491 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 324940288 0 0) 0x7fcc80003520 con 0x7fcc8c0e42d0 2024-03-29T04:10:26.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.347+0000 7fcc9945e640 1 -- 172.21.15.148:0/4040683491 >> v2:172.21.15.148:3300/0 conn(0x7fcc8c0dfef0 msgr2=0x7fcc8c0085d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:26.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.347+0000 7fcc9a460640 1 --2- 172.21.15.148:0/4040683491 >> v2:172.21.15.148:3300/0 conn(0x7fcc8c0dfef0 0x7fcc8c0085d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.347+0000 7fcc9945e640 1 --2- 172.21.15.148:0/4040683491 >> v2:172.21.15.148:3300/0 conn(0x7fcc8c0dfef0 0x7fcc8c0085d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.347+0000 7fcc9945e640 1 -- 172.21.15.148:0/4040683491 >> v1:172.21.15.148:6789/0 conn(0x7fcc8c0e02d0 legacy=0x7fcc8c0e3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.347+0000 7fcc9945e640 1 -- 172.21.15.148:0/4040683491 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8c008e60 con 0x7fcc8c0e42d0 2024-03-29T04:10:26.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.347+0000 7fcc9945e640 1 -- 172.21.15.148:0/4040683491 <== mon.1 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7fcc94050140 con 0x7fcc8c0e42d0 2024-03-29T04:10:26.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.348+0000 7fcc9945e640 1 -- 172.21.15.148:0/4040683491 >> v1:172.21.15.157:6789/0 conn(0x7fcc8c0e42d0 legacy=0x7fcc8c0539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.348+0000 7fcc9945e640 1 --2- 172.21.15.148:0/4040683491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc80003b40 0x7fcc80003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.348+0000 7fcc9945e640 1 --2- 172.21.15.148:0/4040683491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc80004800 0x7fcc80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.348+0000 7fcc9945e640 1 --2- 172.21.15.148:0/4040683491 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc80005210 0x7fcc8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.348+0000 7fcc9945e640 1 -- 172.21.15.148:0/4040683491 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8c0094b0 con 0x7fcc80003b40 2024-03-29T04:10:26.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.348+0000 7fcc9945e640 1 -- 172.21.15.148:0/4040683491 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8000c090 con 0x7fcc80004800 2024-03-29T04:10:26.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.348+0000 7fcc9945e640 1 -- 172.21.15.148:0/4040683491 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcc8000c290 con 0x7fcc80005210 2024-03-29T04:10:26.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.348+0000 7fcc9ac61640 1 --2- 172.21.15.148:0/4040683491 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc80005210 0x7fcc8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.349+0000 7fcc9ac61640 1 -- 172.21.15.148:0/4040683491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc80004800 msgr2=0x7fcc80004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-29T04:10:26.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.349+0000 7fcc9ac61640 1 --2- 172.21.15.148:0/4040683491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc80004800 0x7fcc80004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.349+0000 7fcc9ac61640 1 -- 172.21.15.148:0/4040683491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc80003b40 msgr2=0x7fcc80003f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:10:26.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.349+0000 7fcc9ac61640 1 --2- 172.21.15.148:0/4040683491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc80003b40 0x7fcc80003f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.349+0000 7fcc9ac61640 1 -- 172.21.15.148:0/4040683491 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc80003520 con 0x7fcc80005210 2024-03-29T04:10:26.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.349+0000 7fc28ffff640 1 -- 172.21.15.148:0/2807083736 <== osd.2 v2:172.21.15.148:6808/90026237 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc288078810 con 0x7fc288075d00 2024-03-29T04:10:26.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.349+0000 7fb53311c640 1 -- 172.21.15.148:0/388324995 >> [v2:172.21.15.148:6800/686342976,v1:172.21.15.148:6801/686342976] conn(0x7fb52c149010 msgr2=0x7fb52c1493f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.349+0000 7fb53311c640 1 --2- 172.21.15.148:0/388324995 >> [v2:172.21.15.148:6800/686342976,v1:172.21.15.148:6801/686342976] conn(0x7fb52c149010 0x7fb52c1493f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.349+0000 7fb53311c640 1 -- 172.21.15.148:0/388324995 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb51806c580 msgr2=0x7fb51806ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.349+0000 7fb53311c640 1 --2- 172.21.15.148:0/388324995 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb51806c580 0x7fb51806ea40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb524035000 tx=0x7fb524030000 comp rx=0 tx=0).stop 2024-03-29T04:10:26.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.349+0000 7fb53311c640 1 -- 172.21.15.148:0/388324995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb52c0c75e0 msgr2=0x7fb52c0c92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.349+0000 7fb53311c640 1 --2- 172.21.15.148:0/388324995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb52c0c75e0 0x7fb52c0c92d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb52000e9b0 tx=0x7fb52000ee70 comp rx=0 tx=0).stop 2024-03-29T04:10:26.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.349+0000 7fb4de396640 1 -- 172.21.15.148:0/2551615609 --> [v2:172.21.15.148:6816/466300755,v1:172.21.15.148:6817/466300755] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb4d81444d0 con 0x7fb4c4076980 2024-03-29T04:10:26.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.349+0000 7fb5316b9640 1 -- 172.21.15.148:0/388324995 reap_dead start 2024-03-29T04:10:26.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.349+0000 7fb53311c640 1 -- 172.21.15.148:0/388324995 shutdown_connections 2024-03-29T04:10:26.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.349+0000 7fb53311c640 1 -- 172.21.15.148:0/388324995 >> 172.21.15.148:0/388324995 conn(0x7fb52c0cec80 msgr2=0x7fb52c0b1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.349+0000 7fb53311c640 1 -- 172.21.15.148:0/388324995 shutdown_connections 2024-03-29T04:10:26.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.349+0000 7fcc9ac61640 1 -- 172.21.15.148:0/4040683491 reap_dead start 2024-03-29T04:10:26.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.350+0000 7fb53311c640 1 -- 172.21.15.148:0/388324995 wait complete. 2024-03-29T04:10:26.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.350+0000 7fcc9a460640 1 --2- 172.21.15.148:0/4040683491 >> v2:172.21.15.148:3300/0 conn(0x7fcc8c0dfef0 0x7fcc8c0085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:26.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.350+0000 7fcc9ac61640 1 --2- 172.21.15.148:0/4040683491 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc80005210 0x7fcc8000b9c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcc94000b60 tx=0x7fcc80005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5212366f0b54b78 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.350+0000 7fb4d4ff9640 1 -- 172.21.15.148:0/2551615609 <== osd.3 v2:172.21.15.148:6816/466300755 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb4d81444d0 con 0x7fb4c4076980 2024-03-29T04:10:26.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.350+0000 7fcc9945e640 1 -- 172.21.15.148:0/4040683491 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94068930 con 0x7fcc80005210 2024-03-29T04:10:26.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.350+0000 7fcc9945e640 1 -- 172.21.15.148:0/4040683491 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc9406d1b0 con 0x7fcc80005210 2024-03-29T04:10:26.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.350+0000 7fcc9945e640 1 -- 172.21.15.148:0/4040683491 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9406b5b0 con 0x7fcc80005210 2024-03-29T04:10:26.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.350+0000 7fb4de396640 1 -- 172.21.15.148:0/2551615609 >> [v2:172.21.15.148:6816/466300755,v1:172.21.15.148:6817/466300755] conn(0x7fb4c4076980 msgr2=0x7fb4c4078e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.350+0000 7fb4de396640 1 --2- 172.21.15.148:0/2551615609 >> [v2:172.21.15.148:6816/466300755,v1:172.21.15.148:6817/466300755] conn(0x7fb4c4076980 0x7fb4c4078e20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.351+0000 7fcc9b462640 1 -- 172.21.15.148:0/4040683491 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc80005210 msgr2=0x7fcc8000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.351+0000 7fcc9b462640 1 --2- 172.21.15.148:0/4040683491 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc80005210 0x7fcc8000b9c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcc94000b60 tx=0x7fcc80005830 comp rx=0 tx=0).stop 2024-03-29T04:10:26.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.351+0000 7fcc9b462640 1 -- 172.21.15.148:0/4040683491 shutdown_connections 2024-03-29T04:10:26.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.351+0000 7fcc9b462640 1 --2- 172.21.15.148:0/4040683491 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc80005210 0x7fcc8000b9c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.351+0000 7fcc9b462640 1 -- 172.21.15.148:0/4040683491 >> 172.21.15.148:0/4040683491 conn(0x7fcc8c01ef40 msgr2=0x7fcc8c055c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.352+0000 7fb4de396640 1 -- 172.21.15.148:0/2551615609 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb4c406c5b0 msgr2=0x7fb4c406ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.352+0000 7fb4de396640 1 --2- 172.21.15.148:0/2551615609 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb4c406c5b0 0x7fb4c406ea70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb4d815e8a0 tx=0x7fb4c8009040 comp rx=0 tx=0).stop 2024-03-29T04:10:26.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.352+0000 7fb4de396640 1 -- 172.21.15.148:0/2551615609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4c00051c0 msgr2=0x7fb4d815df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.352+0000 7fb4de396640 1 --2- 172.21.15.148:0/2551615609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4c00051c0 0x7fb4d815df00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc011990 tx=0x7fb4cc011e50 comp rx=0 tx=0).stop 2024-03-29T04:10:26.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.352+0000 7fb4dc933640 1 -- 172.21.15.148:0/2551615609 reap_dead start 2024-03-29T04:10:26.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.352+0000 7fb4de396640 1 -- 172.21.15.148:0/2551615609 shutdown_connections 2024-03-29T04:10:26.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.352+0000 7fb4de396640 1 -- 172.21.15.148:0/2551615609 >> 172.21.15.148:0/2551615609 conn(0x7fb4d80b2c60 msgr2=0x7fb4d8151c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.352+0000 7fcc9b462640 1 -- 172.21.15.148:0/4040683491 shutdown_connections 2024-03-29T04:10:26.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.353+0000 7f4a91590640 1 -- 172.21.15.148:0/2778737557 >> [v2:172.21.15.157:6816/1608291780,v1:172.21.15.157:6817/1608291780] conn(0x7f4a8c0b3810 msgr2=0x7f4a8c0d2a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.353+0000 7f4a91590640 1 --2- 172.21.15.148:0/2778737557 >> [v2:172.21.15.157:6816/1608291780,v1:172.21.15.157:6817/1608291780] conn(0x7f4a8c0b3810 0x7f4a8c0d2a10 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.352+0000 7fcc9b462640 1 -- 172.21.15.148:0/4040683491 wait complete. 2024-03-29T04:10:26.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.353+0000 7f4a91590640 1 -- 172.21.15.148:0/2778737557 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4a7806c600 msgr2=0x7f4a7806eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.353+0000 7f4a91590640 1 --2- 172.21.15.148:0/2778737557 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4a7806c600 0x7f4a7806eac0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c15a760 tx=0x7f4a7c008040 comp rx=0 tx=0).stop 2024-03-29T04:10:26.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.353+0000 7f4a91590640 1 -- 172.21.15.148:0/2778737557 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a8c0d2530 msgr2=0x7f4a8c0ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.353+0000 7f4a91590640 1 --2- 172.21.15.148:0/2778737557 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a8c0d2530 0x7f4a8c0ca660 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400c7b0 tx=0x7f4a8400cc70 comp rx=0 tx=0).stop 2024-03-29T04:10:26.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.353+0000 7f4a8b7fe640 1 -- 172.21.15.148:0/2778737557 reap_dead start 2024-03-29T04:10:26.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.353+0000 7f4a91590640 1 -- 172.21.15.148:0/2778737557 shutdown_connections 2024-03-29T04:10:26.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.353+0000 7f4a91590640 1 -- 172.21.15.148:0/2778737557 >> 172.21.15.148:0/2778737557 conn(0x7f4a8c0d4300 msgr2=0x7f4a8c0ce930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.353+0000 7f4a91590640 1 -- 172.21.15.148:0/2778737557 shutdown_connections 2024-03-29T04:10:26.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.353+0000 7f4a91590640 1 -- 172.21.15.148:0/2778737557 wait complete. 2024-03-29T04:10:26.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.354+0000 7fb4de396640 1 -- 172.21.15.148:0/2551615609 shutdown_connections 2024-03-29T04:10:26.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.354+0000 7fb4de396640 1 -- 172.21.15.148:0/2551615609 wait complete. 2024-03-29T04:10:26.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.353+0000 7fcc9b462640 1 Processor -- start 2024-03-29T04:10:26.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.354+0000 7fcc9b462640 1 -- start start 2024-03-29T04:10:26.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.355+0000 7fcc9b462640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc80005210 0x7fcc8c0502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.355+0000 7fcc9b462640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc8c0e3230 0x7fcc8c17d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.355+0000 7fcc9b462640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc8c051a90 0x7fcc8c00bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.355+0000 7fcc9b462640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8c0e4910 con 0x7fcc8c051a90 2024-03-29T04:10:26.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.355+0000 7fcc9b462640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8c056b50 con 0x7fcc80005210 2024-03-29T04:10:26.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.355+0000 7fcc9b462640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcc8c0e3bc0 con 0x7fcc8c0e3230 2024-03-29T04:10:26.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.355+0000 7fcc9a460640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc80005210 0x7fcc8c0502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.355+0000 7fcc9a460640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc80005210 0x7fcc8c0502a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:39426/0 (socket says 172.21.15.148:39426) 2024-03-29T04:10:26.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.355+0000 7fcc9a460640 1 -- 172.21.15.148:0/1752280919 learned_addr learned my addr 172.21.15.148:0/1752280919 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:26.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.355+0000 7fcc9a460640 1 -- 172.21.15.148:0/1752280919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc8c0e3230 msgr2=0x7fcc8c17d7b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T04:10:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.356+0000 7fcc9a460640 1 --2- 172.21.15.148:0/1752280919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc8c0e3230 0x7fcc8c17d7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.356+0000 7fcc9a460640 1 -- 172.21.15.148:0/1752280919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc8c051a90 msgr2=0x7fcc8c00bf20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T04:10:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.356+0000 7fcc9a460640 1 --2- 172.21.15.148:0/1752280919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc8c051a90 0x7fcc8c00bf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.356+0000 7fcc9a460640 1 -- 172.21.15.148:0/1752280919 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8c0509f0 con 0x7fcc80005210 2024-03-29T04:10:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.356+0000 7fcc9a460640 1 --2- 172.21.15.148:0/1752280919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc80005210 0x7fcc8c0502a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000c520 tx=0x7fcc9000c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.357+0000 7fcc8affd640 1 -- 172.21.15.148:0/1752280919 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9000e870 con 0x7fcc80005210 2024-03-29T04:10:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.357+0000 7fcc9b462640 1 -- 172.21.15.148:0/1752280919 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc8c050cd0 con 0x7fcc80005210 2024-03-29T04:10:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.357+0000 7fcc9b462640 1 -- 172.21.15.148:0/1752280919 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc8c051290 con 0x7fcc80005210 2024-03-29T04:10:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.358+0000 7fcc8affd640 1 -- 172.21.15.148:0/1752280919 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc9000ea30 con 0x7fcc80005210 2024-03-29T04:10:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.361+0000 7fcc8affd640 1 -- 172.21.15.148:0/1752280919 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9000ee20 con 0x7fcc80005210 2024-03-29T04:10:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.361+0000 7fcc9b462640 1 -- 172.21.15.148:0/1752280919 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcc8c0057e0 con 0x7fcc80005210 2024-03-29T04:10:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.361+0000 7fcc8affd640 1 -- 172.21.15.148:0/1752280919 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcc9000d040 con 0x7fcc80005210 2024-03-29T04:10:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.362+0000 7fcc8affd640 1 --2- 172.21.15.148:0/1752280919 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fcc7806c590 0x7fcc7806ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.362+0000 7fcc8affd640 1 -- 172.21.15.148:0/1752280919 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fcc9008b6a0 con 0x7fcc80005210 2024-03-29T04:10:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.362+0000 7fcc99c5f640 1 --2- 172.21.15.148:0/1752280919 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fcc7806c590 0x7fcc7806ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.362+0000 7fcc8affd640 1 --2- 172.21.15.148:0/1752280919 >> [v2:172.21.15.157:6824/2669171853,v1:172.21.15.157:6825/2669171853] conn(0x7fcc78075d10 0x7fcc78078150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.363+0000 7fcc8affd640 1 -- 172.21.15.148:0/1752280919 --> [v2:172.21.15.157:6824/2669171853,v1:172.21.15.157:6825/2669171853] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcc9001be50 con 0x7fcc78075d10 2024-03-29T04:10:26.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.363+0000 7fcc8affd640 1 -- 172.21.15.148:0/1752280919 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcc90092050 con 0x7fcc80005210 2024-03-29T04:10:26.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.363+0000 7fcc9ac61640 1 --2- 172.21.15.148:0/1752280919 >> [v2:172.21.15.157:6824/2669171853,v1:172.21.15.157:6825/2669171853] conn(0x7fcc78075d10 0x7fcc78078150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.363+0000 7fcc99c5f640 1 --2- 172.21.15.148:0/1752280919 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fcc7806c590 0x7fcc7806ea50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c0e2c90 tx=0x7fcc84007970 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.363+0000 7f50974c8640 1 -- 172.21.15.148:0/508044913 --> [v2:172.21.15.157:6808/3576931649,v1:172.21.15.157:6809/3576931649] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f50900687e0 con 0x7f5074075d80 2024-03-29T04:10:26.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.364+0000 7fcc9ac61640 1 --2- 172.21.15.148:0/1752280919 >> [v2:172.21.15.157:6824/2669171853,v1:172.21.15.157:6825/2669171853] conn(0x7fcc78075d10 0x7fcc78078150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.364+0000 7f5085ffb640 1 -- 172.21.15.148:0/508044913 <== osd.6 v2:172.21.15.157:6808/3576931649 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f50900687e0 con 0x7f5074075d80 2024-03-29T04:10:26.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.364+0000 7f50974c8640 1 -- 172.21.15.148:0/508044913 >> [v2:172.21.15.157:6808/3576931649,v1:172.21.15.157:6809/3576931649] conn(0x7f5074075d80 msgr2=0x7f50740781c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.364+0000 7fcc8affd640 1 -- 172.21.15.148:0/1752280919 <== osd.7 v2:172.21.15.157:6824/2669171853 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcc9001be50 con 0x7fcc78075d10 2024-03-29T04:10:26.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.364+0000 7f50974c8640 1 --2- 172.21.15.148:0/508044913 >> [v2:172.21.15.157:6808/3576931649,v1:172.21.15.157:6809/3576931649] conn(0x7f5074075d80 0x7f50740781c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.365+0000 7f50974c8640 1 -- 172.21.15.148:0/508044913 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f507406c600 msgr2=0x7f507406eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.365+0000 7f50974c8640 1 --2- 172.21.15.148:0/508044913 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f507406c600 0x7f507406eac0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f508c036000 tx=0x7f508c030000 comp rx=0 tx=0).stop 2024-03-29T04:10:26.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.365+0000 7f50974c8640 1 -- 172.21.15.148:0/508044913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50900d3430 msgr2=0x7f5090152780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.365+0000 7f50974c8640 1 --2- 172.21.15.148:0/508044913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50900d3430 0x7f5090152780 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f508000d830 tx=0x7f508000dcf0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.365+0000 7f5095a65640 1 -- 172.21.15.148:0/508044913 reap_dead start 2024-03-29T04:10:26.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.365+0000 7f50974c8640 1 -- 172.21.15.148:0/508044913 shutdown_connections 2024-03-29T04:10:26.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.365+0000 7f50974c8640 1 -- 172.21.15.148:0/508044913 >> 172.21.15.148:0/508044913 conn(0x7f50900d4300 msgr2=0x7f50900b5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.365+0000 7f50974c8640 1 -- 172.21.15.148:0/508044913 shutdown_connections 2024-03-29T04:10:26.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.365+0000 7f50974c8640 1 -- 172.21.15.148:0/508044913 wait complete. 2024-03-29T04:10:26.384 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-03-29T04:10:26.384 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-29T04:10:26.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.373+0000 7f856bfff640 1 -- 172.21.15.148:0/3878150777 --> [v2:172.21.15.157:6800/1932472006,v1:172.21.15.157:6801/1932472006] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8564002dd0 con 0x7f856c075c50 2024-03-29T04:10:26.385 INFO:teuthology.orchestra.run.smithi148.stdout:55834574857 2024-03-29T04:10:26.386 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-29T04:10:26.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.374+0000 7f857e7fc640 1 -- 172.21.15.148:0/3878150777 <== osd.4 v2:172.21.15.157:6800/1932472006 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8564002dd0 con 0x7f856c075c50 2024-03-29T04:10:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.375+0000 7f856bfff640 1 -- 172.21.15.148:0/3878150777 >> [v2:172.21.15.157:6800/1932472006,v1:172.21.15.157:6801/1932472006] conn(0x7f856c075c50 msgr2=0x7f856c078090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.375+0000 7f856bfff640 1 --2- 172.21.15.148:0/3878150777 >> [v2:172.21.15.157:6800/1932472006,v1:172.21.15.157:6801/1932472006] conn(0x7f856c075c50 0x7f856c078090 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.375+0000 7f856bfff640 1 -- 172.21.15.148:0/3878150777 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f856c06c560 msgr2=0x7f856c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.375+0000 7f856bfff640 1 --2- 172.21.15.148:0/3878150777 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f856c06c560 0x7f856c06ea20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8580009fd0 tx=0x7f858000b040 comp rx=0 tx=0).stop 2024-03-29T04:10:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.375+0000 7f856bfff640 1 -- 172.21.15.148:0/3878150777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f858813e2c0 msgr2=0x7f8588141030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.375+0000 7f856bfff640 1 --2- 172.21.15.148:0/3878150777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f858813e2c0 0x7f8588141030 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f85880b55b0 tx=0x7f8584001970 comp rx=0 tx=0).stop 2024-03-29T04:10:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.375+0000 7f858e791640 1 -- 172.21.15.148:0/3878150777 reap_dead start 2024-03-29T04:10:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.376+0000 7f856bfff640 1 -- 172.21.15.148:0/3878150777 shutdown_connections 2024-03-29T04:10:26.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.376+0000 7f856bfff640 1 -- 172.21.15.148:0/3878150777 >> 172.21.15.148:0/3878150777 conn(0x7f858806b5b0 msgr2=0x7f85880cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.376+0000 7fefa72d9640 1 -- 172.21.15.148:0/3262938009 --> [v2:172.21.15.148:6824/1406601759,v1:172.21.15.148:6825/1406601759] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fefa00b3b00 con 0x7fef84076c00 2024-03-29T04:10:26.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.376+0000 7f856bfff640 1 -- 172.21.15.148:0/3878150777 shutdown_connections 2024-03-29T04:10:26.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.376+0000 7f856bfff640 1 -- 172.21.15.148:0/3878150777 wait complete. 2024-03-29T04:10:26.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.377+0000 7fef95ffb640 1 -- 172.21.15.148:0/3262938009 <== osd.0 v2:172.21.15.148:6824/1406601759 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fefa00b3b00 con 0x7fef84076c00 2024-03-29T04:10:26.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.377+0000 7fefa72d9640 1 -- 172.21.15.148:0/3262938009 >> [v2:172.21.15.148:6824/1406601759,v1:172.21.15.148:6825/1406601759] conn(0x7fef84076c00 msgr2=0x7fef840790a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.377+0000 7fefa72d9640 1 --2- 172.21.15.148:0/3262938009 >> [v2:172.21.15.148:6824/1406601759,v1:172.21.15.148:6825/1406601759] conn(0x7fef84076c00 0x7fef840790a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.377+0000 7fefa72d9640 1 -- 172.21.15.148:0/3262938009 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fef8406c930 msgr2=0x7fef8406edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.377+0000 7fefa72d9640 1 --2- 172.21.15.148:0/3262938009 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fef8406c930 0x7fef8406edf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fefa014ed40 tx=0x7fef9c008040 comp rx=0 tx=0).stop 2024-03-29T04:10:26.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.378+0000 7fefa72d9640 1 -- 172.21.15.148:0/3262938009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefa00d1bc0 msgr2=0x7fefa015a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.378+0000 7fefa72d9640 1 --2- 172.21.15.148:0/3262938009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefa00d1bc0 0x7fefa015a710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fef9000c750 tx=0x7fef9000cc10 comp rx=0 tx=0).stop 2024-03-29T04:10:26.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.378+0000 7fefa5876640 1 -- 172.21.15.148:0/3262938009 reap_dead start 2024-03-29T04:10:26.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.378+0000 7fefa72d9640 1 -- 172.21.15.148:0/3262938009 shutdown_connections 2024-03-29T04:10:26.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.378+0000 7fefa72d9640 1 -- 172.21.15.148:0/3262938009 >> 172.21.15.148:0/3262938009 conn(0x7fefa00b3d50 msgr2=0x7fefa0075320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.378+0000 7fefa72d9640 1 -- 172.21.15.148:0/3262938009 shutdown_connections 2024-03-29T04:10:26.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.378+0000 7fefa72d9640 1 -- 172.21.15.148:0/3262938009 wait complete. 2024-03-29T04:10:26.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.380+0000 7fc2a89d1640 1 -- 172.21.15.148:0/2807083736 --> [v2:172.21.15.148:6808/90026237,v1:172.21.15.148:6809/90026237] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc284002d50 con 0x7fc288075d00 2024-03-29T04:10:26.391 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-03-29T04:10:26.392 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-29T04:10:26.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.381+0000 7fc28ffff640 1 -- 172.21.15.148:0/2807083736 <== osd.2 v2:172.21.15.148:6808/90026237 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc284002d50 con 0x7fc288075d00 2024-03-29T04:10:26.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.382+0000 7fc2a89d1640 1 -- 172.21.15.148:0/2807083736 >> [v2:172.21.15.148:6808/90026237,v1:172.21.15.148:6809/90026237] conn(0x7fc288075d00 msgr2=0x7fc288078140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.382+0000 7fc2a89d1640 1 --2- 172.21.15.148:0/2807083736 >> [v2:172.21.15.148:6808/90026237,v1:172.21.15.148:6809/90026237] conn(0x7fc288075d00 0x7fc288078140 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.382+0000 7fc2a89d1640 1 -- 172.21.15.148:0/2807083736 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc28806c600 msgr2=0x7fc28806eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.382+0000 7fc2a89d1640 1 --2- 172.21.15.148:0/2807083736 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc28806c600 0x7fc28806eac0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc298036000 tx=0x7fc298008030 comp rx=0 tx=0).stop 2024-03-29T04:10:26.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.382+0000 7fc2a89d1640 1 -- 172.21.15.148:0/2807083736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc29c0e75d0 msgr2=0x7fc29c051550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.382+0000 7fc2a89d1640 1 --2- 172.21.15.148:0/2807083736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc29c0e75d0 0x7fc29c051550 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc2a402d940 tx=0x7fc2a4069280 comp rx=0 tx=0).stop 2024-03-29T04:10:26.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.382+0000 7fc2a3fff640 1 -- 172.21.15.148:0/2807083736 reap_dead start 2024-03-29T04:10:26.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.382+0000 7fc2a89d1640 1 -- 172.21.15.148:0/2807083736 shutdown_connections 2024-03-29T04:10:26.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.382+0000 7fc2a89d1640 1 -- 172.21.15.148:0/2807083736 >> 172.21.15.148:0/2807083736 conn(0x7fc29c03aa20 msgr2=0x7fc29c04d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.395 INFO:teuthology.orchestra.run.smithi148.stdout:55834574857 2024-03-29T04:10:26.395 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-29T04:10:26.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.382+0000 7fc2a89d1640 1 -- 172.21.15.148:0/2807083736 shutdown_connections 2024-03-29T04:10:26.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.382+0000 7fc2a89d1640 1 -- 172.21.15.148:0/2807083736 wait complete. 2024-03-29T04:10:26.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.387+0000 7fcc9b462640 1 -- 172.21.15.148:0/1752280919 --> [v2:172.21.15.157:6824/2669171853,v1:172.21.15.157:6825/2669171853] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcc8c0057e0 con 0x7fcc78075d10 2024-03-29T04:10:26.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.388+0000 7fcc8affd640 1 -- 172.21.15.148:0/1752280919 <== osd.7 v2:172.21.15.157:6824/2669171853 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcc8c0057e0 con 0x7fcc78075d10 2024-03-29T04:10:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.388+0000 7fcc9b462640 1 -- 172.21.15.148:0/1752280919 >> [v2:172.21.15.157:6824/2669171853,v1:172.21.15.157:6825/2669171853] conn(0x7fcc78075d10 msgr2=0x7fcc78078150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.388+0000 7fcc9b462640 1 --2- 172.21.15.148:0/1752280919 >> [v2:172.21.15.157:6824/2669171853,v1:172.21.15.157:6825/2669171853] conn(0x7fcc78075d10 0x7fcc78078150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.388+0000 7fcc9b462640 1 -- 172.21.15.148:0/1752280919 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fcc7806c590 msgr2=0x7fcc7806ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.388+0000 7fcc9b462640 1 --2- 172.21.15.148:0/1752280919 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fcc7806c590 0x7fcc7806ea50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c0e2c90 tx=0x7fcc84007970 comp rx=0 tx=0).stop 2024-03-29T04:10:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.388+0000 7fcc9b462640 1 -- 172.21.15.148:0/1752280919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc80005210 msgr2=0x7fcc8c0502a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.388+0000 7fcc9b462640 1 --2- 172.21.15.148:0/1752280919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc80005210 0x7fcc8c0502a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000c520 tx=0x7fcc9000c9e0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.388+0000 7fcc9ac61640 1 -- 172.21.15.148:0/1752280919 reap_dead start 2024-03-29T04:10:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.388+0000 7fcc9b462640 1 -- 172.21.15.148:0/1752280919 shutdown_connections 2024-03-29T04:10:26.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.388+0000 7fcc9b462640 1 -- 172.21.15.148:0/1752280919 >> 172.21.15.148:0/1752280919 conn(0x7fcc8c01ef40 msgr2=0x7fcc8c055c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.388+0000 7fcc9b462640 1 -- 172.21.15.148:0/1752280919 shutdown_connections 2024-03-29T04:10:26.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.388+0000 7fcc9b462640 1 -- 172.21.15.148:0/1752280919 wait complete. 2024-03-29T04:10:26.398 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-03-29T04:10:26.399 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-29T04:10:26.399 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-03-29T04:10:26.400 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-29T04:10:26.400 INFO:teuthology.orchestra.run.smithi148.stdout:55834574857 2024-03-29T04:10:26.401 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-29T04:10:26.401 INFO:teuthology.orchestra.run.smithi148.stdout:55834574857 2024-03-29T04:10:26.402 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-29T04:10:26.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.620+0000 7fc941bb9640 1 Processor -- start 2024-03-29T04:10:26.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.620+0000 7fc941bb9640 1 -- start start 2024-03-29T04:10:26.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.621+0000 7fc941bb9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9340078d0 0x7fc9340edd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.621+0000 7fc941bb9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc934058560 con 0x7fc93403e830 2024-03-29T04:10:26.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.621+0000 7fc941bb9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc934058840 con 0x7fc9340e30b0 2024-03-29T04:10:26.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.621+0000 7fc941bb9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc934058b20 con 0x7fc9340078d0 2024-03-29T04:10:26.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.621+0000 7fc9413b8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9340078d0 0x7fc9340edd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.621+0000 7fc93bfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc93403e830 0x7fc934053be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47264/0 (socket says 172.21.15.148:47264) 2024-03-29T04:10:26.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.621+0000 7fc93bfff640 1 -- 172.21.15.148:0/644028609 learned_addr learned my addr 172.21.15.148:0/644028609 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:26.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.621+0000 7fc940bb7640 1 --1- 172.21.15.148:0/644028609 >> v1:172.21.15.157:6789/0 conn(0x7fc9340e30b0 0x7fc934057d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.148:38722/0 (socket says 172.21.15.148:38722) 2024-03-29T04:10:26.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.622+0000 7f61fcd1c640 1 Processor -- start 2024-03-29T04:10:26.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.622+0000 7f61fcd1c640 1 -- start start 2024-03-29T04:10:26.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.622+0000 7fc93b7fe640 1 -- 172.21.15.148:0/644028609 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4130994674 0 0) 0x7fc934058840 con 0x7fc9340e30b0 2024-03-29T04:10:26.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.622+0000 7fc93b7fe640 1 -- 172.21.15.148:0/644028609 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc924003520 con 0x7fc9340e30b0 2024-03-29T04:10:26.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.622+0000 7fc93b7fe640 1 -- 172.21.15.148:0/644028609 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3252803097 0 0) 0x7fc934058560 con 0x7fc93403e830 2024-03-29T04:10:26.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.622+0000 7fc93b7fe640 1 -- 172.21.15.148:0/644028609 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc934058840 con 0x7fc93403e830 2024-03-29T04:10:26.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.622+0000 7fc93b7fe640 1 -- 172.21.15.148:0/644028609 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3531502522 0 0) 0x7fc924003520 con 0x7fc9340e30b0 2024-03-29T04:10:26.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.622+0000 7fc93b7fe640 1 -- 172.21.15.148:0/644028609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9340078d0 msgr2=0x7fc9340edd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:26.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.622+0000 7fc93b7fe640 1 --2- 172.21.15.148:0/644028609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc9340078d0 0x7fc9340edd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.622+0000 7fc93b7fe640 1 -- 172.21.15.148:0/644028609 >> v1:172.21.15.148:6789/0 conn(0x7fc93403e830 legacy=0x7fc934053be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.622+0000 7fc93b7fe640 1 -- 172.21.15.148:0/644028609 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9340063a0 con 0x7fc9340e30b0 2024-03-29T04:10:26.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.622+0000 7fc93b7fe640 1 -- 172.21.15.148:0/644028609 <== mon.1 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7fc93c0695f0 con 0x7fc9340e30b0 2024-03-29T04:10:26.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.622+0000 7fc93b7fe640 1 -- 172.21.15.148:0/644028609 >> v1:172.21.15.157:6789/0 conn(0x7fc9340e30b0 legacy=0x7fc934057d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.622+0000 7fc93b7fe640 1 --2- 172.21.15.148:0/644028609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc924003b70 0x7fc924003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.623+0000 7fc93b7fe640 1 --2- 172.21.15.148:0/644028609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc924004800 0x7fc924004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.623+0000 7fc93b7fe640 1 --2- 172.21.15.148:0/644028609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9240051c0 0x7fc92400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.623+0000 7fc93b7fe640 1 -- 172.21.15.148:0/644028609 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc934058b20 con 0x7fc924003b70 2024-03-29T04:10:26.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.623+0000 7fc93b7fe640 1 -- 172.21.15.148:0/644028609 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc92400bf00 con 0x7fc9240051c0 2024-03-29T04:10:26.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.623+0000 7fc93b7fe640 1 -- 172.21.15.148:0/644028609 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc92400c080 con 0x7fc924004800 2024-03-29T04:10:26.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.623+0000 7fed57243640 1 Processor -- start 2024-03-29T04:10:26.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.623+0000 7fed57243640 1 -- start start 2024-03-29T04:10:26.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.624+0000 7fed57243640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fed480e6e10 0x7fed480e7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.624+0000 7fed57243640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed48007330 con 0x7fed480e33e0 2024-03-29T04:10:26.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.624+0000 7fed57243640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed48007610 con 0x7fed48057af0 2024-03-29T04:10:26.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.624+0000 7fed57243640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fed480e77b0 con 0x7fed480e6e10 2024-03-29T04:10:26.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.625+0000 7fed56241640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fed480e33e0 0x7fed48006890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47272/0 (socket says 172.21.15.148:47272) 2024-03-29T04:10:26.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.625+0000 7fed56241640 1 -- 172.21.15.148:0/333900726 learned_addr learned my addr 172.21.15.148:0/333900726 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:26.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.625+0000 7fed55a40640 1 --2- 172.21.15.148:0/333900726 >> v2:172.21.15.148:3300/0 conn(0x7fed480e6e10 0x7fed480e7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.624+0000 7f61fcd1c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61f80d1bc0 0x7f61f80d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.624+0000 7f61f659c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f61f815c370 0x7f61f80cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.148:38734/0 (socket says 172.21.15.148:38734) 2024-03-29T04:10:26.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.624+0000 7f61f659c640 1 -- 172.21.15.148:0/2454643286 learned_addr learned my addr 172.21.15.148:0/2454643286 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:26.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.624+0000 7f61fcd1c640 1 -- 172.21.15.148:0/2454643286 --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61f80b3b00 con 0x7f61f80ce620 2024-03-29T04:10:26.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.624+0000 7f61fcd1c640 1 -- 172.21.15.148:0/2454643286 --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61f80b2730 con 0x7f61f815c370 2024-03-29T04:10:26.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.624+0000 7f61f5d9b640 1 --2- 172.21.15.148:0/2454643286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61f80d1bc0 0x7f61f80d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.624+0000 7f61fcd1c640 1 -- 172.21.15.148:0/2454643286 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f61f80ccd90 con 0x7f61f80d1bc0 2024-03-29T04:10:26.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.623+0000 7fc940bb7640 1 --2- 172.21.15.148:0/644028609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc924003b70 0x7fc924003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.623+0000 7fc93bfff640 1 --2- 172.21.15.148:0/644028609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc924004800 0x7fc924004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.624+0000 7fc940bb7640 1 -- 172.21.15.148:0/644028609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc924004800 msgr2=0x7fc924004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:26.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.624+0000 7fc940bb7640 1 --2- 172.21.15.148:0/644028609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc924004800 0x7fc924004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.624+0000 7fc940bb7640 1 -- 172.21.15.148:0/644028609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9240051c0 msgr2=0x7fc92400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:26.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.624+0000 7fc940bb7640 1 --2- 172.21.15.148:0/644028609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9240051c0 0x7fc92400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.624+0000 7fc940bb7640 1 -- 172.21.15.148:0/644028609 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9340063a0 con 0x7fc924003b70 2024-03-29T04:10:26.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.625+0000 7fc940bb7640 1 --2- 172.21.15.148:0/644028609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc924003b70 0x7fc924003f50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc93c069cd0 tx=0x7fc924005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5703fa8431a8c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.625+0000 7fc93b7fe640 1 -- 172.21.15.148:0/644028609 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c0686c0 con 0x7fc924003b70 2024-03-29T04:10:26.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.625+0000 7fc93b7fe640 1 -- 172.21.15.148:0/644028609 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc93c06d3b0 con 0x7fc924003b70 2024-03-29T04:10:26.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.625+0000 7f61f559a640 1 -- 172.21.15.148:0/2454643286 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4067754145 0 0) 0x7f61f80b2730 con 0x7f61f815c370 2024-03-29T04:10:26.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.626+0000 7fed5523f640 1 -- 172.21.15.148:0/333900726 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2329859543 0 0) 0x7fed48007330 con 0x7fed480e33e0 2024-03-29T04:10:26.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.626+0000 7fed5523f640 1 -- 172.21.15.148:0/333900726 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed3c003520 con 0x7fed480e33e0 2024-03-29T04:10:26.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.626+0000 7fed5523f640 1 -- 172.21.15.148:0/333900726 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2414946758 0 0) 0x7fed48007610 con 0x7fed48057af0 2024-03-29T04:10:26.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.626+0000 7fed5523f640 1 -- 172.21.15.148:0/333900726 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed48007330 con 0x7fed48057af0 2024-03-29T04:10:26.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.626+0000 7fed55a40640 1 -- 172.21.15.148:0/333900726 >> v1:172.21.15.157:6789/0 conn(0x7fed48057af0 legacy=0x7fed48057f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.626+0000 7fed55a40640 1 -- 172.21.15.148:0/333900726 >> v1:172.21.15.148:6789/0 conn(0x7fed480e33e0 legacy=0x7fed48006890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.626+0000 7fc941bb9640 1 -- 172.21.15.148:0/644028609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc924003b70 msgr2=0x7fc924003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.626+0000 7f61f559a640 1 -- 172.21.15.148:0/2454643286 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61e0003520 con 0x7f61f815c370 2024-03-29T04:10:26.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.626+0000 7fc941bb9640 1 --2- 172.21.15.148:0/644028609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc924003b70 0x7fc924003f50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc93c069cd0 tx=0x7fc924005830 comp rx=0 tx=0).stop 2024-03-29T04:10:26.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.627+0000 7fc9413b8640 1 -- 172.21.15.148:0/644028609 reap_dead start 2024-03-29T04:10:26.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.627+0000 7fed55a40640 1 -- 172.21.15.148:0/333900726 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed480572f0 con 0x7fed480e6e10 2024-03-29T04:10:26.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.627+0000 7f61f559a640 1 -- 172.21.15.148:0/2454643286 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 302731049 0 0) 0x7f61f80b3b00 con 0x7f61f80ce620 2024-03-29T04:10:26.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.627+0000 7f61f5d9b640 1 -- 172.21.15.148:0/2454643286 >> v1:172.21.15.157:6789/0 conn(0x7f61f815c370 legacy=0x7f61f80cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.627+0000 7f61f5d9b640 1 -- 172.21.15.148:0/2454643286 >> v1:172.21.15.148:6789/0 conn(0x7f61f80ce620 legacy=0x7f61f80ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.627+0000 7f61f5d9b640 1 -- 172.21.15.148:0/2454643286 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f80b2a10 con 0x7f61f80d1bc0 2024-03-29T04:10:26.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.628+0000 7f61f5d9b640 1 --2- 172.21.15.148:0/2454643286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61f80d1bc0 0x7f61f80d0460 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f61e80097d0 tx=0x7f61e802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e9db0ab8afd76c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.629+0000 7fc941bb9640 1 -- 172.21.15.148:0/644028609 shutdown_connections 2024-03-29T04:10:26.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.629+0000 7fc941bb9640 1 -- 172.21.15.148:0/644028609 >> 172.21.15.148:0/644028609 conn(0x7fc9340559d0 msgr2=0x7fc934055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.629+0000 7f1fbd4c5640 1 Processor -- start 2024-03-29T04:10:26.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.629+0000 7f1fbd4c5640 1 -- start start 2024-03-29T04:10:26.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.630+0000 7fc941bb9640 1 -- 172.21.15.148:0/644028609 shutdown_connections 2024-03-29T04:10:26.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.630+0000 7fc941bb9640 1 -- 172.21.15.148:0/644028609 wait complete. 2024-03-29T04:10:26.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.630+0000 7f61f559a640 1 -- 172.21.15.148:0/2454643286 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e8035020 con 0x7f61f80d1bc0 2024-03-29T04:10:26.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.630+0000 7f61f559a640 1 -- 172.21.15.148:0/2454643286 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61e8031800 con 0x7f61f80d1bc0 2024-03-29T04:10:26.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.630+0000 7f61f559a640 1 -- 172.21.15.148:0/2454643286 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e8031bd0 con 0x7f61f80d1bc0 2024-03-29T04:10:26.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.630+0000 7f1fbd4c5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1fb80b3710 0x7f1fb80b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.630+0000 7f1fbd4c5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fb80b4540 con 0x7f1fb815e2e0 2024-03-29T04:10:26.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.630+0000 7f1fbd4c5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fb80afcb0 con 0x7f1fb80b40b0 2024-03-29T04:10:26.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.630+0000 7f1fbd4c5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f1fb806bac0 con 0x7f1fb80b3710 2024-03-29T04:10:26.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.630+0000 7f1fb6ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f1fb815e2e0 0x7f1fb806e8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47282/0 (socket says 172.21.15.148:47282) 2024-03-29T04:10:26.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.631+0000 7f1fb6ffd640 1 -- 172.21.15.148:0/1795879718 learned_addr learned my addr 172.21.15.148:0/1795879718 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:26.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.631+0000 7f1fb67fc640 1 --2- 172.21.15.148:0/1795879718 >> v2:172.21.15.148:3300/0 conn(0x7f1fb80b3710 0x7f1fb80b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.631+0000 7effd09d3640 1 Processor -- start 2024-03-29T04:10:26.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.631+0000 7f1fb5ffb640 1 -- 172.21.15.148:0/1795879718 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3312536505 0 0) 0x7f1fac001110 con 0x7f1fb815e2e0 2024-03-29T04:10:26.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.631+0000 7f1fb5ffb640 1 -- 172.21.15.148:0/1795879718 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fa0003520 con 0x7f1fb815e2e0 2024-03-29T04:10:26.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.631+0000 7f61fcd1c640 1 -- 172.21.15.148:0/2454643286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61f80d1bc0 msgr2=0x7f61f80d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.631+0000 7f1fb5ffb640 1 -- 172.21.15.148:0/1795879718 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3209648039 0 0) 0x7f1fb80afcb0 con 0x7f1fb80b40b0 2024-03-29T04:10:26.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.631+0000 7f1fb5ffb640 1 -- 172.21.15.148:0/1795879718 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fa0003880 con 0x7f1fb80b40b0 2024-03-29T04:10:26.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.632+0000 7f61fcd1c640 1 --2- 172.21.15.148:0/2454643286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61f80d1bc0 0x7f61f80d0460 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f61e80097d0 tx=0x7f61e802b8a0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.631+0000 7f1fb5ffb640 1 -- 172.21.15.148:0/1795879718 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3160615781 0 0) 0x7f1fa0003520 con 0x7f1fb815e2e0 2024-03-29T04:10:26.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.631+0000 7f1fb5ffb640 1 -- 172.21.15.148:0/1795879718 >> v2:172.21.15.148:3300/0 conn(0x7f1fb80b3710 msgr2=0x7f1fb80b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:26.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.631+0000 7f1fb5ffb640 1 --2- 172.21.15.148:0/1795879718 >> v2:172.21.15.148:3300/0 conn(0x7f1fb80b3710 0x7f1fb80b3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.631+0000 7f1fb5ffb640 1 -- 172.21.15.148:0/1795879718 >> v1:172.21.15.157:6789/0 conn(0x7f1fb80b40b0 legacy=0x7f1fb806bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.631+0000 7f1fb5ffb640 1 -- 172.21.15.148:0/1795879718 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb80aff90 con 0x7f1fb815e2e0 2024-03-29T04:10:26.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.631+0000 7f1fb5ffb640 1 -- 172.21.15.148:0/1795879718 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f1fac001820 con 0x7f1fb815e2e0 2024-03-29T04:10:26.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.631+0000 7f1fb5ffb640 1 -- 172.21.15.148:0/1795879718 >> v1:172.21.15.148:6789/0 conn(0x7f1fb815e2e0 legacy=0x7f1fb806e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.631+0000 7f1fb5ffb640 1 --2- 172.21.15.148:0/1795879718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa0003d50 0x7f1fa0004190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.631+0000 7f1fb5ffb640 1 --2- 172.21.15.148:0/1795879718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fa0004800 0x7f1fa0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.631+0000 7f1fb5ffb640 1 --2- 172.21.15.148:0/1795879718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fa00121c0 0x7f1fa00145b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.631+0000 7f1fb5ffb640 1 -- 172.21.15.148:0/1795879718 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb806bac0 con 0x7f1fa00121c0 2024-03-29T04:10:26.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.631+0000 7f1fb5ffb640 1 -- 172.21.15.148:0/1795879718 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa0014ba0 con 0x7f1fa0003d50 2024-03-29T04:10:26.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.631+0000 7f1fb5ffb640 1 -- 172.21.15.148:0/1795879718 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1fa0014da0 con 0x7f1fa0004800 2024-03-29T04:10:26.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.632+0000 7f1fb67fc640 1 --2- 172.21.15.148:0/1795879718 >> v2:172.21.15.148:3300/0 conn(0x7f1fb80b3710 0x7f1fb80b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:26.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.632+0000 7f1fb77fe640 1 --2- 172.21.15.148:0/1795879718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fa00121c0 0x7f1fa00145b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.632+0000 7f1fb6ffd640 1 --2- 172.21.15.148:0/1795879718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa0003d50 0x7f1fa0004190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.632+0000 7f1fb67fc640 1 --2- 172.21.15.148:0/1795879718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fa0004800 0x7f1fa0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.631+0000 7effd09d3640 1 -- start start 2024-03-29T04:10:26.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.632+0000 7f1fb6ffd640 1 -- 172.21.15.148:0/1795879718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fa0004800 msgr2=0x7f1fa0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:26.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.632+0000 7f1fb6ffd640 1 --2- 172.21.15.148:0/1795879718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fa0004800 0x7f1fa0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.632+0000 7f1fb6ffd640 1 -- 172.21.15.148:0/1795879718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fa00121c0 msgr2=0x7f1fa00145b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:26.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.632+0000 7f1fb6ffd640 1 --2- 172.21.15.148:0/1795879718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fa00121c0 0x7f1fa00145b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.632+0000 7f1fb6ffd640 1 -- 172.21.15.148:0/1795879718 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb80aff90 con 0x7f1fa0003d50 2024-03-29T04:10:26.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.632+0000 7fc941bb9640 1 Processor -- start 2024-03-29T04:10:26.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.632+0000 7fc941bb9640 1 -- start start 2024-03-29T04:10:26.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.633+0000 7f1fb77fe640 1 -- 172.21.15.148:0/1795879718 reap_dead start 2024-03-29T04:10:26.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.633+0000 7f61fcd1c640 1 -- 172.21.15.148:0/2454643286 shutdown_connections 2024-03-29T04:10:26.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.633+0000 7effcad9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7effc40dfee0 0x7effc40085c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47284/0 (socket says 172.21.15.148:47284) 2024-03-29T04:10:26.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.633+0000 7fed55a40640 1 --2- 172.21.15.148:0/333900726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed480e6e10 0x7fed480e7270 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fed400097a0 tx=0x7fed4002b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68bb83b6ba9c29ed server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.633+0000 7effcad9d640 1 -- 172.21.15.148:0/2057464116 learned_addr learned my addr 172.21.15.148:0/2057464116 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:26.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.633+0000 7f61fcd1c640 1 --2- 172.21.15.148:0/2454643286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61f80d1bc0 0x7f61f80d0460 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f61e80097d0 tx=0x7f61e802b8a0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.633+0000 7f61fcd1c640 1 -- 172.21.15.148:0/2454643286 >> 172.21.15.148:0/2454643286 conn(0x7f61f80b3d50 msgr2=0x7f61f80b4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.633+0000 7effd09d3640 1 --2- 172.21.15.148:0/2057464116 >> v2:172.21.15.148:3300/0 conn(0x7effc40e42c0 0x7effc40eaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.633+0000 7fc941bb9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc924003b70 0x7fc9340d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.633+0000 7fc941bb9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9340e78f0 0x7fc93417dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.633+0000 7fed5523f640 1 -- 172.21.15.148:0/333900726 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4002f810 con 0x7fed480e6e10 2024-03-29T04:10:26.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.633+0000 7fed5523f640 1 -- 172.21.15.148:0/333900726 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed4002f9b0 con 0x7fed480e6e10 2024-03-29T04:10:26.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.633+0000 7fc941bb9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9340df180 0x7fc9340df630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.633+0000 7fc941bb9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc9340542f0 con 0x7fc9340e78f0 2024-03-29T04:10:26.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.633+0000 7fc941bb9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc9340e3490 con 0x7fc9340df180 2024-03-29T04:10:26.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.633+0000 7effcb59e640 1 --2- 172.21.15.148:0/2057464116 >> v2:172.21.15.148:3300/0 conn(0x7effc40e42c0 0x7effc40eaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.633+0000 7effd09d3640 1 -- 172.21.15.148:0/2057464116 --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effc4056580 con 0x7effc40dfee0 2024-03-29T04:10:26.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.633+0000 7fe3c0b7c640 1 Processor -- start 2024-03-29T04:10:26.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.633+0000 7fe3c0b7c640 1 -- start start 2024-03-29T04:10:26.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.633+0000 7fc941bb9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc934054470 con 0x7fc924003b70 2024-03-29T04:10:26.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.633+0000 7fc9413b8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9340df180 0x7fc9340df630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.633+0000 7fc9413b8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9340df180 0x7fc9340df630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:39466/0 (socket says 172.21.15.148:39466) 2024-03-29T04:10:26.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.633+0000 7fc9413b8640 1 -- 172.21.15.148:0/3510553119 learned_addr learned my addr 172.21.15.148:0/3510553119 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:26.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.633+0000 7fc93bfff640 1 --2- 172.21.15.148:0/3510553119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9340e78f0 0x7fc93417dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.634+0000 7effd09d3640 1 -- 172.21.15.148:0/2057464116 --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effc4056860 con 0x7effc40e02c0 2024-03-29T04:10:26.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.634+0000 7effd09d3640 1 -- 172.21.15.148:0/2057464116 --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7effc40094a0 con 0x7effc40e42c0 2024-03-29T04:10:26.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.634+0000 7f1fb6ffd640 1 --2- 172.21.15.148:0/1795879718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa0003d50 0x7f1fa0004190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00c460 tx=0x7f1fa0005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e559554d0297367 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.634+0000 7effc9d9b640 1 -- 172.21.15.148:0/2057464116 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2803490410 0 0) 0x7effc4056580 con 0x7effc40dfee0 2024-03-29T04:10:26.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.634+0000 7fc940bb7640 1 --2- 172.21.15.148:0/3510553119 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc924003b70 0x7fc9340d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.634+0000 7effc9d9b640 1 -- 172.21.15.148:0/2057464116 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7effb4005730 con 0x7effc40dfee0 2024-03-29T04:10:26.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.634+0000 7fed5523f640 1 -- 172.21.15.148:0/333900726 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4002fd80 con 0x7fed480e6e10 2024-03-29T04:10:26.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.634+0000 7f1fb5ffb640 1 -- 172.21.15.148:0/1795879718 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac00c0e0 con 0x7f1fa0003d50 2024-03-29T04:10:26.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.634+0000 7f1fb5ffb640 1 -- 172.21.15.148:0/1795879718 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fac030030 con 0x7f1fa0003d50 2024-03-29T04:10:26.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.634+0000 7f1fb5ffb640 1 -- 172.21.15.148:0/1795879718 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac0048a0 con 0x7f1fa0003d50 2024-03-29T04:10:26.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.634+0000 7fed57243640 1 -- 172.21.15.148:0/333900726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed480e6e10 msgr2=0x7fed480e7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.634+0000 7fc9413b8640 1 -- 172.21.15.148:0/3510553119 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc924003b70 msgr2=0x7fc9340d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.634+0000 7fc9413b8640 1 --2- 172.21.15.148:0/3510553119 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc924003b70 0x7fc9340d13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.634+0000 7fc9413b8640 1 -- 172.21.15.148:0/3510553119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9340e78f0 msgr2=0x7fc93417dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.634+0000 7fed57243640 1 --2- 172.21.15.148:0/333900726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed480e6e10 0x7fed480e7270 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fed400097a0 tx=0x7fed4002b9f0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.635+0000 7effcb59e640 1 -- 172.21.15.148:0/2057464116 >> v1:172.21.15.157:6789/0 conn(0x7effc40e02c0 legacy=0x7effc40e3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.635+0000 7effcb59e640 1 -- 172.21.15.148:0/2057464116 >> v1:172.21.15.148:6789/0 conn(0x7effc40dfee0 legacy=0x7effc40085c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.635+0000 7effcb59e640 1 -- 172.21.15.148:0/2057464116 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc4008e50 con 0x7effc40e42c0 2024-03-29T04:10:26.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.635+0000 7fc9413b8640 1 --2- 172.21.15.148:0/3510553119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9340e78f0 0x7fc93417dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.635+0000 7fc9413b8640 1 -- 172.21.15.148:0/3510553119 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9340d1920 con 0x7fc9340df180 2024-03-29T04:10:26.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.635+0000 7fe3c0b7c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe3bc15bfb0 0x7fe3bc0d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.635+0000 7fc93bfff640 1 --2- 172.21.15.148:0/3510553119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9340e78f0 0x7fc93417dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:26.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.635+0000 7fed57243640 1 -- 172.21.15.148:0/333900726 shutdown_connections 2024-03-29T04:10:26.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.635+0000 7fc9413b8640 1 --2- 172.21.15.148:0/3510553119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9340df180 0x7fc9340df630 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc9340e7ce0 tx=0x7fc92c00ec00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.635+0000 7fed57243640 1 --2- 172.21.15.148:0/333900726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed480e6e10 0x7fed480e7270 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.635+0000 7fed57243640 1 -- 172.21.15.148:0/333900726 >> 172.21.15.148:0/333900726 conn(0x7fed4803aa20 msgr2=0x7fed4803ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.635+0000 7f1fbd4c5640 1 -- 172.21.15.148:0/1795879718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa0003d50 msgr2=0x7f1fa0004190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.635+0000 7f1fbd4c5640 1 --2- 172.21.15.148:0/1795879718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa0003d50 0x7f1fa0004190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00c460 tx=0x7f1fa0005830 comp rx=0 tx=0).stop 2024-03-29T04:10:26.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.635+0000 7fe3b9d9b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe3bc0d0610 0x7fe3bc0b4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47286/0 (socket says 172.21.15.148:47286) 2024-03-29T04:10:26.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.635+0000 7fe3b9d9b640 1 -- 172.21.15.148:0/2258188034 learned_addr learned my addr 172.21.15.148:0/2258188034 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:26.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.635+0000 7fe3c0b7c640 1 -- 172.21.15.148:0/2258188034 --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3bc06e970 con 0x7fe3bc0d0610 2024-03-29T04:10:26.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.635+0000 7fed57243640 1 -- 172.21.15.148:0/333900726 shutdown_connections 2024-03-29T04:10:26.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.635+0000 7fed57243640 1 -- 172.21.15.148:0/333900726 wait complete. 2024-03-29T04:10:26.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.637+0000 7fed57243640 1 Processor -- start 2024-03-29T04:10:26.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.637+0000 7fed57243640 1 -- start start 2024-03-29T04:10:26.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.639+0000 7fed57243640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed48057af0 0x7fed48009de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.639+0000 7fed57243640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed480e33e0 0x7fed4800c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.639+0000 7fed57243640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed480e6e10 0x7fed480da2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.639+0000 7fed57243640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fed480e7b80 con 0x7fed480e6e10 2024-03-29T04:10:26.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.639+0000 7fed57243640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fed4803c0b0 con 0x7fed48057af0 2024-03-29T04:10:26.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.639+0000 7fed57243640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fed48006fa0 con 0x7fed480e33e0 2024-03-29T04:10:26.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.639+0000 7fed56241640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed48057af0 0x7fed48009de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.639+0000 7fed56241640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed48057af0 0x7fed48009de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:39472/0 (socket says 172.21.15.148:39472) 2024-03-29T04:10:26.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.639+0000 7fed56241640 1 -- 172.21.15.148:0/3972710765 learned_addr learned my addr 172.21.15.148:0/3972710765 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:26.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.639+0000 7fed55a40640 1 --2- 172.21.15.148:0/3972710765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed480e33e0 0x7fed4800c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.639+0000 7fed56a42640 1 --2- 172.21.15.148:0/3972710765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed480e6e10 0x7fed480da2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.640+0000 7fed56241640 1 -- 172.21.15.148:0/3972710765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed480e33e0 msgr2=0x7fed4800c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.640+0000 7fed56241640 1 --2- 172.21.15.148:0/3972710765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed480e33e0 0x7fed4800c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.640+0000 7fed56241640 1 -- 172.21.15.148:0/3972710765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed480e6e10 msgr2=0x7fed480da2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.640+0000 7fed56241640 1 --2- 172.21.15.148:0/3972710765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed480e6e10 0x7fed480da2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.640+0000 7fed56241640 1 -- 172.21.15.148:0/3972710765 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed4800d0a0 con 0x7fed48057af0 2024-03-29T04:10:26.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.640+0000 7fed56a42640 1 --2- 172.21.15.148:0/3972710765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed480e6e10 0x7fed480da2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:26.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.640+0000 7fed56241640 1 --2- 172.21.15.148:0/3972710765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed48057af0 0x7fed48009de0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fed5006a3b0 tx=0x7fed500756f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.635+0000 7f61fcd1c640 1 -- 172.21.15.148:0/2454643286 shutdown_connections 2024-03-29T04:10:26.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.635+0000 7f61fcd1c640 1 -- 172.21.15.148:0/2454643286 wait complete. 2024-03-29T04:10:26.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.638+0000 7f61fcd1c640 1 Processor -- start 2024-03-29T04:10:26.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.638+0000 7f61fcd1c640 1 -- start start 2024-03-29T04:10:26.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.639+0000 7f61fcd1c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61f80ce620 0x7f61f80c9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.639+0000 7f61fcd1c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61f815c370 0x7f61f8144400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.639+0000 7f61f659c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61f80ce620 0x7f61f80c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.639+0000 7f61f659c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61f80ce620 0x7f61f80c9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.148:44512/0 (socket says 172.21.15.148:44512) 2024-03-29T04:10:26.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.639+0000 7f61f659c640 1 -- 172.21.15.148:0/2560232708 learned_addr learned my addr 172.21.15.148:0/2560232708 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:26.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.639+0000 7f61fcd1c640 1 --2- 172.21.15.148:0/2560232708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61f80ca9f0 0x7f61f80c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.640+0000 7f61fcd1c640 1 -- 172.21.15.148:0/2560232708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f61f80b5050 con 0x7f61f80ca9f0 2024-03-29T04:10:26.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.640+0000 7f61fcd1c640 1 -- 172.21.15.148:0/2560232708 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f61f80cc7d0 con 0x7f61f815c370 2024-03-29T04:10:26.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.640+0000 7f61fcd1c640 1 -- 172.21.15.148:0/2560232708 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f61f80b51d0 con 0x7f61f80ce620 2024-03-29T04:10:26.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.640+0000 7f61f5d9b640 1 --2- 172.21.15.148:0/2560232708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61f815c370 0x7f61f8144400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.640+0000 7f61f6d9d640 1 --2- 172.21.15.148:0/2560232708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61f80ca9f0 0x7f61f80c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.635+0000 7fc9397fa640 1 -- 172.21.15.148:0/3510553119 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c00c680 con 0x7fc9340df180 2024-03-29T04:10:26.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.635+0000 7fc9397fa640 1 -- 172.21.15.148:0/3510553119 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc92c01a070 con 0x7fc9340df180 2024-03-29T04:10:26.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.635+0000 7fc9397fa640 1 -- 172.21.15.148:0/3510553119 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c00ca60 con 0x7fc9340df180 2024-03-29T04:10:26.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.636+0000 7fc941bb9640 1 -- 172.21.15.148:0/3510553119 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc93417e260 con 0x7fc9340df180 2024-03-29T04:10:26.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.636+0000 7fc941bb9640 1 -- 172.21.15.148:0/3510553119 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9340e00e0 con 0x7fc9340df180 2024-03-29T04:10:26.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.636+0000 7effcb59e640 1 --2- 172.21.15.148:0/2057464116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effc40e42c0 0x7effc40eaee0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7effcc0511f0 tx=0x7effcc0a6ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988c88e402aee7a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.637+0000 7effc9d9b640 1 -- 172.21.15.148:0/2057464116 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc0b4070 con 0x7effc40e42c0 2024-03-29T04:10:26.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.637+0000 7effc9d9b640 1 -- 172.21.15.148:0/2057464116 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effcc0a7880 con 0x7effc40e42c0 2024-03-29T04:10:26.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.637+0000 7effc9d9b640 1 -- 172.21.15.148:0/2057464116 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc0015f0 con 0x7effc40e42c0 2024-03-29T04:10:26.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.638+0000 7effd09d3640 1 -- 172.21.15.148:0/2057464116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effc40e42c0 msgr2=0x7effc40eaee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.638+0000 7effd09d3640 1 --2- 172.21.15.148:0/2057464116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effc40e42c0 0x7effc40eaee0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7effcc0511f0 tx=0x7effcc0a6ea0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.638+0000 7effd09d3640 1 -- 172.21.15.148:0/2057464116 shutdown_connections 2024-03-29T04:10:26.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.638+0000 7effd09d3640 1 --2- 172.21.15.148:0/2057464116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effc40e42c0 0x7effc40eaee0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.638+0000 7effd09d3640 1 -- 172.21.15.148:0/2057464116 >> 172.21.15.148:0/2057464116 conn(0x7effc401ef30 msgr2=0x7effc4055c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.638+0000 7effd09d3640 1 -- 172.21.15.148:0/2057464116 shutdown_connections 2024-03-29T04:10:26.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.638+0000 7effd09d3640 1 -- 172.21.15.148:0/2057464116 wait complete. 2024-03-29T04:10:26.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.635+0000 7f1fbd4c5640 1 -- 172.21.15.148:0/1795879718 shutdown_connections 2024-03-29T04:10:26.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.635+0000 7f1fbd4c5640 1 --2- 172.21.15.148:0/1795879718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa0003d50 0x7f1fa0004190 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.635+0000 7f1fbd4c5640 1 -- 172.21.15.148:0/1795879718 >> 172.21.15.148:0/1795879718 conn(0x7f1fb80b2b50 msgr2=0x7f1fb80d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.635+0000 7f1fbd4c5640 1 -- 172.21.15.148:0/1795879718 shutdown_connections 2024-03-29T04:10:26.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.635+0000 7f1fbd4c5640 1 -- 172.21.15.148:0/1795879718 wait complete. 2024-03-29T04:10:26.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.637+0000 7f1fbd4c5640 1 Processor -- start 2024-03-29T04:10:26.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.637+0000 7f1fbd4c5640 1 -- start start 2024-03-29T04:10:26.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.639+0000 7f1fbd4c5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fa0003d50 0x7f1fb81534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.639+0000 7f1fbd4c5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fb8153b70 0x7f1fb8158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.639+0000 7f1fbd4c5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fb8159380 0x7f1fb8154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.639+0000 7f1fbd4c5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb815e840 con 0x7f1fa0003d50 2024-03-29T04:10:26.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.639+0000 7f1fbd4c5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb815e6c0 con 0x7f1fb8159380 2024-03-29T04:10:26.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.639+0000 7f1fbd4c5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1fb80b3b70 con 0x7f1fb8153b70 2024-03-29T04:10:26.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.636+0000 7fe3ba59c640 1 --2- 172.21.15.148:0/2258188034 >> v2:172.21.15.148:3300/0 conn(0x7fe3bc15bfb0 0x7fe3bc0d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.636+0000 7fe3b959a640 1 -- 172.21.15.148:0/2258188034 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2393029490 0 0) 0x7fe3bc06e970 con 0x7fe3bc0d0610 2024-03-29T04:10:26.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.636+0000 7fe3c0b7c640 1 -- 172.21.15.148:0/2258188034 --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3bc0d1d10 con 0x7fe3bc0ce940 2024-03-29T04:10:26.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.636+0000 7fe3c0b7c640 1 -- 172.21.15.148:0/2258188034 --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fe3bc0ce0f0 con 0x7fe3bc15bfb0 2024-03-29T04:10:26.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.636+0000 7fe3b959a640 1 -- 172.21.15.148:0/2258188034 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe3a4003520 con 0x7fe3bc0d0610 2024-03-29T04:10:26.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.636+0000 7fe3b959a640 1 -- 172.21.15.148:0/2258188034 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 892616390 0 0) 0x7fe3a4003520 con 0x7fe3bc0d0610 2024-03-29T04:10:26.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.636+0000 7fe3b959a640 1 -- 172.21.15.148:0/2258188034 >> v2:172.21.15.148:3300/0 conn(0x7fe3bc15bfb0 msgr2=0x7fe3bc0d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:26.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.636+0000 7fe3b959a640 1 --2- 172.21.15.148:0/2258188034 >> v2:172.21.15.148:3300/0 conn(0x7fe3bc15bfb0 0x7fe3bc0d16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.636+0000 7fe3b959a640 1 -- 172.21.15.148:0/2258188034 >> v1:172.21.15.157:6789/0 conn(0x7fe3bc0ce940 legacy=0x7fe3bc06ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:26.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.636+0000 7fe3b959a640 1 -- 172.21.15.148:0/2258188034 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3bc0ce3d0 con 0x7fe3bc0d0610 2024-03-29T04:10:26.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.636+0000 7fe3b959a640 1 -- 172.21.15.148:0/2258188034 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7fe3ac001690 con 0x7fe3bc0d0610 2024-03-29T04:10:26.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.636+0000 7fe3b959a640 1 -- 172.21.15.148:0/2258188034 >> v1:172.21.15.148:6789/0 conn(0x7fe3bc0d0610 legacy=0x7fe3bc0b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.636+0000 7fe3b959a640 1 --2- 172.21.15.148:0/2258188034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3a40039b0 0x7fe3a4003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.636+0000 7fe3b959a640 1 --2- 172.21.15.148:0/2258188034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3a4004800 0x7fe3a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.636+0000 7fe3b959a640 1 --2- 172.21.15.148:0/2258188034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3a40051c0 0x7fe3a40145c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.636+0000 7fe3b959a640 1 -- 172.21.15.148:0/2258188034 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe3bc0ce0f0 con 0x7fe3a4004800 2024-03-29T04:10:26.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.636+0000 7fe3b959a640 1 -- 172.21.15.148:0/2258188034 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a4014b00 con 0x7fe3a40051c0 2024-03-29T04:10:26.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.636+0000 7fe3b959a640 1 -- 172.21.15.148:0/2258188034 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe3a4014c80 con 0x7fe3a40039b0 2024-03-29T04:10:26.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.636+0000 7fe3ba59c640 1 --2- 172.21.15.148:0/2258188034 >> v2:172.21.15.148:3300/0 conn(0x7fe3bc15bfb0 0x7fe3bc0d16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:26.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.637+0000 7fe3b9d9b640 1 --2- 172.21.15.148:0/2258188034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3a4004800 0x7fe3a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.637+0000 7fe3b9d9b640 1 -- 172.21.15.148:0/2258188034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3a40039b0 msgr2=0x7fe3a4003db0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-29T04:10:26.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.637+0000 7fe3b9d9b640 1 --2- 172.21.15.148:0/2258188034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3a40039b0 0x7fe3a4003db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.637+0000 7fe3b9d9b640 1 -- 172.21.15.148:0/2258188034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3a40051c0 msgr2=0x7fe3a40145c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:10:26.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.637+0000 7fe3b9d9b640 1 --2- 172.21.15.148:0/2258188034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3a40051c0 0x7fe3a40145c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.637+0000 7fe3b9d9b640 1 -- 172.21.15.148:0/2258188034 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3bc0ce3d0 con 0x7fe3a4004800 2024-03-29T04:10:26.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.639+0000 7fe3b9d9b640 1 --2- 172.21.15.148:0/2258188034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3a4004800 0x7fe3a4004c80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00c260 tx=0x7fe3a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6745a2827c4b7366 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.639+0000 7fe3b959a640 1 -- 172.21.15.148:0/2258188034 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac007d70 con 0x7fe3a4004800 2024-03-29T04:10:26.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.639+0000 7fe3b959a640 1 -- 172.21.15.148:0/2258188034 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3ac030070 con 0x7fe3a4004800 2024-03-29T04:10:26.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.639+0000 7fe3b959a640 1 -- 172.21.15.148:0/2258188034 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac02cdf0 con 0x7fe3a4004800 2024-03-29T04:10:26.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.640+0000 7fe3bad9d640 1 -- 172.21.15.148:0/2258188034 reap_dead start 2024-03-29T04:10:26.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.640+0000 7fc941bb9640 1 -- 172.21.15.148:0/3510553119 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc91c005290 con 0x7fc9340df180 2024-03-29T04:10:26.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.640+0000 7effd09d3640 1 Processor -- start 2024-03-29T04:10:26.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.640+0000 7f1fb67fc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fb8153b70 0x7f1fb8158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.640+0000 7f1fb67fc640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fb8153b70 0x7f1fb8158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.148:44502/0 (socket says 172.21.15.148:44502) 2024-03-29T04:10:26.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.640+0000 7f1fb67fc640 1 -- 172.21.15.148:0/2978263887 learned_addr learned my addr 172.21.15.148:0/2978263887 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:26.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.640+0000 7f1fb77fe640 1 --2- 172.21.15.148:0/2978263887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fb8159380 0x7f1fb8154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.640+0000 7f1fb6ffd640 1 --2- 172.21.15.148:0/2978263887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fa0003d50 0x7f1fb81534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.640+0000 7fe3c0b7c640 1 -- 172.21.15.148:0/2258188034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3a4004800 msgr2=0x7fe3a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.640+0000 7fe3c0b7c640 1 --2- 172.21.15.148:0/2258188034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3a4004800 0x7fe3a4004c80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00c260 tx=0x7fe3a4005830 comp rx=0 tx=0).stop 2024-03-29T04:10:26.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.641+0000 7fed46ffd640 1 -- 172.21.15.148:0/3972710765 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50074a20 con 0x7fed48057af0 2024-03-29T04:10:26.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.641+0000 7f61f5d9b640 1 -- 172.21.15.148:0/2560232708 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61f80ce620 msgr2=0x7f61f80c9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.641+0000 7f1fb67fc640 1 -- 172.21.15.148:0/2978263887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fb8159380 msgr2=0x7f1fb8154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.641+0000 7f1fb67fc640 1 --2- 172.21.15.148:0/2978263887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fb8159380 0x7f1fb8154740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.641+0000 7f1fb67fc640 1 -- 172.21.15.148:0/2978263887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fa0003d50 msgr2=0x7f1fb81534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.641+0000 7f61f5d9b640 1 --2- 172.21.15.148:0/2560232708 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61f80ce620 0x7f61f80c9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.641+0000 7fed57243640 1 -- 172.21.15.148:0/3972710765 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed4800d380 con 0x7fed48057af0 2024-03-29T04:10:26.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.641+0000 7fed46ffd640 1 -- 172.21.15.148:0/3972710765 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed50074bc0 con 0x7fed48057af0 2024-03-29T04:10:26.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.641+0000 7f61f5d9b640 1 -- 172.21.15.148:0/2560232708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61f80ca9f0 msgr2=0x7f61f80c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.641+0000 7f61f5d9b640 1 --2- 172.21.15.148:0/2560232708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61f80ca9f0 0x7f61f80c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.641+0000 7f61f5d9b640 1 -- 172.21.15.148:0/2560232708 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f80c9950 con 0x7f61f815c370 2024-03-29T04:10:26.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.641+0000 7f61f659c640 1 --2- 172.21.15.148:0/2560232708 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61f80ce620 0x7f61f80c9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:26.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.641+0000 7f61f6d9d640 1 --2- 172.21.15.148:0/2560232708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61f80ca9f0 0x7f61f80c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:26.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.641+0000 7fe3c0b7c640 1 -- 172.21.15.148:0/2258188034 shutdown_connections 2024-03-29T04:10:26.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.641+0000 7fe3c0b7c640 1 --2- 172.21.15.148:0/2258188034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3a4004800 0x7fe3a4004c80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.641+0000 7fe3c0b7c640 1 -- 172.21.15.148:0/2258188034 >> 172.21.15.148:0/2258188034 conn(0x7fe3bc0b3c00 msgr2=0x7fe3bc0b3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.641+0000 7f1fb67fc640 1 --2- 172.21.15.148:0/2978263887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fa0003d50 0x7f1fb81534e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.641+0000 7f1fb67fc640 1 -- 172.21.15.148:0/2978263887 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb8157a00 con 0x7f1fb8153b70 2024-03-29T04:10:26.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.641+0000 7fe3c0b7c640 1 -- 172.21.15.148:0/2258188034 shutdown_connections 2024-03-29T04:10:26.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.641+0000 7f1fb6ffd640 1 --2- 172.21.15.148:0/2978263887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fa0003d50 0x7f1fb81534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:26.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.641+0000 7fed57243640 1 -- 172.21.15.148:0/3972710765 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed4800d940 con 0x7fed48057af0 2024-03-29T04:10:26.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.640+0000 7effd09d3640 1 -- start start 2024-03-29T04:10:26.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.642+0000 7effd09d3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effc40dfee0 0x7effc400c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.642+0000 7fed46ffd640 1 -- 172.21.15.148:0/3972710765 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50074ff0 con 0x7fed48057af0 2024-03-29T04:10:26.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.642+0000 7effd09d3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effc40e42c0 0x7effc40dbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.641+0000 7fe3c0b7c640 1 -- 172.21.15.148:0/2258188034 wait complete. 2024-03-29T04:10:26.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.642+0000 7f1fb67fc640 1 --2- 172.21.15.148:0/2978263887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fb8153b70 0x7f1fb8158e40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1fa80108c0 tx=0x7f1fa8010cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.642+0000 7f61f5d9b640 1 --2- 172.21.15.148:0/2560232708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61f815c370 0x7f61f8144400 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f61e8033000 tx=0x7f61e80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.642+0000 7f1f9f7fe640 1 -- 172.21.15.148:0/2978263887 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa801a020 con 0x7f1fb8153b70 2024-03-29T04:10:26.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.642+0000 7f1f9f7fe640 1 -- 172.21.15.148:0/2978263887 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fa800eb60 con 0x7f1fb8153b70 2024-03-29T04:10:26.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.642+0000 7f1f9f7fe640 1 -- 172.21.15.148:0/2978263887 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa800f040 con 0x7f1fb8153b70 2024-03-29T04:10:26.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.642+0000 7effcad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effc40dfee0 0x7effc400c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.642+0000 7fc9397fa640 1 -- 172.21.15.148:0/3510553119 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc92c00cc00 con 0x7fc9340df180 2024-03-29T04:10:26.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.642+0000 7effca59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effc40e42c0 0x7effc40dbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.642+0000 7effca59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effc40e42c0 0x7effc40dbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48728/0 (socket says 172.21.15.148:48728) 2024-03-29T04:10:26.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.642+0000 7effcad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effc40dfee0 0x7effc400c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:39502/0 (socket says 172.21.15.148:39502) 2024-03-29T04:10:26.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.642+0000 7effd09d3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effc40e2ab0 0x7effc40e2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.642+0000 7effd09d3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7effc40e4900 con 0x7effc40e42c0 2024-03-29T04:10:26.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.642+0000 7effd09d3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7effc4056b40 con 0x7effc40dfee0 2024-03-29T04:10:26.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.642+0000 7effd09d3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7effc40549f0 con 0x7effc40e2ab0 2024-03-29T04:10:26.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.642+0000 7effcad9d640 1 -- 172.21.15.148:0/1495560685 learned_addr learned my addr 172.21.15.148:0/1495560685 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:26.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.642+0000 7fed57243640 1 -- 172.21.15.148:0/3972710765 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed38005290 con 0x7fed48057af0 2024-03-29T04:10:26.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.643+0000 7f1fbd4c5640 1 -- 172.21.15.148:0/2978263887 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fb8157ce0 con 0x7f1fb8153b70 2024-03-29T04:10:26.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.643+0000 7effcb59e640 1 --2- 172.21.15.148:0/1495560685 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effc40e2ab0 0x7effc40e2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.643+0000 7fc9397fa640 1 --2- 172.21.15.148:0/3510553119 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc92806c560 0x7fc92806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.643+0000 7fc9397fa640 1 -- 172.21.15.148:0/3510553119 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc92c08b940 con 0x7fc9340df180 2024-03-29T04:10:26.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.643+0000 7f1fbd4c5640 1 -- 172.21.15.148:0/2978263887 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fb81582a0 con 0x7f1fb8153b70 2024-03-29T04:10:26.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.643+0000 7fc940bb7640 1 --2- 172.21.15.148:0/3510553119 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc92806c560 0x7fc92806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.643+0000 7effcb59e640 1 -- 172.21.15.148:0/1495560685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effc40dfee0 msgr2=0x7effc400c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.643+0000 7effcb59e640 1 --2- 172.21.15.148:0/1495560685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effc40dfee0 0x7effc400c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.643+0000 7effcb59e640 1 -- 172.21.15.148:0/1495560685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effc40e42c0 msgr2=0x7effc40dbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.643+0000 7effcb59e640 1 --2- 172.21.15.148:0/1495560685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effc40e42c0 0x7effc40dbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.643+0000 7effcb59e640 1 -- 172.21.15.148:0/1495560685 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc417dc60 con 0x7effc40e2ab0 2024-03-29T04:10:26.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.643+0000 7effca59c640 1 --2- 172.21.15.148:0/1495560685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effc40e42c0 0x7effc40dbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:26.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.644+0000 7fe3c0b7c640 1 Processor -- start 2024-03-29T04:10:26.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.644+0000 7fe3c0b7c640 1 -- start start 2024-03-29T04:10:26.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.644+0000 7effcb59e640 1 --2- 172.21.15.148:0/1495560685 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effc40e2ab0 0x7effc40e2f40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7effcc069ef0 tx=0x7effcc06b350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.644+0000 7f1fbd4c5640 1 -- 172.21.15.148:0/2978263887 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fb80687e0 con 0x7f1fb8153b70 2024-03-29T04:10:26.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.644+0000 7effb37fe640 1 -- 172.21.15.148:0/1495560685 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc0be070 con 0x7effc40e2ab0 2024-03-29T04:10:26.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.644+0000 7effb37fe640 1 -- 172.21.15.148:0/1495560685 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effcc0afc90 con 0x7effc40e2ab0 2024-03-29T04:10:26.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.644+0000 7fc940bb7640 1 --2- 172.21.15.148:0/3510553119 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc92806c560 0x7fc92806ea20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc924004660 tx=0x7fc93c06f030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.644+0000 7f1f9f7fe640 1 -- 172.21.15.148:0/2978263887 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1fa801d020 con 0x7f1fb8153b70 2024-03-29T04:10:26.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.645+0000 7f1f9f7fe640 1 --2- 172.21.15.148:0/2978263887 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1fa406c630 0x7f1fa406eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.645+0000 7effb37fe640 1 -- 172.21.15.148:0/1495560685 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc0b4040 con 0x7effc40e2ab0 2024-03-29T04:10:26.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.645+0000 7fe3c0b7c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3a4004800 0x7fe3bc157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.645+0000 7fe3c0b7c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3bc0b7a20 0x7fe3bc13ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.645+0000 7fe3c0b7c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3bc0d0f20 0x7fe3bc0d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.645+0000 7f1f9f7fe640 1 -- 172.21.15.148:0/2978263887 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1fa808c300 con 0x7f1fb8153b70 2024-03-29T04:10:26.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.645+0000 7fe3c0b7c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe3bc0d0aa0 con 0x7fe3bc0d0f20 2024-03-29T04:10:26.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.645+0000 7fe3c0b7c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe3bc15c390 con 0x7fe3bc0b7a20 2024-03-29T04:10:26.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.645+0000 7fe3c0b7c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe3bc06f560 con 0x7fe3a4004800 2024-03-29T04:10:26.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.645+0000 7f1fb6ffd640 1 --2- 172.21.15.148:0/2978263887 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1fa406c630 0x7f1fa406eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.645+0000 7effd09d3640 1 -- 172.21.15.148:0/1495560685 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc400b760 con 0x7effc40e2ab0 2024-03-29T04:10:26.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.645+0000 7effd09d3640 1 -- 172.21.15.148:0/1495560685 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7effc417cea0 con 0x7effc40e2ab0 2024-03-29T04:10:26.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.645+0000 7fe3b9d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3bc0b7a20 0x7fe3bc13ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.646+0000 7fe3b9d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3bc0b7a20 0x7fe3bc13ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:39506/0 (socket says 172.21.15.148:39506) 2024-03-29T04:10:26.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.646+0000 7fe3b9d9b640 1 -- 172.21.15.148:0/3165190578 learned_addr learned my addr 172.21.15.148:0/3165190578 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:26.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.646+0000 7fe3ba59c640 1 --2- 172.21.15.148:0/3165190578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3a4004800 0x7fe3bc157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.646+0000 7f1fb6ffd640 1 --2- 172.21.15.148:0/2978263887 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1fa406c630 0x7f1fa406eaf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00c430 tx=0x7f1fac039040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.646+0000 7fe3bad9d640 1 --2- 172.21.15.148:0/3165190578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3bc0d0f20 0x7fe3bc0d13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.646+0000 7fe3b9d9b640 1 -- 172.21.15.148:0/3165190578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3a4004800 msgr2=0x7fe3bc157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.646+0000 7fe3b9d9b640 1 --2- 172.21.15.148:0/3165190578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3a4004800 0x7fe3bc157a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.646+0000 7fe3b9d9b640 1 -- 172.21.15.148:0/3165190578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3bc0d0f20 msgr2=0x7fe3bc0d13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.646+0000 7fe3b9d9b640 1 --2- 172.21.15.148:0/3165190578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3bc0d0f20 0x7fe3bc0d13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.646+0000 7fe3b9d9b640 1 -- 172.21.15.148:0/3165190578 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3bc1427b0 con 0x7fe3bc0b7a20 2024-03-29T04:10:26.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.646+0000 7fe3bad9d640 1 --2- 172.21.15.148:0/3165190578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3bc0d0f20 0x7fe3bc0d13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:26.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.646+0000 7fe3b9d9b640 1 --2- 172.21.15.148:0/3165190578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3bc0b7a20 0x7fe3bc13ef00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00c350 tx=0x7fe3ac001990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.646+0000 7f61e6ffd640 1 -- 172.21.15.148:0/2560232708 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e8041070 con 0x7f61f815c370 2024-03-29T04:10:26.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.647+0000 7fc9397fa640 1 -- 172.21.15.148:0/3510553119 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc92c023020 con 0x7fc9340df180 2024-03-29T04:10:26.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.646+0000 7effd09d3640 1 -- 172.21.15.148:0/1495560685 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effc40057d0 con 0x7effc40e2ab0 2024-03-29T04:10:26.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.647+0000 7f61e6ffd640 1 -- 172.21.15.148:0/2560232708 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61e803ac80 con 0x7f61f815c370 2024-03-29T04:10:26.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.647+0000 7f61e6ffd640 1 -- 172.21.15.148:0/2560232708 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e8035040 con 0x7f61f815c370 2024-03-29T04:10:26.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.647+0000 7f61fcd1c640 1 -- 172.21.15.148:0/2560232708 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f80c9c30 con 0x7f61f815c370 2024-03-29T04:10:26.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.647+0000 7f61fcd1c640 1 -- 172.21.15.148:0/2560232708 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f80ca1f0 con 0x7f61f815c370 2024-03-29T04:10:26.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.647+0000 7fe3aaffd640 1 -- 172.21.15.148:0/3165190578 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac045070 con 0x7fe3bc0b7a20 2024-03-29T04:10:26.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.647+0000 7fe3aaffd640 1 -- 172.21.15.148:0/3165190578 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3ac004cb0 con 0x7fe3bc0b7a20 2024-03-29T04:10:26.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.647+0000 7fe3aaffd640 1 -- 172.21.15.148:0/3165190578 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac004e60 con 0x7fe3bc0b7a20 2024-03-29T04:10:26.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.647+0000 7fe3c0b7c640 1 -- 172.21.15.148:0/3165190578 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3bc157f70 con 0x7fe3bc0b7a20 2024-03-29T04:10:26.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.647+0000 7fe3c0b7c640 1 -- 172.21.15.148:0/3165190578 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3bc1424d0 con 0x7fe3bc0b7a20 2024-03-29T04:10:26.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.648+0000 7fed46ffd640 1 -- 172.21.15.148:0/3972710765 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fed5006b7c0 con 0x7fed48057af0 2024-03-29T04:10:26.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.648+0000 7fed46ffd640 1 --2- 172.21.15.148:0/3972710765 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fed3406c580 0x7fed3406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.649+0000 7fed55a40640 1 --2- 172.21.15.148:0/3972710765 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fed3406c580 0x7fed3406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.649+0000 7f61fcd1c640 1 -- 172.21.15.148:0/2560232708 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61f80687e0 con 0x7f61f815c370 2024-03-29T04:10:26.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.649+0000 7fed46ffd640 1 -- 172.21.15.148:0/3972710765 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fed50106dd0 con 0x7fed48057af0 2024-03-29T04:10:26.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.649+0000 7fe3c0b7c640 1 -- 172.21.15.148:0/3165190578 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3bc068cd0 con 0x7fe3bc0b7a20 2024-03-29T04:10:26.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.649+0000 7fed55a40640 1 --2- 172.21.15.148:0/3972710765 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fed3406c580 0x7fed3406ea40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fed40036000 tx=0x7fed40008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.650+0000 7f1f9f7fe640 1 -- 172.21.15.148:0/2978263887 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1fa8024020 con 0x7f1fb8153b70 2024-03-29T04:10:26.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.652+0000 7effb37fe640 1 -- 172.21.15.148:0/1495560685 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7effcc0c1070 con 0x7effc40e2ab0 2024-03-29T04:10:26.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.652+0000 7fed46ffd640 1 -- 172.21.15.148:0/3972710765 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fed500d5120 con 0x7fed48057af0 2024-03-29T04:10:26.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.652+0000 7effb37fe640 1 --2- 172.21.15.148:0/1495560685 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7effb806c580 0x7effb806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.652+0000 7effb37fe640 1 -- 172.21.15.148:0/1495560685 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7effcc128d00 con 0x7effc40e2ab0 2024-03-29T04:10:26.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.652+0000 7effcad9d640 1 --2- 172.21.15.148:0/1495560685 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7effb806c580 0x7effb806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.653+0000 7effcad9d640 1 --2- 172.21.15.148:0/1495560685 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7effb806c580 0x7effb806ea40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7effbc001db0 tx=0x7effbc001d40 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.653+0000 7f61e6ffd640 1 -- 172.21.15.148:0/2560232708 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f61e804c060 con 0x7f61f815c370 2024-03-29T04:10:26.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.654+0000 7f61e6ffd640 1 --2- 172.21.15.148:0/2560232708 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f61d806c530 0x7f61d806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.654+0000 7f61f659c640 1 --2- 172.21.15.148:0/2560232708 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f61d806c530 0x7f61d806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.655+0000 7f61f659c640 1 --2- 172.21.15.148:0/2560232708 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f61d806c530 0x7f61d806e9f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f61ec0011e0 tx=0x7f61ec009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.655+0000 7f61e6ffd640 1 -- 172.21.15.148:0/2560232708 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f61e80ad750 con 0x7f61f815c370 2024-03-29T04:10:26.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.656+0000 7effb37fe640 1 -- 172.21.15.148:0/1495560685 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7effcc0c8070 con 0x7effc40e2ab0 2024-03-29T04:10:26.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.656+0000 7fc466ac4640 1 Processor -- start 2024-03-29T04:10:26.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.656+0000 7fc466ac4640 1 -- start start 2024-03-29T04:10:26.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.656+0000 7fe3aaffd640 1 -- 172.21.15.148:0/3165190578 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe3ac0072b0 con 0x7fe3bc0b7a20 2024-03-29T04:10:26.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.657+0000 7fe3aaffd640 1 --2- 172.21.15.148:0/3165190578 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe39c06c600 0x7fe39c06eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.657+0000 7fe3aaffd640 1 -- 172.21.15.148:0/3165190578 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe3ac0ad520 con 0x7fe3bc0b7a20 2024-03-29T04:10:26.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.657+0000 7fc466ac4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc460072520 0x7fc4600b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.657+0000 7fc466ac4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc46015f580 0x7fc460072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.657+0000 7fc466ac4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc460161260 0x7fc4601616f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.657+0000 7fc466ac4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fc4600b59f0 con 0x7fc46015f580 2024-03-29T04:10:26.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.657+0000 7fc466ac4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc460072e40 con 0x7fc460161260 2024-03-29T04:10:26.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.657+0000 7fc466ac4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc46006dfc0 con 0x7fc460072520 2024-03-29T04:10:26.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.657+0000 7fc465061640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc460161260 0x7fc4601616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.657+0000 7fc45ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc46015f580 0x7fc460072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.657+0000 7fc464860640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc460072520 0x7fc4600b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.658+0000 7fc45ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc46015f580 0x7fc460072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48752/0 (socket says 172.21.15.148:48752) 2024-03-29T04:10:26.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.658+0000 7fc45ffff640 1 -- 172.21.15.148:0/1689873018 learned_addr learned my addr 172.21.15.148:0/1689873018 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:26.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.658+0000 7fc45ffff640 1 -- 172.21.15.148:0/1689873018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc460072520 msgr2=0x7fc4600b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:26.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.658+0000 7fc45ffff640 1 --2- 172.21.15.148:0/1689873018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc460072520 0x7fc4600b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.658+0000 7fc45ffff640 1 -- 172.21.15.148:0/1689873018 >> v2:172.21.15.157:3300/0 conn(0x7fc460161260 msgr2=0x7fc4601616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:26.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.658+0000 7fc45ffff640 1 --2- 172.21.15.148:0/1689873018 >> v2:172.21.15.157:3300/0 conn(0x7fc460161260 0x7fc4601616f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.658+0000 7fc45ffff640 1 -- 172.21.15.148:0/1689873018 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4600b5c20 con 0x7fc46015f580 2024-03-29T04:10:26.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.659+0000 7fc45ffff640 1 --2- 172.21.15.148:0/1689873018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc46015f580 0x7fc460072900 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc4600724b0 tx=0x7fc44c02b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc522e387d73075 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.659+0000 7fe3ba59c640 1 --2- 172.21.15.148:0/3165190578 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe39c06c600 0x7fe39c06eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.660+0000 7fc45f7fe640 1 -- 172.21.15.148:0/1689873018 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c035020 con 0x7fc46015f580 2024-03-29T04:10:26.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.660+0000 7fc45f7fe640 1 -- 172.21.15.148:0/1689873018 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc44c031880 con 0x7fc46015f580 2024-03-29T04:10:26.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.660+0000 7fc45f7fe640 1 -- 172.21.15.148:0/1689873018 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c031cb0 con 0x7fc46015f580 2024-03-29T04:10:26.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.660+0000 7fe3ba59c640 1 --2- 172.21.15.148:0/3165190578 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe39c06c600 0x7fe39c06eac0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0006fd0 tx=0x7fe3b001b210 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.661+0000 7fc466ac4640 1 -- 172.21.15.148:0/1689873018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc46015f580 msgr2=0x7fc460072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.661+0000 7fc466ac4640 1 --2- 172.21.15.148:0/1689873018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc46015f580 0x7fc460072900 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc4600724b0 tx=0x7fc44c02b940 comp rx=0 tx=0).stop 2024-03-29T04:10:26.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.661+0000 7f61e6ffd640 1 -- 172.21.15.148:0/2560232708 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f61e807bba0 con 0x7f61f815c370 2024-03-29T04:10:26.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.661+0000 7fc466ac4640 1 -- 172.21.15.148:0/1689873018 shutdown_connections 2024-03-29T04:10:26.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.661+0000 7fc466ac4640 1 --2- 172.21.15.148:0/1689873018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc46015f580 0x7fc460072900 secure :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc4600724b0 tx=0x7fc44c02b940 comp rx=0 tx=0).stop 2024-03-29T04:10:26.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.661+0000 7fc466ac4640 1 --2- 172.21.15.148:0/1689873018 >> v2:172.21.15.157:3300/0 conn(0x7fc460161260 0x7fc4601616f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.661+0000 7fc466ac4640 1 --2- 172.21.15.148:0/1689873018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc460072520 0x7fc4600b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.661+0000 7fc466ac4640 1 -- 172.21.15.148:0/1689873018 >> 172.21.15.148:0/1689873018 conn(0x7fc4600d0f50 msgr2=0x7fc4600d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.661+0000 7fc466ac4640 1 -- 172.21.15.148:0/1689873018 shutdown_connections 2024-03-29T04:10:26.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.661+0000 7fc466ac4640 1 -- 172.21.15.148:0/1689873018 wait complete. 2024-03-29T04:10:26.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.663+0000 7fc466ac4640 1 Processor -- start 2024-03-29T04:10:26.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.664+0000 7fc466ac4640 1 -- start start 2024-03-29T04:10:26.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.664+0000 7fe3aaffd640 1 -- 172.21.15.148:0/3165190578 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe3ac07b190 con 0x7fe3bc0b7a20 2024-03-29T04:10:26.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.665+0000 7fc466ac4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc460072520 0x7fc4601457b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.666+0000 7fc464860640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc460072520 0x7fc4601457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.666+0000 7fc464860640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc460072520 0x7fc4601457b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48760/0 (socket says 172.21.15.148:48760) 2024-03-29T04:10:26.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.666+0000 7fc464860640 1 -- 172.21.15.148:0/3176080785 learned_addr learned my addr 172.21.15.148:0/3176080785 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:26.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.666+0000 7fc466ac4640 1 --2- 172.21.15.148:0/3176080785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc460161260 0x7fc460147ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.666+0000 7fc466ac4640 1 --2- 172.21.15.148:0/3176080785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc460156bf0 0x7fc4601570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.666+0000 7fc466ac4640 1 -- 172.21.15.148:0/3176080785 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc4600b61e0 con 0x7fc460072520 2024-03-29T04:10:26.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.666+0000 7fc466ac4640 1 -- 172.21.15.148:0/3176080785 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc4600b4540 con 0x7fc460156bf0 2024-03-29T04:10:26.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.666+0000 7fc466ac4640 1 -- 172.21.15.148:0/3176080785 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc46015fa00 con 0x7fc460161260 2024-03-29T04:10:26.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.666+0000 7fc464860640 1 -- 172.21.15.148:0/3176080785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc460161260 msgr2=0x7fc460147ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-29T04:10:26.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.666+0000 7fc464860640 1 --2- 172.21.15.148:0/3176080785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc460161260 0x7fc460147ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.666+0000 7fc464860640 1 -- 172.21.15.148:0/3176080785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc460156bf0 msgr2=0x7fc4601570a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-29T04:10:26.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.666+0000 7fc464860640 1 --2- 172.21.15.148:0/3176080785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc460156bf0 0x7fc4601570a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.666+0000 7fc464860640 1 -- 172.21.15.148:0/3176080785 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc460157760 con 0x7fc460072520 2024-03-29T04:10:26.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.667+0000 7fc464860640 1 --2- 172.21.15.148:0/3176080785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc460072520 0x7fc4601457b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc45400d920 tx=0x7fc45400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.667+0000 7fc45d7fa640 1 -- 172.21.15.148:0/3176080785 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45400b980 con 0x7fc460072520 2024-03-29T04:10:26.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.668+0000 7fc45d7fa640 1 -- 172.21.15.148:0/3176080785 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc45402a070 con 0x7fc460072520 2024-03-29T04:10:26.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.668+0000 7fc466ac4640 1 -- 172.21.15.148:0/3176080785 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46015a1f0 con 0x7fc460072520 2024-03-29T04:10:26.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.670+0000 7fc466ac4640 1 -- 172.21.15.148:0/3176080785 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc46015a7b0 con 0x7fc460072520 2024-03-29T04:10:26.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.672+0000 7fc45d7fa640 1 -- 172.21.15.148:0/3176080785 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45400bd20 con 0x7fc460072520 2024-03-29T04:10:26.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.673+0000 7fc45d7fa640 1 -- 172.21.15.148:0/3176080785 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc45402f460 con 0x7fc460072520 2024-03-29T04:10:26.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.673+0000 7fc44affd640 1 -- 172.21.15.148:0/3176080785 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4600687e0 con 0x7fc460072520 2024-03-29T04:10:26.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.674+0000 7fc45d7fa640 1 --2- 172.21.15.148:0/3176080785 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc44406c630 0x7fc44406eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.674+0000 7fc45d7fa640 1 -- 172.21.15.148:0/3176080785 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc4540a0070 con 0x7fc460072520 2024-03-29T04:10:26.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.675+0000 7fc45ffff640 1 --2- 172.21.15.148:0/3176080785 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc44406c630 0x7fc44406eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.675+0000 7fc45ffff640 1 --2- 172.21.15.148:0/3176080785 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc44406c630 0x7fc44406eaf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc44c036000 tx=0x7fc44c008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.683+0000 7fc45d7fa640 1 -- 172.21.15.148:0/3176080785 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc45406adf0 con 0x7fc460072520 2024-03-29T04:10:26.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.686+0000 7f40cf463640 1 Processor -- start 2024-03-29T04:10:26.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.687+0000 7f40cf463640 1 -- start start 2024-03-29T04:10:26.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.689+0000 7f40cf463640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f40c815c130 0x7f40c806f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.689+0000 7f40cf463640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40c815c510 0x7f40c806ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.689+0000 7f40cf463640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40c80b00d0 con 0x7f40c806bfc0 2024-03-29T04:10:26.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.689+0000 7f40cf463640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f40c80ccf10 con 0x7f40c815c130 2024-03-29T04:10:26.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.689+0000 7f40cf463640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f40c80ccc30 con 0x7f40c815c510 2024-03-29T04:10:26.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.689+0000 7f40cda00640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f40c806bfc0 0x7f40c806c3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47300/0 (socket says 172.21.15.148:47300) 2024-03-29T04:10:26.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.689+0000 7f40cda00640 1 -- 172.21.15.148:0/1019579326 learned_addr learned my addr 172.21.15.148:0/1019579326 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:26.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.690+0000 7f40cd1ff640 1 --2- 172.21.15.148:0/1019579326 >> v2:172.21.15.157:3300/0 conn(0x7f40c815c130 0x7f40c806f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.690+0000 7f40bffff640 1 -- 172.21.15.148:0/1019579326 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2396500995 0 0) 0x7f40c80b00d0 con 0x7f40c806bfc0 2024-03-29T04:10:26.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.690+0000 7f40cc9fe640 1 --2- 172.21.15.148:0/1019579326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40c815c510 0x7f40c806ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.690+0000 7f40bffff640 1 -- 172.21.15.148:0/1019579326 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40b4003520 con 0x7f40c806bfc0 2024-03-29T04:10:26.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.690+0000 7f40bffff640 1 -- 172.21.15.148:0/1019579326 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3129929653 0 0) 0x7f40b4003520 con 0x7f40c806bfc0 2024-03-29T04:10:26.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.690+0000 7f40bffff640 1 -- 172.21.15.148:0/1019579326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40c815c510 msgr2=0x7f40c806ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:26.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.690+0000 7f40bffff640 1 --2- 172.21.15.148:0/1019579326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40c815c510 0x7f40c806ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.690+0000 7f40bffff640 1 -- 172.21.15.148:0/1019579326 >> v2:172.21.15.157:3300/0 conn(0x7f40c815c130 msgr2=0x7f40c806f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:26.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.690+0000 7f40bffff640 1 --2- 172.21.15.148:0/1019579326 >> v2:172.21.15.157:3300/0 conn(0x7f40c815c130 0x7f40c806f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.690+0000 7f40bffff640 1 -- 172.21.15.148:0/1019579326 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c80cae20 con 0x7f40c806bfc0 2024-03-29T04:10:26.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.690+0000 7f40bffff640 1 -- 172.21.15.148:0/1019579326 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f40b8001670 con 0x7f40c806bfc0 2024-03-29T04:10:26.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.690+0000 7f40bffff640 1 -- 172.21.15.148:0/1019579326 >> v1:172.21.15.148:6789/0 conn(0x7f40c806bfc0 legacy=0x7f40c806c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.690+0000 7f40bffff640 1 --2- 172.21.15.148:0/1019579326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40b40039b0 0x7f40b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.690+0000 7f40bffff640 1 --2- 172.21.15.148:0/1019579326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40b4004800 0x7f40b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.690+0000 7f40bffff640 1 --2- 172.21.15.148:0/1019579326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40b40052e0 0x7f40b40141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.690+0000 7f40bffff640 1 -- 172.21.15.148:0/1019579326 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f40c80ccf10 con 0x7f40b4004800 2024-03-29T04:10:26.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.690+0000 7f40bffff640 1 -- 172.21.15.148:0/1019579326 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f40c80ccc30 con 0x7f40b40039b0 2024-03-29T04:10:26.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.690+0000 7f40bffff640 1 -- 172.21.15.148:0/1019579326 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f40b40149d0 con 0x7f40b40052e0 2024-03-29T04:10:26.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.691+0000 7f40cda00640 1 --2- 172.21.15.148:0/1019579326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40b40052e0 0x7f40b40141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.691+0000 7f40cda00640 1 -- 172.21.15.148:0/1019579326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40b40039b0 msgr2=0x7f40b4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:26.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.691+0000 7f40cda00640 1 --2- 172.21.15.148:0/1019579326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40b40039b0 0x7f40b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.691+0000 7f40cda00640 1 -- 172.21.15.148:0/1019579326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40b4004800 msgr2=0x7f40b4004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:10:26.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.691+0000 7f40cda00640 1 --2- 172.21.15.148:0/1019579326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40b4004800 0x7f40b4004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.691+0000 7f40cda00640 1 -- 172.21.15.148:0/1019579326 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b4003520 con 0x7f40b40052e0 2024-03-29T04:10:26.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.692+0000 7f40cda00640 1 --2- 172.21.15.148:0/1019579326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40b40052e0 0x7f40b40141d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f40b8001eb0 tx=0x7f40b4005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38e833b745969737 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.693+0000 7f40bffff640 1 -- 172.21.15.148:0/1019579326 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b8007ce0 con 0x7f40b40052e0 2024-03-29T04:10:26.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.693+0000 7f40bffff640 1 -- 172.21.15.148:0/1019579326 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40b8030070 con 0x7f40b40052e0 2024-03-29T04:10:26.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.693+0000 7f40bffff640 1 -- 172.21.15.148:0/1019579326 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b8005d30 con 0x7f40b40052e0 2024-03-29T04:10:26.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.694+0000 7f40cf463640 1 -- 172.21.15.148:0/1019579326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40b40052e0 msgr2=0x7f40b40141d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.694+0000 7f40cf463640 1 --2- 172.21.15.148:0/1019579326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40b40052e0 0x7f40b40141d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f40b8001eb0 tx=0x7f40b4005830 comp rx=0 tx=0).stop 2024-03-29T04:10:26.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.694+0000 7f40cda00640 1 -- 172.21.15.148:0/1019579326 reap_dead start 2024-03-29T04:10:26.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.697+0000 7f40cf463640 1 -- 172.21.15.148:0/1019579326 shutdown_connections 2024-03-29T04:10:26.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.697+0000 7f40cf463640 1 -- 172.21.15.148:0/1019579326 >> 172.21.15.148:0/1019579326 conn(0x7f40c80cd120 msgr2=0x7f40c80cd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.698+0000 7f40cf463640 1 -- 172.21.15.148:0/1019579326 shutdown_connections 2024-03-29T04:10:26.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.702+0000 7f40cf463640 1 -- 172.21.15.148:0/1019579326 wait complete. 2024-03-29T04:10:26.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.703+0000 7f40cf463640 1 Processor -- start 2024-03-29T04:10:26.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.704+0000 7f40cf463640 1 -- start start 2024-03-29T04:10:26.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.705+0000 7f40cf463640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40b40052e0 0x7f40c81528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.705+0000 7f40cf463640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40c80cef20 0x7f40c80cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.705+0000 7f40cf463640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40c8157f30 0x7f40c81583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.705+0000 7f40cf463640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f40c80cc9d0 con 0x7f40c80cef20 2024-03-29T04:10:26.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.705+0000 7f40cf463640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f40c80cc850 con 0x7f40c8157f30 2024-03-29T04:10:26.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.705+0000 7f40cf463640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f40c806ed40 con 0x7f40b40052e0 2024-03-29T04:10:26.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.706+0000 7f40cd1ff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40b40052e0 0x7f40c81528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.706+0000 7f40cc9fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40c80cef20 0x7f40c80cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.706+0000 7f40cc9fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40c80cef20 0x7f40c80cf3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48774/0 (socket says 172.21.15.148:48774) 2024-03-29T04:10:26.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.706+0000 7f40cda00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40c8157f30 0x7f40c81583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.706+0000 7f40cda00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40c8157f30 0x7f40c81583c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:39562/0 (socket says 172.21.15.148:39562) 2024-03-29T04:10:26.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.706+0000 7f40cc9fe640 1 -- 172.21.15.148:0/3418778694 learned_addr learned my addr 172.21.15.148:0/3418778694 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:26.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.707+0000 7f40cc9fe640 1 -- 172.21.15.148:0/3418778694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40b40052e0 msgr2=0x7f40c81528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.707+0000 7f40cc9fe640 1 --2- 172.21.15.148:0/3418778694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40b40052e0 0x7f40c81528c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.707+0000 7f40cc9fe640 1 -- 172.21.15.148:0/3418778694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40c8157f30 msgr2=0x7f40c81583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.707+0000 7f40cc9fe640 1 --2- 172.21.15.148:0/3418778694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40c8157f30 0x7f40c81583c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.707+0000 7f40cc9fe640 1 -- 172.21.15.148:0/3418778694 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c80cf9e0 con 0x7f40c80cef20 2024-03-29T04:10:26.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.708+0000 7f40cda00640 1 --2- 172.21.15.148:0/3418778694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40c8157f30 0x7f40c81583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:26.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.708+0000 7f40cc9fe640 1 --2- 172.21.15.148:0/3418778694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40c80cef20 0x7f40c80cf3a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f40c400c750 tx=0x7f40c400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.708+0000 7f40bdffb640 1 -- 172.21.15.148:0/3418778694 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c400ce60 con 0x7f40c80cef20 2024-03-29T04:10:26.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.708+0000 7f40cf463640 1 -- 172.21.15.148:0/3418778694 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40c80cfcc0 con 0x7f40c80cef20 2024-03-29T04:10:26.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.708+0000 7f40bdffb640 1 -- 172.21.15.148:0/3418778694 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40c4004230 con 0x7f40c80cef20 2024-03-29T04:10:26.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.708+0000 7f40bdffb640 1 -- 172.21.15.148:0/3418778694 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c4003610 con 0x7f40c80cef20 2024-03-29T04:10:26.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.712+0000 7f40cf463640 1 -- 172.21.15.148:0/3418778694 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40c815ce40 con 0x7f40c80cef20 2024-03-29T04:10:26.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.715+0000 7f40bdffb640 1 -- 172.21.15.148:0/3418778694 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f40c40037b0 con 0x7f40c80cef20 2024-03-29T04:10:26.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.719+0000 7f40bdffb640 1 --2- 172.21.15.148:0/3418778694 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f40ac06c730 0x7f40ac06ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.716+0000 7f40b37fe640 1 -- 172.21.15.148:0/3418778694 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40c815d040 con 0x7f40c80cef20 2024-03-29T04:10:26.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.719+0000 7f40bdffb640 1 -- 172.21.15.148:0/3418778694 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f40c40043e0 con 0x7f40c80cef20 2024-03-29T04:10:26.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.720+0000 7f40cd1ff640 1 --2- 172.21.15.148:0/3418778694 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f40ac06c730 0x7f40ac06ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.721+0000 7f40cd1ff640 1 --2- 172.21.15.148:0/3418778694 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f40ac06c730 0x7f40ac06ebf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f40c00070b0 tx=0x7f40c000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:26.729+0000 7f40bdffb640 1 -- 172.21.15.148:0/3418778694 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f40c815d040 con 0x7f40c80cef20 2024-03-29T04:10:27.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.028+0000 7f61fcd1c640 1 -- 172.21.15.148:0/2560232708 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f61f80d3030 con 0x7f61f815c370 2024-03-29T04:10:27.011 INFO:teuthology.orchestra.run.smithi148.stdout:55834574856 2024-03-29T04:10:27.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.030+0000 7f61e6ffd640 1 -- 172.21.15.148:0/2560232708 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f61e807b4a0 con 0x7f61f815c370 2024-03-29T04:10:27.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.042+0000 7fc941bb9640 1 -- 172.21.15.148:0/3510553119 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fc91c005850 con 0x7fc9340df180 2024-03-29T04:10:27.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.043+0000 7fc9397fa640 1 -- 172.21.15.148:0/3510553119 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc9340d1920 con 0x7fc9340df180 2024-03-29T04:10:27.022 INFO:teuthology.orchestra.run.smithi148.stdout:55834574857 2024-03-29T04:10:27.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.046+0000 7f61fcd1c640 1 -- 172.21.15.148:0/2560232708 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f61d806c530 msgr2=0x7f61d806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.046+0000 7f61fcd1c640 1 --2- 172.21.15.148:0/2560232708 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f61d806c530 0x7f61d806e9f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f61ec0011e0 tx=0x7f61ec009040 comp rx=0 tx=0).stop 2024-03-29T04:10:27.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.046+0000 7fc922ffd640 1 -- 172.21.15.148:0/3510553119 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc92806c560 msgr2=0x7fc92806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.046+0000 7fc922ffd640 1 --2- 172.21.15.148:0/3510553119 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc92806c560 0x7fc92806ea20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc924004660 tx=0x7fc93c06f030 comp rx=0 tx=0).stop 2024-03-29T04:10:27.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.046+0000 7fc922ffd640 1 -- 172.21.15.148:0/3510553119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9340df180 msgr2=0x7fc9340df630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.046+0000 7fc922ffd640 1 --2- 172.21.15.148:0/3510553119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9340df180 0x7fc9340df630 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc9340e7ce0 tx=0x7fc92c00ec00 comp rx=0 tx=0).stop 2024-03-29T04:10:27.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.046+0000 7f61fcd1c640 1 -- 172.21.15.148:0/2560232708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61f815c370 msgr2=0x7f61f8144400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.046+0000 7f61fcd1c640 1 --2- 172.21.15.148:0/2560232708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61f815c370 0x7f61f8144400 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f61e8033000 tx=0x7f61e80046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.047+0000 7f61fcd1c640 1 -- 172.21.15.148:0/2560232708 shutdown_connections 2024-03-29T04:10:27.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.046+0000 7fc922ffd640 1 -- 172.21.15.148:0/3510553119 shutdown_connections 2024-03-29T04:10:27.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.046+0000 7fc922ffd640 1 --2- 172.21.15.148:0/3510553119 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc92806c560 0x7fc92806ea20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.046+0000 7fc922ffd640 1 --2- 172.21.15.148:0/3510553119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9340df180 0x7fc9340df630 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.046+0000 7fc922ffd640 1 --2- 172.21.15.148:0/3510553119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9340e78f0 0x7fc93417dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.046+0000 7fc922ffd640 1 --2- 172.21.15.148:0/3510553119 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc924003b70 0x7fc9340d13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.046+0000 7fc922ffd640 1 -- 172.21.15.148:0/3510553119 >> 172.21.15.148:0/3510553119 conn(0x7fc9340559d0 msgr2=0x7fc934057370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:27.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.047+0000 7f61fcd1c640 1 --2- 172.21.15.148:0/2560232708 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f61d806c530 0x7f61d806e9f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.047+0000 7f61fcd1c640 1 --2- 172.21.15.148:0/2560232708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61f80ca9f0 0x7f61f80c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.047+0000 7f61fcd1c640 1 --2- 172.21.15.148:0/2560232708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61f815c370 0x7f61f8144400 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f61e8033000 tx=0x7f61e80046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.047+0000 7f61fcd1c640 1 --2- 172.21.15.148:0/2560232708 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61f80ce620 0x7f61f80c9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.047+0000 7f61fcd1c640 1 -- 172.21.15.148:0/2560232708 >> 172.21.15.148:0/2560232708 conn(0x7f61f80b3d50 msgr2=0x7f61f80b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:27.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.047+0000 7f61fcd1c640 1 -- 172.21.15.148:0/2560232708 shutdown_connections 2024-03-29T04:10:27.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.047+0000 7f61fcd1c640 1 -- 172.21.15.148:0/2560232708 wait complete. 2024-03-29T04:10:27.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.048+0000 7fc922ffd640 1 -- 172.21.15.148:0/3510553119 shutdown_connections 2024-03-29T04:10:27.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.048+0000 7fc922ffd640 1 -- 172.21.15.148:0/3510553119 wait complete. 2024-03-29T04:10:27.044 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.3 2024-03-29T04:10:27.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.069+0000 7effd09d3640 1 -- 172.21.15.148:0/1495560685 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7effc400f3b0 con 0x7effc40e2ab0 2024-03-29T04:10:27.050 INFO:teuthology.orchestra.run.smithi148.stdout:55834574857 2024-03-29T04:10:27.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.070+0000 7effb37fe640 1 -- 172.21.15.148:0/1495560685 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7effcc0f6970 con 0x7effc40e2ab0 2024-03-29T04:10:27.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.073+0000 7effd09d3640 1 -- 172.21.15.148:0/1495560685 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7effb806c580 msgr2=0x7effb806ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.073+0000 7effd09d3640 1 --2- 172.21.15.148:0/1495560685 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7effb806c580 0x7effb806ea40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7effbc001db0 tx=0x7effbc001d40 comp rx=0 tx=0).stop 2024-03-29T04:10:27.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.073+0000 7effd09d3640 1 -- 172.21.15.148:0/1495560685 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effc40e2ab0 msgr2=0x7effc40e2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.073+0000 7effd09d3640 1 --2- 172.21.15.148:0/1495560685 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effc40e2ab0 0x7effc40e2f40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7effcc069ef0 tx=0x7effcc06b350 comp rx=0 tx=0).stop 2024-03-29T04:10:27.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.073+0000 7effd09d3640 1 -- 172.21.15.148:0/1495560685 shutdown_connections 2024-03-29T04:10:27.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.073+0000 7effd09d3640 1 --2- 172.21.15.148:0/1495560685 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7effb806c580 0x7effb806ea40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.073+0000 7effd09d3640 1 --2- 172.21.15.148:0/1495560685 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7effc40e2ab0 0x7effc40e2f40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.073+0000 7effd09d3640 1 --2- 172.21.15.148:0/1495560685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effc40e42c0 0x7effc40dbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.073+0000 7effd09d3640 1 --2- 172.21.15.148:0/1495560685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effc40dfee0 0x7effc400c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.074+0000 7effd09d3640 1 -- 172.21.15.148:0/1495560685 >> 172.21.15.148:0/1495560685 conn(0x7effc401ef30 msgr2=0x7effc40dee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:27.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.074+0000 7fed57243640 1 -- 172.21.15.148:0/3972710765 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fed38005850 con 0x7fed48057af0 2024-03-29T04:10:27.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.074+0000 7effd09d3640 1 -- 172.21.15.148:0/1495560685 shutdown_connections 2024-03-29T04:10:27.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.074+0000 7effd09d3640 1 -- 172.21.15.148:0/1495560685 wait complete. 2024-03-29T04:10:27.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.074+0000 7fed46ffd640 1 -- 172.21.15.148:0/3972710765 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fed500d4a20 con 0x7fed48057af0 2024-03-29T04:10:27.055 INFO:teuthology.orchestra.run.smithi148.stdout:55834574857 2024-03-29T04:10:27.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.078+0000 7fed44ff9640 1 -- 172.21.15.148:0/3972710765 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fed3406c580 msgr2=0x7fed3406ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.078+0000 7fed44ff9640 1 --2- 172.21.15.148:0/3972710765 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fed3406c580 0x7fed3406ea40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fed40036000 tx=0x7fed40008030 comp rx=0 tx=0).stop 2024-03-29T04:10:27.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.078+0000 7fed44ff9640 1 -- 172.21.15.148:0/3972710765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed48057af0 msgr2=0x7fed48009de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.078+0000 7fed44ff9640 1 --2- 172.21.15.148:0/3972710765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed48057af0 0x7fed48009de0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fed5006a3b0 tx=0x7fed500756f0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.078+0000 7fed44ff9640 1 -- 172.21.15.148:0/3972710765 shutdown_connections 2024-03-29T04:10:27.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.078+0000 7fed44ff9640 1 --2- 172.21.15.148:0/3972710765 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fed3406c580 0x7fed3406ea40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.078+0000 7fed44ff9640 1 --2- 172.21.15.148:0/3972710765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed480e6e10 0x7fed480da2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.078+0000 7fed44ff9640 1 --2- 172.21.15.148:0/3972710765 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed480e33e0 0x7fed4800c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.078+0000 7fed44ff9640 1 --2- 172.21.15.148:0/3972710765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed48057af0 0x7fed48009de0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.078+0000 7fed44ff9640 1 -- 172.21.15.148:0/3972710765 >> 172.21.15.148:0/3972710765 conn(0x7fed4803aa20 msgr2=0x7fed480399a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:27.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.079+0000 7fed44ff9640 1 -- 172.21.15.148:0/3972710765 shutdown_connections 2024-03-29T04:10:27.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.079+0000 7fed44ff9640 1 -- 172.21.15.148:0/3972710765 wait complete. 2024-03-29T04:10:27.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.082+0000 7f1fbd4c5640 1 -- 172.21.15.148:0/2978263887 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f1fb80d1470 con 0x7f1fb8153b70 2024-03-29T04:10:27.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.083+0000 7f1f9f7fe640 1 -- 172.21.15.148:0/2978263887 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1fa805a5c0 con 0x7f1fb8153b70 2024-03-29T04:10:27.062 INFO:teuthology.orchestra.run.smithi148.stdout:55834574856 2024-03-29T04:10:27.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.087+0000 7f1fbd4c5640 1 -- 172.21.15.148:0/2978263887 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1fa406c630 msgr2=0x7f1fa406eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.087+0000 7f1fbd4c5640 1 --2- 172.21.15.148:0/2978263887 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1fa406c630 0x7f1fa406eaf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00c430 tx=0x7f1fac039040 comp rx=0 tx=0).stop 2024-03-29T04:10:27.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.087+0000 7f1fbd4c5640 1 -- 172.21.15.148:0/2978263887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fb8153b70 msgr2=0x7f1fb8158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.087+0000 7f1fbd4c5640 1 --2- 172.21.15.148:0/2978263887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fb8153b70 0x7f1fb8158e40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1fa80108c0 tx=0x7f1fa8010cc0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.087+0000 7f1fbd4c5640 1 -- 172.21.15.148:0/2978263887 shutdown_connections 2024-03-29T04:10:27.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.087+0000 7f1fbd4c5640 1 --2- 172.21.15.148:0/2978263887 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1fa406c630 0x7f1fa406eaf0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.087+0000 7f1fbd4c5640 1 --2- 172.21.15.148:0/2978263887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fb8159380 0x7f1fb8154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.087+0000 7f1fbd4c5640 1 --2- 172.21.15.148:0/2978263887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fb8153b70 0x7f1fb8158e40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.087+0000 7f1fbd4c5640 1 --2- 172.21.15.148:0/2978263887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fa0003d50 0x7f1fb81534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.087+0000 7f1fbd4c5640 1 -- 172.21.15.148:0/2978263887 >> 172.21.15.148:0/2978263887 conn(0x7f1fb80b2b50 msgr2=0x7f1fb80b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:27.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.088+0000 7f1fbd4c5640 1 -- 172.21.15.148:0/2978263887 shutdown_connections 2024-03-29T04:10:27.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.088+0000 7f1fbd4c5640 1 -- 172.21.15.148:0/2978263887 wait complete. 2024-03-29T04:10:27.070 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.4 2024-03-29T04:10:27.072 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.5 2024-03-29T04:10:27.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.097+0000 7fc44affd640 1 -- 172.21.15.148:0/3176080785 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fc4600ce7b0 con 0x7fc460072520 2024-03-29T04:10:27.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.098+0000 7fc45d7fa640 1 -- 172.21.15.148:0/3176080785 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc45406a6f0 con 0x7fc460072520 2024-03-29T04:10:27.077 INFO:teuthology.orchestra.run.smithi148.stdout:55834574856 2024-03-29T04:10:27.078 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.1 2024-03-29T04:10:27.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.100+0000 7fc44affd640 1 -- 172.21.15.148:0/3176080785 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc44406c630 msgr2=0x7fc44406eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.100+0000 7fc44affd640 1 --2- 172.21.15.148:0/3176080785 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc44406c630 0x7fc44406eaf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc44c036000 tx=0x7fc44c008030 comp rx=0 tx=0).stop 2024-03-29T04:10:27.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.100+0000 7fc44affd640 1 -- 172.21.15.148:0/3176080785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc460072520 msgr2=0x7fc4601457b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.100+0000 7fc44affd640 1 --2- 172.21.15.148:0/3176080785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc460072520 0x7fc4601457b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc45400d920 tx=0x7fc45400dde0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.100+0000 7fc44affd640 1 -- 172.21.15.148:0/3176080785 shutdown_connections 2024-03-29T04:10:27.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.100+0000 7fc44affd640 1 --2- 172.21.15.148:0/3176080785 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc44406c630 0x7fc44406eaf0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.100+0000 7fc44affd640 1 --2- 172.21.15.148:0/3176080785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc460156bf0 0x7fc4601570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.100+0000 7fc44affd640 1 --2- 172.21.15.148:0/3176080785 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc460161260 0x7fc460147ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.100+0000 7fc44affd640 1 --2- 172.21.15.148:0/3176080785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc460072520 0x7fc4601457b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.100+0000 7fc44affd640 1 -- 172.21.15.148:0/3176080785 >> 172.21.15.148:0/3176080785 conn(0x7fc4600d0f50 msgr2=0x7fc46015dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:27.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.101+0000 7fc44affd640 1 -- 172.21.15.148:0/3176080785 shutdown_connections 2024-03-29T04:10:27.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.101+0000 7fc44affd640 1 -- 172.21.15.148:0/3176080785 wait complete. 2024-03-29T04:10:27.082 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.2 2024-03-29T04:10:27.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.115+0000 7f40b37fe640 1 -- 172.21.15.148:0/3418778694 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f40c80687e0 con 0x7f40c80cef20 2024-03-29T04:10:27.095 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.6 2024-03-29T04:10:27.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.116+0000 7f40bdffb640 1 -- 172.21.15.148:0/3418778694 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f40c815d040 con 0x7f40c80cef20 2024-03-29T04:10:27.095 INFO:teuthology.orchestra.run.smithi148.stdout:55834574857 2024-03-29T04:10:27.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.118+0000 7f40b37fe640 1 -- 172.21.15.148:0/3418778694 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f40ac06c730 msgr2=0x7f40ac06ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.118+0000 7f40b37fe640 1 --2- 172.21.15.148:0/3418778694 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f40ac06c730 0x7f40ac06ebf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f40c00070b0 tx=0x7f40c000b040 comp rx=0 tx=0).stop 2024-03-29T04:10:27.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.118+0000 7f40b37fe640 1 -- 172.21.15.148:0/3418778694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40c80cef20 msgr2=0x7f40c80cf3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.118+0000 7f40b37fe640 1 --2- 172.21.15.148:0/3418778694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40c80cef20 0x7f40c80cf3a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f40c400c750 tx=0x7f40c400cc10 comp rx=0 tx=0).stop 2024-03-29T04:10:27.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.118+0000 7f40b37fe640 1 -- 172.21.15.148:0/3418778694 shutdown_connections 2024-03-29T04:10:27.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.118+0000 7f40b37fe640 1 --2- 172.21.15.148:0/3418778694 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f40ac06c730 0x7f40ac06ebf0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.118+0000 7f40b37fe640 1 --2- 172.21.15.148:0/3418778694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40c8157f30 0x7f40c81583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.118+0000 7f40b37fe640 1 --2- 172.21.15.148:0/3418778694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40c80cef20 0x7f40c80cf3a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.118+0000 7f40b37fe640 1 --2- 172.21.15.148:0/3418778694 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40b40052e0 0x7f40c81528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.118+0000 7f40b37fe640 1 -- 172.21.15.148:0/3418778694 >> 172.21.15.148:0/3418778694 conn(0x7f40c80cd120 msgr2=0x7f40c80b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:27.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.118+0000 7f40b37fe640 1 -- 172.21.15.148:0/3418778694 shutdown_connections 2024-03-29T04:10:27.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.118+0000 7f40b37fe640 1 -- 172.21.15.148:0/3418778694 wait complete. 2024-03-29T04:10:27.108 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.0 2024-03-29T04:10:27.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.226+0000 7fe3c0b7c640 1 -- 172.21.15.148:0/3165190578 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fe3bc155110 con 0x7fe3bc0b7a20 2024-03-29T04:10:27.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.226+0000 7fe3aaffd640 1 -- 172.21.15.148:0/3165190578 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe3ac07f490 con 0x7fe3bc0b7a20 2024-03-29T04:10:27.205 INFO:teuthology.orchestra.run.smithi148.stdout:55834574856 2024-03-29T04:10:27.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.228+0000 7fe3c0b7c640 1 -- 172.21.15.148:0/3165190578 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe39c06c600 msgr2=0x7fe39c06eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.228+0000 7fe3c0b7c640 1 --2- 172.21.15.148:0/3165190578 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe39c06c600 0x7fe39c06eac0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0006fd0 tx=0x7fe3b001b210 comp rx=0 tx=0).stop 2024-03-29T04:10:27.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.228+0000 7fe3c0b7c640 1 -- 172.21.15.148:0/3165190578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3bc0b7a20 msgr2=0x7fe3bc13ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.228+0000 7fe3c0b7c640 1 --2- 172.21.15.148:0/3165190578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3bc0b7a20 0x7fe3bc13ef00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00c350 tx=0x7fe3ac001990 comp rx=0 tx=0).stop 2024-03-29T04:10:27.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.228+0000 7fe3c0b7c640 1 -- 172.21.15.148:0/3165190578 shutdown_connections 2024-03-29T04:10:27.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.228+0000 7fe3c0b7c640 1 --2- 172.21.15.148:0/3165190578 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe39c06c600 0x7fe39c06eac0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.228+0000 7fe3c0b7c640 1 --2- 172.21.15.148:0/3165190578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3bc0d0f20 0x7fe3bc0d13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.228+0000 7fe3c0b7c640 1 --2- 172.21.15.148:0/3165190578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3bc0b7a20 0x7fe3bc13ef00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.228+0000 7fe3c0b7c640 1 --2- 172.21.15.148:0/3165190578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3a4004800 0x7fe3bc157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.228+0000 7fe3c0b7c640 1 -- 172.21.15.148:0/3165190578 >> 172.21.15.148:0/3165190578 conn(0x7fe3bc0b3c00 msgr2=0x7fe3bc15d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:27.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.228+0000 7fe3c0b7c640 1 -- 172.21.15.148:0/3165190578 shutdown_connections 2024-03-29T04:10:27.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:27.228+0000 7fe3c0b7c640 1 -- 172.21.15.148:0/3165190578 wait complete. 2024-03-29T04:10:27.219 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.7 2024-03-29T04:10:28.046 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-29T04:10:28.071 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-29T04:10:28.073 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-29T04:10:28.078 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-29T04:10:28.083 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-29T04:10:28.096 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-29T04:10:28.109 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-29T04:10:28.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.208+0000 7f90a0315640 1 Processor -- start 2024-03-29T04:10:28.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.214+0000 7f90a0315640 1 -- start start 2024-03-29T04:10:28.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.216+0000 7f90a0315640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f90980b3170 0x7f90980cc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.216+0000 7f90a0315640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9098162920 0x7f909806d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.216+0000 7f90a0315640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9098071600 con 0x7f9098162540 2024-03-29T04:10:28.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.216+0000 7f90a0315640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f90980718e0 con 0x7f9098162920 2024-03-29T04:10:28.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.216+0000 7f90a0315640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9098071bc0 con 0x7f90980b3170 2024-03-29T04:10:28.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.217+0000 7f909d8b0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f90980b3170 0x7f90980cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.217+0000 7f909e8b2640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9098162920 0x7f909806d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.217+0000 7f909e8b2640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9098162920 0x7f909806d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48780/0 (socket says 172.21.15.148:48780) 2024-03-29T04:10:28.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.217+0000 7f909e8b2640 1 -- 172.21.15.148:0/1346246500 learned_addr learned my addr 172.21.15.148:0/1346246500 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:28.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.218+0000 7f909e8b2640 1 -- 172.21.15.148:0/1346246500 >> v2:172.21.15.157:3300/0 conn(0x7f90980b3170 msgr2=0x7f90980cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:28.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.218+0000 7f909e8b2640 1 --2- 172.21.15.148:0/1346246500 >> v2:172.21.15.157:3300/0 conn(0x7f90980b3170 0x7f90980cc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.218+0000 7f909e8b2640 1 -- 172.21.15.148:0/1346246500 >> v1:172.21.15.148:6789/0 conn(0x7f9098162540 legacy=0x7f90980b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.218+0000 7f909e8b2640 1 -- 172.21.15.148:0/1346246500 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9098071ea0 con 0x7f9098162920 2024-03-29T04:10:28.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.219+0000 7f909e8b2640 1 --2- 172.21.15.148:0/1346246500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9098162920 0x7f909806d7d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f90980710e0 tx=0x7f909002c1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0788328cdf6f91f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:28.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.220+0000 7f909d0af640 1 -- 172.21.15.148:0/1346246500 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909002cb90 con 0x7f9098162920 2024-03-29T04:10:28.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.220+0000 7f909d0af640 1 -- 172.21.15.148:0/1346246500 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f909002cd30 con 0x7f9098162920 2024-03-29T04:10:28.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.220+0000 7f909d0af640 1 -- 172.21.15.148:0/1346246500 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9090034610 con 0x7f9098162920 2024-03-29T04:10:28.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.221+0000 7f90a0315640 1 -- 172.21.15.148:0/1346246500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9098162920 msgr2=0x7f909806d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.221+0000 7f90a0315640 1 --2- 172.21.15.148:0/1346246500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9098162920 0x7f909806d7d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f90980710e0 tx=0x7f909002c1f0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.222+0000 7f90a0315640 1 -- 172.21.15.148:0/1346246500 shutdown_connections 2024-03-29T04:10:28.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.222+0000 7f90a0315640 1 --2- 172.21.15.148:0/1346246500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9098162920 0x7f909806d7d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.222+0000 7f90a0315640 1 --2- 172.21.15.148:0/1346246500 >> v2:172.21.15.157:3300/0 conn(0x7f90980b3170 0x7f90980cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.222+0000 7f90a0315640 1 -- 172.21.15.148:0/1346246500 >> 172.21.15.148:0/1346246500 conn(0x7f909806b5b0 msgr2=0x7f909806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:28.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.222+0000 7f90a0315640 1 -- 172.21.15.148:0/1346246500 shutdown_connections 2024-03-29T04:10:28.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.222+0000 7f90a0315640 1 -- 172.21.15.148:0/1346246500 wait complete. 2024-03-29T04:10:28.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.224+0000 7f90a0315640 1 Processor -- start 2024-03-29T04:10:28.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.224+0000 7f90a0315640 1 -- start start 2024-03-29T04:10:28.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.226+0000 7f90a0315640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90980b3170 0x7f90980aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.226+0000 7f90a0315640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9098162540 0x7f9098165350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.226+0000 7f90a0315640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9098162920 0x7f909815f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.226+0000 7f909d8b0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9098162540 0x7f9098165350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.226+0000 7f909e0b1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90980b3170 0x7f90980aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.226+0000 7f909d8b0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9098162540 0x7f9098165350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:39578/0 (socket says 172.21.15.148:39578) 2024-03-29T04:10:28.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.226+0000 7f909e0b1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90980b3170 0x7f90980aaab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.148:44574/0 (socket says 172.21.15.148:44574) 2024-03-29T04:10:28.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.226+0000 7f909e8b2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9098162920 0x7f909815f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.226+0000 7f909e8b2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9098162920 0x7f909815f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48786/0 (socket says 172.21.15.148:48786) 2024-03-29T04:10:28.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.226+0000 7f909d8b0640 1 -- 172.21.15.148:0/1783233009 learned_addr learned my addr 172.21.15.148:0/1783233009 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:28.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.226+0000 7f90a0315640 1 -- 172.21.15.148:0/1783233009 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f909806de90 con 0x7f9098162920 2024-03-29T04:10:28.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.226+0000 7f90a0315640 1 -- 172.21.15.148:0/1783233009 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f909806dd10 con 0x7f9098162540 2024-03-29T04:10:28.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.226+0000 7f90a0315640 1 -- 172.21.15.148:0/1783233009 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f909806e010 con 0x7f90980b3170 2024-03-29T04:10:28.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.227+0000 7f909e0b1640 1 -- 172.21.15.148:0/1783233009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9098162540 msgr2=0x7f9098165350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.227+0000 7f909e0b1640 1 --2- 172.21.15.148:0/1783233009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9098162540 0x7f9098165350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.227+0000 7f909e0b1640 1 -- 172.21.15.148:0/1783233009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9098162920 msgr2=0x7f909815f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.227+0000 7f909e0b1640 1 --2- 172.21.15.148:0/1783233009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9098162920 0x7f909815f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.227+0000 7f909e0b1640 1 -- 172.21.15.148:0/1783233009 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9098165aa0 con 0x7f90980b3170 2024-03-29T04:10:28.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.227+0000 7f909e8b2640 1 --2- 172.21.15.148:0/1783233009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9098162920 0x7f909815f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:28.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.228+0000 7f909e0b1640 1 --2- 172.21.15.148:0/1783233009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90980b3170 0x7f90980aaab0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f909400c340 tx=0x7f909400c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:28.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.228+0000 7f908effd640 1 -- 172.21.15.148:0/1783233009 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90940052d0 con 0x7f90980b3170 2024-03-29T04:10:28.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.228+0000 7f908effd640 1 -- 172.21.15.148:0/1783233009 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9094005490 con 0x7f90980b3170 2024-03-29T04:10:28.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.229+0000 7f90a0315640 1 -- 172.21.15.148:0/1783233009 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9098165d80 con 0x7f90980b3170 2024-03-29T04:10:28.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.229+0000 7f908effd640 1 -- 172.21.15.148:0/1783233009 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9094007260 con 0x7f90980b3170 2024-03-29T04:10:28.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.229+0000 7f90a0315640 1 -- 172.21.15.148:0/1783233009 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9098166340 con 0x7f90980b3170 2024-03-29T04:10:28.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.231+0000 7f90a0315640 1 -- 172.21.15.148:0/1783233009 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9080005290 con 0x7f90980b3170 2024-03-29T04:10:28.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.232+0000 7f908effd640 1 -- 172.21.15.148:0/1783233009 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9094007400 con 0x7f90980b3170 2024-03-29T04:10:28.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.235+0000 7f908effd640 1 --2- 172.21.15.148:0/1783233009 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f908406c670 0x7f908406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.236+0000 7f909d8b0640 1 --2- 172.21.15.148:0/1783233009 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f908406c670 0x7f908406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.237+0000 7f908effd640 1 -- 172.21.15.148:0/1783233009 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f909408ae70 con 0x7f90980b3170 2024-03-29T04:10:28.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.237+0000 7f908effd640 1 -- 172.21.15.148:0/1783233009 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9094058a60 con 0x7f90980b3170 2024-03-29T04:10:28.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.238+0000 7f909d8b0640 1 --2- 172.21.15.148:0/1783233009 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f908406c670 0x7f908406eb30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f90982043a0 tx=0x7f9088009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:28.219 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-29T04:10:28.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.265+0000 7f6ca209a640 1 Processor -- start 2024-03-29T04:10:28.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.265+0000 7f6ca209a640 1 -- start start 2024-03-29T04:10:28.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.266+0000 7f6ca209a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c9c0b34b0 0x7f6c9c16d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.266+0000 7f6ca209a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c9c0b1d00 con 0x7f6c9c06da00 2024-03-29T04:10:28.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.266+0000 7f6ca209a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c9c0b1fe0 con 0x7f6c9c1628d0 2024-03-29T04:10:28.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.266+0000 7f6ca209a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6c9c162cb0 con 0x7f6c9c0b34b0 2024-03-29T04:10:28.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.267+0000 7f6c9bfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c9c0b34b0 0x7f6c9c16d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.267+0000 7f6c9bfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c9c0b34b0 0x7f6c9c16d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.148:44590/0 (socket says 172.21.15.148:44590) 2024-03-29T04:10:28.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.268+0000 7f6c9bfff640 1 -- 172.21.15.148:0/63671873 learned_addr learned my addr 172.21.15.148:0/63671873 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:28.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.269+0000 7f6c9a7fc640 1 -- 172.21.15.148:0/63671873 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1932262745 0 0) 0x7f6c9c0b1fe0 con 0x7f6c9c1628d0 2024-03-29T04:10:28.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.270+0000 7f6c9a7fc640 1 -- 172.21.15.148:0/63671873 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c84003520 con 0x7f6c9c1628d0 2024-03-29T04:10:28.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.270+0000 7f6c9a7fc640 1 -- 172.21.15.148:0/63671873 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1702435679 0 0) 0x7f6c9c0b1d00 con 0x7f6c9c06da00 2024-03-29T04:10:28.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.270+0000 7f6c9a7fc640 1 -- 172.21.15.148:0/63671873 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c9c0b1fe0 con 0x7f6c9c06da00 2024-03-29T04:10:28.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.271+0000 7f6c9a7fc640 1 -- 172.21.15.148:0/63671873 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1297317182 0 0) 0x7f6c84003520 con 0x7f6c9c1628d0 2024-03-29T04:10:28.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.271+0000 7f6c9a7fc640 1 -- 172.21.15.148:0/63671873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c9c0b34b0 msgr2=0x7f6c9c16d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:28.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.271+0000 7f6c9a7fc640 1 --2- 172.21.15.148:0/63671873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c9c0b34b0 0x7f6c9c16d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.271+0000 7f6c9a7fc640 1 -- 172.21.15.148:0/63671873 >> v1:172.21.15.148:6789/0 conn(0x7f6c9c06da00 legacy=0x7f6c9c0b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.272+0000 7f6c9a7fc640 1 -- 172.21.15.148:0/63671873 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c9c0b22c0 con 0x7f6c9c1628d0 2024-03-29T04:10:28.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.272+0000 7f6c9a7fc640 1 -- 172.21.15.148:0/63671873 <== mon.1 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f6c8c0019f0 con 0x7f6c9c1628d0 2024-03-29T04:10:28.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.272+0000 7f6c9a7fc640 1 -- 172.21.15.148:0/63671873 >> v1:172.21.15.157:6789/0 conn(0x7f6c9c1628d0 legacy=0x7f6c9c0b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.273+0000 7f6c9a7fc640 1 --2- 172.21.15.148:0/63671873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c84003b40 0x7f6c84003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.273+0000 7f6c9a7fc640 1 --2- 172.21.15.148:0/63671873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c84004800 0x7f6c84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.273+0000 7f6c9a7fc640 1 --2- 172.21.15.148:0/63671873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c840051c0 0x7f6c8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.274+0000 7f6c9a7fc640 1 -- 172.21.15.148:0/63671873 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c162cb0 con 0x7f6c840051c0 2024-03-29T04:10:28.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.274+0000 7f6c9b7fe640 1 --2- 172.21.15.148:0/63671873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c84003b40 0x7f6c84003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.274+0000 7f6c9a7fc640 1 -- 172.21.15.148:0/63671873 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8400c090 con 0x7f6c84003b40 2024-03-29T04:10:28.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.274+0000 7f6c9a7fc640 1 -- 172.21.15.148:0/63671873 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6c8400c290 con 0x7f6c84004800 2024-03-29T04:10:28.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.274+0000 7f6c9bfff640 1 --2- 172.21.15.148:0/63671873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c840051c0 0x7f6c8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.275+0000 7f6c9b7fe640 1 -- 172.21.15.148:0/63671873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c84004800 msgr2=0x7f6c84004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-29T04:10:28.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.275+0000 7f6c9b7fe640 1 --2- 172.21.15.148:0/63671873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c84004800 0x7f6c84004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.275+0000 7f6c9b7fe640 1 -- 172.21.15.148:0/63671873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c840051c0 msgr2=0x7f6c8400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:28.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.275+0000 7f6c9b7fe640 1 --2- 172.21.15.148:0/63671873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c840051c0 0x7f6c8400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.275+0000 7f6c9b7fe640 1 -- 172.21.15.148:0/63671873 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c84003520 con 0x7f6c84003b40 2024-03-29T04:10:28.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.276+0000 7f6c9bfff640 1 --2- 172.21.15.148:0/63671873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c840051c0 0x7f6c8400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:28.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.276+0000 7f8ba0bdf640 1 Processor -- start 2024-03-29T04:10:28.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.276+0000 7f8ba0bdf640 1 -- start start 2024-03-29T04:10:28.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.277+0000 7f75e22ce640 1 Processor -- start 2024-03-29T04:10:28.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.277+0000 7f75e22ce640 1 -- start start 2024-03-29T04:10:28.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.277+0000 7f8ba0bdf640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8b9400c0a0 0x7f8b94038080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.277+0000 7f6c9b7fe640 1 --2- 172.21.15.148:0/63671873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c84003b40 0x7f6c84003f40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00c4c0 tx=0x7f6c84005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a2f97740c42d21a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:28.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.278+0000 7f8ba0bdf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b94053c70 0x7f8b940df220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.278+0000 7f8ba0bdf640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b940df760 con 0x7f8b94033ef0 2024-03-29T04:10:28.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.278+0000 7f8b9bfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8b94033ef0 0x7f8b9404fa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47324/0 (socket says 172.21.15.148:47324) 2024-03-29T04:10:28.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.278+0000 7f8b9bfff640 1 -- 172.21.15.148:0/1428375930 learned_addr learned my addr 172.21.15.148:0/1428375930 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:28.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.278+0000 7f75e22ce640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f75dc163900 0x7f75dc166cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.278+0000 7f8ba0bdf640 1 -- 172.21.15.148:0/1428375930 --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f8b940544c0 con 0x7f8b9400c0a0 2024-03-29T04:10:28.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.278+0000 7f8ba0bdf640 1 -- 172.21.15.148:0/1428375930 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8b940541e0 con 0x7f8b94053c70 2024-03-29T04:10:28.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f75e22ce640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75dc06c5c0 0x7f75dc06ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f75e22ce640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75dc0b36d0 con 0x7f75dc163520 2024-03-29T04:10:28.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f75e22ce640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f75dc06cf70 con 0x7f75dc163900 2024-03-29T04:10:28.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f75e22ce640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f75dc159cd0 con 0x7f75dc06c5c0 2024-03-29T04:10:28.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f8b9a7fc640 1 -- 172.21.15.148:0/1428375930 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3431574946 0 0) 0x7f8b940df760 con 0x7f8b94033ef0 2024-03-29T04:10:28.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f8b9b7fe640 1 --2- 172.21.15.148:0/1428375930 >> v2:172.21.15.148:3300/0 conn(0x7f8b9400c0a0 0x7f8b94038080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f8b9a7fc640 1 -- 172.21.15.148:0/1428375930 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b8c003520 con 0x7f8b94033ef0 2024-03-29T04:10:28.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f75dbfff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f75dc163520 0x7f75dc0b2ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.148:38794/0 (socket says 172.21.15.148:38794) 2024-03-29T04:10:28.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f75dbfff640 1 -- 172.21.15.148:0/1176157735 learned_addr learned my addr 172.21.15.148:0/1176157735 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:28.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f75e086b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75dc06c5c0 0x7f75dc06ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f75db7fe640 1 --2- 172.21.15.148:0/1176157735 >> v2:172.21.15.157:3300/0 conn(0x7f75dc163900 0x7f75dc166cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f8b9a7fc640 1 -- 172.21.15.148:0/1428375930 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 423026449 0 0) 0x7f8b8c003520 con 0x7f8b94033ef0 2024-03-29T04:10:28.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f8b9a7fc640 1 -- 172.21.15.148:0/1428375930 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b94053c70 msgr2=0x7f8b940df220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:28.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f8b9affd640 1 --2- 172.21.15.148:0/1428375930 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b94053c70 0x7f8b940df220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f8b9a7fc640 1 --2- 172.21.15.148:0/1428375930 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b94053c70 0x7f8b940df220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f8b9a7fc640 1 -- 172.21.15.148:0/1428375930 >> v2:172.21.15.148:3300/0 conn(0x7f8b9400c0a0 msgr2=0x7f8b94038080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:28.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f8b9a7fc640 1 --2- 172.21.15.148:0/1428375930 >> v2:172.21.15.148:3300/0 conn(0x7f8b9400c0a0 0x7f8b94038080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.280+0000 7f6c9a7fc640 1 -- 172.21.15.148:0/63671873 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c039070 con 0x7f6c84003b40 2024-03-29T04:10:28.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.280+0000 7f6c9a7fc640 1 -- 172.21.15.148:0/63671873 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c8c02cc80 con 0x7f6c84003b40 2024-03-29T04:10:28.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f8b9a7fc640 1 -- 172.21.15.148:0/1428375930 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b94005f30 con 0x7f8b94033ef0 2024-03-29T04:10:28.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f75db7fe640 1 -- 172.21.15.148:0/1176157735 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75dc06c5c0 msgr2=0x7f75dc06ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:28.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f75db7fe640 1 --2- 172.21.15.148:0/1176157735 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75dc06c5c0 0x7f75dc06ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f75db7fe640 1 -- 172.21.15.148:0/1176157735 >> v1:172.21.15.157:6789/0 conn(0x7f75dc163520 legacy=0x7f75dc0b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f75db7fe640 1 -- 172.21.15.148:0/1176157735 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75dc0b39b0 con 0x7f75dc163900 2024-03-29T04:10:28.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f8b9a7fc640 1 -- 172.21.15.148:0/1428375930 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f8b9c050140 con 0x7f8b94033ef0 2024-03-29T04:10:28.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f8b9a7fc640 1 -- 172.21.15.148:0/1428375930 >> v1:172.21.15.148:6789/0 conn(0x7f8b94033ef0 legacy=0x7f8b9404fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.281+0000 7f6ca209a640 1 -- 172.21.15.148:0/63671873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c84003b40 msgr2=0x7f6c84003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.281+0000 7f6ca209a640 1 --2- 172.21.15.148:0/63671873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c84003b40 0x7f6c84003f40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00c4c0 tx=0x7f6c84005830 comp rx=0 tx=0).stop 2024-03-29T04:10:28.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.281+0000 7f6c9bfff640 1 -- 172.21.15.148:0/63671873 reap_dead start 2024-03-29T04:10:28.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.281+0000 7f75db7fe640 1 --2- 172.21.15.148:0/1176157735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75dc163900 0x7f75dc166cc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f75d00097a0 tx=0x7f75d002b660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c34f5c76115d968 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:28.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f8b9a7fc640 1 --2- 172.21.15.148:0/1428375930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b8c0039b0 0x7f8b8c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f8b9a7fc640 1 --2- 172.21.15.148:0/1428375930 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b8c004350 0x7f8b8c004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f8b9a7fc640 1 --2- 172.21.15.148:0/1428375930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b8c005020 0x7f8b8c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f8b9a7fc640 1 -- 172.21.15.148:0/1428375930 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8b940544c0 con 0x7f8b8c005020 2024-03-29T04:10:28.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.279+0000 7f8b9a7fc640 1 -- 172.21.15.148:0/1428375930 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8b940541e0 con 0x7f8b8c0039b0 2024-03-29T04:10:28.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.280+0000 7f8b9a7fc640 1 -- 172.21.15.148:0/1428375930 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8b8c00bfb0 con 0x7f8b8c004350 2024-03-29T04:10:28.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.280+0000 7f8b9bfff640 1 --2- 172.21.15.148:0/1428375930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b8c005020 0x7f8b8c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.280+0000 7f8b9bfff640 1 -- 172.21.15.148:0/1428375930 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b8c004350 msgr2=0x7f8b8c004820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:10:28.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.280+0000 7f8b9bfff640 1 --2- 172.21.15.148:0/1428375930 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b8c004350 0x7f8b8c004820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.280+0000 7f8b9bfff640 1 -- 172.21.15.148:0/1428375930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b8c0039b0 msgr2=0x7f8b8c003dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:10:28.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.280+0000 7f8b9bfff640 1 --2- 172.21.15.148:0/1428375930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b8c0039b0 0x7f8b8c003dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.280+0000 7f8b9bfff640 1 -- 172.21.15.148:0/1428375930 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b94005f30 con 0x7f8b8c005020 2024-03-29T04:10:28.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.281+0000 7f8b9affd640 1 --2- 172.21.15.148:0/1428375930 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b94053c70 0x7f8b940df220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:28.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.281+0000 7f6ca209a640 1 -- 172.21.15.148:0/63671873 shutdown_connections 2024-03-29T04:10:28.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.281+0000 7f6ca209a640 1 -- 172.21.15.148:0/63671873 >> 172.21.15.148:0/63671873 conn(0x7f6c9c0ceb80 msgr2=0x7f6c9c0cef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:28.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.281+0000 7f75daffd640 1 -- 172.21.15.148:0/1176157735 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0037070 con 0x7f75dc163900 2024-03-29T04:10:28.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.281+0000 7f75daffd640 1 -- 172.21.15.148:0/1176157735 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75d0030710 con 0x7f75dc163900 2024-03-29T04:10:28.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.281+0000 7f8b9bfff640 1 --2- 172.21.15.148:0/1428375930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b8c005020 0x7f8b8c00b9c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c069cd0 tx=0x7f8b8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944c51c36be910ac server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:28.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.282+0000 7f8b9a7fc640 1 -- 172.21.15.148:0/1428375930 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c0ad070 con 0x7f8b8c005020 2024-03-29T04:10:28.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.282+0000 7f8b9a7fc640 1 -- 172.21.15.148:0/1428375930 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b9c06d070 con 0x7f8b8c005020 2024-03-29T04:10:28.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.282+0000 7f75e22ce640 1 -- 172.21.15.148:0/1176157735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75dc163900 msgr2=0x7f75dc166cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.282+0000 7f6ca209a640 1 -- 172.21.15.148:0/63671873 shutdown_connections 2024-03-29T04:10:28.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.282+0000 7f75e22ce640 1 --2- 172.21.15.148:0/1176157735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75dc163900 0x7f75dc166cc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f75d00097a0 tx=0x7f75d002b660 comp rx=0 tx=0).stop 2024-03-29T04:10:28.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.282+0000 7f6ca209a640 1 -- 172.21.15.148:0/63671873 wait complete. 2024-03-29T04:10:28.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.282+0000 7f8b9a7fc640 1 -- 172.21.15.148:0/1428375930 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c068cd0 con 0x7f8b8c005020 2024-03-29T04:10:28.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.283+0000 7f8ba0bdf640 1 -- 172.21.15.148:0/1428375930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b8c005020 msgr2=0x7f8b8c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.283+0000 7f8ba0bdf640 1 --2- 172.21.15.148:0/1428375930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b8c005020 0x7f8b8c00b9c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c069cd0 tx=0x7f8b8c005830 comp rx=0 tx=0).stop 2024-03-29T04:10:28.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.283+0000 7f8b9bfff640 1 -- 172.21.15.148:0/1428375930 reap_dead start 2024-03-29T04:10:28.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.283+0000 7f8ba0bdf640 1 -- 172.21.15.148:0/1428375930 shutdown_connections 2024-03-29T04:10:28.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.283+0000 7f8ba0bdf640 1 -- 172.21.15.148:0/1428375930 >> 172.21.15.148:0/1428375930 conn(0x7f8b94009fb0 msgr2=0x7f8b9400a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:28.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.283+0000 7f75e22ce640 1 -- 172.21.15.148:0/1176157735 shutdown_connections 2024-03-29T04:10:28.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.283+0000 7f75e22ce640 1 --2- 172.21.15.148:0/1176157735 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75dc06c5c0 0x7f75dc06ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.283+0000 7f75e22ce640 1 --2- 172.21.15.148:0/1176157735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75dc163900 0x7f75dc166cc0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.283+0000 7f75e22ce640 1 -- 172.21.15.148:0/1176157735 >> 172.21.15.148:0/1176157735 conn(0x7f75dc0cfed0 msgr2=0x7f75dc0d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:28.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.284+0000 7f75e22ce640 1 -- 172.21.15.148:0/1176157735 shutdown_connections 2024-03-29T04:10:28.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.284+0000 7f75e22ce640 1 -- 172.21.15.148:0/1176157735 wait complete. 2024-03-29T04:10:28.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.284+0000 7f8ba0bdf640 1 -- 172.21.15.148:0/1428375930 shutdown_connections 2024-03-29T04:10:28.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.285+0000 7f8ba0bdf640 1 -- 172.21.15.148:0/1428375930 wait complete. 2024-03-29T04:10:28.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.285+0000 7f6ca209a640 1 Processor -- start 2024-03-29T04:10:28.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.285+0000 7f5343ca5640 1 Processor -- start 2024-03-29T04:10:28.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.285+0000 7f5343ca5640 1 -- start start 2024-03-29T04:10:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.286+0000 7f75e22ce640 1 Processor -- start 2024-03-29T04:10:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.286+0000 7f75e22ce640 1 -- start start 2024-03-29T04:10:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.286+0000 7f6ca209a640 1 -- start start 2024-03-29T04:10:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.287+0000 7f6ca209a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c84003b40 0x7f6c9c161760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.287+0000 7f6ca209a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c9c1f4aa0 0x7f6c9c1578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.287+0000 7f6ca209a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c9c0c70a0 0x7f6c9c15ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.287+0000 7f6ca209a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c0b5840 con 0x7f6c84003b40 2024-03-29T04:10:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.287+0000 7f6ca209a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c0b2880 con 0x7f6c9c0c70a0 2024-03-29T04:10:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.287+0000 7f6ca209a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6c9c0b1920 con 0x7f6c9c1f4aa0 2024-03-29T04:10:28.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.287+0000 7f6c9bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c9c0c70a0 0x7f6c9c15ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.287+0000 7f6c9bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c9c0c70a0 0x7f6c9c15ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:39622/0 (socket says 172.21.15.148:39622) 2024-03-29T04:10:28.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.287+0000 7f6c9bfff640 1 -- 172.21.15.148:0/3050129365 learned_addr learned my addr 172.21.15.148:0/3050129365 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:28.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.287+0000 7f6c9affd640 1 --2- 172.21.15.148:0/3050129365 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c9c1f4aa0 0x7f6c9c1578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.286+0000 7f5343ca5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f533c06f4e0 0x7f533c143a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.286+0000 7f5343ca5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f533c06c5c0 0x7f533c06ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.286+0000 7f5343ca5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f533c15acb0 0x7f533c15b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.286+0000 7f5343ca5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f533c06cf60 con 0x7f533c15acb0 2024-03-29T04:10:28.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.286+0000 7f5343ca5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f533c0b2ee0 con 0x7f533c06c5c0 2024-03-29T04:10:28.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.286+0000 7f5343ca5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f533c1610a0 con 0x7f533c06f4e0 2024-03-29T04:10:28.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.287+0000 7f5341240640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f533c06c5c0 0x7f533c06ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.287+0000 7f5341240640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f533c06c5c0 0x7f533c06ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:39608/0 (socket says 172.21.15.148:39608) 2024-03-29T04:10:28.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.287+0000 7f5341240640 1 -- 172.21.15.148:0/4125568781 learned_addr learned my addr 172.21.15.148:0/4125568781 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:28.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f75e22ce640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75dc06c5c0 0x7f75dc06d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f75e22ce640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75dc163520 0x7f75dc145ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.286+0000 7f8ba0bdf640 1 Processor -- start 2024-03-29T04:10:28.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f75e22ce640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75dc163900 0x7f75dc146420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f75dbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75dc06c5c0 0x7f75dc06d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f75dbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75dc06c5c0 0x7f75dc06d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48842/0 (socket says 172.21.15.148:48842) 2024-03-29T04:10:28.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f75db7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75dc163520 0x7f75dc145ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f75db7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75dc163520 0x7f75dc145ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:39630/0 (socket says 172.21.15.148:39630) 2024-03-29T04:10:28.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.287+0000 7f8ba0bdf640 1 -- start start 2024-03-29T04:10:28.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f75dbfff640 1 -- 172.21.15.148:0/3711669944 learned_addr learned my addr 172.21.15.148:0/3711669944 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:28.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f75e086b640 1 --2- 172.21.15.148:0/3711669944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75dc163900 0x7f75dc146420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f75e22ce640 1 -- 172.21.15.148:0/3711669944 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f75dc1667a0 con 0x7f75dc06c5c0 2024-03-29T04:10:28.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f6c9bfff640 1 -- 172.21.15.148:0/3050129365 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c9c1f4aa0 msgr2=0x7f6c9c1578c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f6c9bfff640 1 --2- 172.21.15.148:0/3050129365 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c9c1f4aa0 0x7f6c9c1578c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f6c9bfff640 1 -- 172.21.15.148:0/3050129365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c84003b40 msgr2=0x7f6c9c161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f6c9bfff640 1 --2- 172.21.15.148:0/3050129365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c84003b40 0x7f6c9c161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f6c9bfff640 1 -- 172.21.15.148:0/3050129365 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c9c1f4f60 con 0x7f6c9c0c70a0 2024-03-29T04:10:28.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f75e22ce640 1 -- 172.21.15.148:0/3711669944 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f75dc0b3c90 con 0x7f75dc163520 2024-03-29T04:10:28.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f6c9bfff640 1 --2- 172.21.15.148:0/3050129365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c9c0c70a0 0x7f6c9c15ff20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6c90011490 tx=0x7f6c90011950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:28.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f75e22ce640 1 -- 172.21.15.148:0/3711669944 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f75dc167200 con 0x7f75dc163900 2024-03-29T04:10:28.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f8ba0bdf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b94048620 0x7f8b940355b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.287+0000 7f5341a41640 1 --2- 172.21.15.148:0/4125568781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f533c06f4e0 0x7f533c143a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.287+0000 7f5342242640 1 --2- 172.21.15.148:0/4125568781 >> v2:172.21.15.148:3300/0 conn(0x7f533c15acb0 0x7f533c15b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f5341240640 1 -- 172.21.15.148:0/4125568781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f533c06f4e0 msgr2=0x7f533c143a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:28.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f5341240640 1 --2- 172.21.15.148:0/4125568781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f533c06f4e0 0x7f533c143a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f5341240640 1 -- 172.21.15.148:0/4125568781 >> v2:172.21.15.148:3300/0 conn(0x7f533c15acb0 msgr2=0x7f533c15b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:28.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f5341240640 1 --2- 172.21.15.148:0/4125568781 >> v2:172.21.15.148:3300/0 conn(0x7f533c15acb0 0x7f533c15b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f5341240640 1 -- 172.21.15.148:0/4125568781 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f533c0b3130 con 0x7f533c06c5c0 2024-03-29T04:10:28.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.289+0000 7f75dbfff640 1 -- 172.21.15.148:0/3711669944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75dc163900 msgr2=0x7f75dc146420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.289+0000 7f75dbfff640 1 --2- 172.21.15.148:0/3711669944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75dc163900 0x7f75dc146420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.289+0000 7f75dbfff640 1 -- 172.21.15.148:0/3711669944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75dc163520 msgr2=0x7f75dc145ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.289+0000 7f75dbfff640 1 --2- 172.21.15.148:0/3711669944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75dc163520 0x7f75dc145ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.289+0000 7f75dbfff640 1 -- 172.21.15.148:0/3711669944 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75dc146d60 con 0x7f75dc06c5c0 2024-03-29T04:10:28.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.289+0000 7f5341240640 1 --2- 172.21.15.148:0/4125568781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f533c06c5c0 0x7f533c06ca20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f533c0b2ca0 tx=0x7f533802b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67a3babf3de56ac3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:28.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.289+0000 7f75dbfff640 1 --2- 172.21.15.148:0/3711669944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75dc06c5c0 0x7f75dc06d830 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00c5f0 tx=0x7f75cc00cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:28.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.289+0000 7f75e086b640 1 --2- 172.21.15.148:0/3711669944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75dc163900 0x7f75dc146420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:28.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.289+0000 7f75db7fe640 1 --2- 172.21.15.148:0/3711669944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75dc163520 0x7f75dc145ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:28.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f8ba0bdf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b940eb610 0x7f8b94036060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f8ba0bdf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b940365a0 0x7f8b940343e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.290+0000 7f5340a3f640 1 -- 172.21.15.148:0/4125568781 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5338034070 con 0x7f533c06c5c0 2024-03-29T04:10:28.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.290+0000 7f5340a3f640 1 -- 172.21.15.148:0/4125568781 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5338030850 con 0x7f533c06c5c0 2024-03-29T04:10:28.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.290+0000 7f5340a3f640 1 -- 172.21.15.148:0/4125568781 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5338030c20 con 0x7f533c06c5c0 2024-03-29T04:10:28.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f8ba0bdf640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8b94037910 con 0x7f8b940eb610 2024-03-29T04:10:28.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f8ba0bdf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8b940064f0 con 0x7f8b940365a0 2024-03-29T04:10:28.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.290+0000 7f75d8ff9640 1 -- 172.21.15.148:0/3711669944 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc018070 con 0x7f75dc06c5c0 2024-03-29T04:10:28.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.290+0000 7f75d8ff9640 1 -- 172.21.15.148:0/3711669944 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75cc0074e0 con 0x7f75dc06c5c0 2024-03-29T04:10:28.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.290+0000 7f75d8ff9640 1 -- 172.21.15.148:0/3711669944 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc007690 con 0x7f75dc06c5c0 2024-03-29T04:10:28.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f8ba0bdf640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8b94037a90 con 0x7f8b94048620 2024-03-29T04:10:28.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.289+0000 7f8b9bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b940365a0 0x7f8b940343e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.290+0000 7f75e22ce640 1 -- 172.21.15.148:0/3711669944 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75dc147040 con 0x7f75dc06c5c0 2024-03-29T04:10:28.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.290+0000 7f75e22ce640 1 -- 172.21.15.148:0/3711669944 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75dc147600 con 0x7f75dc06c5c0 2024-03-29T04:10:28.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.289+0000 7f8b9bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b940365a0 0x7f8b940343e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:39640/0 (socket says 172.21.15.148:39640) 2024-03-29T04:10:28.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.289+0000 7f8b9bfff640 1 -- 172.21.15.148:0/2462312331 learned_addr learned my addr 172.21.15.148:0/2462312331 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:28.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.289+0000 7f8b9affd640 1 --2- 172.21.15.148:0/2462312331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b940eb610 0x7f8b94036060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.289+0000 7f8b9bfff640 1 -- 172.21.15.148:0/2462312331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b94048620 msgr2=0x7f8b940355b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-29T04:10:28.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.289+0000 7f8b9bfff640 1 --2- 172.21.15.148:0/2462312331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b94048620 0x7f8b940355b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.289+0000 7f8b9bfff640 1 -- 172.21.15.148:0/2462312331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b940eb610 msgr2=0x7f8b94036060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.289+0000 7f8b9bfff640 1 --2- 172.21.15.148:0/2462312331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b940eb610 0x7f8b94036060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.289+0000 7f8b9bfff640 1 -- 172.21.15.148:0/2462312331 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b940ebdf0 con 0x7f8b940365a0 2024-03-29T04:10:28.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.289+0000 7f8b9affd640 1 --2- 172.21.15.148:0/2462312331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b940eb610 0x7f8b94036060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:28.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.290+0000 7f8b9bfff640 1 --2- 172.21.15.148:0/2462312331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b940365a0 0x7f8b940343e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c069cd0 tx=0x7f8b9c069340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:28.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.291+0000 7f8b87fff640 1 -- 172.21.15.148:0/2462312331 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c0ad040 con 0x7f8b940365a0 2024-03-29T04:10:28.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.291+0000 7f8b87fff640 1 -- 172.21.15.148:0/2462312331 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b9c050140 con 0x7f8b940365a0 2024-03-29T04:10:28.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.291+0000 7f5343ca5640 1 -- 172.21.15.148:0/4125568781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f533c06c5c0 msgr2=0x7f533c06ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.291+0000 7f5343ca5640 1 --2- 172.21.15.148:0/4125568781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f533c06c5c0 0x7f533c06ca20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f533c0b2ca0 tx=0x7f533802b990 comp rx=0 tx=0).stop 2024-03-29T04:10:28.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.291+0000 7f8ba0bdf640 1 -- 172.21.15.148:0/2462312331 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b940ec0d0 con 0x7f8b940365a0 2024-03-29T04:10:28.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.291+0000 7f8ba0bdf640 1 -- 172.21.15.148:0/2462312331 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b940ec690 con 0x7f8b940365a0 2024-03-29T04:10:28.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.291+0000 7f8b87fff640 1 -- 172.21.15.148:0/2462312331 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c068cd0 con 0x7f8b940365a0 2024-03-29T04:10:28.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.292+0000 7f5343ca5640 1 -- 172.21.15.148:0/4125568781 shutdown_connections 2024-03-29T04:10:28.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.292+0000 7f5343ca5640 1 --2- 172.21.15.148:0/4125568781 >> v2:172.21.15.148:3300/0 conn(0x7f533c15acb0 0x7f533c15b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.292+0000 7f5343ca5640 1 --2- 172.21.15.148:0/4125568781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f533c06c5c0 0x7f533c06ca20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.292+0000 7f5343ca5640 1 --2- 172.21.15.148:0/4125568781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f533c06f4e0 0x7f533c143a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.292+0000 7f5343ca5640 1 -- 172.21.15.148:0/4125568781 >> 172.21.15.148:0/4125568781 conn(0x7f533c0cfed0 msgr2=0x7f533c0d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:28.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f6c83fff640 1 -- 172.21.15.148:0/3050129365 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9000f040 con 0x7f6c9c0c70a0 2024-03-29T04:10:28.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.288+0000 7f6c83fff640 1 -- 172.21.15.148:0/3050129365 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c9001c070 con 0x7f6c9c0c70a0 2024-03-29T04:10:28.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.290+0000 7f6ca209a640 1 -- 172.21.15.148:0/3050129365 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c9c1597c0 con 0x7f6c9c0c70a0 2024-03-29T04:10:28.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.290+0000 7f6ca209a640 1 -- 172.21.15.148:0/3050129365 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c9c159d80 con 0x7f6c9c0c70a0 2024-03-29T04:10:28.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.291+0000 7f6c83fff640 1 -- 172.21.15.148:0/3050129365 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c90015b50 con 0x7f6c9c0c70a0 2024-03-29T04:10:28.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.293+0000 7f6ca209a640 1 -- 172.21.15.148:0/3050129365 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c9c0687e0 con 0x7f6c9c0c70a0 2024-03-29T04:10:28.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.294+0000 7f5343ca5640 1 -- 172.21.15.148:0/4125568781 shutdown_connections 2024-03-29T04:10:28.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.293+0000 7f75d8ff9640 1 -- 172.21.15.148:0/3711669944 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f75cc01b070 con 0x7f75dc06c5c0 2024-03-29T04:10:28.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.293+0000 7f75d8ff9640 1 --2- 172.21.15.148:0/3711669944 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f75c806c600 0x7f75c806eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.294+0000 7f75d8ff9640 1 -- 172.21.15.148:0/3711669944 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f75cc08afd0 con 0x7f75dc06c5c0 2024-03-29T04:10:28.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.294+0000 7f8b87fff640 1 -- 172.21.15.148:0/2462312331 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8b9c06b5b0 con 0x7f8b940365a0 2024-03-29T04:10:28.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.294+0000 7f5343ca5640 1 -- 172.21.15.148:0/4125568781 wait complete. 2024-03-29T04:10:28.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.294+0000 7f8b87fff640 1 --2- 172.21.15.148:0/2462312331 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f8b8006c620 0x7f8b8006eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.295+0000 7f8b9b7fe640 1 --2- 172.21.15.148:0/2462312331 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f8b8006c620 0x7f8b8006eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.296+0000 7f8b9b7fe640 1 --2- 172.21.15.148:0/2462312331 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f8b8006c620 0x7f8b8006eae0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8b900099d0 tx=0x7f8b90009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:28.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.296+0000 7f75db7fe640 1 --2- 172.21.15.148:0/3711669944 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f75c806c600 0x7f75c806eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.296+0000 7f8b87fff640 1 -- 172.21.15.148:0/2462312331 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f8b9c04fe60 con 0x7f8b940365a0 2024-03-29T04:10:28.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.296+0000 7f5343ca5640 1 Processor -- start 2024-03-29T04:10:28.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.296+0000 7f8ba0bdf640 1 -- 172.21.15.148:0/2462312331 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b7c005290 con 0x7f8b940365a0 2024-03-29T04:10:28.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.296+0000 7f5343ca5640 1 -- start start 2024-03-29T04:10:28.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.296+0000 7f75db7fe640 1 --2- 172.21.15.148:0/3711669944 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f75c806c600 0x7f75c806eac0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f75d0034000 tx=0x7f75d0008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:28.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.297+0000 7f75e22ce640 1 -- 172.21.15.148:0/3711669944 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75bc005290 con 0x7f75dc06c5c0 2024-03-29T04:10:28.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.297+0000 7f5343ca5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f533c06c5c0 0x7f533c071660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.297+0000 7f5343ca5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f533c06f4e0 0x7f533c155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.297+0000 7f5343ca5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f533c15acb0 0x7f533c1560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.297+0000 7f5343ca5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f533c0b39d0 con 0x7f533c06f4e0 2024-03-29T04:10:28.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.297+0000 7f5343ca5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f533c0b36f0 con 0x7f533c06c5c0 2024-03-29T04:10:28.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.297+0000 7f5343ca5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f533c143fa0 con 0x7f533c15acb0 2024-03-29T04:10:28.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.297+0000 7f5341a41640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f533c06c5c0 0x7f533c071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.297+0000 7f5341a41640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f533c06c5c0 0x7f533c071660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:39648/0 (socket says 172.21.15.148:39648) 2024-03-29T04:10:28.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.297+0000 7f5341a41640 1 -- 172.21.15.148:0/1585829725 learned_addr learned my addr 172.21.15.148:0/1585829725 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:28.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.297+0000 7f5341240640 1 --2- 172.21.15.148:0/1585829725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f533c06f4e0 0x7f533c155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.298+0000 7f5342242640 1 --2- 172.21.15.148:0/1585829725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f533c15acb0 0x7f533c1560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.298+0000 7f5341a41640 1 -- 172.21.15.148:0/1585829725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f533c15acb0 msgr2=0x7f533c1560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.298+0000 7f5341a41640 1 --2- 172.21.15.148:0/1585829725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f533c15acb0 0x7f533c1560a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.298+0000 7f5341a41640 1 -- 172.21.15.148:0/1585829725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f533c06f4e0 msgr2=0x7f533c155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.298+0000 7f5341a41640 1 --2- 172.21.15.148:0/1585829725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f533c06f4e0 0x7f533c155b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.298+0000 7f5341a41640 1 -- 172.21.15.148:0/1585829725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f533c1565e0 con 0x7f533c06c5c0 2024-03-29T04:10:28.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.298+0000 7f5341a41640 1 --2- 172.21.15.148:0/1585829725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f533c06c5c0 0x7f533c071660 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f532c00e9e0 tx=0x7f532c00eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:28.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.299+0000 7f6c83fff640 1 -- 172.21.15.148:0/3050129365 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6c90015cf0 con 0x7f6c9c0c70a0 2024-03-29T04:10:28.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.301+0000 7f53327fc640 1 -- 172.21.15.148:0/1585829725 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c017070 con 0x7f533c06c5c0 2024-03-29T04:10:28.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.301+0000 7f5343ca5640 1 -- 172.21.15.148:0/1585829725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f533c135fc0 con 0x7f533c06c5c0 2024-03-29T04:10:28.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.302+0000 7f5343ca5640 1 -- 172.21.15.148:0/1585829725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f533c134870 con 0x7f533c06c5c0 2024-03-29T04:10:28.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.302+0000 7f8b87fff640 1 -- 172.21.15.148:0/2462312331 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8b9c0c3030 con 0x7f8b940365a0 2024-03-29T04:10:28.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.302+0000 7f6c83fff640 1 --2- 172.21.15.148:0/3050129365 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6c8806c5b0 0x7f6c8806ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.303+0000 7f5343ca5640 1 -- 172.21.15.148:0/1585829725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f533c0687e0 con 0x7f533c06c5c0 2024-03-29T04:10:28.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.303+0000 7f6c9b7fe640 1 --2- 172.21.15.148:0/3050129365 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6c8806c5b0 0x7f6c8806ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.304+0000 7f6c83fff640 1 -- 172.21.15.148:0/3050129365 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6c9008b7b0 con 0x7f6c9c0c70a0 2024-03-29T04:10:28.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.305+0000 7f6c9b7fe640 1 --2- 172.21.15.148:0/3050129365 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6c8806c5b0 0x7f6c8806ea70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c036000 tx=0x7f6c8c030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:28.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.307+0000 7f6c83fff640 1 -- 172.21.15.148:0/3050129365 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6c90059c00 con 0x7f6c9c0c70a0 2024-03-29T04:10:28.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.308+0000 7f75d8ff9640 1 -- 172.21.15.148:0/3711669944 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f75cc059290 con 0x7f75dc06c5c0 2024-03-29T04:10:28.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.307+0000 7f53327fc640 1 -- 172.21.15.148:0/1585829725 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f532c005990 con 0x7f533c06c5c0 2024-03-29T04:10:28.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.307+0000 7f53327fc640 1 -- 172.21.15.148:0/1585829725 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c021060 con 0x7f533c06c5c0 2024-03-29T04:10:28.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.308+0000 7f53327fc640 1 -- 172.21.15.148:0/1585829725 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f532c01a070 con 0x7f533c06c5c0 2024-03-29T04:10:28.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.309+0000 7f53327fc640 1 --2- 172.21.15.148:0/1585829725 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f532006c530 0x7f532006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.309+0000 7f53327fc640 1 -- 172.21.15.148:0/1585829725 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f532c08c200 con 0x7f533c06c5c0 2024-03-29T04:10:28.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.309+0000 7f5341240640 1 --2- 172.21.15.148:0/1585829725 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f532006c530 0x7f532006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.310+0000 7f5341240640 1 --2- 172.21.15.148:0/1585829725 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f532006c530 0x7f532006e9f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5338036000 tx=0x7f5338008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:28.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.314+0000 7f53327fc640 1 -- 172.21.15.148:0/1585829725 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f533c135fc0 con 0x7f533c06c5c0 2024-03-29T04:10:28.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.332+0000 7f21aaeba640 1 Processor -- start 2024-03-29T04:10:28.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.332+0000 7f21aaeba640 1 -- start start 2024-03-29T04:10:28.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.334+0000 7f21aaeba640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f219c0e6900 0x7f219c056350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.334+0000 7f21aaeba640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f219c0e6ce0 0x7f219c054ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.334+0000 7f21aaeba640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f219c0550e0 0x7f219c0e7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.334+0000 7f21aaeba640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f219c055570 con 0x7f219c0e6900 2024-03-29T04:10:28.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.334+0000 7f21aaeba640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f219c006b20 con 0x7f219c0550e0 2024-03-29T04:10:28.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.334+0000 7f21aaeba640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f219c006e00 con 0x7f219c0e6ce0 2024-03-29T04:10:28.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.334+0000 7f21a9eb8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f219c0e6900 0x7f219c056350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.334+0000 7f21a9eb8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f219c0e6900 0x7f219c056350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48872/0 (socket says 172.21.15.148:48872) 2024-03-29T04:10:28.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.334+0000 7f21a9eb8640 1 -- 172.21.15.148:0/1833617533 learned_addr learned my addr 172.21.15.148:0/1833617533 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:28.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.334+0000 7f21aa6b9640 1 --2- 172.21.15.148:0/1833617533 >> v2:172.21.15.157:3300/0 conn(0x7f219c0550e0 0x7f219c0e7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.334+0000 7f21a96b7640 1 --2- 172.21.15.148:0/1833617533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f219c0e6ce0 0x7f219c054ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.335+0000 7f21a9eb8640 1 -- 172.21.15.148:0/1833617533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f219c0e6ce0 msgr2=0x7f219c054ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:28.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.335+0000 7f21a9eb8640 1 --2- 172.21.15.148:0/1833617533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f219c0e6ce0 0x7f219c054ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.335+0000 7f21a9eb8640 1 -- 172.21.15.148:0/1833617533 >> v2:172.21.15.157:3300/0 conn(0x7f219c0550e0 msgr2=0x7f219c0e7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:28.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.335+0000 7f21a9eb8640 1 --2- 172.21.15.148:0/1833617533 >> v2:172.21.15.157:3300/0 conn(0x7f219c0550e0 0x7f219c0e7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.335+0000 7f21a9eb8640 1 -- 172.21.15.148:0/1833617533 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219c0070e0 con 0x7f219c0e6900 2024-03-29T04:10:28.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.335+0000 7f21aa6b9640 1 --2- 172.21.15.148:0/1833617533 >> v2:172.21.15.157:3300/0 conn(0x7f219c0550e0 0x7f219c0e7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:28.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.336+0000 7f21a9eb8640 1 --2- 172.21.15.148:0/1833617533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f219c0e6900 0x7f219c056350 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f21a406a3b0 tx=0x7f21a40a6f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a592b018ec866941 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:28.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.337+0000 7f21a8eb6640 1 -- 172.21.15.148:0/1833617533 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a40b4070 con 0x7f219c0e6900 2024-03-29T04:10:28.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.337+0000 7f21a8eb6640 1 -- 172.21.15.148:0/1833617533 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21a40a7db0 con 0x7f219c0e6900 2024-03-29T04:10:28.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.337+0000 7f21a8eb6640 1 -- 172.21.15.148:0/1833617533 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a40af680 con 0x7f219c0e6900 2024-03-29T04:10:28.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.336+0000 7f5d7bfff640 1 Processor -- start 2024-03-29T04:10:28.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.336+0000 7f5d7bfff640 1 -- start start 2024-03-29T04:10:28.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.337+0000 7f5d7bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5d7c145a20 0x7f5d7c15d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.337+0000 7f5d7bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5d7c0b6da0 0x7f5d7c070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.337+0000 7f5d7bfff640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d7c145e00 con 0x7f5d7c071430 2024-03-29T04:10:28.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.337+0000 7f5d7bfff640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f5d7c0b6500 con 0x7f5d7c145a20 2024-03-29T04:10:28.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.337+0000 7f5d7bfff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5d7c06eae0 con 0x7f5d7c0b6da0 2024-03-29T04:10:28.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.337+0000 7f5d7affd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5d7c145a20 0x7f5d7c15d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.337+0000 7f5d7affd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5d7c145a20 0x7f5d7c15d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48878/0 (socket says 172.21.15.148:48878) 2024-03-29T04:10:28.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.337+0000 7f5d7affd640 1 -- 172.21.15.148:0/4120989922 learned_addr learned my addr 172.21.15.148:0/4120989922 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:28.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.338+0000 7f5d7affd640 1 -- 172.21.15.148:0/4120989922 >> v2:172.21.15.157:3300/0 conn(0x7f5d7c0b6da0 msgr2=0x7f5d7c070ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-29T04:10:28.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.338+0000 7f5d7affd640 1 --2- 172.21.15.148:0/4120989922 >> v2:172.21.15.157:3300/0 conn(0x7f5d7c0b6da0 0x7f5d7c070ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.338+0000 7f5d7affd640 1 -- 172.21.15.148:0/4120989922 >> v1:172.21.15.157:6789/0 conn(0x7f5d7c071430 legacy=0x7f5d7c15fc10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:10:28.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.338+0000 7f5d7affd640 1 -- 172.21.15.148:0/4120989922 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d7c06edc0 con 0x7f5d7c145a20 2024-03-29T04:10:28.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.340+0000 7f21aaeba640 1 -- 172.21.15.148:0/1833617533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f219c0e6900 msgr2=0x7f219c056350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.339+0000 7f5d7affd640 1 --2- 172.21.15.148:0/4120989922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d7c145a20 0x7f5d7c15d060 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0b4b90 tx=0x7f5d6c02bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ee3631909acf42 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:28.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.340+0000 7f21aaeba640 1 --2- 172.21.15.148:0/1833617533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f219c0e6900 0x7f219c056350 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f21a406a3b0 tx=0x7f21a40a6f40 comp rx=0 tx=0).stop 2024-03-29T04:10:28.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.340+0000 7f21aaeba640 1 -- 172.21.15.148:0/1833617533 shutdown_connections 2024-03-29T04:10:28.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.340+0000 7f21aaeba640 1 --2- 172.21.15.148:0/1833617533 >> v2:172.21.15.157:3300/0 conn(0x7f219c0550e0 0x7f219c0e7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.340+0000 7f21aaeba640 1 --2- 172.21.15.148:0/1833617533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f219c0e6ce0 0x7f219c054ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.340+0000 7f21aaeba640 1 --2- 172.21.15.148:0/1833617533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f219c0e6900 0x7f219c056350 secure :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f21a406a3b0 tx=0x7f21a40a6f40 comp rx=0 tx=0).stop 2024-03-29T04:10:28.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.340+0000 7f21aaeba640 1 -- 172.21.15.148:0/1833617533 >> 172.21.15.148:0/1833617533 conn(0x7f219c01ef10 msgr2=0x7f219c055c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:28.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.340+0000 7f5d79ffb640 1 -- 172.21.15.148:0/4120989922 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c039070 con 0x7f5d7c145a20 2024-03-29T04:10:28.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.340+0000 7f5d79ffb640 1 -- 172.21.15.148:0/4120989922 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d6c02c730 con 0x7f5d7c145a20 2024-03-29T04:10:28.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.341+0000 7f21aaeba640 1 -- 172.21.15.148:0/1833617533 shutdown_connections 2024-03-29T04:10:28.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.341+0000 7f5d7bfff640 1 -- 172.21.15.148:0/4120989922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d7c145a20 msgr2=0x7f5d7c15d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.341+0000 7f5d7bfff640 1 --2- 172.21.15.148:0/4120989922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d7c145a20 0x7f5d7c15d060 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0b4b90 tx=0x7f5d6c02bb90 comp rx=0 tx=0).stop 2024-03-29T04:10:28.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.342+0000 7f21aaeba640 1 -- 172.21.15.148:0/1833617533 wait complete. 2024-03-29T04:10:28.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.342+0000 7f5d7bfff640 1 -- 172.21.15.148:0/4120989922 shutdown_connections 2024-03-29T04:10:28.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.342+0000 7f5d7bfff640 1 --2- 172.21.15.148:0/4120989922 >> v2:172.21.15.157:3300/0 conn(0x7f5d7c0b6da0 0x7f5d7c070ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.342+0000 7f5d7bfff640 1 --2- 172.21.15.148:0/4120989922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d7c145a20 0x7f5d7c15d060 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.342+0000 7f5d7bfff640 1 -- 172.21.15.148:0/4120989922 >> 172.21.15.148:0/4120989922 conn(0x7f5d7c0b3d00 msgr2=0x7f5d7c0b4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:28.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.343+0000 7f5d7bfff640 1 -- 172.21.15.148:0/4120989922 shutdown_connections 2024-03-29T04:10:28.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.343+0000 7f21aaeba640 1 Processor -- start 2024-03-29T04:10:28.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.344+0000 7f21aaeba640 1 -- start start 2024-03-29T04:10:28.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.345+0000 7f21aaeba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f219c0550e0 0x7f219c03f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.345+0000 7f21aaeba640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f219c0e6ce0 0x7f219c04c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.345+0000 7f21aaeba640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f219c0e04b0 0x7f219c17d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.345+0000 7f21aaeba640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f219c03b770 con 0x7f219c0e04b0 2024-03-29T04:10:28.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.345+0000 7f21aaeba640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f219c050290 con 0x7f219c0550e0 2024-03-29T04:10:28.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.345+0000 7f21aaeba640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f219c0558b0 con 0x7f219c0e6ce0 2024-03-29T04:10:28.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.345+0000 7f21aa6b9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f219c0e04b0 0x7f219c17d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.345+0000 7f21aa6b9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f219c0e04b0 0x7f219c17d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48894/0 (socket says 172.21.15.148:48894) 2024-03-29T04:10:28.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.345+0000 7f21aa6b9640 1 -- 172.21.15.148:0/1712739274 learned_addr learned my addr 172.21.15.148:0/1712739274 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:28.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.343+0000 7f5d7bfff640 1 -- 172.21.15.148:0/4120989922 wait complete. 2024-03-29T04:10:28.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.346+0000 7f21aa6b9640 1 -- 172.21.15.148:0/1712739274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f219c0e6ce0 msgr2=0x7f219c04c550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T04:10:28.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.344+0000 7f5d7bfff640 1 Processor -- start 2024-03-29T04:10:28.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.345+0000 7f5d7bfff640 1 -- start start 2024-03-29T04:10:28.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.346+0000 7f5d7bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d7c071430 0x7f5d7c08b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.346+0000 7f5d7bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d7c0b6da0 0x7f5d7c144c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.346+0000 7f5d7bfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d7c145a20 0x7f5d7c141230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.346+0000 7f5d7bfff640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7c06f660 con 0x7f5d7c071430 2024-03-29T04:10:28.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.346+0000 7f5d7bfff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7c06f380 con 0x7f5d7c0b6da0 2024-03-29T04:10:28.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.346+0000 7f5d7bfff640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5d7c15d5a0 con 0x7f5d7c145a20 2024-03-29T04:10:28.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.346+0000 7f5d7affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d7c071430 0x7f5d7c08b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.346+0000 7f5d7a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d7c0b6da0 0x7f5d7c144c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.346+0000 7f5d7a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d7c0b6da0 0x7f5d7c144c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:39680/0 (socket says 172.21.15.148:39680) 2024-03-29T04:10:28.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.346+0000 7f5d7a7fc640 1 -- 172.21.15.148:0/947502034 learned_addr learned my addr 172.21.15.148:0/947502034 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:28.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.347+0000 7f5d7a7fc640 1 -- 172.21.15.148:0/947502034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d7c145a20 msgr2=0x7f5d7c141230 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-29T04:10:28.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.347+0000 7f5d7a7fc640 1 --2- 172.21.15.148:0/947502034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d7c145a20 0x7f5d7c141230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.347+0000 7f5d7a7fc640 1 -- 172.21.15.148:0/947502034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d7c071430 msgr2=0x7f5d7c08b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.347+0000 7f5d7a7fc640 1 --2- 172.21.15.148:0/947502034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d7c071430 0x7f5d7c08b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.347+0000 7f5d7a7fc640 1 -- 172.21.15.148:0/947502034 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d7c08bde0 con 0x7f5d7c0b6da0 2024-03-29T04:10:28.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.347+0000 7f5d7affd640 1 --2- 172.21.15.148:0/947502034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d7c071430 0x7f5d7c08b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:28.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.347+0000 7f5d7a7fc640 1 --2- 172.21.15.148:0/947502034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d7c0b6da0 0x7f5d7c144c00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400b9e0 tx=0x7f5d7400bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:28.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.346+0000 7f21aa6b9640 1 --2- 172.21.15.148:0/1712739274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f219c0e6ce0 0x7f219c04c550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.347+0000 7f5d63fff640 1 -- 172.21.15.148:0/947502034 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d74019070 con 0x7f5d7c0b6da0 2024-03-29T04:10:28.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.347+0000 7f5d7bfff640 1 -- 172.21.15.148:0/947502034 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d7c145140 con 0x7f5d7c0b6da0 2024-03-29T04:10:28.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.346+0000 7f21aa6b9640 1 -- 172.21.15.148:0/1712739274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f219c0550e0 msgr2=0x7f219c03f9b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-29T04:10:28.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.346+0000 7f21aa6b9640 1 --2- 172.21.15.148:0/1712739274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f219c0550e0 0x7f219c03f9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.347+0000 7f5d7bfff640 1 -- 172.21.15.148:0/947502034 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d7c151770 con 0x7f5d7c0b6da0 2024-03-29T04:10:28.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.346+0000 7f21aa6b9640 1 -- 172.21.15.148:0/1712739274 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219c04ca90 con 0x7f219c0e04b0 2024-03-29T04:10:28.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.346+0000 7f21aa6b9640 1 --2- 172.21.15.148:0/1712739274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f219c0e04b0 0x7f219c17d6c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f21a000eb60 tx=0x7f21a000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:28.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.348+0000 7f5d63fff640 1 -- 172.21.15.148:0/947502034 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d7400cba0 con 0x7f5d7c0b6da0 2024-03-29T04:10:28.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.348+0000 7f5d63fff640 1 -- 172.21.15.148:0/947502034 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d74014440 con 0x7f5d7c0b6da0 2024-03-29T04:10:28.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.350+0000 7f5d63fff640 1 -- 172.21.15.148:0/947502034 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5d7401c070 con 0x7f5d7c0b6da0 2024-03-29T04:10:28.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.351+0000 7f5d63fff640 1 --2- 172.21.15.148:0/947502034 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5d6806c630 0x7f5d6806eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.349+0000 7f219a7fc640 1 -- 172.21.15.148:0/1712739274 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a0017070 con 0x7f219c0e04b0 2024-03-29T04:10:28.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.349+0000 7f219a7fc640 1 -- 172.21.15.148:0/1712739274 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21a000ed70 con 0x7f219c0e04b0 2024-03-29T04:10:28.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.349+0000 7f219a7fc640 1 -- 172.21.15.148:0/1712739274 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a000ce10 con 0x7f219c0e04b0 2024-03-29T04:10:28.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.351+0000 7f5d63fff640 1 -- 172.21.15.148:0/947502034 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5d7408b580 con 0x7f5d7c0b6da0 2024-03-29T04:10:28.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.351+0000 7f5d7affd640 1 --2- 172.21.15.148:0/947502034 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5d6806c630 0x7f5d6806eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.353+0000 7f5d7affd640 1 --2- 172.21.15.148:0/947502034 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5d6806c630 0x7f5d6806eaf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c036000 tx=0x7f5d6c030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:28.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.353+0000 7f5d7bfff640 1 -- 172.21.15.148:0/947502034 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d5c005290 con 0x7f5d7c0b6da0 2024-03-29T04:10:28.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.352+0000 7f21aaeba640 1 -- 172.21.15.148:0/1712739274 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f219c0498d0 con 0x7f219c0e04b0 2024-03-29T04:10:28.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.352+0000 7f21aaeba640 1 -- 172.21.15.148:0/1712739274 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f219c049e90 con 0x7f219c0e04b0 2024-03-29T04:10:28.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.356+0000 7f219a7fc640 1 -- 172.21.15.148:0/1712739274 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f21a0005990 con 0x7f219c0e04b0 2024-03-29T04:10:28.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.356+0000 7f21aaeba640 1 -- 172.21.15.148:0/1712739274 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f219c0057b0 con 0x7f219c0e04b0 2024-03-29T04:10:28.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.356+0000 7f219a7fc640 1 --2- 172.21.15.148:0/1712739274 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f218806c600 0x7f218806eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.357+0000 7f219a7fc640 1 -- 172.21.15.148:0/1712739274 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f21a008b490 con 0x7f219c0e04b0 2024-03-29T04:10:28.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.357+0000 7f21a9eb8640 1 --2- 172.21.15.148:0/1712739274 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f218806c600 0x7f218806eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.357+0000 7f21a9eb8640 1 --2- 172.21.15.148:0/1712739274 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f218806c600 0x7f218806eac0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f219c00afc0 tx=0x7f21a40ab000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:28.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.361+0000 7f5d63fff640 1 -- 172.21.15.148:0/947502034 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5d74059280 con 0x7f5d7c0b6da0 2024-03-29T04:10:28.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.367+0000 7f219a7fc640 1 -- 172.21.15.148:0/1712739274 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f219c04ca90 con 0x7f219c0e04b0 2024-03-29T04:10:28.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.440+0000 7f248181b640 1 Processor -- start 2024-03-29T04:10:28.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.444+0000 7f248181b640 1 -- start start 2024-03-29T04:10:28.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.445+0000 7f248181b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f247c144cc0 0x7f247c06d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.445+0000 7f248181b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f247c0b5f40 0x7f247c06dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.445+0000 7f247b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f247c0b5f40 0x7f247c06dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.445+0000 7f247b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f247c0b5f40 0x7f247c06dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48910/0 (socket says 172.21.15.148:48910) 2024-03-29T04:10:28.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.445+0000 7f247b7fe640 1 -- 172.21.15.148:0/1423579698 learned_addr learned my addr 172.21.15.148:0/1423579698 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:28.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.445+0000 7f248181b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f247c0b4e90 con 0x7f247c1448e0 2024-03-29T04:10:28.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.445+0000 7f2473fff640 1 --2- 172.21.15.148:0/1423579698 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f247c144cc0 0x7f247c06d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.445+0000 7f248181b640 1 -- 172.21.15.148:0/1423579698 --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f247c0b3e80 con 0x7f247c0b5f40 2024-03-29T04:10:28.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.445+0000 7f248181b640 1 -- 172.21.15.148:0/1423579698 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f247c0b40d0 con 0x7f247c144cc0 2024-03-29T04:10:28.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.446+0000 7f247b7fe640 1 -- 172.21.15.148:0/1423579698 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f247c144cc0 msgr2=0x7f247c06d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:28.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.446+0000 7f247b7fe640 1 --2- 172.21.15.148:0/1423579698 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f247c144cc0 0x7f247c06d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.446+0000 7f247b7fe640 1 -- 172.21.15.148:0/1423579698 >> v1:172.21.15.148:6789/0 conn(0x7f247c1448e0 legacy=0x7f247c0b5830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:10:28.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.446+0000 7f247b7fe640 1 -- 172.21.15.148:0/1423579698 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247c0b43b0 con 0x7f247c0b5f40 2024-03-29T04:10:28.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.447+0000 7f247b7fe640 1 --2- 172.21.15.148:0/1423579698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f247c0b5f40 0x7f247c06dc10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2474009e50 tx=0x7f247402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5dbf88d61fdbbf server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:28.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.454+0000 7f247a7fc640 1 -- 172.21.15.148:0/1423579698 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247402c9e0 con 0x7f247c0b5f40 2024-03-29T04:10:28.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.454+0000 7f247a7fc640 1 -- 172.21.15.148:0/1423579698 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f247402cb80 con 0x7f247c0b5f40 2024-03-29T04:10:28.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.454+0000 7f247a7fc640 1 -- 172.21.15.148:0/1423579698 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24740343f0 con 0x7f247c0b5f40 2024-03-29T04:10:28.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.457+0000 7f248181b640 1 -- 172.21.15.148:0/1423579698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f247c0b5f40 msgr2=0x7f247c06dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.457+0000 7f248181b640 1 --2- 172.21.15.148:0/1423579698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f247c0b5f40 0x7f247c06dc10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2474009e50 tx=0x7f247402be40 comp rx=0 tx=0).stop 2024-03-29T04:10:28.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.461+0000 7f248181b640 1 -- 172.21.15.148:0/1423579698 shutdown_connections 2024-03-29T04:10:28.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.461+0000 7f248181b640 1 --2- 172.21.15.148:0/1423579698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f247c0b5f40 0x7f247c06dc10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.461+0000 7f248181b640 1 --2- 172.21.15.148:0/1423579698 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f247c144cc0 0x7f247c06d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.461+0000 7f248181b640 1 -- 172.21.15.148:0/1423579698 >> 172.21.15.148:0/1423579698 conn(0x7f247c072450 msgr2=0x7f247c072860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:28.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.473+0000 7f248181b640 1 -- 172.21.15.148:0/1423579698 shutdown_connections 2024-03-29T04:10:28.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.477+0000 7f248181b640 1 -- 172.21.15.148:0/1423579698 wait complete. 2024-03-29T04:10:28.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.478+0000 7f248181b640 1 Processor -- start 2024-03-29T04:10:28.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.479+0000 7f248181b640 1 -- start start 2024-03-29T04:10:28.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.480+0000 7f248181b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f247c0b5f40 0x7f247c13a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.480+0000 7f248181b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f247c1448e0 0x7f247c149300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.480+0000 7f248181b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f247c13c340 0x7f247c06ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.480+0000 7f247affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f247c0b5f40 0x7f247c13a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.480+0000 7f248181b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f247c0708d0 con 0x7f247c0b5f40 2024-03-29T04:10:28.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.480+0000 7f248181b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f247c0b4970 con 0x7f247c1448e0 2024-03-29T04:10:28.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.480+0000 7f248181b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f247c06b5b0 con 0x7f247c13c340 2024-03-29T04:10:28.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.480+0000 7f247affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f247c0b5f40 0x7f247c13a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48914/0 (socket says 172.21.15.148:48914) 2024-03-29T04:10:28.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.480+0000 7f247affd640 1 -- 172.21.15.148:0/141604117 learned_addr learned my addr 172.21.15.148:0/141604117 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:28.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.481+0000 7f247affd640 1 -- 172.21.15.148:0/141604117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f247c13c340 msgr2=0x7f247c06ca30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-29T04:10:28.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.481+0000 7f247affd640 1 --2- 172.21.15.148:0/141604117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f247c13c340 0x7f247c06ca30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.481+0000 7f247affd640 1 -- 172.21.15.148:0/141604117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f247c1448e0 msgr2=0x7f247c149300 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-29T04:10:28.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.481+0000 7f247affd640 1 --2- 172.21.15.148:0/141604117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f247c1448e0 0x7f247c149300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.481+0000 7f247affd640 1 -- 172.21.15.148:0/141604117 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247c13aea0 con 0x7f247c0b5f40 2024-03-29T04:10:28.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.481+0000 7f247affd640 1 --2- 172.21.15.148:0/141604117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f247c0b5f40 0x7f247c13a960 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f246400b450 tx=0x7f246400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:28.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.483+0000 7f2478ff9640 1 -- 172.21.15.148:0/141604117 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2464015020 con 0x7f247c0b5f40 2024-03-29T04:10:28.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.483+0000 7f2478ff9640 1 -- 172.21.15.148:0/141604117 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2464011890 con 0x7f247c0b5f40 2024-03-29T04:10:28.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.484+0000 7f2478ff9640 1 -- 172.21.15.148:0/141604117 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2464011c60 con 0x7f247c0b5f40 2024-03-29T04:10:28.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.484+0000 7f248181b640 1 -- 172.21.15.148:0/141604117 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f247c13cd90 con 0x7f247c0b5f40 2024-03-29T04:10:28.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.484+0000 7f248181b640 1 -- 172.21.15.148:0/141604117 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f247c15d110 con 0x7f247c0b5f40 2024-03-29T04:10:28.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.485+0000 7f248181b640 1 -- 172.21.15.148:0/141604117 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f247c0687e0 con 0x7f247c0b5f40 2024-03-29T04:10:28.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.498+0000 7f2478ff9640 1 -- 172.21.15.148:0/141604117 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2464018020 con 0x7f247c0b5f40 2024-03-29T04:10:28.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.498+0000 7f2478ff9640 1 --2- 172.21.15.148:0/141604117 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f245c06c630 0x7f245c06eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.498+0000 7f2478ff9640 1 -- 172.21.15.148:0/141604117 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f246408bb90 con 0x7f247c0b5f40 2024-03-29T04:10:28.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.500+0000 7f2473fff640 1 --2- 172.21.15.148:0/141604117 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f245c06c630 0x7f245c06eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.501+0000 7f2473fff640 1 --2- 172.21.15.148:0/141604117 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f245c06c630 0x7f245c06eaf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f247c156e30 tx=0x7f246c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:28.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.510+0000 7f2478ff9640 1 -- 172.21.15.148:0/141604117 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f246401f020 con 0x7f247c0b5f40 2024-03-29T04:10:28.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.632+0000 7f90a0315640 1 -- 172.21.15.148:0/1783233009 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f9080005580 con 0x7f90980b3170 2024-03-29T04:10:28.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.639+0000 7f908effd640 1 -- 172.21.15.148:0/1783233009 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f909405cd60 con 0x7f90980b3170 2024-03-29T04:10:28.621 INFO:teuthology.orchestra.run.smithi148.stdout:55834574859 2024-03-29T04:10:28.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.643+0000 7f90a0315640 1 -- 172.21.15.148:0/1783233009 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f908406c670 msgr2=0x7f908406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.643+0000 7f90a0315640 1 --2- 172.21.15.148:0/1783233009 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f908406c670 0x7f908406eb30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f90982043a0 tx=0x7f9088009190 comp rx=0 tx=0).stop 2024-03-29T04:10:28.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.643+0000 7f90a0315640 1 -- 172.21.15.148:0/1783233009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90980b3170 msgr2=0x7f90980aaab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.643+0000 7f90a0315640 1 --2- 172.21.15.148:0/1783233009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90980b3170 0x7f90980aaab0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f909400c340 tx=0x7f909400c800 comp rx=0 tx=0).stop 2024-03-29T04:10:28.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.649+0000 7f90a0315640 1 -- 172.21.15.148:0/1783233009 shutdown_connections 2024-03-29T04:10:28.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.649+0000 7f90a0315640 1 --2- 172.21.15.148:0/1783233009 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f908406c670 0x7f908406eb30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.649+0000 7f90a0315640 1 --2- 172.21.15.148:0/1783233009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9098162920 0x7f909815f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.649+0000 7f90a0315640 1 --2- 172.21.15.148:0/1783233009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9098162540 0x7f9098165350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.649+0000 7f90a0315640 1 --2- 172.21.15.148:0/1783233009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90980b3170 0x7f90980aaab0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.649+0000 7f90a0315640 1 -- 172.21.15.148:0/1783233009 >> 172.21.15.148:0/1783233009 conn(0x7f909806b5b0 msgr2=0x7f90980672c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:28.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.649+0000 7f90a0315640 1 -- 172.21.15.148:0/1783233009 shutdown_connections 2024-03-29T04:10:28.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.649+0000 7f90a0315640 1 -- 172.21.15.148:0/1783233009 wait complete. 2024-03-29T04:10:28.644 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.3 2024-03-29T04:10:28.644 DEBUG:teuthology.parallel:result is None 2024-03-29T04:10:28.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.709+0000 7f8ba0bdf640 1 -- 172.21.15.148:0/2462312331 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f8b7c005580 con 0x7f8b940365a0 2024-03-29T04:10:28.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.709+0000 7f8b87fff640 1 -- 172.21.15.148:0/2462312331 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8b9c0f6200 con 0x7f8b940365a0 2024-03-29T04:10:28.689 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-03-29T04:10:28.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.712+0000 7f8ba0bdf640 1 -- 172.21.15.148:0/2462312331 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f8b8006c620 msgr2=0x7f8b8006eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.712+0000 7f8ba0bdf640 1 --2- 172.21.15.148:0/2462312331 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f8b8006c620 0x7f8b8006eae0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8b900099d0 tx=0x7f8b90009190 comp rx=0 tx=0).stop 2024-03-29T04:10:28.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.712+0000 7f8ba0bdf640 1 -- 172.21.15.148:0/2462312331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b940365a0 msgr2=0x7f8b940343e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.712+0000 7f8ba0bdf640 1 --2- 172.21.15.148:0/2462312331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b940365a0 0x7f8b940343e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c069cd0 tx=0x7f8b9c069340 comp rx=0 tx=0).stop 2024-03-29T04:10:28.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.712+0000 7f8ba0bdf640 1 -- 172.21.15.148:0/2462312331 shutdown_connections 2024-03-29T04:10:28.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.712+0000 7f8ba0bdf640 1 --2- 172.21.15.148:0/2462312331 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f8b8006c620 0x7f8b8006eae0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.712+0000 7f8ba0bdf640 1 --2- 172.21.15.148:0/2462312331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b940365a0 0x7f8b940343e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.712+0000 7f8ba0bdf640 1 --2- 172.21.15.148:0/2462312331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b940eb610 0x7f8b94036060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.712+0000 7f8ba0bdf640 1 --2- 172.21.15.148:0/2462312331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b94048620 0x7f8b940355b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.712+0000 7f8ba0bdf640 1 -- 172.21.15.148:0/2462312331 >> 172.21.15.148:0/2462312331 conn(0x7f8b94009fb0 msgr2=0x7f8b940df220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:28.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.714+0000 7f8ba0bdf640 1 -- 172.21.15.148:0/2462312331 shutdown_connections 2024-03-29T04:10:28.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.714+0000 7f8ba0bdf640 1 -- 172.21.15.148:0/2462312331 wait complete. 2024-03-29T04:10:28.711 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.2 2024-03-29T04:10:28.711 DEBUG:teuthology.parallel:result is None 2024-03-29T04:10:28.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.741+0000 7f6ca209a640 1 -- 172.21.15.148:0/3050129365 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f6c9c0ce8f0 con 0x7f6c9c0c70a0 2024-03-29T04:10:28.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.742+0000 7f6c83fff640 1 -- 172.21.15.148:0/3050129365 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6c90059500 con 0x7f6c9c0c70a0 2024-03-29T04:10:28.721 INFO:teuthology.orchestra.run.smithi148.stdout:55834574859 2024-03-29T04:10:28.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.744+0000 7f5343ca5640 1 -- 172.21.15.148:0/1585829725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f533c15fbd0 con 0x7f533c06c5c0 2024-03-29T04:10:28.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.745+0000 7f6ca209a640 1 -- 172.21.15.148:0/3050129365 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6c8806c5b0 msgr2=0x7f6c8806ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.745+0000 7f53327fc640 1 -- 172.21.15.148:0/1585829725 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f532c059e50 con 0x7f533c06c5c0 2024-03-29T04:10:28.724 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-03-29T04:10:28.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.745+0000 7f6ca209a640 1 --2- 172.21.15.148:0/3050129365 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6c8806c5b0 0x7f6c8806ea70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c036000 tx=0x7f6c8c030000 comp rx=0 tx=0).stop 2024-03-29T04:10:28.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.745+0000 7f6ca209a640 1 -- 172.21.15.148:0/3050129365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c9c0c70a0 msgr2=0x7f6c9c15ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.745+0000 7f6ca209a640 1 --2- 172.21.15.148:0/3050129365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c9c0c70a0 0x7f6c9c15ff20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6c90011490 tx=0x7f6c90011950 comp rx=0 tx=0).stop 2024-03-29T04:10:28.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.745+0000 7f6ca209a640 1 -- 172.21.15.148:0/3050129365 shutdown_connections 2024-03-29T04:10:28.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.745+0000 7f6ca209a640 1 --2- 172.21.15.148:0/3050129365 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6c8806c5b0 0x7f6c8806ea70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.746+0000 7f6ca209a640 1 --2- 172.21.15.148:0/3050129365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c9c0c70a0 0x7f6c9c15ff20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.746+0000 7f6ca209a640 1 --2- 172.21.15.148:0/3050129365 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c9c1f4aa0 0x7f6c9c1578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.746+0000 7f6ca209a640 1 --2- 172.21.15.148:0/3050129365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c84003b40 0x7f6c9c161760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.746+0000 7f6ca209a640 1 -- 172.21.15.148:0/3050129365 >> 172.21.15.148:0/3050129365 conn(0x7f6c9c0ceb80 msgr2=0x7f6c9c0b4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:28.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.746+0000 7f6ca209a640 1 -- 172.21.15.148:0/3050129365 shutdown_connections 2024-03-29T04:10:28.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.746+0000 7f6ca209a640 1 -- 172.21.15.148:0/3050129365 wait complete. 2024-03-29T04:10:28.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.748+0000 7f5343ca5640 1 -- 172.21.15.148:0/1585829725 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f532006c530 msgr2=0x7f532006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.748+0000 7f5343ca5640 1 --2- 172.21.15.148:0/1585829725 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f532006c530 0x7f532006e9f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5338036000 tx=0x7f5338008030 comp rx=0 tx=0).stop 2024-03-29T04:10:28.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.748+0000 7f5343ca5640 1 -- 172.21.15.148:0/1585829725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f533c06c5c0 msgr2=0x7f533c071660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.748+0000 7f5343ca5640 1 --2- 172.21.15.148:0/1585829725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f533c06c5c0 0x7f533c071660 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f532c00e9e0 tx=0x7f532c00eea0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.748+0000 7f5343ca5640 1 -- 172.21.15.148:0/1585829725 shutdown_connections 2024-03-29T04:10:28.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.748+0000 7f5343ca5640 1 --2- 172.21.15.148:0/1585829725 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f532006c530 0x7f532006e9f0 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5338036000 tx=0x7f5338008030 comp rx=0 tx=0).stop 2024-03-29T04:10:28.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.748+0000 7f5343ca5640 1 --2- 172.21.15.148:0/1585829725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f533c15acb0 0x7f533c1560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.748+0000 7f5343ca5640 1 --2- 172.21.15.148:0/1585829725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f533c06f4e0 0x7f533c155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.748+0000 7f5343ca5640 1 --2- 172.21.15.148:0/1585829725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f533c06c5c0 0x7f533c071660 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.748+0000 7f5343ca5640 1 -- 172.21.15.148:0/1585829725 >> 172.21.15.148:0/1585829725 conn(0x7f533c0cfed0 msgr2=0x7f533c15f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:28.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.748+0000 7f5343ca5640 1 -- 172.21.15.148:0/1585829725 shutdown_connections 2024-03-29T04:10:28.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.748+0000 7f5343ca5640 1 -- 172.21.15.148:0/1585829725 wait complete. 2024-03-29T04:10:28.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.749+0000 7f75e22ce640 1 -- 172.21.15.148:0/3711669944 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f75bc005580 con 0x7f75dc06c5c0 2024-03-29T04:10:28.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.749+0000 7f75d8ff9640 1 -- 172.21.15.148:0/3711669944 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f75cc058b90 con 0x7f75dc06c5c0 2024-03-29T04:10:28.729 INFO:teuthology.orchestra.run.smithi148.stdout:55834574859 2024-03-29T04:10:28.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.753+0000 7f75e22ce640 1 -- 172.21.15.148:0/3711669944 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f75c806c600 msgr2=0x7f75c806eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.753+0000 7f75e22ce640 1 --2- 172.21.15.148:0/3711669944 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f75c806c600 0x7f75c806eac0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f75d0034000 tx=0x7f75d0008030 comp rx=0 tx=0).stop 2024-03-29T04:10:28.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.753+0000 7f75e22ce640 1 -- 172.21.15.148:0/3711669944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75dc06c5c0 msgr2=0x7f75dc06d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.753+0000 7f75e22ce640 1 --2- 172.21.15.148:0/3711669944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75dc06c5c0 0x7f75dc06d830 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00c5f0 tx=0x7f75cc00cab0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.753+0000 7f75e22ce640 1 -- 172.21.15.148:0/3711669944 shutdown_connections 2024-03-29T04:10:28.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.753+0000 7f75e22ce640 1 --2- 172.21.15.148:0/3711669944 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f75c806c600 0x7f75c806eac0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.753+0000 7f75e22ce640 1 --2- 172.21.15.148:0/3711669944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75dc163900 0x7f75dc146420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.753+0000 7f75e22ce640 1 --2- 172.21.15.148:0/3711669944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75dc163520 0x7f75dc145ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.753+0000 7f75e22ce640 1 --2- 172.21.15.148:0/3711669944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75dc06c5c0 0x7f75dc06d830 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.753+0000 7f75e22ce640 1 -- 172.21.15.148:0/3711669944 >> 172.21.15.148:0/3711669944 conn(0x7f75dc0cfed0 msgr2=0x7f75dc0d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:28.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.753+0000 7f75e22ce640 1 -- 172.21.15.148:0/3711669944 shutdown_connections 2024-03-29T04:10:28.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.753+0000 7f75e22ce640 1 -- 172.21.15.148:0/3711669944 wait complete. 2024-03-29T04:10:28.744 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.4 2024-03-29T04:10:28.745 DEBUG:teuthology.parallel:result is None 2024-03-29T04:10:28.745 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.5 2024-03-29T04:10:28.745 DEBUG:teuthology.parallel:result is None 2024-03-29T04:10:28.749 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.1 2024-03-29T04:10:28.749 DEBUG:teuthology.parallel:result is None 2024-03-29T04:10:28.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.775+0000 7f21aaeba640 1 -- 172.21.15.148:0/1712739274 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f219c00f390 con 0x7f219c0e04b0 2024-03-29T04:10:28.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.775+0000 7f219a7fc640 1 -- 172.21.15.148:0/1712739274 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f21a0059750 con 0x7f219c0e04b0 2024-03-29T04:10:28.754 INFO:teuthology.orchestra.run.smithi148.stdout:55834574859 2024-03-29T04:10:28.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.777+0000 7f21aaeba640 1 -- 172.21.15.148:0/1712739274 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f218806c600 msgr2=0x7f218806eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.777+0000 7f21aaeba640 1 --2- 172.21.15.148:0/1712739274 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f218806c600 0x7f218806eac0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f219c00afc0 tx=0x7f21a40ab000 comp rx=0 tx=0).stop 2024-03-29T04:10:28.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.777+0000 7f21aaeba640 1 -- 172.21.15.148:0/1712739274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f219c0e04b0 msgr2=0x7f219c17d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.777+0000 7f21aaeba640 1 --2- 172.21.15.148:0/1712739274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f219c0e04b0 0x7f219c17d6c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f21a000eb60 tx=0x7f21a000c520 comp rx=0 tx=0).stop 2024-03-29T04:10:28.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.777+0000 7f21aaeba640 1 -- 172.21.15.148:0/1712739274 shutdown_connections 2024-03-29T04:10:28.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.777+0000 7f21aaeba640 1 --2- 172.21.15.148:0/1712739274 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f218806c600 0x7f218806eac0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.777+0000 7f21aaeba640 1 --2- 172.21.15.148:0/1712739274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f219c0e04b0 0x7f219c17d6c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.777+0000 7f21aaeba640 1 --2- 172.21.15.148:0/1712739274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f219c0e6ce0 0x7f219c04c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.777+0000 7f21aaeba640 1 --2- 172.21.15.148:0/1712739274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f219c0550e0 0x7f219c03f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.777+0000 7f21aaeba640 1 -- 172.21.15.148:0/1712739274 >> 172.21.15.148:0/1712739274 conn(0x7f219c01ef10 msgr2=0x7f219c055c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:28.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.777+0000 7f21aaeba640 1 -- 172.21.15.148:0/1712739274 shutdown_connections 2024-03-29T04:10:28.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.777+0000 7f21aaeba640 1 -- 172.21.15.148:0/1712739274 wait complete. 2024-03-29T04:10:28.771 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.0 2024-03-29T04:10:28.771 DEBUG:teuthology.parallel:result is None 2024-03-29T04:10:28.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.794+0000 7f5d7bfff640 1 -- 172.21.15.148:0/947502034 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f5d5c005580 con 0x7f5d7c0b6da0 2024-03-29T04:10:28.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.795+0000 7f5d63fff640 1 -- 172.21.15.148:0/947502034 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5d7405d580 con 0x7f5d7c0b6da0 2024-03-29T04:10:28.774 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-03-29T04:10:28.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.797+0000 7f5d7bfff640 1 -- 172.21.15.148:0/947502034 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5d6806c630 msgr2=0x7f5d6806eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.797+0000 7f5d7bfff640 1 --2- 172.21.15.148:0/947502034 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5d6806c630 0x7f5d6806eaf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c036000 tx=0x7f5d6c030000 comp rx=0 tx=0).stop 2024-03-29T04:10:28.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.797+0000 7f5d7bfff640 1 -- 172.21.15.148:0/947502034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d7c0b6da0 msgr2=0x7f5d7c144c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.797+0000 7f5d7bfff640 1 --2- 172.21.15.148:0/947502034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d7c0b6da0 0x7f5d7c144c00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400b9e0 tx=0x7f5d7400bea0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.797+0000 7f5d7bfff640 1 -- 172.21.15.148:0/947502034 shutdown_connections 2024-03-29T04:10:28.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.797+0000 7f5d7bfff640 1 --2- 172.21.15.148:0/947502034 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5d6806c630 0x7f5d6806eaf0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.797+0000 7f5d7bfff640 1 --2- 172.21.15.148:0/947502034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d7c145a20 0x7f5d7c141230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.797+0000 7f5d7bfff640 1 --2- 172.21.15.148:0/947502034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d7c0b6da0 0x7f5d7c144c00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.797+0000 7f5d7bfff640 1 --2- 172.21.15.148:0/947502034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d7c071430 0x7f5d7c08b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.797+0000 7f5d7bfff640 1 -- 172.21.15.148:0/947502034 >> 172.21.15.148:0/947502034 conn(0x7f5d7c0b3d00 msgr2=0x7f5d7c15ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:28.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.797+0000 7f5d7bfff640 1 -- 172.21.15.148:0/947502034 shutdown_connections 2024-03-29T04:10:28.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.797+0000 7f5d7bfff640 1 -- 172.21.15.148:0/947502034 wait complete. 2024-03-29T04:10:28.787 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.6 2024-03-29T04:10:28.787 DEBUG:teuthology.parallel:result is None 2024-03-29T04:10:28.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.861+0000 7f248181b640 1 -- 172.21.15.148:0/141604117 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f247c06dba0 con 0x7f247c0b5f40 2024-03-29T04:10:28.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.862+0000 7f2478ff9640 1 -- 172.21.15.148:0/141604117 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f246400d040 con 0x7f247c0b5f40 2024-03-29T04:10:28.841 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-03-29T04:10:28.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.864+0000 7f248181b640 1 -- 172.21.15.148:0/141604117 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f245c06c630 msgr2=0x7f245c06eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.864+0000 7f248181b640 1 --2- 172.21.15.148:0/141604117 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f245c06c630 0x7f245c06eaf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f247c156e30 tx=0x7f246c009190 comp rx=0 tx=0).stop 2024-03-29T04:10:28.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.864+0000 7f248181b640 1 -- 172.21.15.148:0/141604117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f247c0b5f40 msgr2=0x7f247c13a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.864+0000 7f248181b640 1 --2- 172.21.15.148:0/141604117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f247c0b5f40 0x7f247c13a960 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f246400b450 tx=0x7f246400b910 comp rx=0 tx=0).stop 2024-03-29T04:10:28.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.864+0000 7f248181b640 1 -- 172.21.15.148:0/141604117 shutdown_connections 2024-03-29T04:10:28.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.864+0000 7f248181b640 1 --2- 172.21.15.148:0/141604117 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f245c06c630 0x7f245c06eaf0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.864+0000 7f248181b640 1 --2- 172.21.15.148:0/141604117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f247c13c340 0x7f247c06ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.864+0000 7f248181b640 1 --2- 172.21.15.148:0/141604117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f247c1448e0 0x7f247c149300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.864+0000 7f248181b640 1 --2- 172.21.15.148:0/141604117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f247c0b5f40 0x7f247c13a960 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.864+0000 7f248181b640 1 -- 172.21.15.148:0/141604117 >> 172.21.15.148:0/141604117 conn(0x7f247c072450 msgr2=0x7f247c143700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:28.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.864+0000 7f248181b640 1 -- 172.21.15.148:0/141604117 shutdown_connections 2024-03-29T04:10:28.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.864+0000 7f248181b640 1 -- 172.21.15.148:0/141604117 wait complete. 2024-03-29T04:10:28.854 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.7 2024-03-29T04:10:28.854 DEBUG:teuthology.parallel:result is None 2024-03-29T04:10:28.854 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-29T04:10:28.854 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:10:28.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.979+0000 7f84b1f24640 1 Processor -- start 2024-03-29T04:10:28.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.979+0000 7f84b1f24640 1 -- start start 2024-03-29T04:10:28.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.980+0000 7f84b1f24640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84ac0af530 0x7f84ac0af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.980+0000 7f84b1f24640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84ac06cc00 con 0x7f84ac15f690 2024-03-29T04:10:28.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.980+0000 7f84b1f24640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84ac06cee0 con 0x7f84ac158c80 2024-03-29T04:10:28.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.980+0000 7f84b1f24640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f84ac06c9b0 con 0x7f84ac0af530 2024-03-29T04:10:28.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.980+0000 7f84aaffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f84ac15f690 0x7f84ac162a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47326/0 (socket says 172.21.15.148:47326) 2024-03-29T04:10:28.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.980+0000 7f84aaffd640 1 -- 172.21.15.148:0/2448141047 learned_addr learned my addr 172.21.15.148:0/2448141047 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:28.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.980+0000 7f84abfff640 1 --2- 172.21.15.148:0/2448141047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84ac0af530 0x7f84ac0af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.980+0000 7f84aa7fc640 1 -- 172.21.15.148:0/2448141047 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1143951422 0 0) 0x7f84ac06cc00 con 0x7f84ac15f690 2024-03-29T04:10:28.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.980+0000 7f84aa7fc640 1 -- 172.21.15.148:0/2448141047 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8494003520 con 0x7f84ac15f690 2024-03-29T04:10:28.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.980+0000 7f84aa7fc640 1 -- 172.21.15.148:0/2448141047 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1114158184 0 0) 0x7f84ac06cee0 con 0x7f84ac158c80 2024-03-29T04:10:28.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.980+0000 7f84aa7fc640 1 -- 172.21.15.148:0/2448141047 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84ac06cc00 con 0x7f84ac158c80 2024-03-29T04:10:28.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.980+0000 7f84abfff640 1 -- 172.21.15.148:0/2448141047 >> v1:172.21.15.157:6789/0 conn(0x7f84ac158c80 legacy=0x7f84ac0b1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.980+0000 7f84abfff640 1 -- 172.21.15.148:0/2448141047 >> v1:172.21.15.148:6789/0 conn(0x7f84ac15f690 legacy=0x7f84ac162a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.981+0000 7f84abfff640 1 -- 172.21.15.148:0/2448141047 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84ac162200 con 0x7f84ac0af530 2024-03-29T04:10:28.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.981+0000 7f84abfff640 1 --2- 172.21.15.148:0/2448141047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84ac0af530 0x7f84ac0af9c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f84a4009df0 tx=0x7f84a402bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b30d59fe16ebc246 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:28.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.981+0000 7f84aa7fc640 1 -- 172.21.15.148:0/2448141047 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a4039070 con 0x7f84ac0af530 2024-03-29T04:10:28.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.981+0000 7f84aa7fc640 1 -- 172.21.15.148:0/2448141047 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84a402cb90 con 0x7f84ac0af530 2024-03-29T04:10:28.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.982+0000 7f84aa7fc640 1 -- 172.21.15.148:0/2448141047 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a4034440 con 0x7f84ac0af530 2024-03-29T04:10:28.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.982+0000 7f84b1f24640 1 -- 172.21.15.148:0/2448141047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84ac0af530 msgr2=0x7f84ac0af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.982+0000 7f84b1f24640 1 --2- 172.21.15.148:0/2448141047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84ac0af530 0x7f84ac0af9c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f84a4009df0 tx=0x7f84a402bde0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.982+0000 7f84b1f24640 1 -- 172.21.15.148:0/2448141047 shutdown_connections 2024-03-29T04:10:28.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.982+0000 7f84b1f24640 1 --2- 172.21.15.148:0/2448141047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84ac0af530 0x7f84ac0af9c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.982+0000 7f84b1f24640 1 -- 172.21.15.148:0/2448141047 >> 172.21.15.148:0/2448141047 conn(0x7f84ac0b5050 msgr2=0x7f84ac0b5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:28.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.982+0000 7f84b1f24640 1 -- 172.21.15.148:0/2448141047 shutdown_connections 2024-03-29T04:10:28.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.982+0000 7f84b1f24640 1 -- 172.21.15.148:0/2448141047 wait complete. 2024-03-29T04:10:28.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.984+0000 7f84b1f24640 1 Processor -- start 2024-03-29T04:10:28.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.984+0000 7f84b1f24640 1 -- start start 2024-03-29T04:10:28.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.985+0000 7f84b1f24640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84ac0af530 0x7f84ac1e90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.985+0000 7f84b1f24640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84ac158c80 0x7f84ac154c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.985+0000 7f84b1f24640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84ac15f690 0x7f84ac1614c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.986+0000 7f84b1f24640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f84ac159060 con 0x7f84ac15f690 2024-03-29T04:10:28.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.986+0000 7f84b1f24640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f84ac163110 con 0x7f84ac0af530 2024-03-29T04:10:28.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.986+0000 7f84b1f24640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f84ac06b5b0 con 0x7f84ac158c80 2024-03-29T04:10:28.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.986+0000 7f84abfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84ac15f690 0x7f84ac1614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.986+0000 7f84abfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84ac15f690 0x7f84ac1614c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48930/0 (socket says 172.21.15.148:48930) 2024-03-29T04:10:28.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.986+0000 7f84abfff640 1 -- 172.21.15.148:0/136771198 learned_addr learned my addr 172.21.15.148:0/136771198 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:28.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.986+0000 7f84aaffd640 1 --2- 172.21.15.148:0/136771198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84ac158c80 0x7f84ac154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.986+0000 7f84ab7fe640 1 --2- 172.21.15.148:0/136771198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84ac0af530 0x7f84ac1e90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.986+0000 7f84abfff640 1 -- 172.21.15.148:0/136771198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84ac158c80 msgr2=0x7f84ac154c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.986+0000 7f84abfff640 1 --2- 172.21.15.148:0/136771198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84ac158c80 0x7f84ac154c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.986+0000 7f84abfff640 1 -- 172.21.15.148:0/136771198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84ac0af530 msgr2=0x7f84ac1e90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:28.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.986+0000 7f84abfff640 1 --2- 172.21.15.148:0/136771198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84ac0af530 0x7f84ac1e90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:28.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.986+0000 7f84abfff640 1 -- 172.21.15.148:0/136771198 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84ac1e8040 con 0x7f84ac15f690 2024-03-29T04:10:28.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.986+0000 7f84abfff640 1 --2- 172.21.15.148:0/136771198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84ac15f690 0x7f84ac1614c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f84a4009df0 tx=0x7f84a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:28.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.986+0000 7f8493fff640 1 -- 172.21.15.148:0/136771198 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a4044020 con 0x7f84ac15f690 2024-03-29T04:10:28.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.986+0000 7f8493fff640 1 -- 172.21.15.148:0/136771198 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84a402cd40 con 0x7f84ac15f690 2024-03-29T04:10:28.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.986+0000 7f8493fff640 1 -- 172.21.15.148:0/136771198 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a4039040 con 0x7f84ac15f690 2024-03-29T04:10:28.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.986+0000 7f84b1f24640 1 -- 172.21.15.148:0/136771198 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84ac1e82e0 con 0x7f84ac15f690 2024-03-29T04:10:28.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.986+0000 7f84b1f24640 1 -- 172.21.15.148:0/136771198 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84ac1e88a0 con 0x7f84ac15f690 2024-03-29T04:10:28.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.988+0000 7f8493fff640 1 -- 172.21.15.148:0/136771198 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f84a4047020 con 0x7f84ac15f690 2024-03-29T04:10:28.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.988+0000 7f84b1f24640 1 -- 172.21.15.148:0/136771198 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84ac0687e0 con 0x7f84ac15f690 2024-03-29T04:10:28.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.988+0000 7f8493fff640 1 --2- 172.21.15.148:0/136771198 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f849806c560 0x7f849806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:28.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.988+0000 7f8493fff640 1 -- 172.21.15.148:0/136771198 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f84a40ae240 con 0x7f84ac15f690 2024-03-29T04:10:28.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.988+0000 7f84ab7fe640 1 --2- 172.21.15.148:0/136771198 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f849806c560 0x7f849806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:28.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.989+0000 7f84ab7fe640 1 --2- 172.21.15.148:0/136771198 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f849806c560 0x7f849806ea20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f84a0001650 tx=0x7f84a0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:28.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:28.993+0000 7f8493fff640 1 -- 172.21.15.148:0/136771198 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f84a404e030 con 0x7f84ac15f690 2024-03-29T04:10:29.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.171+0000 7f84b1f24640 1 -- 172.21.15.148:0/136771198 --> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f84ac0723f0 con 0x7f849806c560 2024-03-29T04:10:29.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.179+0000 7f8493fff640 1 -- 172.21.15.148:0/136771198 <== mgr.4113 v2:172.21.15.148:6832/3271445075 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332533 (secure 0 0 0) 0x7f84ac0723f0 con 0x7f849806c560 2024-03-29T04:10:29.162 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:29.163 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-03-29T04:10:29.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.182+0000 7f84b1f24640 1 -- 172.21.15.148:0/136771198 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f849806c560 msgr2=0x7f849806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:29.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.182+0000 7f84b1f24640 1 --2- 172.21.15.148:0/136771198 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f849806c560 0x7f849806ea20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f84a0001650 tx=0x7f84a0009040 comp rx=0 tx=0).stop 2024-03-29T04:10:29.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.182+0000 7f84b1f24640 1 -- 172.21.15.148:0/136771198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84ac15f690 msgr2=0x7f84ac1614c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:29.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.182+0000 7f84b1f24640 1 --2- 172.21.15.148:0/136771198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84ac15f690 0x7f84ac1614c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f84a4009df0 tx=0x7f84a4004a40 comp rx=0 tx=0).stop 2024-03-29T04:10:29.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.182+0000 7f84b1f24640 1 -- 172.21.15.148:0/136771198 shutdown_connections 2024-03-29T04:10:29.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.182+0000 7f84b1f24640 1 --2- 172.21.15.148:0/136771198 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f849806c560 0x7f849806ea20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:29.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.182+0000 7f84b1f24640 1 --2- 172.21.15.148:0/136771198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84ac15f690 0x7f84ac1614c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:29.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.182+0000 7f84b1f24640 1 --2- 172.21.15.148:0/136771198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84ac158c80 0x7f84ac154c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:29.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.182+0000 7f84b1f24640 1 --2- 172.21.15.148:0/136771198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84ac0af530 0x7f84ac1e90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:29.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.182+0000 7f84b1f24640 1 -- 172.21.15.148:0/136771198 >> 172.21.15.148:0/136771198 conn(0x7f84ac0b5050 msgr2=0x7f84ac06dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:29.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.182+0000 7f84b1f24640 1 -- 172.21.15.148:0/136771198 shutdown_connections 2024-03-29T04:10:29.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.182+0000 7f84b1f24640 1 -- 172.21.15.148:0/136771198 wait complete. 2024-03-29T04:10:29.173 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-03-29T04:10:27.888129+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236032,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738688,"statfs":{"total":773094113280,"available":772852416512,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002255"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911869+0000","last_change":"2024-03-29T04:09:59.541493+0000","last_active":"2024-03-29T04:10:05.911869+0000","last_peered":"2024-03-29T04:10:05.911869+0000","last_clean":"2024-03-29T04:10:05.911869+0000","last_became_active":"2024-03-29T04:09:59.541208+0000","last_became_peered":"2024-03-29T04:09:59.541208+0000","last_unstale":"2024-03-29T04:10:05.911869+0000","last_undegraded":"2024-03-29T04:10:05.911869+0000","last_fullsized":"2024-03-29T04:10:05.911869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:55:08.672579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910799+0000","last_change":"2024-03-29T04:10:00.548950+0000","last_active":"2024-03-29T04:10:05.910799+0000","last_peered":"2024-03-29T04:10:05.910799+0000","last_clean":"2024-03-29T04:10:05.910799+0000","last_became_active":"2024-03-29T04:10:00.548670+0000","last_became_peered":"2024-03-29T04:10:00.548670+0000","last_unstale":"2024-03-29T04:10:05.910799+0000","last_undegraded":"2024-03-29T04:10:05.910799+0000","last_fullsized":"2024-03-29T04:10:05.910799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:16:02.331457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911034+0000","last_change":"2024-03-29T04:10:04.649988+0000","last_active":"2024-03-29T04:10:05.911034+0000","last_peered":"2024-03-29T04:10:05.911034+0000","last_clean":"2024-03-29T04:10:05.911034+0000","last_became_active":"2024-03-29T04:10:04.649557+0000","last_became_peered":"2024-03-29T04:10:04.649557+0000","last_unstale":"2024-03-29T04:10:05.911034+0000","last_undegraded":"2024-03-29T04:10:05.911034+0000","last_fullsized":"2024-03-29T04:10:05.911034+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:07:56.606322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911032+0000","last_change":"2024-03-29T04:09:59.542509+0000","last_active":"2024-03-29T04:10:05.911032+0000","last_peered":"2024-03-29T04:10:05.911032+0000","last_clean":"2024-03-29T04:10:05.911032+0000","last_became_active":"2024-03-29T04:09:59.542221+0000","last_became_peered":"2024-03-29T04:09:59.542221+0000","last_unstale":"2024-03-29T04:10:05.911032+0000","last_undegraded":"2024-03-29T04:10:05.911032+0000","last_fullsized":"2024-03-29T04:10:05.911032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:58:10.121554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910298+0000","last_change":"2024-03-29T04:10:00.536505+0000","last_active":"2024-03-29T04:10:05.910298+0000","last_peered":"2024-03-29T04:10:05.910298+0000","last_clean":"2024-03-29T04:10:05.910298+0000","last_became_active":"2024-03-29T04:10:00.536200+0000","last_became_peered":"2024-03-29T04:10:00.536200+0000","last_unstale":"2024-03-29T04:10:05.910298+0000","last_undegraded":"2024-03-29T04:10:05.910298+0000","last_fullsized":"2024-03-29T04:10:05.910298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:29.233534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557208+0000","last_change":"2024-03-29T04:10:04.576319+0000","last_active":"2024-03-29T04:10:05.557208+0000","last_peered":"2024-03-29T04:10:05.557208+0000","last_clean":"2024-03-29T04:10:05.557208+0000","last_became_active":"2024-03-29T04:10:04.575508+0000","last_became_peered":"2024-03-29T04:10:04.575508+0000","last_unstale":"2024-03-29T04:10:05.557208+0000","last_undegraded":"2024-03-29T04:10:05.557208+0000","last_fullsized":"2024-03-29T04:10:05.557208+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:34.129098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911504+0000","last_change":"2024-03-29T04:09:59.545281+0000","last_active":"2024-03-29T04:10:05.911504+0000","last_peered":"2024-03-29T04:10:05.911504+0000","last_clean":"2024-03-29T04:10:05.911504+0000","last_became_active":"2024-03-29T04:09:59.544991+0000","last_became_peered":"2024-03-29T04:09:59.544991+0000","last_unstale":"2024-03-29T04:10:05.911504+0000","last_undegraded":"2024-03-29T04:10:05.911504+0000","last_fullsized":"2024-03-29T04:10:05.911504+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:35:01.484589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556861+0000","last_change":"2024-03-29T04:10:00.542510+0000","last_active":"2024-03-29T04:10:05.556861+0000","last_peered":"2024-03-29T04:10:05.556861+0000","last_clean":"2024-03-29T04:10:05.556861+0000","last_became_active":"2024-03-29T04:10:00.542014+0000","last_became_peered":"2024-03-29T04:10:00.542014+0000","last_unstale":"2024-03-29T04:10:05.556861+0000","last_undegraded":"2024-03-29T04:10:05.556861+0000","last_fullsized":"2024-03-29T04:10:05.556861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:30.096622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556709+0000","last_change":"2024-03-29T04:10:04.574735+0000","last_active":"2024-03-29T04:10:05.556709+0000","last_peered":"2024-03-29T04:10:05.556709+0000","last_clean":"2024-03-29T04:10:05.556709+0000","last_became_active":"2024-03-29T04:10:04.573995+0000","last_became_peered":"2024-03-29T04:10:04.573995+0000","last_unstale":"2024-03-29T04:10:05.556709+0000","last_undegraded":"2024-03-29T04:10:05.556709+0000","last_fullsized":"2024-03-29T04:10:05.556709+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:52.246321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911346+0000","last_change":"2024-03-29T04:09:59.542326+0000","last_active":"2024-03-29T04:10:05.911346+0000","last_peered":"2024-03-29T04:10:05.911346+0000","last_clean":"2024-03-29T04:10:05.911346+0000","last_became_active":"2024-03-29T04:09:59.540873+0000","last_became_peered":"2024-03-29T04:09:59.540873+0000","last_unstale":"2024-03-29T04:10:05.911346+0000","last_undegraded":"2024-03-29T04:10:05.911346+0000","last_fullsized":"2024-03-29T04:10:05.911346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:02.879344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557486+0000","last_change":"2024-03-29T04:10:00.543297+0000","last_active":"2024-03-29T04:10:05.557486+0000","last_peered":"2024-03-29T04:10:05.557486+0000","last_clean":"2024-03-29T04:10:05.557486+0000","last_became_active":"2024-03-29T04:10:00.541793+0000","last_became_peered":"2024-03-29T04:10:00.541793+0000","last_unstale":"2024-03-29T04:10:05.557486+0000","last_undegraded":"2024-03-29T04:10:05.557486+0000","last_fullsized":"2024-03-29T04:10:05.557486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:13.002495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557550+0000","last_change":"2024-03-29T04:10:04.572366+0000","last_active":"2024-03-29T04:10:05.557550+0000","last_peered":"2024-03-29T04:10:05.557550+0000","last_clean":"2024-03-29T04:10:05.557550+0000","last_became_active":"2024-03-29T04:10:04.572066+0000","last_became_peered":"2024-03-29T04:10:04.572066+0000","last_unstale":"2024-03-29T04:10:05.557550+0000","last_undegraded":"2024-03-29T04:10:05.557550+0000","last_fullsized":"2024-03-29T04:10:05.557550+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:00:55.353300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910448+0000","last_change":"2024-03-29T04:10:00.547621+0000","last_active":"2024-03-29T04:10:05.910448+0000","last_peered":"2024-03-29T04:10:05.910448+0000","last_clean":"2024-03-29T04:10:05.910448+0000","last_became_active":"2024-03-29T04:10:00.547282+0000","last_became_peered":"2024-03-29T04:10:00.547282+0000","last_unstale":"2024-03-29T04:10:05.910448+0000","last_undegraded":"2024-03-29T04:10:05.910448+0000","last_fullsized":"2024-03-29T04:10:05.910448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:41.517875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911279+0000","last_change":"2024-03-29T04:09:59.539362+0000","last_active":"2024-03-29T04:10:05.911279+0000","last_peered":"2024-03-29T04:10:05.911279+0000","last_clean":"2024-03-29T04:10:05.911279+0000","last_became_active":"2024-03-29T04:09:59.539062+0000","last_became_peered":"2024-03-29T04:09:59.539062+0000","last_unstale":"2024-03-29T04:10:05.911279+0000","last_undegraded":"2024-03-29T04:10:05.911279+0000","last_fullsized":"2024-03-29T04:10:05.911279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:33:50.904931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.558475+0000","last_change":"2024-03-29T04:10:04.579133+0000","last_active":"2024-03-29T04:10:05.558475+0000","last_peered":"2024-03-29T04:10:05.558475+0000","last_clean":"2024-03-29T04:10:05.558475+0000","last_became_active":"2024-03-29T04:10:04.578748+0000","last_became_peered":"2024-03-29T04:10:04.578748+0000","last_unstale":"2024-03-29T04:10:05.558475+0000","last_undegraded":"2024-03-29T04:10:05.558475+0000","last_fullsized":"2024-03-29T04:10:05.558475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:06.998903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910015+0000","last_change":"2024-03-29T04:10:00.547887+0000","last_active":"2024-03-29T04:10:05.910015+0000","last_peered":"2024-03-29T04:10:05.910015+0000","last_clean":"2024-03-29T04:10:05.910015+0000","last_became_active":"2024-03-29T04:10:00.547419+0000","last_became_peered":"2024-03-29T04:10:00.547419+0000","last_unstale":"2024-03-29T04:10:05.910015+0000","last_undegraded":"2024-03-29T04:10:05.910015+0000","last_fullsized":"2024-03-29T04:10:05.910015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:46.707085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912399+0000","last_change":"2024-03-29T04:09:59.544306+0000","last_active":"2024-03-29T04:10:05.912399+0000","last_peered":"2024-03-29T04:10:05.912399+0000","last_clean":"2024-03-29T04:10:05.912399+0000","last_became_active":"2024-03-29T04:09:59.544028+0000","last_became_peered":"2024-03-29T04:09:59.544028+0000","last_unstale":"2024-03-29T04:10:05.912399+0000","last_undegraded":"2024-03-29T04:10:05.912399+0000","last_fullsized":"2024-03-29T04:10:05.912399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:33:50.904931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.558045+0000","last_change":"2024-03-29T04:10:04.577178+0000","last_active":"2024-03-29T04:10:05.558045+0000","last_peered":"2024-03-29T04:10:05.558045+0000","last_clean":"2024-03-29T04:10:05.558045+0000","last_became_active":"2024-03-29T04:10:04.576788+0000","last_became_peered":"2024-03-29T04:10:04.576788+0000","last_unstale":"2024-03-29T04:10:05.558045+0000","last_undegraded":"2024-03-29T04:10:05.558045+0000","last_fullsized":"2024-03-29T04:10:05.558045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:26:14.412794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910704+0000","last_change":"2024-03-29T04:10:00.543623+0000","last_active":"2024-03-29T04:10:05.910704+0000","last_peered":"2024-03-29T04:10:05.910704+0000","last_clean":"2024-03-29T04:10:05.910704+0000","last_became_active":"2024-03-29T04:10:00.542670+0000","last_became_peered":"2024-03-29T04:10:00.542670+0000","last_unstale":"2024-03-29T04:10:05.910704+0000","last_undegraded":"2024-03-29T04:10:05.910704+0000","last_fullsized":"2024-03-29T04:10:05.910704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:48.213845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910045+0000","last_change":"2024-03-29T04:09:59.540613+0000","last_active":"2024-03-29T04:10:05.910045+0000","last_peered":"2024-03-29T04:10:05.910045+0000","last_clean":"2024-03-29T04:10:05.910045+0000","last_became_active":"2024-03-29T04:09:59.540340+0000","last_became_peered":"2024-03-29T04:09:59.540340+0000","last_unstale":"2024-03-29T04:10:05.910045+0000","last_undegraded":"2024-03-29T04:10:05.910045+0000","last_fullsized":"2024-03-29T04:10:05.910045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:05.307537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911330+0000","last_change":"2024-03-29T04:10:04.646383+0000","last_active":"2024-03-29T04:10:05.911330+0000","last_peered":"2024-03-29T04:10:05.911330+0000","last_clean":"2024-03-29T04:10:05.911330+0000","last_became_active":"2024-03-29T04:10:04.646046+0000","last_became_peered":"2024-03-29T04:10:04.646046+0000","last_unstale":"2024-03-29T04:10:05.911330+0000","last_undegraded":"2024-03-29T04:10:05.911330+0000","last_fullsized":"2024-03-29T04:10:05.911330+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:17.034970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557229+0000","last_change":"2024-03-29T04:10:00.537777+0000","last_active":"2024-03-29T04:10:05.557229+0000","last_peered":"2024-03-29T04:10:05.557229+0000","last_clean":"2024-03-29T04:10:05.557229+0000","last_became_active":"2024-03-29T04:10:00.537423+0000","last_became_peered":"2024-03-29T04:10:00.537423+0000","last_unstale":"2024-03-29T04:10:05.557229+0000","last_undegraded":"2024-03-29T04:10:05.557229+0000","last_fullsized":"2024-03-29T04:10:05.557229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:33:51.909341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910053+0000","last_change":"2024-03-29T04:09:59.543268+0000","last_active":"2024-03-29T04:10:05.910053+0000","last_peered":"2024-03-29T04:10:05.910053+0000","last_clean":"2024-03-29T04:10:05.910053+0000","last_became_active":"2024-03-29T04:09:59.542982+0000","last_became_peered":"2024-03-29T04:09:59.542982+0000","last_unstale":"2024-03-29T04:10:05.910053+0000","last_undegraded":"2024-03-29T04:10:05.910053+0000","last_fullsized":"2024-03-29T04:10:05.910053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:01.925223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557817+0000","last_change":"2024-03-29T04:10:04.577232+0000","last_active":"2024-03-29T04:10:05.557817+0000","last_peered":"2024-03-29T04:10:05.557817+0000","last_clean":"2024-03-29T04:10:05.557817+0000","last_became_active":"2024-03-29T04:10:04.576758+0000","last_became_peered":"2024-03-29T04:10:04.576758+0000","last_unstale":"2024-03-29T04:10:05.557817+0000","last_undegraded":"2024-03-29T04:10:05.557817+0000","last_fullsized":"2024-03-29T04:10:05.557817+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:26:08.662188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912635+0000","last_change":"2024-03-29T04:10:00.546235+0000","last_active":"2024-03-29T04:10:05.912635+0000","last_peered":"2024-03-29T04:10:05.912635+0000","last_clean":"2024-03-29T04:10:05.912635+0000","last_became_active":"2024-03-29T04:10:00.545908+0000","last_became_peered":"2024-03-29T04:10:00.545908+0000","last_unstale":"2024-03-29T04:10:05.912635+0000","last_undegraded":"2024-03-29T04:10:05.912635+0000","last_fullsized":"2024-03-29T04:10:05.912635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:16:02.331457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911074+0000","last_change":"2024-03-29T04:09:59.541657+0000","last_active":"2024-03-29T04:10:05.911074+0000","last_peered":"2024-03-29T04:10:05.911074+0000","last_clean":"2024-03-29T04:10:05.911074+0000","last_became_active":"2024-03-29T04:09:59.541305+0000","last_became_peered":"2024-03-29T04:09:59.541305+0000","last_unstale":"2024-03-29T04:10:05.911074+0000","last_undegraded":"2024-03-29T04:10:05.911074+0000","last_fullsized":"2024-03-29T04:10:05.911074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:25:06.123311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556981+0000","last_change":"2024-03-29T04:10:04.646174+0000","last_active":"2024-03-29T04:10:05.556981+0000","last_peered":"2024-03-29T04:10:05.556981+0000","last_clean":"2024-03-29T04:10:05.556981+0000","last_became_active":"2024-03-29T04:10:04.645802+0000","last_became_peered":"2024-03-29T04:10:04.645802+0000","last_unstale":"2024-03-29T04:10:05.556981+0000","last_undegraded":"2024-03-29T04:10:05.556981+0000","last_fullsized":"2024-03-29T04:10:05.556981+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:35:04.485257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911614+0000","last_change":"2024-03-29T04:10:00.543254+0000","last_active":"2024-03-29T04:10:05.911614+0000","last_peered":"2024-03-29T04:10:05.911614+0000","last_clean":"2024-03-29T04:10:05.911614+0000","last_became_active":"2024-03-29T04:10:00.542958+0000","last_became_peered":"2024-03-29T04:10:00.542958+0000","last_unstale":"2024-03-29T04:10:05.911614+0000","last_undegraded":"2024-03-29T04:10:05.911614+0000","last_fullsized":"2024-03-29T04:10:05.911614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:00:51.320824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911924+0000","last_change":"2024-03-29T04:09:59.544320+0000","last_active":"2024-03-29T04:10:05.911924+0000","last_peered":"2024-03-29T04:10:05.911924+0000","last_clean":"2024-03-29T04:10:05.911924+0000","last_became_active":"2024-03-29T04:09:59.543928+0000","last_became_peered":"2024-03-29T04:09:59.543928+0000","last_unstale":"2024-03-29T04:10:05.911924+0000","last_undegraded":"2024-03-29T04:10:05.911924+0000","last_fullsized":"2024-03-29T04:10:05.911924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:02.879344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910274+0000","last_change":"2024-03-29T04:10:04.646905+0000","last_active":"2024-03-29T04:10:05.910274+0000","last_peered":"2024-03-29T04:10:05.910274+0000","last_clean":"2024-03-29T04:10:05.910274+0000","last_became_active":"2024-03-29T04:10:04.646482+0000","last_became_peered":"2024-03-29T04:10:04.646482+0000","last_unstale":"2024-03-29T04:10:05.910274+0000","last_undegraded":"2024-03-29T04:10:05.910274+0000","last_fullsized":"2024-03-29T04:10:05.910274+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:38:53.191487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910879+0000","last_change":"2024-03-29T04:10:00.541465+0000","last_active":"2024-03-29T04:10:05.910879+0000","last_peered":"2024-03-29T04:10:05.910879+0000","last_clean":"2024-03-29T04:10:05.910879+0000","last_became_active":"2024-03-29T04:10:00.541197+0000","last_became_peered":"2024-03-29T04:10:00.541197+0000","last_unstale":"2024-03-29T04:10:05.910879+0000","last_undegraded":"2024-03-29T04:10:05.910879+0000","last_fullsized":"2024-03-29T04:10:05.910879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:52:12.957170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911673+0000","last_change":"2024-03-29T04:09:59.545759+0000","last_active":"2024-03-29T04:10:05.911673+0000","last_peered":"2024-03-29T04:10:05.911673+0000","last_clean":"2024-03-29T04:10:05.911673+0000","last_became_active":"2024-03-29T04:09:59.545373+0000","last_became_peered":"2024-03-29T04:09:59.545373+0000","last_unstale":"2024-03-29T04:10:05.911673+0000","last_undegraded":"2024-03-29T04:10:05.911673+0000","last_fullsized":"2024-03-29T04:10:05.911673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:58:10.121554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911095+0000","last_change":"2024-03-29T04:10:04.644123+0000","last_active":"2024-03-29T04:10:05.911095+0000","last_peered":"2024-03-29T04:10:05.911095+0000","last_clean":"2024-03-29T04:10:05.911095+0000","last_became_active":"2024-03-29T04:10:04.643701+0000","last_became_peered":"2024-03-29T04:10:04.643701+0000","last_unstale":"2024-03-29T04:10:05.911095+0000","last_undegraded":"2024-03-29T04:10:05.911095+0000","last_fullsized":"2024-03-29T04:10:05.911095+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:26:14.412794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557781+0000","last_change":"2024-03-29T04:10:00.543518+0000","last_active":"2024-03-29T04:10:05.557781+0000","last_peered":"2024-03-29T04:10:05.557781+0000","last_clean":"2024-03-29T04:10:05.557781+0000","last_became_active":"2024-03-29T04:10:00.542373+0000","last_became_peered":"2024-03-29T04:10:00.542373+0000","last_unstale":"2024-03-29T04:10:05.557781+0000","last_undegraded":"2024-03-29T04:10:05.557781+0000","last_fullsized":"2024-03-29T04:10:05.557781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:02.966428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910966+0000","last_change":"2024-03-29T04:09:59.542356+0000","last_active":"2024-03-29T04:10:05.910966+0000","last_peered":"2024-03-29T04:10:05.910966+0000","last_clean":"2024-03-29T04:10:05.910966+0000","last_became_active":"2024-03-29T04:09:59.541917+0000","last_became_peered":"2024-03-29T04:09:59.541917+0000","last_unstale":"2024-03-29T04:10:05.910966+0000","last_undegraded":"2024-03-29T04:10:05.910966+0000","last_fullsized":"2024-03-29T04:10:05.910966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:01.925223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912224+0000","last_change":"2024-03-29T04:10:04.578218+0000","last_active":"2024-03-29T04:10:05.912224+0000","last_peered":"2024-03-29T04:10:05.912224+0000","last_clean":"2024-03-29T04:10:05.912224+0000","last_became_active":"2024-03-29T04:10:04.577221+0000","last_became_peered":"2024-03-29T04:10:04.577221+0000","last_unstale":"2024-03-29T04:10:05.912224+0000","last_undegraded":"2024-03-29T04:10:05.912224+0000","last_fullsized":"2024-03-29T04:10:05.912224+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:44:17.161094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.558130+0000","last_change":"2024-03-29T04:10:00.536176+0000","last_active":"2024-03-29T04:10:05.558130+0000","last_peered":"2024-03-29T04:10:05.558130+0000","last_clean":"2024-03-29T04:10:05.558130+0000","last_became_active":"2024-03-29T04:10:00.535854+0000","last_became_peered":"2024-03-29T04:10:00.535854+0000","last_unstale":"2024-03-29T04:10:05.558130+0000","last_undegraded":"2024-03-29T04:10:05.558130+0000","last_fullsized":"2024-03-29T04:10:05.558130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:25:07.127721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557572+0000","last_change":"2024-03-29T04:09:59.542230+0000","last_active":"2024-03-29T04:10:05.557572+0000","last_peered":"2024-03-29T04:10:05.557572+0000","last_clean":"2024-03-29T04:10:05.557572+0000","last_became_active":"2024-03-29T04:09:59.541972+0000","last_became_peered":"2024-03-29T04:09:59.541972+0000","last_unstale":"2024-03-29T04:10:05.557572+0000","last_undegraded":"2024-03-29T04:10:05.557572+0000","last_fullsized":"2024-03-29T04:10:05.557572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:01.925223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910829+0000","last_change":"2024-03-29T04:10:04.574263+0000","last_active":"2024-03-29T04:10:05.910829+0000","last_peered":"2024-03-29T04:10:05.910829+0000","last_clean":"2024-03-29T04:10:05.910829+0000","last_became_active":"2024-03-29T04:10:04.573843+0000","last_became_peered":"2024-03-29T04:10:04.573843+0000","last_unstale":"2024-03-29T04:10:05.910829+0000","last_undegraded":"2024-03-29T04:10:05.910829+0000","last_fullsized":"2024-03-29T04:10:05.910829+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:06.998903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912194+0000","last_change":"2024-03-29T04:10:00.540824+0000","last_active":"2024-03-29T04:10:05.912194+0000","last_peered":"2024-03-29T04:10:05.912194+0000","last_clean":"2024-03-29T04:10:05.912194+0000","last_became_active":"2024-03-29T04:10:00.540505+0000","last_became_peered":"2024-03-29T04:10:00.540505+0000","last_unstale":"2024-03-29T04:10:05.912194+0000","last_undegraded":"2024-03-29T04:10:05.912194+0000","last_fullsized":"2024-03-29T04:10:05.912194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:41.517875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.559855+0000","last_change":"2024-03-29T04:09:59.540415+0000","last_active":"2024-03-29T04:10:05.559855+0000","last_peered":"2024-03-29T04:10:05.559855+0000","last_clean":"2024-03-29T04:10:05.559855+0000","last_became_active":"2024-03-29T04:09:59.540130+0000","last_became_peered":"2024-03-29T04:09:59.540130+0000","last_unstale":"2024-03-29T04:10:05.559855+0000","last_undegraded":"2024-03-29T04:10:05.559855+0000","last_fullsized":"2024-03-29T04:10:05.559855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:05.307537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911492+0000","last_change":"2024-03-29T04:10:04.575156+0000","last_active":"2024-03-29T04:10:05.911492+0000","last_peered":"2024-03-29T04:10:05.911492+0000","last_clean":"2024-03-29T04:10:05.911492+0000","last_became_active":"2024-03-29T04:10:04.573816+0000","last_became_peered":"2024-03-29T04:10:04.573816+0000","last_unstale":"2024-03-29T04:10:05.911492+0000","last_undegraded":"2024-03-29T04:10:05.911492+0000","last_fullsized":"2024-03-29T04:10:05.911492+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:23:51.029457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.909950+0000","last_change":"2024-03-29T04:10:00.531739+0000","last_active":"2024-03-29T04:10:05.909950+0000","last_peered":"2024-03-29T04:10:05.909950+0000","last_clean":"2024-03-29T04:10:05.909950+0000","last_became_active":"2024-03-29T04:10:00.531395+0000","last_became_peered":"2024-03-29T04:10:00.531395+0000","last_unstale":"2024-03-29T04:10:05.909950+0000","last_undegraded":"2024-03-29T04:10:05.909950+0000","last_fullsized":"2024-03-29T04:10:05.909950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:02:32.544769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910076+0000","last_change":"2024-03-29T04:09:59.541400+0000","last_active":"2024-03-29T04:10:05.910076+0000","last_peered":"2024-03-29T04:10:05.910076+0000","last_clean":"2024-03-29T04:10:05.910076+0000","last_became_active":"2024-03-29T04:09:59.541114+0000","last_became_peered":"2024-03-29T04:09:59.541114+0000","last_unstale":"2024-03-29T04:10:05.910076+0000","last_undegraded":"2024-03-29T04:10:05.910076+0000","last_fullsized":"2024-03-29T04:10:05.910076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:25:06.123311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912614+0000","last_change":"2024-03-29T04:10:04.645661+0000","last_active":"2024-03-29T04:10:05.912614+0000","last_peered":"2024-03-29T04:10:05.912614+0000","last_clean":"2024-03-29T04:10:05.912614+0000","last_became_active":"2024-03-29T04:10:04.645258+0000","last_became_peered":"2024-03-29T04:10:04.645258+0000","last_unstale":"2024-03-29T04:10:05.912614+0000","last_undegraded":"2024-03-29T04:10:05.912614+0000","last_fullsized":"2024-03-29T04:10:05.912614+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:44:17.161094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910699+0000","last_change":"2024-03-29T04:10:00.537081+0000","last_active":"2024-03-29T04:10:05.910699+0000","last_peered":"2024-03-29T04:10:05.910699+0000","last_clean":"2024-03-29T04:10:05.910699+0000","last_became_active":"2024-03-29T04:10:00.536267+0000","last_became_peered":"2024-03-29T04:10:00.536267+0000","last_unstale":"2024-03-29T04:10:05.910699+0000","last_undegraded":"2024-03-29T04:10:05.910699+0000","last_fullsized":"2024-03-29T04:10:05.910699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:41.517875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557767+0000","last_change":"2024-03-29T04:09:59.535631+0000","last_active":"2024-03-29T04:10:05.557767+0000","last_peered":"2024-03-29T04:10:05.557767+0000","last_clean":"2024-03-29T04:10:05.557767+0000","last_became_active":"2024-03-29T04:09:59.535323+0000","last_became_peered":"2024-03-29T04:09:59.535323+0000","last_unstale":"2024-03-29T04:10:05.557767+0000","last_undegraded":"2024-03-29T04:10:05.557767+0000","last_fullsized":"2024-03-29T04:10:05.557767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:05.307537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910183+0000","last_change":"2024-03-29T04:09:49.002570+0000","last_active":"2024-03-29T04:10:05.910183+0000","last_peered":"2024-03-29T04:10:05.910183+0000","last_clean":"2024-03-29T04:10:05.910183+0000","last_became_active":"2024-03-29T04:09:49.002235+0000","last_became_peered":"2024-03-29T04:09:49.002235+0000","last_unstale":"2024-03-29T04:10:05.910183+0000","last_undegraded":"2024-03-29T04:10:05.910183+0000","last_fullsized":"2024-03-29T04:10:05.910183+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:47.900733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:47.900733+0000","last_clean_scrub_stamp":"2024-03-29T04:09:47.900733+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:44:04.608265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911029+0000","last_change":"2024-03-29T04:10:04.646698+0000","last_active":"2024-03-29T04:10:05.911029+0000","last_peered":"2024-03-29T04:10:05.911029+0000","last_clean":"2024-03-29T04:10:05.911029+0000","last_became_active":"2024-03-29T04:10:04.646221+0000","last_became_peered":"2024-03-29T04:10:04.646221+0000","last_unstale":"2024-03-29T04:10:05.911029+0000","last_undegraded":"2024-03-29T04:10:05.911029+0000","last_fullsized":"2024-03-29T04:10:05.911029+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:17.034970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911371+0000","last_change":"2024-03-29T04:10:00.540055+0000","last_active":"2024-03-29T04:10:05.911371+0000","last_peered":"2024-03-29T04:10:05.911371+0000","last_clean":"2024-03-29T04:10:05.911371+0000","last_became_active":"2024-03-29T04:10:00.539766+0000","last_became_peered":"2024-03-29T04:10:00.539766+0000","last_unstale":"2024-03-29T04:10:05.911371+0000","last_undegraded":"2024-03-29T04:10:05.911371+0000","last_fullsized":"2024-03-29T04:10:05.911371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:46.804154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.560157+0000","last_change":"2024-03-29T04:09:59.536860+0000","last_active":"2024-03-29T04:10:05.560157+0000","last_peered":"2024-03-29T04:10:05.560157+0000","last_clean":"2024-03-29T04:10:05.560157+0000","last_became_active":"2024-03-29T04:09:59.536504+0000","last_became_peered":"2024-03-29T04:09:59.536504+0000","last_unstale":"2024-03-29T04:10:05.560157+0000","last_undegraded":"2024-03-29T04:10:05.560157+0000","last_fullsized":"2024-03-29T04:10:05.560157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:02.879344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910369+0000","last_change":"2024-03-29T04:10:04.644446+0000","last_active":"2024-03-29T04:10:05.910369+0000","last_peered":"2024-03-29T04:10:05.910369+0000","last_clean":"2024-03-29T04:10:05.910369+0000","last_became_active":"2024-03-29T04:10:04.643982+0000","last_became_peered":"2024-03-29T04:10:04.643982+0000","last_unstale":"2024-03-29T04:10:05.910369+0000","last_undegraded":"2024-03-29T04:10:05.910369+0000","last_fullsized":"2024-03-29T04:10:05.910369+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:26:14.412794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556955+0000","last_change":"2024-03-29T04:10:00.544801+0000","last_active":"2024-03-29T04:10:05.556955+0000","last_peered":"2024-03-29T04:10:05.556955+0000","last_clean":"2024-03-29T04:10:05.556955+0000","last_became_active":"2024-03-29T04:10:00.544481+0000","last_became_peered":"2024-03-29T04:10:00.544481+0000","last_unstale":"2024-03-29T04:10:05.556955+0000","last_undegraded":"2024-03-29T04:10:05.556955+0000","last_fullsized":"2024-03-29T04:10:05.556955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:00:51.320824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911910+0000","last_change":"2024-03-29T04:09:59.541406+0000","last_active":"2024-03-29T04:10:05.911910+0000","last_peered":"2024-03-29T04:10:05.911910+0000","last_clean":"2024-03-29T04:10:05.911910+0000","last_became_active":"2024-03-29T04:09:59.541099+0000","last_became_peered":"2024-03-29T04:09:59.541099+0000","last_unstale":"2024-03-29T04:10:05.911910+0000","last_undegraded":"2024-03-29T04:10:05.911910+0000","last_fullsized":"2024-03-29T04:10:05.911910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:25:06.123311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911325+0000","last_change":"2024-03-29T04:10:04.579423+0000","last_active":"2024-03-29T04:10:05.911325+0000","last_peered":"2024-03-29T04:10:05.911325+0000","last_clean":"2024-03-29T04:10:05.911325+0000","last_became_active":"2024-03-29T04:10:04.579062+0000","last_became_peered":"2024-03-29T04:10:04.579062+0000","last_unstale":"2024-03-29T04:10:05.911325+0000","last_undegraded":"2024-03-29T04:10:05.911325+0000","last_fullsized":"2024-03-29T04:10:05.911325+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:06.998903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911863+0000","last_change":"2024-03-29T04:10:00.547282+0000","last_active":"2024-03-29T04:10:05.911863+0000","last_peered":"2024-03-29T04:10:05.911863+0000","last_clean":"2024-03-29T04:10:05.911863+0000","last_became_active":"2024-03-29T04:10:00.547026+0000","last_became_peered":"2024-03-29T04:10:00.547026+0000","last_unstale":"2024-03-29T04:10:05.911863+0000","last_undegraded":"2024-03-29T04:10:05.911863+0000","last_fullsized":"2024-03-29T04:10:05.911863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:34:06.367730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.560063+0000","last_change":"2024-03-29T04:09:59.540515+0000","last_active":"2024-03-29T04:10:05.560063+0000","last_peered":"2024-03-29T04:10:05.560063+0000","last_clean":"2024-03-29T04:10:05.560063+0000","last_became_active":"2024-03-29T04:09:59.540198+0000","last_became_peered":"2024-03-29T04:09:59.540198+0000","last_unstale":"2024-03-29T04:10:05.560063+0000","last_undegraded":"2024-03-29T04:10:05.560063+0000","last_fullsized":"2024-03-29T04:10:05.560063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:01.925223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556468+0000","last_change":"2024-03-29T04:10:04.644258+0000","last_active":"2024-03-29T04:10:05.556468+0000","last_peered":"2024-03-29T04:10:05.556468+0000","last_clean":"2024-03-29T04:10:05.556468+0000","last_became_active":"2024-03-29T04:10:04.643890+0000","last_became_peered":"2024-03-29T04:10:04.643890+0000","last_unstale":"2024-03-29T04:10:05.556468+0000","last_undegraded":"2024-03-29T04:10:05.556468+0000","last_fullsized":"2024-03-29T04:10:05.556468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:46:14.769141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911624+0000","last_change":"2024-03-29T04:10:00.549814+0000","last_active":"2024-03-29T04:10:05.911624+0000","last_peered":"2024-03-29T04:10:05.911624+0000","last_clean":"2024-03-29T04:10:05.911624+0000","last_became_active":"2024-03-29T04:10:00.549571+0000","last_became_peered":"2024-03-29T04:10:00.549571+0000","last_unstale":"2024-03-29T04:10:05.911624+0000","last_undegraded":"2024-03-29T04:10:05.911624+0000","last_fullsized":"2024-03-29T04:10:05.911624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:00:51.320824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557978+0000","last_change":"2024-03-29T04:09:59.538979+0000","last_active":"2024-03-29T04:10:05.557978+0000","last_peered":"2024-03-29T04:10:05.557978+0000","last_clean":"2024-03-29T04:10:05.557978+0000","last_became_active":"2024-03-29T04:09:59.538707+0000","last_became_peered":"2024-03-29T04:09:59.538707+0000","last_unstale":"2024-03-29T04:10:05.557978+0000","last_undegraded":"2024-03-29T04:10:05.557978+0000","last_fullsized":"2024-03-29T04:10:05.557978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:07:35.127071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556613+0000","last_change":"2024-03-29T04:10:04.574569+0000","last_active":"2024-03-29T04:10:05.556613+0000","last_peered":"2024-03-29T04:10:05.556613+0000","last_clean":"2024-03-29T04:10:05.556613+0000","last_became_active":"2024-03-29T04:10:04.574048+0000","last_became_peered":"2024-03-29T04:10:04.574048+0000","last_unstale":"2024-03-29T04:10:05.556613+0000","last_undegraded":"2024-03-29T04:10:05.556613+0000","last_fullsized":"2024-03-29T04:10:05.556613+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:33.266010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910658+0000","last_change":"2024-03-29T04:10:00.549205+0000","last_active":"2024-03-29T04:10:05.910658+0000","last_peered":"2024-03-29T04:10:05.910658+0000","last_clean":"2024-03-29T04:10:05.910658+0000","last_became_active":"2024-03-29T04:10:00.548863+0000","last_became_peered":"2024-03-29T04:10:00.548863+0000","last_unstale":"2024-03-29T04:10:05.910658+0000","last_undegraded":"2024-03-29T04:10:05.910658+0000","last_fullsized":"2024-03-29T04:10:05.910658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:30.096622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.559751+0000","last_change":"2024-03-29T04:09:59.538016+0000","last_active":"2024-03-29T04:10:05.559751+0000","last_peered":"2024-03-29T04:10:05.559751+0000","last_clean":"2024-03-29T04:10:05.559751+0000","last_became_active":"2024-03-29T04:09:59.537671+0000","last_became_peered":"2024-03-29T04:09:59.537671+0000","last_unstale":"2024-03-29T04:10:05.559751+0000","last_undegraded":"2024-03-29T04:10:05.559751+0000","last_fullsized":"2024-03-29T04:10:05.559751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:25:00.672418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912466+0000","last_change":"2024-03-29T04:10:04.574244+0000","last_active":"2024-03-29T04:10:05.912466+0000","last_peered":"2024-03-29T04:10:05.912466+0000","last_clean":"2024-03-29T04:10:05.912466+0000","last_became_active":"2024-03-29T04:10:04.573865+0000","last_became_peered":"2024-03-29T04:10:04.573865+0000","last_unstale":"2024-03-29T04:10:05.912466+0000","last_undegraded":"2024-03-29T04:10:05.912466+0000","last_fullsized":"2024-03-29T04:10:05.912466+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:23:51.029457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910228+0000","last_change":"2024-03-29T04:10:00.547804+0000","last_active":"2024-03-29T04:10:05.910228+0000","last_peered":"2024-03-29T04:10:05.910228+0000","last_clean":"2024-03-29T04:10:05.910228+0000","last_became_active":"2024-03-29T04:10:00.547500+0000","last_became_peered":"2024-03-29T04:10:00.547500+0000","last_unstale":"2024-03-29T04:10:05.910228+0000","last_undegraded":"2024-03-29T04:10:05.910228+0000","last_fullsized":"2024-03-29T04:10:05.910228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:52:12.957170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910669+0000","last_change":"2024-03-29T04:09:59.538286+0000","last_active":"2024-03-29T04:10:05.910669+0000","last_peered":"2024-03-29T04:10:05.910669+0000","last_clean":"2024-03-29T04:10:05.910669+0000","last_became_active":"2024-03-29T04:09:59.537970+0000","last_became_peered":"2024-03-29T04:09:59.537970+0000","last_unstale":"2024-03-29T04:10:05.910669+0000","last_undegraded":"2024-03-29T04:10:05.910669+0000","last_fullsized":"2024-03-29T04:10:05.910669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:05.307537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911473+0000","last_change":"2024-03-29T04:10:04.647055+0000","last_active":"2024-03-29T04:10:05.911473+0000","last_peered":"2024-03-29T04:10:05.911473+0000","last_clean":"2024-03-29T04:10:05.911473+0000","last_became_active":"2024-03-29T04:10:04.646604+0000","last_became_peered":"2024-03-29T04:10:04.646604+0000","last_unstale":"2024-03-29T04:10:05.911473+0000","last_undegraded":"2024-03-29T04:10:05.911473+0000","last_fullsized":"2024-03-29T04:10:05.911473+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:32:59.580503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911224+0000","last_change":"2024-03-29T04:10:00.548354+0000","last_active":"2024-03-29T04:10:05.911224+0000","last_peered":"2024-03-29T04:10:05.911224+0000","last_clean":"2024-03-29T04:10:05.911224+0000","last_became_active":"2024-03-29T04:10:00.548084+0000","last_became_peered":"2024-03-29T04:10:00.548084+0000","last_unstale":"2024-03-29T04:10:05.911224+0000","last_undegraded":"2024-03-29T04:10:05.911224+0000","last_fullsized":"2024-03-29T04:10:05.911224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:02.966428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557470+0000","last_change":"2024-03-29T04:09:59.538823+0000","last_active":"2024-03-29T04:10:05.557470+0000","last_peered":"2024-03-29T04:10:05.557470+0000","last_clean":"2024-03-29T04:10:05.557470+0000","last_became_active":"2024-03-29T04:09:59.538570+0000","last_became_peered":"2024-03-29T04:09:59.538570+0000","last_unstale":"2024-03-29T04:10:05.557470+0000","last_undegraded":"2024-03-29T04:10:05.557470+0000","last_fullsized":"2024-03-29T04:10:05.557470+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:02.879344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911403+0000","last_change":"2024-03-29T04:10:04.644206+0000","last_active":"2024-03-29T04:10:05.911403+0000","last_peered":"2024-03-29T04:10:05.911403+0000","last_clean":"2024-03-29T04:10:05.911403+0000","last_became_active":"2024-03-29T04:10:04.643690+0000","last_became_peered":"2024-03-29T04:10:04.643690+0000","last_unstale":"2024-03-29T04:10:05.911403+0000","last_undegraded":"2024-03-29T04:10:05.911403+0000","last_fullsized":"2024-03-29T04:10:05.911403+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:10:07.163684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910609+0000","last_change":"2024-03-29T04:10:00.541166+0000","last_active":"2024-03-29T04:10:05.910609+0000","last_peered":"2024-03-29T04:10:05.910609+0000","last_clean":"2024-03-29T04:10:05.910609+0000","last_became_active":"2024-03-29T04:10:00.540895+0000","last_became_peered":"2024-03-29T04:10:00.540895+0000","last_unstale":"2024-03-29T04:10:05.910609+0000","last_undegraded":"2024-03-29T04:10:05.910609+0000","last_fullsized":"2024-03-29T04:10:05.910609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:21:11.329145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911540+0000","last_change":"2024-03-29T04:09:59.538433+0000","last_active":"2024-03-29T04:10:05.911540+0000","last_peered":"2024-03-29T04:10:05.911540+0000","last_clean":"2024-03-29T04:10:05.911540+0000","last_became_active":"2024-03-29T04:09:59.538039+0000","last_became_peered":"2024-03-29T04:09:59.538039+0000","last_unstale":"2024-03-29T04:10:05.911540+0000","last_undegraded":"2024-03-29T04:10:05.911540+0000","last_fullsized":"2024-03-29T04:10:05.911540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:45:51.875624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557426+0000","last_change":"2024-03-29T04:10:04.647019+0000","last_active":"2024-03-29T04:10:05.557426+0000","last_peered":"2024-03-29T04:10:05.557426+0000","last_clean":"2024-03-29T04:10:05.557426+0000","last_became_active":"2024-03-29T04:10:04.646642+0000","last_became_peered":"2024-03-29T04:10:04.646642+0000","last_unstale":"2024-03-29T04:10:05.557426+0000","last_undegraded":"2024-03-29T04:10:05.557426+0000","last_fullsized":"2024-03-29T04:10:05.557426+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:07:56.606322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911280+0000","last_change":"2024-03-29T04:10:00.544134+0000","last_active":"2024-03-29T04:10:05.911280+0000","last_peered":"2024-03-29T04:10:05.911280+0000","last_clean":"2024-03-29T04:10:05.911280+0000","last_became_active":"2024-03-29T04:10:00.543534+0000","last_became_peered":"2024-03-29T04:10:00.543534+0000","last_unstale":"2024-03-29T04:10:05.911280+0000","last_undegraded":"2024-03-29T04:10:05.911280+0000","last_fullsized":"2024-03-29T04:10:05.911280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:26:04.629712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:10.817289+0000","last_change":"2024-03-29T04:09:59.544458+0000","last_active":"2024-03-29T04:10:10.817289+0000","last_peered":"2024-03-29T04:10:10.817289+0000","last_clean":"2024-03-29T04:10:10.817289+0000","last_became_active":"2024-03-29T04:09:59.544137+0000","last_became_peered":"2024-03-29T04:09:59.544137+0000","last_unstale":"2024-03-29T04:10:10.817289+0000","last_undegraded":"2024-03-29T04:10:10.817289+0000","last_fullsized":"2024-03-29T04:10:10.817289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:02.879344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912395+0000","last_change":"2024-03-29T04:10:04.576867+0000","last_active":"2024-03-29T04:10:05.912395+0000","last_peered":"2024-03-29T04:10:05.912395+0000","last_clean":"2024-03-29T04:10:05.912395+0000","last_became_active":"2024-03-29T04:10:04.576416+0000","last_became_peered":"2024-03-29T04:10:04.576416+0000","last_unstale":"2024-03-29T04:10:05.912395+0000","last_undegraded":"2024-03-29T04:10:05.912395+0000","last_fullsized":"2024-03-29T04:10:05.912395+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:34:10.400206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.558230+0000","last_change":"2024-03-29T04:10:00.547277+0000","last_active":"2024-03-29T04:10:05.558230+0000","last_peered":"2024-03-29T04:10:05.558230+0000","last_clean":"2024-03-29T04:10:05.558230+0000","last_became_active":"2024-03-29T04:10:00.547024+0000","last_became_peered":"2024-03-29T04:10:00.547024+0000","last_unstale":"2024-03-29T04:10:05.558230+0000","last_undegraded":"2024-03-29T04:10:05.558230+0000","last_fullsized":"2024-03-29T04:10:05.558230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:58:11.125964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911813+0000","last_change":"2024-03-29T04:09:59.541483+0000","last_active":"2024-03-29T04:10:05.911813+0000","last_peered":"2024-03-29T04:10:05.911813+0000","last_clean":"2024-03-29T04:10:05.911813+0000","last_became_active":"2024-03-29T04:09:59.541196+0000","last_became_peered":"2024-03-29T04:09:59.541196+0000","last_unstale":"2024-03-29T04:10:05.911813+0000","last_undegraded":"2024-03-29T04:10:05.911813+0000","last_fullsized":"2024-03-29T04:10:05.911813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:35:01.484589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557918+0000","last_change":"2024-03-29T04:10:04.574437+0000","last_active":"2024-03-29T04:10:05.557918+0000","last_peered":"2024-03-29T04:10:05.557918+0000","last_clean":"2024-03-29T04:10:05.557918+0000","last_became_active":"2024-03-29T04:10:04.573898+0000","last_became_peered":"2024-03-29T04:10:04.573898+0000","last_unstale":"2024-03-29T04:10:05.557918+0000","last_undegraded":"2024-03-29T04:10:05.557918+0000","last_fullsized":"2024-03-29T04:10:05.557918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:50.836630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911733+0000","last_change":"2024-03-29T04:10:00.536536+0000","last_active":"2024-03-29T04:10:05.911733+0000","last_peered":"2024-03-29T04:10:05.911733+0000","last_clean":"2024-03-29T04:10:05.911733+0000","last_became_active":"2024-03-29T04:10:00.536254+0000","last_became_peered":"2024-03-29T04:10:00.536254+0000","last_unstale":"2024-03-29T04:10:05.911733+0000","last_undegraded":"2024-03-29T04:10:05.911733+0000","last_fullsized":"2024-03-29T04:10:05.911733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:48.213845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557997+0000","last_change":"2024-03-29T04:09:59.541406+0000","last_active":"2024-03-29T04:10:05.557997+0000","last_peered":"2024-03-29T04:10:05.557997+0000","last_clean":"2024-03-29T04:10:05.557997+0000","last_became_active":"2024-03-29T04:09:59.541101+0000","last_became_peered":"2024-03-29T04:09:59.541101+0000","last_unstale":"2024-03-29T04:10:05.557997+0000","last_undegraded":"2024-03-29T04:10:05.557997+0000","last_fullsized":"2024-03-29T04:10:05.557997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:25:06.123311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910592+0000","last_change":"2024-03-29T04:10:04.577680+0000","last_active":"2024-03-29T04:10:05.910592+0000","last_peered":"2024-03-29T04:10:05.910592+0000","last_clean":"2024-03-29T04:10:05.910592+0000","last_became_active":"2024-03-29T04:10:04.577291+0000","last_became_peered":"2024-03-29T04:10:04.577291+0000","last_unstale":"2024-03-29T04:10:05.910592+0000","last_undegraded":"2024-03-29T04:10:05.910592+0000","last_fullsized":"2024-03-29T04:10:05.910592+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:46:14.769141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557898+0000","last_change":"2024-03-29T04:10:00.542773+0000","last_active":"2024-03-29T04:10:05.557898+0000","last_peered":"2024-03-29T04:10:05.557898+0000","last_clean":"2024-03-29T04:10:05.557898+0000","last_became_active":"2024-03-29T04:10:00.542081+0000","last_became_peered":"2024-03-29T04:10:00.542081+0000","last_unstale":"2024-03-29T04:10:05.557898+0000","last_undegraded":"2024-03-29T04:10:05.557898+0000","last_fullsized":"2024-03-29T04:10:05.557898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:26:04.629712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910800+0000","last_change":"2024-03-29T04:09:59.543623+0000","last_active":"2024-03-29T04:10:05.910800+0000","last_peered":"2024-03-29T04:10:05.910800+0000","last_clean":"2024-03-29T04:10:05.910800+0000","last_became_active":"2024-03-29T04:09:59.542122+0000","last_became_peered":"2024-03-29T04:09:59.542122+0000","last_unstale":"2024-03-29T04:10:05.910800+0000","last_undegraded":"2024-03-29T04:10:05.910800+0000","last_fullsized":"2024-03-29T04:10:05.910800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:01.925223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911195+0000","last_change":"2024-03-29T04:10:04.648725+0000","last_active":"2024-03-29T04:10:05.911195+0000","last_peered":"2024-03-29T04:10:05.911195+0000","last_clean":"2024-03-29T04:10:05.911195+0000","last_became_active":"2024-03-29T04:10:04.648125+0000","last_became_peered":"2024-03-29T04:10:04.648125+0000","last_unstale":"2024-03-29T04:10:05.911195+0000","last_undegraded":"2024-03-29T04:10:05.911195+0000","last_fullsized":"2024-03-29T04:10:05.911195+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:16:31.571145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912466+0000","last_change":"2024-03-29T04:10:04.646516+0000","last_active":"2024-03-29T04:10:05.912466+0000","last_peered":"2024-03-29T04:10:05.912466+0000","last_clean":"2024-03-29T04:10:05.912466+0000","last_became_active":"2024-03-29T04:10:04.646118+0000","last_became_peered":"2024-03-29T04:10:04.646118+0000","last_unstale":"2024-03-29T04:10:05.912466+0000","last_undegraded":"2024-03-29T04:10:05.912466+0000","last_fullsized":"2024-03-29T04:10:05.912466+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:17.034970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.909860+0000","last_change":"2024-03-29T04:10:00.542208+0000","last_active":"2024-03-29T04:10:05.909860+0000","last_peered":"2024-03-29T04:10:05.909860+0000","last_clean":"2024-03-29T04:10:05.909860+0000","last_became_active":"2024-03-29T04:10:00.541927+0000","last_became_peered":"2024-03-29T04:10:00.541927+0000","last_unstale":"2024-03-29T04:10:05.909860+0000","last_undegraded":"2024-03-29T04:10:05.909860+0000","last_fullsized":"2024-03-29T04:10:05.909860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:16:02.331457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556546+0000","last_change":"2024-03-29T04:09:59.539257+0000","last_active":"2024-03-29T04:10:05.556546+0000","last_peered":"2024-03-29T04:10:05.556546+0000","last_clean":"2024-03-29T04:10:05.556546+0000","last_became_active":"2024-03-29T04:09:59.538298+0000","last_became_peered":"2024-03-29T04:09:59.538298+0000","last_unstale":"2024-03-29T04:10:05.556546+0000","last_undegraded":"2024-03-29T04:10:05.556546+0000","last_fullsized":"2024-03-29T04:10:05.556546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:33:50.904931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911024+0000","last_change":"2024-03-29T04:10:04.575619+0000","last_active":"2024-03-29T04:10:05.911024+0000","last_peered":"2024-03-29T04:10:05.911024+0000","last_clean":"2024-03-29T04:10:05.911024+0000","last_became_active":"2024-03-29T04:10:04.575138+0000","last_became_peered":"2024-03-29T04:10:04.575138+0000","last_unstale":"2024-03-29T04:10:05.911024+0000","last_undegraded":"2024-03-29T04:10:05.911024+0000","last_fullsized":"2024-03-29T04:10:05.911024+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:06.998903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910644+0000","last_change":"2024-03-29T04:10:00.543853+0000","last_active":"2024-03-29T04:10:05.910644+0000","last_peered":"2024-03-29T04:10:05.910644+0000","last_clean":"2024-03-29T04:10:05.910644+0000","last_became_active":"2024-03-29T04:10:00.543030+0000","last_became_peered":"2024-03-29T04:10:00.543030+0000","last_unstale":"2024-03-29T04:10:05.910644+0000","last_undegraded":"2024-03-29T04:10:05.910644+0000","last_fullsized":"2024-03-29T04:10:05.910644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:29.233534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911422+0000","last_change":"2024-03-29T04:09:59.537737+0000","last_active":"2024-03-29T04:10:05.911422+0000","last_peered":"2024-03-29T04:10:05.911422+0000","last_clean":"2024-03-29T04:10:05.911422+0000","last_became_active":"2024-03-29T04:09:59.537453+0000","last_became_peered":"2024-03-29T04:09:59.537453+0000","last_unstale":"2024-03-29T04:10:05.911422+0000","last_undegraded":"2024-03-29T04:10:05.911422+0000","last_fullsized":"2024-03-29T04:10:05.911422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:05.307537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910688+0000","last_change":"2024-03-29T04:09:59.538269+0000","last_active":"2024-03-29T04:10:05.910688+0000","last_peered":"2024-03-29T04:10:05.910688+0000","last_clean":"2024-03-29T04:10:05.910688+0000","last_became_active":"2024-03-29T04:09:59.537365+0000","last_became_peered":"2024-03-29T04:09:59.537365+0000","last_unstale":"2024-03-29T04:10:05.910688+0000","last_undegraded":"2024-03-29T04:10:05.910688+0000","last_fullsized":"2024-03-29T04:10:05.910688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:33:50.904931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911456+0000","last_change":"2024-03-29T04:10:00.532121+0000","last_active":"2024-03-29T04:10:05.911456+0000","last_peered":"2024-03-29T04:10:05.911456+0000","last_clean":"2024-03-29T04:10:05.911456+0000","last_became_active":"2024-03-29T04:10:00.531664+0000","last_became_peered":"2024-03-29T04:10:00.531664+0000","last_unstale":"2024-03-29T04:10:05.911456+0000","last_undegraded":"2024-03-29T04:10:05.911456+0000","last_fullsized":"2024-03-29T04:10:05.911456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:13:32.378332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.560130+0000","last_change":"2024-03-29T04:10:04.574558+0000","last_active":"2024-03-29T04:10:05.560130+0000","last_peered":"2024-03-29T04:10:05.560130+0000","last_clean":"2024-03-29T04:10:05.560130+0000","last_became_active":"2024-03-29T04:10:04.574156+0000","last_became_peered":"2024-03-29T04:10:04.574156+0000","last_unstale":"2024-03-29T04:10:05.560130+0000","last_undegraded":"2024-03-29T04:10:05.560130+0000","last_fullsized":"2024-03-29T04:10:05.560130+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:02:36.577244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910098+0000","last_change":"2024-03-29T04:09:59.536444+0000","last_active":"2024-03-29T04:10:05.910098+0000","last_peered":"2024-03-29T04:10:05.910098+0000","last_clean":"2024-03-29T04:10:05.910098+0000","last_became_active":"2024-03-29T04:09:59.536165+0000","last_became_peered":"2024-03-29T04:09:59.536165+0000","last_unstale":"2024-03-29T04:10:05.910098+0000","last_undegraded":"2024-03-29T04:10:05.910098+0000","last_fullsized":"2024-03-29T04:10:05.910098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:37:46.716233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.558124+0000","last_change":"2024-03-29T04:10:00.544569+0000","last_active":"2024-03-29T04:10:05.558124+0000","last_peered":"2024-03-29T04:10:05.558124+0000","last_clean":"2024-03-29T04:10:05.558124+0000","last_became_active":"2024-03-29T04:10:00.544290+0000","last_became_peered":"2024-03-29T04:10:00.544290+0000","last_unstale":"2024-03-29T04:10:05.558124+0000","last_undegraded":"2024-03-29T04:10:05.558124+0000","last_fullsized":"2024-03-29T04:10:05.558124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:02.929633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910585+0000","last_change":"2024-03-29T04:10:04.580538+0000","last_active":"2024-03-29T04:10:05.910585+0000","last_peered":"2024-03-29T04:10:05.910585+0000","last_clean":"2024-03-29T04:10:05.910585+0000","last_became_active":"2024-03-29T04:10:04.580221+0000","last_became_peered":"2024-03-29T04:10:04.580221+0000","last_unstale":"2024-03-29T04:10:05.910585+0000","last_undegraded":"2024-03-29T04:10:05.910585+0000","last_fullsized":"2024-03-29T04:10:05.910585+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:01:23.958242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910087+0000","last_change":"2024-03-29T04:10:00.546325+0000","last_active":"2024-03-29T04:10:05.910087+0000","last_peered":"2024-03-29T04:10:05.910087+0000","last_clean":"2024-03-29T04:10:05.910087+0000","last_became_active":"2024-03-29T04:10:00.545988+0000","last_became_peered":"2024-03-29T04:10:00.545988+0000","last_unstale":"2024-03-29T04:10:05.910087+0000","last_undegraded":"2024-03-29T04:10:05.910087+0000","last_fullsized":"2024-03-29T04:10:05.910087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:16:02.331457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.559836+0000","last_change":"2024-03-29T04:10:00.541221+0000","last_active":"2024-03-29T04:10:05.559836+0000","last_peered":"2024-03-29T04:10:05.559836+0000","last_clean":"2024-03-29T04:10:05.559836+0000","last_became_active":"2024-03-29T04:10:00.540946+0000","last_became_peered":"2024-03-29T04:10:00.540946+0000","last_unstale":"2024-03-29T04:10:05.559836+0000","last_undegraded":"2024-03-29T04:10:05.559836+0000","last_fullsized":"2024-03-29T04:10:05.559836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:35:02.488999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557358+0000","last_change":"2024-03-29T04:10:00.546304+0000","last_active":"2024-03-29T04:10:05.557358+0000","last_peered":"2024-03-29T04:10:05.557358+0000","last_clean":"2024-03-29T04:10:05.557358+0000","last_became_active":"2024-03-29T04:10:00.545529+0000","last_became_peered":"2024-03-29T04:10:00.545529+0000","last_unstale":"2024-03-29T04:10:05.557358+0000","last_undegraded":"2024-03-29T04:10:05.557358+0000","last_fullsized":"2024-03-29T04:10:05.557358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:44:13.128618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911371+0000","last_change":"2024-03-29T04:10:00.542033+0000","last_active":"2024-03-29T04:10:05.911371+0000","last_peered":"2024-03-29T04:10:05.911371+0000","last_clean":"2024-03-29T04:10:05.911371+0000","last_became_active":"2024-03-29T04:10:00.541764+0000","last_became_peered":"2024-03-29T04:10:00.541764+0000","last_unstale":"2024-03-29T04:10:05.911371+0000","last_undegraded":"2024-03-29T04:10:05.911371+0000","last_fullsized":"2024-03-29T04:10:05.911371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:46.707085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911616+0000","last_change":"2024-03-29T04:10:00.543634+0000","last_active":"2024-03-29T04:10:05.911616+0000","last_peered":"2024-03-29T04:10:05.911616+0000","last_clean":"2024-03-29T04:10:05.911616+0000","last_became_active":"2024-03-29T04:10:00.543343+0000","last_became_peered":"2024-03-29T04:10:00.543343+0000","last_unstale":"2024-03-29T04:10:05.911616+0000","last_undegraded":"2024-03-29T04:10:05.911616+0000","last_fullsized":"2024-03-29T04:10:05.911616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:35:00.452782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911632+0000","last_change":"2024-03-29T04:10:00.554863+0000","last_active":"2024-03-29T04:10:05.911632+0000","last_peered":"2024-03-29T04:10:05.911632+0000","last_clean":"2024-03-29T04:10:05.911632+0000","last_became_active":"2024-03-29T04:10:00.554560+0000","last_became_peered":"2024-03-29T04:10:00.554560+0000","last_unstale":"2024-03-29T04:10:05.911632+0000","last_undegraded":"2024-03-29T04:10:05.911632+0000","last_fullsized":"2024-03-29T04:10:05.911632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:29.233534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910973+0000","last_change":"2024-03-29T04:10:00.541999+0000","last_active":"2024-03-29T04:10:05.910973+0000","last_peered":"2024-03-29T04:10:05.910973+0000","last_clean":"2024-03-29T04:10:05.910973+0000","last_became_active":"2024-03-29T04:10:00.541699+0000","last_became_peered":"2024-03-29T04:10:00.541699+0000","last_unstale":"2024-03-29T04:10:05.910973+0000","last_undegraded":"2024-03-29T04:10:05.910973+0000","last_fullsized":"2024-03-29T04:10:05.910973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:48.213845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911525+0000","last_change":"2024-03-29T04:10:00.537506+0000","last_active":"2024-03-29T04:10:05.911525+0000","last_peered":"2024-03-29T04:10:05.911525+0000","last_clean":"2024-03-29T04:10:05.911525+0000","last_became_active":"2024-03-29T04:10:00.537239+0000","last_became_peered":"2024-03-29T04:10:00.537239+0000","last_unstale":"2024-03-29T04:10:05.911525+0000","last_undegraded":"2024-03-29T04:10:05.911525+0000","last_fullsized":"2024-03-29T04:10:05.911525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:30.096622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556535+0000","last_change":"2024-03-29T04:10:00.544443+0000","last_active":"2024-03-29T04:10:05.556535+0000","last_peered":"2024-03-29T04:10:05.556535+0000","last_clean":"2024-03-29T04:10:05.556535+0000","last_became_active":"2024-03-29T04:10:00.543427+0000","last_became_peered":"2024-03-29T04:10:00.543427+0000","last_unstale":"2024-03-29T04:10:05.556535+0000","last_undegraded":"2024-03-29T04:10:05.556535+0000","last_fullsized":"2024-03-29T04:10:05.556535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:48.213845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911292+0000","last_change":"2024-03-29T04:10:00.538050+0000","last_active":"2024-03-29T04:10:05.911292+0000","last_peered":"2024-03-29T04:10:05.911292+0000","last_clean":"2024-03-29T04:10:05.911292+0000","last_became_active":"2024-03-29T04:10:00.537771+0000","last_became_peered":"2024-03-29T04:10:00.537771+0000","last_unstale":"2024-03-29T04:10:05.911292+0000","last_undegraded":"2024-03-29T04:10:05.911292+0000","last_fullsized":"2024-03-29T04:10:05.911292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:21:11.329145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557260+0000","last_change":"2024-03-29T04:10:00.546534+0000","last_active":"2024-03-29T04:10:05.557260+0000","last_peered":"2024-03-29T04:10:05.557260+0000","last_clean":"2024-03-29T04:10:05.557260+0000","last_became_active":"2024-03-29T04:10:00.546028+0000","last_became_peered":"2024-03-29T04:10:00.546028+0000","last_unstale":"2024-03-29T04:10:05.557260+0000","last_undegraded":"2024-03-29T04:10:05.557260+0000","last_fullsized":"2024-03-29T04:10:05.557260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:23:46.996981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911142+0000","last_change":"2024-03-29T04:10:00.539866+0000","last_active":"2024-03-29T04:10:05.911142+0000","last_peered":"2024-03-29T04:10:05.911142+0000","last_clean":"2024-03-29T04:10:05.911142+0000","last_became_active":"2024-03-29T04:10:00.539562+0000","last_became_peered":"2024-03-29T04:10:00.539562+0000","last_unstale":"2024-03-29T04:10:05.911142+0000","last_undegraded":"2024-03-29T04:10:05.911142+0000","last_fullsized":"2024-03-29T04:10:05.911142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:52:12.957170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910850+0000","last_change":"2024-03-29T04:10:00.542361+0000","last_active":"2024-03-29T04:10:05.910850+0000","last_peered":"2024-03-29T04:10:05.910850+0000","last_clean":"2024-03-29T04:10:05.910850+0000","last_became_active":"2024-03-29T04:10:00.541625+0000","last_became_peered":"2024-03-29T04:10:00.541625+0000","last_unstale":"2024-03-29T04:10:05.910850+0000","last_undegraded":"2024-03-29T04:10:05.910850+0000","last_fullsized":"2024-03-29T04:10:05.910850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:46.707085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911557+0000","last_change":"2024-03-29T04:10:00.539117+0000","last_active":"2024-03-29T04:10:05.911557+0000","last_peered":"2024-03-29T04:10:05.911557+0000","last_clean":"2024-03-29T04:10:05.911557+0000","last_became_active":"2024-03-29T04:10:00.538833+0000","last_became_peered":"2024-03-29T04:10:00.538833+0000","last_unstale":"2024-03-29T04:10:05.911557+0000","last_undegraded":"2024-03-29T04:10:05.911557+0000","last_fullsized":"2024-03-29T04:10:05.911557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:13.002495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.560038+0000","last_change":"2024-03-29T04:10:00.551402+0000","last_active":"2024-03-29T04:10:05.560038+0000","last_peered":"2024-03-29T04:10:05.560038+0000","last_clean":"2024-03-29T04:10:05.560038+0000","last_became_active":"2024-03-29T04:10:00.551092+0000","last_became_peered":"2024-03-29T04:10:00.551092+0000","last_unstale":"2024-03-29T04:10:05.560038+0000","last_undegraded":"2024-03-29T04:10:05.560038+0000","last_fullsized":"2024-03-29T04:10:05.560038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:58:11.125964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.558449+0000","last_change":"2024-03-29T04:10:00.542777+0000","last_active":"2024-03-29T04:10:05.558449+0000","last_peered":"2024-03-29T04:10:05.558449+0000","last_clean":"2024-03-29T04:10:05.558449+0000","last_became_active":"2024-03-29T04:10:00.542241+0000","last_became_peered":"2024-03-29T04:10:00.542241+0000","last_unstale":"2024-03-29T04:10:05.558449+0000","last_undegraded":"2024-03-29T04:10:05.558449+0000","last_fullsized":"2024-03-29T04:10:05.558449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:46.804154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556402+0000","last_change":"2024-03-29T04:10:00.545811+0000","last_active":"2024-03-29T04:10:05.556402+0000","last_peered":"2024-03-29T04:10:05.556402+0000","last_clean":"2024-03-29T04:10:05.556402+0000","last_became_active":"2024-03-29T04:10:00.545501+0000","last_became_peered":"2024-03-29T04:10:00.545501+0000","last_unstale":"2024-03-29T04:10:05.556402+0000","last_undegraded":"2024-03-29T04:10:05.556402+0000","last_fullsized":"2024-03-29T04:10:05.556402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:29.233534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.909960+0000","last_change":"2024-03-29T04:10:00.545471+0000","last_active":"2024-03-29T04:10:05.909960+0000","last_peered":"2024-03-29T04:10:05.909960+0000","last_clean":"2024-03-29T04:10:05.909960+0000","last_became_active":"2024-03-29T04:10:00.545173+0000","last_became_peered":"2024-03-29T04:10:00.545173+0000","last_unstale":"2024-03-29T04:10:05.909960+0000","last_undegraded":"2024-03-29T04:10:05.909960+0000","last_fullsized":"2024-03-29T04:10:05.909960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:13:32.378332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910520+0000","last_change":"2024-03-29T04:10:00.545485+0000","last_active":"2024-03-29T04:10:05.910520+0000","last_peered":"2024-03-29T04:10:05.910520+0000","last_clean":"2024-03-29T04:10:05.910520+0000","last_became_active":"2024-03-29T04:10:00.545139+0000","last_became_peered":"2024-03-29T04:10:00.545139+0000","last_unstale":"2024-03-29T04:10:05.910520+0000","last_undegraded":"2024-03-29T04:10:05.910520+0000","last_fullsized":"2024-03-29T04:10:05.910520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:14:59.937144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.909924+0000","last_change":"2024-03-29T04:10:00.550441+0000","last_active":"2024-03-29T04:10:05.909924+0000","last_peered":"2024-03-29T04:10:05.909924+0000","last_clean":"2024-03-29T04:10:05.909924+0000","last_became_active":"2024-03-29T04:10:00.550150+0000","last_became_peered":"2024-03-29T04:10:00.550150+0000","last_unstale":"2024-03-29T04:10:05.909924+0000","last_undegraded":"2024-03-29T04:10:05.909924+0000","last_fullsized":"2024-03-29T04:10:05.909924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:52:12.957170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.558164+0000","last_change":"2024-03-29T04:10:00.540368+0000","last_active":"2024-03-29T04:10:05.558164+0000","last_peered":"2024-03-29T04:10:05.558164+0000","last_clean":"2024-03-29T04:10:05.558164+0000","last_became_active":"2024-03-29T04:10:00.538360+0000","last_became_peered":"2024-03-29T04:10:00.538360+0000","last_unstale":"2024-03-29T04:10:05.558164+0000","last_undegraded":"2024-03-29T04:10:05.558164+0000","last_fullsized":"2024-03-29T04:10:05.558164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:35:02.488999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911011+0000","last_change":"2024-03-29T04:10:00.550531+0000","last_active":"2024-03-29T04:10:05.911011+0000","last_peered":"2024-03-29T04:10:05.911011+0000","last_clean":"2024-03-29T04:10:05.911011+0000","last_became_active":"2024-03-29T04:10:00.550079+0000","last_became_peered":"2024-03-29T04:10:00.550079+0000","last_unstale":"2024-03-29T04:10:05.911011+0000","last_undegraded":"2024-03-29T04:10:05.911011+0000","last_fullsized":"2024-03-29T04:10:05.911011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:13:32.378332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910949+0000","last_change":"2024-03-29T04:10:00.550387+0000","last_active":"2024-03-29T04:10:05.910949+0000","last_peered":"2024-03-29T04:10:05.910949+0000","last_clean":"2024-03-29T04:10:05.910949+0000","last_became_active":"2024-03-29T04:10:00.550063+0000","last_became_peered":"2024-03-29T04:10:00.550063+0000","last_unstale":"2024-03-29T04:10:05.910949+0000","last_undegraded":"2024-03-29T04:10:05.910949+0000","last_fullsized":"2024-03-29T04:10:05.910949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:29.233534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911804+0000","last_change":"2024-03-29T04:10:00.541680+0000","last_active":"2024-03-29T04:10:05.911804+0000","last_peered":"2024-03-29T04:10:05.911804+0000","last_clean":"2024-03-29T04:10:05.911804+0000","last_became_active":"2024-03-29T04:10:00.541354+0000","last_became_peered":"2024-03-29T04:10:00.541354+0000","last_unstale":"2024-03-29T04:10:05.911804+0000","last_undegraded":"2024-03-29T04:10:05.911804+0000","last_fullsized":"2024-03-29T04:10:05.911804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:44:13.128618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910384+0000","last_change":"2024-03-29T04:10:00.544081+0000","last_active":"2024-03-29T04:10:05.910384+0000","last_peered":"2024-03-29T04:10:05.910384+0000","last_clean":"2024-03-29T04:10:05.910384+0000","last_became_active":"2024-03-29T04:10:00.543353+0000","last_became_peered":"2024-03-29T04:10:00.543353+0000","last_unstale":"2024-03-29T04:10:05.910384+0000","last_undegraded":"2024-03-29T04:10:05.910384+0000","last_fullsized":"2024-03-29T04:10:05.910384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:13:32.378332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910548+0000","last_change":"2024-03-29T04:10:00.543517+0000","last_active":"2024-03-29T04:10:05.910548+0000","last_peered":"2024-03-29T04:10:05.910548+0000","last_clean":"2024-03-29T04:10:05.910548+0000","last_became_active":"2024-03-29T04:10:00.542854+0000","last_became_peered":"2024-03-29T04:10:00.542854+0000","last_unstale":"2024-03-29T04:10:05.910548+0000","last_undegraded":"2024-03-29T04:10:05.910548+0000","last_fullsized":"2024-03-29T04:10:05.910548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:35:00.452782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557167+0000","last_change":"2024-03-29T04:10:00.546548+0000","last_active":"2024-03-29T04:10:05.557167+0000","last_peered":"2024-03-29T04:10:05.557167+0000","last_clean":"2024-03-29T04:10:05.557167+0000","last_became_active":"2024-03-29T04:10:00.546267+0000","last_became_peered":"2024-03-29T04:10:00.546267+0000","last_unstale":"2024-03-29T04:10:05.557167+0000","last_undegraded":"2024-03-29T04:10:05.557167+0000","last_fullsized":"2024-03-29T04:10:05.557167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:26:10.380318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911101+0000","last_change":"2024-03-29T04:10:00.539796+0000","last_active":"2024-03-29T04:10:05.911101+0000","last_peered":"2024-03-29T04:10:05.911101+0000","last_clean":"2024-03-29T04:10:05.911101+0000","last_became_active":"2024-03-29T04:10:00.539532+0000","last_became_peered":"2024-03-29T04:10:00.539532+0000","last_unstale":"2024-03-29T04:10:05.911101+0000","last_undegraded":"2024-03-29T04:10:05.911101+0000","last_fullsized":"2024-03-29T04:10:05.911101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:26:10.380318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556840+0000","last_change":"2024-03-29T04:10:00.549077+0000","last_active":"2024-03-29T04:10:05.556840+0000","last_peered":"2024-03-29T04:10:05.556840+0000","last_clean":"2024-03-29T04:10:05.556840+0000","last_became_active":"2024-03-29T04:10:00.548266+0000","last_became_peered":"2024-03-29T04:10:00.548266+0000","last_unstale":"2024-03-29T04:10:05.556840+0000","last_undegraded":"2024-03-29T04:10:05.556840+0000","last_fullsized":"2024-03-29T04:10:05.556840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:35:00.452782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910792+0000","last_change":"2024-03-29T04:10:00.535635+0000","last_active":"2024-03-29T04:10:05.910792+0000","last_peered":"2024-03-29T04:10:05.910792+0000","last_clean":"2024-03-29T04:10:05.910792+0000","last_became_active":"2024-03-29T04:10:00.535357+0000","last_became_peered":"2024-03-29T04:10:00.535357+0000","last_unstale":"2024-03-29T04:10:05.910792+0000","last_undegraded":"2024-03-29T04:10:05.910792+0000","last_fullsized":"2024-03-29T04:10:05.910792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:41.517875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911835+0000","last_change":"2024-03-29T04:10:00.547651+0000","last_active":"2024-03-29T04:10:05.911835+0000","last_peered":"2024-03-29T04:10:05.911835+0000","last_clean":"2024-03-29T04:10:05.911835+0000","last_became_active":"2024-03-29T04:10:00.547369+0000","last_became_peered":"2024-03-29T04:10:00.547369+0000","last_unstale":"2024-03-29T04:10:05.911835+0000","last_undegraded":"2024-03-29T04:10:05.911835+0000","last_fullsized":"2024-03-29T04:10:05.911835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:46.707085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911744+0000","last_change":"2024-03-29T04:10:00.545663+0000","last_active":"2024-03-29T04:10:05.911744+0000","last_peered":"2024-03-29T04:10:05.911744+0000","last_clean":"2024-03-29T04:10:05.911744+0000","last_became_active":"2024-03-29T04:10:00.545394+0000","last_became_peered":"2024-03-29T04:10:00.545394+0000","last_unstale":"2024-03-29T04:10:05.911744+0000","last_undegraded":"2024-03-29T04:10:05.911744+0000","last_fullsized":"2024-03-29T04:10:05.911744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:23:46.996981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:10:29.178 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:10:29.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.336+0000 7f3880e4a640 1 Processor -- start 2024-03-29T04:10:29.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.336+0000 7f3880e4a640 1 -- start start 2024-03-29T04:10:29.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.337+0000 7f3880e4a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f387c0ba030 0x7f387c0d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:29.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.337+0000 7f3880e4a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f387c0d10d0 0x7f387c0d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:29.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.337+0000 7f3880e4a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f387c15e7c0 con 0x7f387c15e3e0 2024-03-29T04:10:29.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.337+0000 7f3880e4a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f387c0d3840 con 0x7f387c0d10d0 2024-03-29T04:10:29.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.337+0000 7f3880e4a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f387c0d2c30 con 0x7f387c0ba030 2024-03-29T04:10:29.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.337+0000 7f387a59c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f387c15e3e0 0x7f387c0d2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47336/0 (socket says 172.21.15.148:47336) 2024-03-29T04:10:29.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.337+0000 7f387ad9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f387c0d10d0 0x7f387c0d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:29.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.337+0000 7f387a59c640 1 -- 172.21.15.148:0/2734220847 learned_addr learned my addr 172.21.15.148:0/2734220847 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:29.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.337+0000 7f3873fff640 1 --2- 172.21.15.148:0/2734220847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f387c0ba030 0x7f387c0d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:29.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.338+0000 7f3879d9b640 1 -- 172.21.15.148:0/2734220847 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3896920284 0 0) 0x7f387c15e7c0 con 0x7f387c15e3e0 2024-03-29T04:10:29.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.338+0000 7f3879d9b640 1 -- 172.21.15.148:0/2734220847 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3868003520 con 0x7f387c15e3e0 2024-03-29T04:10:29.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.338+0000 7f387ad9d640 1 -- 172.21.15.148:0/2734220847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f387c0ba030 msgr2=0x7f387c0d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:29.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.338+0000 7f387ad9d640 1 --2- 172.21.15.148:0/2734220847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f387c0ba030 0x7f387c0d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:29.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.338+0000 7f387ad9d640 1 -- 172.21.15.148:0/2734220847 >> v1:172.21.15.148:6789/0 conn(0x7f387c15e3e0 legacy=0x7f387c0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:29.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.338+0000 7f387ad9d640 1 -- 172.21.15.148:0/2734220847 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f387c0d2f10 con 0x7f387c0d10d0 2024-03-29T04:10:29.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.338+0000 7f387ad9d640 1 --2- 172.21.15.148:0/2734220847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f387c0d10d0 0x7f387c0d1580 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3874009d90 tx=0x7f387402bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1510fc0ec7e4e099 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:29.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.339+0000 7f3879d9b640 1 -- 172.21.15.148:0/2734220847 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3874039070 con 0x7f387c0d10d0 2024-03-29T04:10:29.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.339+0000 7f3879d9b640 1 -- 172.21.15.148:0/2734220847 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f387402cce0 con 0x7f387c0d10d0 2024-03-29T04:10:29.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.339+0000 7f3879d9b640 1 -- 172.21.15.148:0/2734220847 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3874034660 con 0x7f387c0d10d0 2024-03-29T04:10:29.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.339+0000 7f3880e4a640 1 -- 172.21.15.148:0/2734220847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f387c0d10d0 msgr2=0x7f387c0d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:29.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.339+0000 7f3880e4a640 1 --2- 172.21.15.148:0/2734220847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f387c0d10d0 0x7f387c0d1580 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3874009d90 tx=0x7f387402bf30 comp rx=0 tx=0).stop 2024-03-29T04:10:29.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.339+0000 7f3880e4a640 1 -- 172.21.15.148:0/2734220847 shutdown_connections 2024-03-29T04:10:29.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.339+0000 7f3880e4a640 1 --2- 172.21.15.148:0/2734220847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f387c0d10d0 0x7f387c0d1580 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:29.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.339+0000 7f3880e4a640 1 --2- 172.21.15.148:0/2734220847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f387c0ba030 0x7f387c0d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:29.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.339+0000 7f3880e4a640 1 -- 172.21.15.148:0/2734220847 >> 172.21.15.148:0/2734220847 conn(0x7f387c0b6210 msgr2=0x7f387c0b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:29.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.339+0000 7f3880e4a640 1 -- 172.21.15.148:0/2734220847 shutdown_connections 2024-03-29T04:10:29.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.340+0000 7f3880e4a640 1 -- 172.21.15.148:0/2734220847 wait complete. 2024-03-29T04:10:29.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.341+0000 7f3880e4a640 1 Processor -- start 2024-03-29T04:10:29.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.341+0000 7f3880e4a640 1 -- start start 2024-03-29T04:10:29.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.343+0000 7f3880e4a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f387c0ba030 0x7f387c0ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:29.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.343+0000 7f3880e4a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f387c0d10d0 0x7f387c0c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:29.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.343+0000 7f3880e4a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f387c15e3e0 0x7f387c154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:29.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.343+0000 7f3880e4a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f387c167f50 con 0x7f387c0ba030 2024-03-29T04:10:29.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.343+0000 7f3880e4a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f387c0d34d0 con 0x7f387c15e3e0 2024-03-29T04:10:29.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.343+0000 7f3880e4a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f387c1680d0 con 0x7f387c0d10d0 2024-03-29T04:10:29.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.343+0000 7f387a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f387c0ba030 0x7f387c0ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:29.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.343+0000 7f387a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f387c0ba030 0x7f387c0ca7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48948/0 (socket says 172.21.15.148:48948) 2024-03-29T04:10:29.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.343+0000 7f387a59c640 1 -- 172.21.15.148:0/3159672448 learned_addr learned my addr 172.21.15.148:0/3159672448 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:29.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.343+0000 7f3873fff640 1 --2- 172.21.15.148:0/3159672448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f387c0d10d0 0x7f387c0c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:29.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.343+0000 7f387ad9d640 1 --2- 172.21.15.148:0/3159672448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f387c15e3e0 0x7f387c154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:29.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.343+0000 7f387a59c640 1 -- 172.21.15.148:0/3159672448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f387c0d10d0 msgr2=0x7f387c0c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:29.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.343+0000 7f387a59c640 1 --2- 172.21.15.148:0/3159672448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f387c0d10d0 0x7f387c0c8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:29.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.343+0000 7f387a59c640 1 -- 172.21.15.148:0/3159672448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f387c15e3e0 msgr2=0x7f387c154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:29.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.343+0000 7f387a59c640 1 --2- 172.21.15.148:0/3159672448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f387c15e3e0 0x7f387c154a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:29.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.343+0000 7f387a59c640 1 -- 172.21.15.148:0/3159672448 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f387c1539c0 con 0x7f387c0ba030 2024-03-29T04:10:29.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.343+0000 7f3873fff640 1 --2- 172.21.15.148:0/3159672448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f387c0d10d0 0x7f387c0c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:29.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.343+0000 7f387ad9d640 1 --2- 172.21.15.148:0/3159672448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f387c15e3e0 0x7f387c154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:29.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.343+0000 7f387a59c640 1 --2- 172.21.15.148:0/3159672448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f387c0ba030 0x7f387c0ca7e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f386c00c390 tx=0x7f386c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:29.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.343+0000 7f3872ffd640 1 -- 172.21.15.148:0/3159672448 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386c017070 con 0x7f387c0ba030 2024-03-29T04:10:29.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.343+0000 7f3872ffd640 1 -- 172.21.15.148:0/3159672448 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f386c005ce0 con 0x7f387c0ba030 2024-03-29T04:10:29.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.343+0000 7f3880e4a640 1 -- 172.21.15.148:0/3159672448 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f387c153ca0 con 0x7f387c0ba030 2024-03-29T04:10:29.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.343+0000 7f3872ffd640 1 -- 172.21.15.148:0/3159672448 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386c0072f0 con 0x7f387c0ba030 2024-03-29T04:10:29.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.343+0000 7f3880e4a640 1 -- 172.21.15.148:0/3159672448 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f387c154260 con 0x7f387c0ba030 2024-03-29T04:10:29.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.344+0000 7f3880e4a640 1 -- 172.21.15.148:0/3159672448 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f387c0cf6c0 con 0x7f387c0ba030 2024-03-29T04:10:29.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.345+0000 7f3872ffd640 1 -- 172.21.15.148:0/3159672448 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f386c01a070 con 0x7f387c0ba030 2024-03-29T04:10:29.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.345+0000 7f3872ffd640 1 --2- 172.21.15.148:0/3159672448 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f385c06c530 0x7f385c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:29.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.346+0000 7f3873fff640 1 --2- 172.21.15.148:0/3159672448 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f385c06c530 0x7f385c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:29.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.346+0000 7f3872ffd640 1 -- 172.21.15.148:0/3159672448 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f386c08ad30 con 0x7f387c0ba030 2024-03-29T04:10:29.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.346+0000 7f3873fff640 1 --2- 172.21.15.148:0/3159672448 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f385c06c530 0x7f385c06e9f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f387c06ccd0 tx=0x7f3864008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:29.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.350+0000 7f3872ffd640 1 -- 172.21.15.148:0/3159672448 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f387c0cf6c0 con 0x7f387c0ba030 2024-03-29T04:10:29.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.529+0000 7f3880e4a640 1 -- 172.21.15.148:0/3159672448 --> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f387c0b6a00 con 0x7f385c06c530 2024-03-29T04:10:29.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.537+0000 7f3872ffd640 1 -- 172.21.15.148:0/3159672448 <== mgr.4113 v2:172.21.15.148:6832/3271445075 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332533 (secure 0 0 0) 0x7f387c0b6a00 con 0x7f385c06c530 2024-03-29T04:10:29.520 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:29.522 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-03-29T04:10:29.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.541+0000 7f3880e4a640 1 -- 172.21.15.148:0/3159672448 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f385c06c530 msgr2=0x7f385c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:29.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.541+0000 7f3880e4a640 1 --2- 172.21.15.148:0/3159672448 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f385c06c530 0x7f385c06e9f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f387c06ccd0 tx=0x7f3864008040 comp rx=0 tx=0).stop 2024-03-29T04:10:29.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.541+0000 7f3880e4a640 1 -- 172.21.15.148:0/3159672448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f387c0ba030 msgr2=0x7f387c0ca7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:29.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.541+0000 7f3880e4a640 1 --2- 172.21.15.148:0/3159672448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f387c0ba030 0x7f387c0ca7e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f386c00c390 tx=0x7f386c00c850 comp rx=0 tx=0).stop 2024-03-29T04:10:29.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.541+0000 7f3880e4a640 1 -- 172.21.15.148:0/3159672448 shutdown_connections 2024-03-29T04:10:29.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.541+0000 7f3880e4a640 1 --2- 172.21.15.148:0/3159672448 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f385c06c530 0x7f385c06e9f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:29.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.541+0000 7f3880e4a640 1 --2- 172.21.15.148:0/3159672448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f387c15e3e0 0x7f387c154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:29.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.541+0000 7f3880e4a640 1 --2- 172.21.15.148:0/3159672448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f387c0d10d0 0x7f387c0c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:29.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.541+0000 7f3880e4a640 1 --2- 172.21.15.148:0/3159672448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f387c0ba030 0x7f387c0ca7e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:29.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.541+0000 7f3880e4a640 1 -- 172.21.15.148:0/3159672448 >> 172.21.15.148:0/3159672448 conn(0x7f387c0b6210 msgr2=0x7f387c0b7110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:29.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.541+0000 7f3880e4a640 1 -- 172.21.15.148:0/3159672448 shutdown_connections 2024-03-29T04:10:29.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.541+0000 7f3880e4a640 1 -- 172.21.15.148:0/3159672448 wait complete. 2024-03-29T04:10:29.531 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-03-29T04:10:27.888129+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236032,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738688,"statfs":{"total":773094113280,"available":772852416512,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002255"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911869+0000","last_change":"2024-03-29T04:09:59.541493+0000","last_active":"2024-03-29T04:10:05.911869+0000","last_peered":"2024-03-29T04:10:05.911869+0000","last_clean":"2024-03-29T04:10:05.911869+0000","last_became_active":"2024-03-29T04:09:59.541208+0000","last_became_peered":"2024-03-29T04:09:59.541208+0000","last_unstale":"2024-03-29T04:10:05.911869+0000","last_undegraded":"2024-03-29T04:10:05.911869+0000","last_fullsized":"2024-03-29T04:10:05.911869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:55:08.672579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910799+0000","last_change":"2024-03-29T04:10:00.548950+0000","last_active":"2024-03-29T04:10:05.910799+0000","last_peered":"2024-03-29T04:10:05.910799+0000","last_clean":"2024-03-29T04:10:05.910799+0000","last_became_active":"2024-03-29T04:10:00.548670+0000","last_became_peered":"2024-03-29T04:10:00.548670+0000","last_unstale":"2024-03-29T04:10:05.910799+0000","last_undegraded":"2024-03-29T04:10:05.910799+0000","last_fullsized":"2024-03-29T04:10:05.910799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:16:02.331457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911034+0000","last_change":"2024-03-29T04:10:04.649988+0000","last_active":"2024-03-29T04:10:05.911034+0000","last_peered":"2024-03-29T04:10:05.911034+0000","last_clean":"2024-03-29T04:10:05.911034+0000","last_became_active":"2024-03-29T04:10:04.649557+0000","last_became_peered":"2024-03-29T04:10:04.649557+0000","last_unstale":"2024-03-29T04:10:05.911034+0000","last_undegraded":"2024-03-29T04:10:05.911034+0000","last_fullsized":"2024-03-29T04:10:05.911034+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:07:56.606322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911032+0000","last_change":"2024-03-29T04:09:59.542509+0000","last_active":"2024-03-29T04:10:05.911032+0000","last_peered":"2024-03-29T04:10:05.911032+0000","last_clean":"2024-03-29T04:10:05.911032+0000","last_became_active":"2024-03-29T04:09:59.542221+0000","last_became_peered":"2024-03-29T04:09:59.542221+0000","last_unstale":"2024-03-29T04:10:05.911032+0000","last_undegraded":"2024-03-29T04:10:05.911032+0000","last_fullsized":"2024-03-29T04:10:05.911032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:58:10.121554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910298+0000","last_change":"2024-03-29T04:10:00.536505+0000","last_active":"2024-03-29T04:10:05.910298+0000","last_peered":"2024-03-29T04:10:05.910298+0000","last_clean":"2024-03-29T04:10:05.910298+0000","last_became_active":"2024-03-29T04:10:00.536200+0000","last_became_peered":"2024-03-29T04:10:00.536200+0000","last_unstale":"2024-03-29T04:10:05.910298+0000","last_undegraded":"2024-03-29T04:10:05.910298+0000","last_fullsized":"2024-03-29T04:10:05.910298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:29.233534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557208+0000","last_change":"2024-03-29T04:10:04.576319+0000","last_active":"2024-03-29T04:10:05.557208+0000","last_peered":"2024-03-29T04:10:05.557208+0000","last_clean":"2024-03-29T04:10:05.557208+0000","last_became_active":"2024-03-29T04:10:04.575508+0000","last_became_peered":"2024-03-29T04:10:04.575508+0000","last_unstale":"2024-03-29T04:10:05.557208+0000","last_undegraded":"2024-03-29T04:10:05.557208+0000","last_fullsized":"2024-03-29T04:10:05.557208+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:34.129098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911504+0000","last_change":"2024-03-29T04:09:59.545281+0000","last_active":"2024-03-29T04:10:05.911504+0000","last_peered":"2024-03-29T04:10:05.911504+0000","last_clean":"2024-03-29T04:10:05.911504+0000","last_became_active":"2024-03-29T04:09:59.544991+0000","last_became_peered":"2024-03-29T04:09:59.544991+0000","last_unstale":"2024-03-29T04:10:05.911504+0000","last_undegraded":"2024-03-29T04:10:05.911504+0000","last_fullsized":"2024-03-29T04:10:05.911504+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:35:01.484589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556861+0000","last_change":"2024-03-29T04:10:00.542510+0000","last_active":"2024-03-29T04:10:05.556861+0000","last_peered":"2024-03-29T04:10:05.556861+0000","last_clean":"2024-03-29T04:10:05.556861+0000","last_became_active":"2024-03-29T04:10:00.542014+0000","last_became_peered":"2024-03-29T04:10:00.542014+0000","last_unstale":"2024-03-29T04:10:05.556861+0000","last_undegraded":"2024-03-29T04:10:05.556861+0000","last_fullsized":"2024-03-29T04:10:05.556861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:30.096622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556709+0000","last_change":"2024-03-29T04:10:04.574735+0000","last_active":"2024-03-29T04:10:05.556709+0000","last_peered":"2024-03-29T04:10:05.556709+0000","last_clean":"2024-03-29T04:10:05.556709+0000","last_became_active":"2024-03-29T04:10:04.573995+0000","last_became_peered":"2024-03-29T04:10:04.573995+0000","last_unstale":"2024-03-29T04:10:05.556709+0000","last_undegraded":"2024-03-29T04:10:05.556709+0000","last_fullsized":"2024-03-29T04:10:05.556709+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:52.246321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911346+0000","last_change":"2024-03-29T04:09:59.542326+0000","last_active":"2024-03-29T04:10:05.911346+0000","last_peered":"2024-03-29T04:10:05.911346+0000","last_clean":"2024-03-29T04:10:05.911346+0000","last_became_active":"2024-03-29T04:09:59.540873+0000","last_became_peered":"2024-03-29T04:09:59.540873+0000","last_unstale":"2024-03-29T04:10:05.911346+0000","last_undegraded":"2024-03-29T04:10:05.911346+0000","last_fullsized":"2024-03-29T04:10:05.911346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:02.879344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557486+0000","last_change":"2024-03-29T04:10:00.543297+0000","last_active":"2024-03-29T04:10:05.557486+0000","last_peered":"2024-03-29T04:10:05.557486+0000","last_clean":"2024-03-29T04:10:05.557486+0000","last_became_active":"2024-03-29T04:10:00.541793+0000","last_became_peered":"2024-03-29T04:10:00.541793+0000","last_unstale":"2024-03-29T04:10:05.557486+0000","last_undegraded":"2024-03-29T04:10:05.557486+0000","last_fullsized":"2024-03-29T04:10:05.557486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:13.002495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557550+0000","last_change":"2024-03-29T04:10:04.572366+0000","last_active":"2024-03-29T04:10:05.557550+0000","last_peered":"2024-03-29T04:10:05.557550+0000","last_clean":"2024-03-29T04:10:05.557550+0000","last_became_active":"2024-03-29T04:10:04.572066+0000","last_became_peered":"2024-03-29T04:10:04.572066+0000","last_unstale":"2024-03-29T04:10:05.557550+0000","last_undegraded":"2024-03-29T04:10:05.557550+0000","last_fullsized":"2024-03-29T04:10:05.557550+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:00:55.353300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910448+0000","last_change":"2024-03-29T04:10:00.547621+0000","last_active":"2024-03-29T04:10:05.910448+0000","last_peered":"2024-03-29T04:10:05.910448+0000","last_clean":"2024-03-29T04:10:05.910448+0000","last_became_active":"2024-03-29T04:10:00.547282+0000","last_became_peered":"2024-03-29T04:10:00.547282+0000","last_unstale":"2024-03-29T04:10:05.910448+0000","last_undegraded":"2024-03-29T04:10:05.910448+0000","last_fullsized":"2024-03-29T04:10:05.910448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:41.517875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911279+0000","last_change":"2024-03-29T04:09:59.539362+0000","last_active":"2024-03-29T04:10:05.911279+0000","last_peered":"2024-03-29T04:10:05.911279+0000","last_clean":"2024-03-29T04:10:05.911279+0000","last_became_active":"2024-03-29T04:09:59.539062+0000","last_became_peered":"2024-03-29T04:09:59.539062+0000","last_unstale":"2024-03-29T04:10:05.911279+0000","last_undegraded":"2024-03-29T04:10:05.911279+0000","last_fullsized":"2024-03-29T04:10:05.911279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:33:50.904931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.558475+0000","last_change":"2024-03-29T04:10:04.579133+0000","last_active":"2024-03-29T04:10:05.558475+0000","last_peered":"2024-03-29T04:10:05.558475+0000","last_clean":"2024-03-29T04:10:05.558475+0000","last_became_active":"2024-03-29T04:10:04.578748+0000","last_became_peered":"2024-03-29T04:10:04.578748+0000","last_unstale":"2024-03-29T04:10:05.558475+0000","last_undegraded":"2024-03-29T04:10:05.558475+0000","last_fullsized":"2024-03-29T04:10:05.558475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:06.998903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910015+0000","last_change":"2024-03-29T04:10:00.547887+0000","last_active":"2024-03-29T04:10:05.910015+0000","last_peered":"2024-03-29T04:10:05.910015+0000","last_clean":"2024-03-29T04:10:05.910015+0000","last_became_active":"2024-03-29T04:10:00.547419+0000","last_became_peered":"2024-03-29T04:10:00.547419+0000","last_unstale":"2024-03-29T04:10:05.910015+0000","last_undegraded":"2024-03-29T04:10:05.910015+0000","last_fullsized":"2024-03-29T04:10:05.910015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:46.707085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912399+0000","last_change":"2024-03-29T04:09:59.544306+0000","last_active":"2024-03-29T04:10:05.912399+0000","last_peered":"2024-03-29T04:10:05.912399+0000","last_clean":"2024-03-29T04:10:05.912399+0000","last_became_active":"2024-03-29T04:09:59.544028+0000","last_became_peered":"2024-03-29T04:09:59.544028+0000","last_unstale":"2024-03-29T04:10:05.912399+0000","last_undegraded":"2024-03-29T04:10:05.912399+0000","last_fullsized":"2024-03-29T04:10:05.912399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:33:50.904931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.558045+0000","last_change":"2024-03-29T04:10:04.577178+0000","last_active":"2024-03-29T04:10:05.558045+0000","last_peered":"2024-03-29T04:10:05.558045+0000","last_clean":"2024-03-29T04:10:05.558045+0000","last_became_active":"2024-03-29T04:10:04.576788+0000","last_became_peered":"2024-03-29T04:10:04.576788+0000","last_unstale":"2024-03-29T04:10:05.558045+0000","last_undegraded":"2024-03-29T04:10:05.558045+0000","last_fullsized":"2024-03-29T04:10:05.558045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:26:14.412794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910704+0000","last_change":"2024-03-29T04:10:00.543623+0000","last_active":"2024-03-29T04:10:05.910704+0000","last_peered":"2024-03-29T04:10:05.910704+0000","last_clean":"2024-03-29T04:10:05.910704+0000","last_became_active":"2024-03-29T04:10:00.542670+0000","last_became_peered":"2024-03-29T04:10:00.542670+0000","last_unstale":"2024-03-29T04:10:05.910704+0000","last_undegraded":"2024-03-29T04:10:05.910704+0000","last_fullsized":"2024-03-29T04:10:05.910704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:48.213845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910045+0000","last_change":"2024-03-29T04:09:59.540613+0000","last_active":"2024-03-29T04:10:05.910045+0000","last_peered":"2024-03-29T04:10:05.910045+0000","last_clean":"2024-03-29T04:10:05.910045+0000","last_became_active":"2024-03-29T04:09:59.540340+0000","last_became_peered":"2024-03-29T04:09:59.540340+0000","last_unstale":"2024-03-29T04:10:05.910045+0000","last_undegraded":"2024-03-29T04:10:05.910045+0000","last_fullsized":"2024-03-29T04:10:05.910045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:05.307537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911330+0000","last_change":"2024-03-29T04:10:04.646383+0000","last_active":"2024-03-29T04:10:05.911330+0000","last_peered":"2024-03-29T04:10:05.911330+0000","last_clean":"2024-03-29T04:10:05.911330+0000","last_became_active":"2024-03-29T04:10:04.646046+0000","last_became_peered":"2024-03-29T04:10:04.646046+0000","last_unstale":"2024-03-29T04:10:05.911330+0000","last_undegraded":"2024-03-29T04:10:05.911330+0000","last_fullsized":"2024-03-29T04:10:05.911330+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:17.034970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557229+0000","last_change":"2024-03-29T04:10:00.537777+0000","last_active":"2024-03-29T04:10:05.557229+0000","last_peered":"2024-03-29T04:10:05.557229+0000","last_clean":"2024-03-29T04:10:05.557229+0000","last_became_active":"2024-03-29T04:10:00.537423+0000","last_became_peered":"2024-03-29T04:10:00.537423+0000","last_unstale":"2024-03-29T04:10:05.557229+0000","last_undegraded":"2024-03-29T04:10:05.557229+0000","last_fullsized":"2024-03-29T04:10:05.557229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:33:51.909341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910053+0000","last_change":"2024-03-29T04:09:59.543268+0000","last_active":"2024-03-29T04:10:05.910053+0000","last_peered":"2024-03-29T04:10:05.910053+0000","last_clean":"2024-03-29T04:10:05.910053+0000","last_became_active":"2024-03-29T04:09:59.542982+0000","last_became_peered":"2024-03-29T04:09:59.542982+0000","last_unstale":"2024-03-29T04:10:05.910053+0000","last_undegraded":"2024-03-29T04:10:05.910053+0000","last_fullsized":"2024-03-29T04:10:05.910053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:01.925223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557817+0000","last_change":"2024-03-29T04:10:04.577232+0000","last_active":"2024-03-29T04:10:05.557817+0000","last_peered":"2024-03-29T04:10:05.557817+0000","last_clean":"2024-03-29T04:10:05.557817+0000","last_became_active":"2024-03-29T04:10:04.576758+0000","last_became_peered":"2024-03-29T04:10:04.576758+0000","last_unstale":"2024-03-29T04:10:05.557817+0000","last_undegraded":"2024-03-29T04:10:05.557817+0000","last_fullsized":"2024-03-29T04:10:05.557817+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:26:08.662188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912635+0000","last_change":"2024-03-29T04:10:00.546235+0000","last_active":"2024-03-29T04:10:05.912635+0000","last_peered":"2024-03-29T04:10:05.912635+0000","last_clean":"2024-03-29T04:10:05.912635+0000","last_became_active":"2024-03-29T04:10:00.545908+0000","last_became_peered":"2024-03-29T04:10:00.545908+0000","last_unstale":"2024-03-29T04:10:05.912635+0000","last_undegraded":"2024-03-29T04:10:05.912635+0000","last_fullsized":"2024-03-29T04:10:05.912635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:16:02.331457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911074+0000","last_change":"2024-03-29T04:09:59.541657+0000","last_active":"2024-03-29T04:10:05.911074+0000","last_peered":"2024-03-29T04:10:05.911074+0000","last_clean":"2024-03-29T04:10:05.911074+0000","last_became_active":"2024-03-29T04:09:59.541305+0000","last_became_peered":"2024-03-29T04:09:59.541305+0000","last_unstale":"2024-03-29T04:10:05.911074+0000","last_undegraded":"2024-03-29T04:10:05.911074+0000","last_fullsized":"2024-03-29T04:10:05.911074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:25:06.123311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556981+0000","last_change":"2024-03-29T04:10:04.646174+0000","last_active":"2024-03-29T04:10:05.556981+0000","last_peered":"2024-03-29T04:10:05.556981+0000","last_clean":"2024-03-29T04:10:05.556981+0000","last_became_active":"2024-03-29T04:10:04.645802+0000","last_became_peered":"2024-03-29T04:10:04.645802+0000","last_unstale":"2024-03-29T04:10:05.556981+0000","last_undegraded":"2024-03-29T04:10:05.556981+0000","last_fullsized":"2024-03-29T04:10:05.556981+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:35:04.485257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911614+0000","last_change":"2024-03-29T04:10:00.543254+0000","last_active":"2024-03-29T04:10:05.911614+0000","last_peered":"2024-03-29T04:10:05.911614+0000","last_clean":"2024-03-29T04:10:05.911614+0000","last_became_active":"2024-03-29T04:10:00.542958+0000","last_became_peered":"2024-03-29T04:10:00.542958+0000","last_unstale":"2024-03-29T04:10:05.911614+0000","last_undegraded":"2024-03-29T04:10:05.911614+0000","last_fullsized":"2024-03-29T04:10:05.911614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:00:51.320824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911924+0000","last_change":"2024-03-29T04:09:59.544320+0000","last_active":"2024-03-29T04:10:05.911924+0000","last_peered":"2024-03-29T04:10:05.911924+0000","last_clean":"2024-03-29T04:10:05.911924+0000","last_became_active":"2024-03-29T04:09:59.543928+0000","last_became_peered":"2024-03-29T04:09:59.543928+0000","last_unstale":"2024-03-29T04:10:05.911924+0000","last_undegraded":"2024-03-29T04:10:05.911924+0000","last_fullsized":"2024-03-29T04:10:05.911924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:02.879344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910274+0000","last_change":"2024-03-29T04:10:04.646905+0000","last_active":"2024-03-29T04:10:05.910274+0000","last_peered":"2024-03-29T04:10:05.910274+0000","last_clean":"2024-03-29T04:10:05.910274+0000","last_became_active":"2024-03-29T04:10:04.646482+0000","last_became_peered":"2024-03-29T04:10:04.646482+0000","last_unstale":"2024-03-29T04:10:05.910274+0000","last_undegraded":"2024-03-29T04:10:05.910274+0000","last_fullsized":"2024-03-29T04:10:05.910274+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:38:53.191487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910879+0000","last_change":"2024-03-29T04:10:00.541465+0000","last_active":"2024-03-29T04:10:05.910879+0000","last_peered":"2024-03-29T04:10:05.910879+0000","last_clean":"2024-03-29T04:10:05.910879+0000","last_became_active":"2024-03-29T04:10:00.541197+0000","last_became_peered":"2024-03-29T04:10:00.541197+0000","last_unstale":"2024-03-29T04:10:05.910879+0000","last_undegraded":"2024-03-29T04:10:05.910879+0000","last_fullsized":"2024-03-29T04:10:05.910879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:52:12.957170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911673+0000","last_change":"2024-03-29T04:09:59.545759+0000","last_active":"2024-03-29T04:10:05.911673+0000","last_peered":"2024-03-29T04:10:05.911673+0000","last_clean":"2024-03-29T04:10:05.911673+0000","last_became_active":"2024-03-29T04:09:59.545373+0000","last_became_peered":"2024-03-29T04:09:59.545373+0000","last_unstale":"2024-03-29T04:10:05.911673+0000","last_undegraded":"2024-03-29T04:10:05.911673+0000","last_fullsized":"2024-03-29T04:10:05.911673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:58:10.121554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911095+0000","last_change":"2024-03-29T04:10:04.644123+0000","last_active":"2024-03-29T04:10:05.911095+0000","last_peered":"2024-03-29T04:10:05.911095+0000","last_clean":"2024-03-29T04:10:05.911095+0000","last_became_active":"2024-03-29T04:10:04.643701+0000","last_became_peered":"2024-03-29T04:10:04.643701+0000","last_unstale":"2024-03-29T04:10:05.911095+0000","last_undegraded":"2024-03-29T04:10:05.911095+0000","last_fullsized":"2024-03-29T04:10:05.911095+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:26:14.412794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557781+0000","last_change":"2024-03-29T04:10:00.543518+0000","last_active":"2024-03-29T04:10:05.557781+0000","last_peered":"2024-03-29T04:10:05.557781+0000","last_clean":"2024-03-29T04:10:05.557781+0000","last_became_active":"2024-03-29T04:10:00.542373+0000","last_became_peered":"2024-03-29T04:10:00.542373+0000","last_unstale":"2024-03-29T04:10:05.557781+0000","last_undegraded":"2024-03-29T04:10:05.557781+0000","last_fullsized":"2024-03-29T04:10:05.557781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:02.966428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910966+0000","last_change":"2024-03-29T04:09:59.542356+0000","last_active":"2024-03-29T04:10:05.910966+0000","last_peered":"2024-03-29T04:10:05.910966+0000","last_clean":"2024-03-29T04:10:05.910966+0000","last_became_active":"2024-03-29T04:09:59.541917+0000","last_became_peered":"2024-03-29T04:09:59.541917+0000","last_unstale":"2024-03-29T04:10:05.910966+0000","last_undegraded":"2024-03-29T04:10:05.910966+0000","last_fullsized":"2024-03-29T04:10:05.910966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:01.925223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912224+0000","last_change":"2024-03-29T04:10:04.578218+0000","last_active":"2024-03-29T04:10:05.912224+0000","last_peered":"2024-03-29T04:10:05.912224+0000","last_clean":"2024-03-29T04:10:05.912224+0000","last_became_active":"2024-03-29T04:10:04.577221+0000","last_became_peered":"2024-03-29T04:10:04.577221+0000","last_unstale":"2024-03-29T04:10:05.912224+0000","last_undegraded":"2024-03-29T04:10:05.912224+0000","last_fullsized":"2024-03-29T04:10:05.912224+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:44:17.161094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.558130+0000","last_change":"2024-03-29T04:10:00.536176+0000","last_active":"2024-03-29T04:10:05.558130+0000","last_peered":"2024-03-29T04:10:05.558130+0000","last_clean":"2024-03-29T04:10:05.558130+0000","last_became_active":"2024-03-29T04:10:00.535854+0000","last_became_peered":"2024-03-29T04:10:00.535854+0000","last_unstale":"2024-03-29T04:10:05.558130+0000","last_undegraded":"2024-03-29T04:10:05.558130+0000","last_fullsized":"2024-03-29T04:10:05.558130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:25:07.127721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557572+0000","last_change":"2024-03-29T04:09:59.542230+0000","last_active":"2024-03-29T04:10:05.557572+0000","last_peered":"2024-03-29T04:10:05.557572+0000","last_clean":"2024-03-29T04:10:05.557572+0000","last_became_active":"2024-03-29T04:09:59.541972+0000","last_became_peered":"2024-03-29T04:09:59.541972+0000","last_unstale":"2024-03-29T04:10:05.557572+0000","last_undegraded":"2024-03-29T04:10:05.557572+0000","last_fullsized":"2024-03-29T04:10:05.557572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:01.925223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910829+0000","last_change":"2024-03-29T04:10:04.574263+0000","last_active":"2024-03-29T04:10:05.910829+0000","last_peered":"2024-03-29T04:10:05.910829+0000","last_clean":"2024-03-29T04:10:05.910829+0000","last_became_active":"2024-03-29T04:10:04.573843+0000","last_became_peered":"2024-03-29T04:10:04.573843+0000","last_unstale":"2024-03-29T04:10:05.910829+0000","last_undegraded":"2024-03-29T04:10:05.910829+0000","last_fullsized":"2024-03-29T04:10:05.910829+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:06.998903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912194+0000","last_change":"2024-03-29T04:10:00.540824+0000","last_active":"2024-03-29T04:10:05.912194+0000","last_peered":"2024-03-29T04:10:05.912194+0000","last_clean":"2024-03-29T04:10:05.912194+0000","last_became_active":"2024-03-29T04:10:00.540505+0000","last_became_peered":"2024-03-29T04:10:00.540505+0000","last_unstale":"2024-03-29T04:10:05.912194+0000","last_undegraded":"2024-03-29T04:10:05.912194+0000","last_fullsized":"2024-03-29T04:10:05.912194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:41.517875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.559855+0000","last_change":"2024-03-29T04:09:59.540415+0000","last_active":"2024-03-29T04:10:05.559855+0000","last_peered":"2024-03-29T04:10:05.559855+0000","last_clean":"2024-03-29T04:10:05.559855+0000","last_became_active":"2024-03-29T04:09:59.540130+0000","last_became_peered":"2024-03-29T04:09:59.540130+0000","last_unstale":"2024-03-29T04:10:05.559855+0000","last_undegraded":"2024-03-29T04:10:05.559855+0000","last_fullsized":"2024-03-29T04:10:05.559855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:05.307537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911492+0000","last_change":"2024-03-29T04:10:04.575156+0000","last_active":"2024-03-29T04:10:05.911492+0000","last_peered":"2024-03-29T04:10:05.911492+0000","last_clean":"2024-03-29T04:10:05.911492+0000","last_became_active":"2024-03-29T04:10:04.573816+0000","last_became_peered":"2024-03-29T04:10:04.573816+0000","last_unstale":"2024-03-29T04:10:05.911492+0000","last_undegraded":"2024-03-29T04:10:05.911492+0000","last_fullsized":"2024-03-29T04:10:05.911492+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:23:51.029457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.909950+0000","last_change":"2024-03-29T04:10:00.531739+0000","last_active":"2024-03-29T04:10:05.909950+0000","last_peered":"2024-03-29T04:10:05.909950+0000","last_clean":"2024-03-29T04:10:05.909950+0000","last_became_active":"2024-03-29T04:10:00.531395+0000","last_became_peered":"2024-03-29T04:10:00.531395+0000","last_unstale":"2024-03-29T04:10:05.909950+0000","last_undegraded":"2024-03-29T04:10:05.909950+0000","last_fullsized":"2024-03-29T04:10:05.909950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:02:32.544769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910076+0000","last_change":"2024-03-29T04:09:59.541400+0000","last_active":"2024-03-29T04:10:05.910076+0000","last_peered":"2024-03-29T04:10:05.910076+0000","last_clean":"2024-03-29T04:10:05.910076+0000","last_became_active":"2024-03-29T04:09:59.541114+0000","last_became_peered":"2024-03-29T04:09:59.541114+0000","last_unstale":"2024-03-29T04:10:05.910076+0000","last_undegraded":"2024-03-29T04:10:05.910076+0000","last_fullsized":"2024-03-29T04:10:05.910076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:25:06.123311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912614+0000","last_change":"2024-03-29T04:10:04.645661+0000","last_active":"2024-03-29T04:10:05.912614+0000","last_peered":"2024-03-29T04:10:05.912614+0000","last_clean":"2024-03-29T04:10:05.912614+0000","last_became_active":"2024-03-29T04:10:04.645258+0000","last_became_peered":"2024-03-29T04:10:04.645258+0000","last_unstale":"2024-03-29T04:10:05.912614+0000","last_undegraded":"2024-03-29T04:10:05.912614+0000","last_fullsized":"2024-03-29T04:10:05.912614+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:44:17.161094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910699+0000","last_change":"2024-03-29T04:10:00.537081+0000","last_active":"2024-03-29T04:10:05.910699+0000","last_peered":"2024-03-29T04:10:05.910699+0000","last_clean":"2024-03-29T04:10:05.910699+0000","last_became_active":"2024-03-29T04:10:00.536267+0000","last_became_peered":"2024-03-29T04:10:00.536267+0000","last_unstale":"2024-03-29T04:10:05.910699+0000","last_undegraded":"2024-03-29T04:10:05.910699+0000","last_fullsized":"2024-03-29T04:10:05.910699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:41.517875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557767+0000","last_change":"2024-03-29T04:09:59.535631+0000","last_active":"2024-03-29T04:10:05.557767+0000","last_peered":"2024-03-29T04:10:05.557767+0000","last_clean":"2024-03-29T04:10:05.557767+0000","last_became_active":"2024-03-29T04:09:59.535323+0000","last_became_peered":"2024-03-29T04:09:59.535323+0000","last_unstale":"2024-03-29T04:10:05.557767+0000","last_undegraded":"2024-03-29T04:10:05.557767+0000","last_fullsized":"2024-03-29T04:10:05.557767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:05.307537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910183+0000","last_change":"2024-03-29T04:09:49.002570+0000","last_active":"2024-03-29T04:10:05.910183+0000","last_peered":"2024-03-29T04:10:05.910183+0000","last_clean":"2024-03-29T04:10:05.910183+0000","last_became_active":"2024-03-29T04:09:49.002235+0000","last_became_peered":"2024-03-29T04:09:49.002235+0000","last_unstale":"2024-03-29T04:10:05.910183+0000","last_undegraded":"2024-03-29T04:10:05.910183+0000","last_fullsized":"2024-03-29T04:10:05.910183+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:47.900733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:47.900733+0000","last_clean_scrub_stamp":"2024-03-29T04:09:47.900733+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:44:04.608265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911029+0000","last_change":"2024-03-29T04:10:04.646698+0000","last_active":"2024-03-29T04:10:05.911029+0000","last_peered":"2024-03-29T04:10:05.911029+0000","last_clean":"2024-03-29T04:10:05.911029+0000","last_became_active":"2024-03-29T04:10:04.646221+0000","last_became_peered":"2024-03-29T04:10:04.646221+0000","last_unstale":"2024-03-29T04:10:05.911029+0000","last_undegraded":"2024-03-29T04:10:05.911029+0000","last_fullsized":"2024-03-29T04:10:05.911029+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:17.034970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911371+0000","last_change":"2024-03-29T04:10:00.540055+0000","last_active":"2024-03-29T04:10:05.911371+0000","last_peered":"2024-03-29T04:10:05.911371+0000","last_clean":"2024-03-29T04:10:05.911371+0000","last_became_active":"2024-03-29T04:10:00.539766+0000","last_became_peered":"2024-03-29T04:10:00.539766+0000","last_unstale":"2024-03-29T04:10:05.911371+0000","last_undegraded":"2024-03-29T04:10:05.911371+0000","last_fullsized":"2024-03-29T04:10:05.911371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:46.804154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.560157+0000","last_change":"2024-03-29T04:09:59.536860+0000","last_active":"2024-03-29T04:10:05.560157+0000","last_peered":"2024-03-29T04:10:05.560157+0000","last_clean":"2024-03-29T04:10:05.560157+0000","last_became_active":"2024-03-29T04:09:59.536504+0000","last_became_peered":"2024-03-29T04:09:59.536504+0000","last_unstale":"2024-03-29T04:10:05.560157+0000","last_undegraded":"2024-03-29T04:10:05.560157+0000","last_fullsized":"2024-03-29T04:10:05.560157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:02.879344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910369+0000","last_change":"2024-03-29T04:10:04.644446+0000","last_active":"2024-03-29T04:10:05.910369+0000","last_peered":"2024-03-29T04:10:05.910369+0000","last_clean":"2024-03-29T04:10:05.910369+0000","last_became_active":"2024-03-29T04:10:04.643982+0000","last_became_peered":"2024-03-29T04:10:04.643982+0000","last_unstale":"2024-03-29T04:10:05.910369+0000","last_undegraded":"2024-03-29T04:10:05.910369+0000","last_fullsized":"2024-03-29T04:10:05.910369+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:26:14.412794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556955+0000","last_change":"2024-03-29T04:10:00.544801+0000","last_active":"2024-03-29T04:10:05.556955+0000","last_peered":"2024-03-29T04:10:05.556955+0000","last_clean":"2024-03-29T04:10:05.556955+0000","last_became_active":"2024-03-29T04:10:00.544481+0000","last_became_peered":"2024-03-29T04:10:00.544481+0000","last_unstale":"2024-03-29T04:10:05.556955+0000","last_undegraded":"2024-03-29T04:10:05.556955+0000","last_fullsized":"2024-03-29T04:10:05.556955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:00:51.320824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911910+0000","last_change":"2024-03-29T04:09:59.541406+0000","last_active":"2024-03-29T04:10:05.911910+0000","last_peered":"2024-03-29T04:10:05.911910+0000","last_clean":"2024-03-29T04:10:05.911910+0000","last_became_active":"2024-03-29T04:09:59.541099+0000","last_became_peered":"2024-03-29T04:09:59.541099+0000","last_unstale":"2024-03-29T04:10:05.911910+0000","last_undegraded":"2024-03-29T04:10:05.911910+0000","last_fullsized":"2024-03-29T04:10:05.911910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:25:06.123311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911325+0000","last_change":"2024-03-29T04:10:04.579423+0000","last_active":"2024-03-29T04:10:05.911325+0000","last_peered":"2024-03-29T04:10:05.911325+0000","last_clean":"2024-03-29T04:10:05.911325+0000","last_became_active":"2024-03-29T04:10:04.579062+0000","last_became_peered":"2024-03-29T04:10:04.579062+0000","last_unstale":"2024-03-29T04:10:05.911325+0000","last_undegraded":"2024-03-29T04:10:05.911325+0000","last_fullsized":"2024-03-29T04:10:05.911325+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:06.998903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911863+0000","last_change":"2024-03-29T04:10:00.547282+0000","last_active":"2024-03-29T04:10:05.911863+0000","last_peered":"2024-03-29T04:10:05.911863+0000","last_clean":"2024-03-29T04:10:05.911863+0000","last_became_active":"2024-03-29T04:10:00.547026+0000","last_became_peered":"2024-03-29T04:10:00.547026+0000","last_unstale":"2024-03-29T04:10:05.911863+0000","last_undegraded":"2024-03-29T04:10:05.911863+0000","last_fullsized":"2024-03-29T04:10:05.911863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:34:06.367730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.560063+0000","last_change":"2024-03-29T04:09:59.540515+0000","last_active":"2024-03-29T04:10:05.560063+0000","last_peered":"2024-03-29T04:10:05.560063+0000","last_clean":"2024-03-29T04:10:05.560063+0000","last_became_active":"2024-03-29T04:09:59.540198+0000","last_became_peered":"2024-03-29T04:09:59.540198+0000","last_unstale":"2024-03-29T04:10:05.560063+0000","last_undegraded":"2024-03-29T04:10:05.560063+0000","last_fullsized":"2024-03-29T04:10:05.560063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:01.925223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556468+0000","last_change":"2024-03-29T04:10:04.644258+0000","last_active":"2024-03-29T04:10:05.556468+0000","last_peered":"2024-03-29T04:10:05.556468+0000","last_clean":"2024-03-29T04:10:05.556468+0000","last_became_active":"2024-03-29T04:10:04.643890+0000","last_became_peered":"2024-03-29T04:10:04.643890+0000","last_unstale":"2024-03-29T04:10:05.556468+0000","last_undegraded":"2024-03-29T04:10:05.556468+0000","last_fullsized":"2024-03-29T04:10:05.556468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:46:14.769141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911624+0000","last_change":"2024-03-29T04:10:00.549814+0000","last_active":"2024-03-29T04:10:05.911624+0000","last_peered":"2024-03-29T04:10:05.911624+0000","last_clean":"2024-03-29T04:10:05.911624+0000","last_became_active":"2024-03-29T04:10:00.549571+0000","last_became_peered":"2024-03-29T04:10:00.549571+0000","last_unstale":"2024-03-29T04:10:05.911624+0000","last_undegraded":"2024-03-29T04:10:05.911624+0000","last_fullsized":"2024-03-29T04:10:05.911624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:00:51.320824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557978+0000","last_change":"2024-03-29T04:09:59.538979+0000","last_active":"2024-03-29T04:10:05.557978+0000","last_peered":"2024-03-29T04:10:05.557978+0000","last_clean":"2024-03-29T04:10:05.557978+0000","last_became_active":"2024-03-29T04:09:59.538707+0000","last_became_peered":"2024-03-29T04:09:59.538707+0000","last_unstale":"2024-03-29T04:10:05.557978+0000","last_undegraded":"2024-03-29T04:10:05.557978+0000","last_fullsized":"2024-03-29T04:10:05.557978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:07:35.127071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556613+0000","last_change":"2024-03-29T04:10:04.574569+0000","last_active":"2024-03-29T04:10:05.556613+0000","last_peered":"2024-03-29T04:10:05.556613+0000","last_clean":"2024-03-29T04:10:05.556613+0000","last_became_active":"2024-03-29T04:10:04.574048+0000","last_became_peered":"2024-03-29T04:10:04.574048+0000","last_unstale":"2024-03-29T04:10:05.556613+0000","last_undegraded":"2024-03-29T04:10:05.556613+0000","last_fullsized":"2024-03-29T04:10:05.556613+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:33.266010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910658+0000","last_change":"2024-03-29T04:10:00.549205+0000","last_active":"2024-03-29T04:10:05.910658+0000","last_peered":"2024-03-29T04:10:05.910658+0000","last_clean":"2024-03-29T04:10:05.910658+0000","last_became_active":"2024-03-29T04:10:00.548863+0000","last_became_peered":"2024-03-29T04:10:00.548863+0000","last_unstale":"2024-03-29T04:10:05.910658+0000","last_undegraded":"2024-03-29T04:10:05.910658+0000","last_fullsized":"2024-03-29T04:10:05.910658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:30.096622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.559751+0000","last_change":"2024-03-29T04:09:59.538016+0000","last_active":"2024-03-29T04:10:05.559751+0000","last_peered":"2024-03-29T04:10:05.559751+0000","last_clean":"2024-03-29T04:10:05.559751+0000","last_became_active":"2024-03-29T04:09:59.537671+0000","last_became_peered":"2024-03-29T04:09:59.537671+0000","last_unstale":"2024-03-29T04:10:05.559751+0000","last_undegraded":"2024-03-29T04:10:05.559751+0000","last_fullsized":"2024-03-29T04:10:05.559751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:25:00.672418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912466+0000","last_change":"2024-03-29T04:10:04.574244+0000","last_active":"2024-03-29T04:10:05.912466+0000","last_peered":"2024-03-29T04:10:05.912466+0000","last_clean":"2024-03-29T04:10:05.912466+0000","last_became_active":"2024-03-29T04:10:04.573865+0000","last_became_peered":"2024-03-29T04:10:04.573865+0000","last_unstale":"2024-03-29T04:10:05.912466+0000","last_undegraded":"2024-03-29T04:10:05.912466+0000","last_fullsized":"2024-03-29T04:10:05.912466+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:23:51.029457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910228+0000","last_change":"2024-03-29T04:10:00.547804+0000","last_active":"2024-03-29T04:10:05.910228+0000","last_peered":"2024-03-29T04:10:05.910228+0000","last_clean":"2024-03-29T04:10:05.910228+0000","last_became_active":"2024-03-29T04:10:00.547500+0000","last_became_peered":"2024-03-29T04:10:00.547500+0000","last_unstale":"2024-03-29T04:10:05.910228+0000","last_undegraded":"2024-03-29T04:10:05.910228+0000","last_fullsized":"2024-03-29T04:10:05.910228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:52:12.957170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910669+0000","last_change":"2024-03-29T04:09:59.538286+0000","last_active":"2024-03-29T04:10:05.910669+0000","last_peered":"2024-03-29T04:10:05.910669+0000","last_clean":"2024-03-29T04:10:05.910669+0000","last_became_active":"2024-03-29T04:09:59.537970+0000","last_became_peered":"2024-03-29T04:09:59.537970+0000","last_unstale":"2024-03-29T04:10:05.910669+0000","last_undegraded":"2024-03-29T04:10:05.910669+0000","last_fullsized":"2024-03-29T04:10:05.910669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:05.307537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911473+0000","last_change":"2024-03-29T04:10:04.647055+0000","last_active":"2024-03-29T04:10:05.911473+0000","last_peered":"2024-03-29T04:10:05.911473+0000","last_clean":"2024-03-29T04:10:05.911473+0000","last_became_active":"2024-03-29T04:10:04.646604+0000","last_became_peered":"2024-03-29T04:10:04.646604+0000","last_unstale":"2024-03-29T04:10:05.911473+0000","last_undegraded":"2024-03-29T04:10:05.911473+0000","last_fullsized":"2024-03-29T04:10:05.911473+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:32:59.580503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911224+0000","last_change":"2024-03-29T04:10:00.548354+0000","last_active":"2024-03-29T04:10:05.911224+0000","last_peered":"2024-03-29T04:10:05.911224+0000","last_clean":"2024-03-29T04:10:05.911224+0000","last_became_active":"2024-03-29T04:10:00.548084+0000","last_became_peered":"2024-03-29T04:10:00.548084+0000","last_unstale":"2024-03-29T04:10:05.911224+0000","last_undegraded":"2024-03-29T04:10:05.911224+0000","last_fullsized":"2024-03-29T04:10:05.911224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:02.966428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557470+0000","last_change":"2024-03-29T04:09:59.538823+0000","last_active":"2024-03-29T04:10:05.557470+0000","last_peered":"2024-03-29T04:10:05.557470+0000","last_clean":"2024-03-29T04:10:05.557470+0000","last_became_active":"2024-03-29T04:09:59.538570+0000","last_became_peered":"2024-03-29T04:09:59.538570+0000","last_unstale":"2024-03-29T04:10:05.557470+0000","last_undegraded":"2024-03-29T04:10:05.557470+0000","last_fullsized":"2024-03-29T04:10:05.557470+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:02.879344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911403+0000","last_change":"2024-03-29T04:10:04.644206+0000","last_active":"2024-03-29T04:10:05.911403+0000","last_peered":"2024-03-29T04:10:05.911403+0000","last_clean":"2024-03-29T04:10:05.911403+0000","last_became_active":"2024-03-29T04:10:04.643690+0000","last_became_peered":"2024-03-29T04:10:04.643690+0000","last_unstale":"2024-03-29T04:10:05.911403+0000","last_undegraded":"2024-03-29T04:10:05.911403+0000","last_fullsized":"2024-03-29T04:10:05.911403+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:10:07.163684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910609+0000","last_change":"2024-03-29T04:10:00.541166+0000","last_active":"2024-03-29T04:10:05.910609+0000","last_peered":"2024-03-29T04:10:05.910609+0000","last_clean":"2024-03-29T04:10:05.910609+0000","last_became_active":"2024-03-29T04:10:00.540895+0000","last_became_peered":"2024-03-29T04:10:00.540895+0000","last_unstale":"2024-03-29T04:10:05.910609+0000","last_undegraded":"2024-03-29T04:10:05.910609+0000","last_fullsized":"2024-03-29T04:10:05.910609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:21:11.329145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911540+0000","last_change":"2024-03-29T04:09:59.538433+0000","last_active":"2024-03-29T04:10:05.911540+0000","last_peered":"2024-03-29T04:10:05.911540+0000","last_clean":"2024-03-29T04:10:05.911540+0000","last_became_active":"2024-03-29T04:09:59.538039+0000","last_became_peered":"2024-03-29T04:09:59.538039+0000","last_unstale":"2024-03-29T04:10:05.911540+0000","last_undegraded":"2024-03-29T04:10:05.911540+0000","last_fullsized":"2024-03-29T04:10:05.911540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:45:51.875624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557426+0000","last_change":"2024-03-29T04:10:04.647019+0000","last_active":"2024-03-29T04:10:05.557426+0000","last_peered":"2024-03-29T04:10:05.557426+0000","last_clean":"2024-03-29T04:10:05.557426+0000","last_became_active":"2024-03-29T04:10:04.646642+0000","last_became_peered":"2024-03-29T04:10:04.646642+0000","last_unstale":"2024-03-29T04:10:05.557426+0000","last_undegraded":"2024-03-29T04:10:05.557426+0000","last_fullsized":"2024-03-29T04:10:05.557426+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:07:56.606322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911280+0000","last_change":"2024-03-29T04:10:00.544134+0000","last_active":"2024-03-29T04:10:05.911280+0000","last_peered":"2024-03-29T04:10:05.911280+0000","last_clean":"2024-03-29T04:10:05.911280+0000","last_became_active":"2024-03-29T04:10:00.543534+0000","last_became_peered":"2024-03-29T04:10:00.543534+0000","last_unstale":"2024-03-29T04:10:05.911280+0000","last_undegraded":"2024-03-29T04:10:05.911280+0000","last_fullsized":"2024-03-29T04:10:05.911280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:26:04.629712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:10.817289+0000","last_change":"2024-03-29T04:09:59.544458+0000","last_active":"2024-03-29T04:10:10.817289+0000","last_peered":"2024-03-29T04:10:10.817289+0000","last_clean":"2024-03-29T04:10:10.817289+0000","last_became_active":"2024-03-29T04:09:59.544137+0000","last_became_peered":"2024-03-29T04:09:59.544137+0000","last_unstale":"2024-03-29T04:10:10.817289+0000","last_undegraded":"2024-03-29T04:10:10.817289+0000","last_fullsized":"2024-03-29T04:10:10.817289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:02.879344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912395+0000","last_change":"2024-03-29T04:10:04.576867+0000","last_active":"2024-03-29T04:10:05.912395+0000","last_peered":"2024-03-29T04:10:05.912395+0000","last_clean":"2024-03-29T04:10:05.912395+0000","last_became_active":"2024-03-29T04:10:04.576416+0000","last_became_peered":"2024-03-29T04:10:04.576416+0000","last_unstale":"2024-03-29T04:10:05.912395+0000","last_undegraded":"2024-03-29T04:10:05.912395+0000","last_fullsized":"2024-03-29T04:10:05.912395+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:34:10.400206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.558230+0000","last_change":"2024-03-29T04:10:00.547277+0000","last_active":"2024-03-29T04:10:05.558230+0000","last_peered":"2024-03-29T04:10:05.558230+0000","last_clean":"2024-03-29T04:10:05.558230+0000","last_became_active":"2024-03-29T04:10:00.547024+0000","last_became_peered":"2024-03-29T04:10:00.547024+0000","last_unstale":"2024-03-29T04:10:05.558230+0000","last_undegraded":"2024-03-29T04:10:05.558230+0000","last_fullsized":"2024-03-29T04:10:05.558230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:58:11.125964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911813+0000","last_change":"2024-03-29T04:09:59.541483+0000","last_active":"2024-03-29T04:10:05.911813+0000","last_peered":"2024-03-29T04:10:05.911813+0000","last_clean":"2024-03-29T04:10:05.911813+0000","last_became_active":"2024-03-29T04:09:59.541196+0000","last_became_peered":"2024-03-29T04:09:59.541196+0000","last_unstale":"2024-03-29T04:10:05.911813+0000","last_undegraded":"2024-03-29T04:10:05.911813+0000","last_fullsized":"2024-03-29T04:10:05.911813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:35:01.484589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557918+0000","last_change":"2024-03-29T04:10:04.574437+0000","last_active":"2024-03-29T04:10:05.557918+0000","last_peered":"2024-03-29T04:10:05.557918+0000","last_clean":"2024-03-29T04:10:05.557918+0000","last_became_active":"2024-03-29T04:10:04.573898+0000","last_became_peered":"2024-03-29T04:10:04.573898+0000","last_unstale":"2024-03-29T04:10:05.557918+0000","last_undegraded":"2024-03-29T04:10:05.557918+0000","last_fullsized":"2024-03-29T04:10:05.557918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:50.836630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911733+0000","last_change":"2024-03-29T04:10:00.536536+0000","last_active":"2024-03-29T04:10:05.911733+0000","last_peered":"2024-03-29T04:10:05.911733+0000","last_clean":"2024-03-29T04:10:05.911733+0000","last_became_active":"2024-03-29T04:10:00.536254+0000","last_became_peered":"2024-03-29T04:10:00.536254+0000","last_unstale":"2024-03-29T04:10:05.911733+0000","last_undegraded":"2024-03-29T04:10:05.911733+0000","last_fullsized":"2024-03-29T04:10:05.911733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:48.213845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557997+0000","last_change":"2024-03-29T04:09:59.541406+0000","last_active":"2024-03-29T04:10:05.557997+0000","last_peered":"2024-03-29T04:10:05.557997+0000","last_clean":"2024-03-29T04:10:05.557997+0000","last_became_active":"2024-03-29T04:09:59.541101+0000","last_became_peered":"2024-03-29T04:09:59.541101+0000","last_unstale":"2024-03-29T04:10:05.557997+0000","last_undegraded":"2024-03-29T04:10:05.557997+0000","last_fullsized":"2024-03-29T04:10:05.557997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:25:06.123311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910592+0000","last_change":"2024-03-29T04:10:04.577680+0000","last_active":"2024-03-29T04:10:05.910592+0000","last_peered":"2024-03-29T04:10:05.910592+0000","last_clean":"2024-03-29T04:10:05.910592+0000","last_became_active":"2024-03-29T04:10:04.577291+0000","last_became_peered":"2024-03-29T04:10:04.577291+0000","last_unstale":"2024-03-29T04:10:05.910592+0000","last_undegraded":"2024-03-29T04:10:05.910592+0000","last_fullsized":"2024-03-29T04:10:05.910592+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:46:14.769141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557898+0000","last_change":"2024-03-29T04:10:00.542773+0000","last_active":"2024-03-29T04:10:05.557898+0000","last_peered":"2024-03-29T04:10:05.557898+0000","last_clean":"2024-03-29T04:10:05.557898+0000","last_became_active":"2024-03-29T04:10:00.542081+0000","last_became_peered":"2024-03-29T04:10:00.542081+0000","last_unstale":"2024-03-29T04:10:05.557898+0000","last_undegraded":"2024-03-29T04:10:05.557898+0000","last_fullsized":"2024-03-29T04:10:05.557898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:26:04.629712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910800+0000","last_change":"2024-03-29T04:09:59.543623+0000","last_active":"2024-03-29T04:10:05.910800+0000","last_peered":"2024-03-29T04:10:05.910800+0000","last_clean":"2024-03-29T04:10:05.910800+0000","last_became_active":"2024-03-29T04:09:59.542122+0000","last_became_peered":"2024-03-29T04:09:59.542122+0000","last_unstale":"2024-03-29T04:10:05.910800+0000","last_undegraded":"2024-03-29T04:10:05.910800+0000","last_fullsized":"2024-03-29T04:10:05.910800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:01.925223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911195+0000","last_change":"2024-03-29T04:10:04.648725+0000","last_active":"2024-03-29T04:10:05.911195+0000","last_peered":"2024-03-29T04:10:05.911195+0000","last_clean":"2024-03-29T04:10:05.911195+0000","last_became_active":"2024-03-29T04:10:04.648125+0000","last_became_peered":"2024-03-29T04:10:04.648125+0000","last_unstale":"2024-03-29T04:10:05.911195+0000","last_undegraded":"2024-03-29T04:10:05.911195+0000","last_fullsized":"2024-03-29T04:10:05.911195+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:16:31.571145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.912466+0000","last_change":"2024-03-29T04:10:04.646516+0000","last_active":"2024-03-29T04:10:05.912466+0000","last_peered":"2024-03-29T04:10:05.912466+0000","last_clean":"2024-03-29T04:10:05.912466+0000","last_became_active":"2024-03-29T04:10:04.646118+0000","last_became_peered":"2024-03-29T04:10:04.646118+0000","last_unstale":"2024-03-29T04:10:05.912466+0000","last_undegraded":"2024-03-29T04:10:05.912466+0000","last_fullsized":"2024-03-29T04:10:05.912466+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:17.034970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.909860+0000","last_change":"2024-03-29T04:10:00.542208+0000","last_active":"2024-03-29T04:10:05.909860+0000","last_peered":"2024-03-29T04:10:05.909860+0000","last_clean":"2024-03-29T04:10:05.909860+0000","last_became_active":"2024-03-29T04:10:00.541927+0000","last_became_peered":"2024-03-29T04:10:00.541927+0000","last_unstale":"2024-03-29T04:10:05.909860+0000","last_undegraded":"2024-03-29T04:10:05.909860+0000","last_fullsized":"2024-03-29T04:10:05.909860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:16:02.331457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556546+0000","last_change":"2024-03-29T04:09:59.539257+0000","last_active":"2024-03-29T04:10:05.556546+0000","last_peered":"2024-03-29T04:10:05.556546+0000","last_clean":"2024-03-29T04:10:05.556546+0000","last_became_active":"2024-03-29T04:09:59.538298+0000","last_became_peered":"2024-03-29T04:09:59.538298+0000","last_unstale":"2024-03-29T04:10:05.556546+0000","last_undegraded":"2024-03-29T04:10:05.556546+0000","last_fullsized":"2024-03-29T04:10:05.556546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:33:50.904931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911024+0000","last_change":"2024-03-29T04:10:04.575619+0000","last_active":"2024-03-29T04:10:05.911024+0000","last_peered":"2024-03-29T04:10:05.911024+0000","last_clean":"2024-03-29T04:10:05.911024+0000","last_became_active":"2024-03-29T04:10:04.575138+0000","last_became_peered":"2024-03-29T04:10:04.575138+0000","last_unstale":"2024-03-29T04:10:05.911024+0000","last_undegraded":"2024-03-29T04:10:05.911024+0000","last_fullsized":"2024-03-29T04:10:05.911024+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:06.998903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910644+0000","last_change":"2024-03-29T04:10:00.543853+0000","last_active":"2024-03-29T04:10:05.910644+0000","last_peered":"2024-03-29T04:10:05.910644+0000","last_clean":"2024-03-29T04:10:05.910644+0000","last_became_active":"2024-03-29T04:10:00.543030+0000","last_became_peered":"2024-03-29T04:10:00.543030+0000","last_unstale":"2024-03-29T04:10:05.910644+0000","last_undegraded":"2024-03-29T04:10:05.910644+0000","last_fullsized":"2024-03-29T04:10:05.910644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:29.233534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911422+0000","last_change":"2024-03-29T04:09:59.537737+0000","last_active":"2024-03-29T04:10:05.911422+0000","last_peered":"2024-03-29T04:10:05.911422+0000","last_clean":"2024-03-29T04:10:05.911422+0000","last_became_active":"2024-03-29T04:09:59.537453+0000","last_became_peered":"2024-03-29T04:09:59.537453+0000","last_unstale":"2024-03-29T04:10:05.911422+0000","last_undegraded":"2024-03-29T04:10:05.911422+0000","last_fullsized":"2024-03-29T04:10:05.911422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:05.307537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910688+0000","last_change":"2024-03-29T04:09:59.538269+0000","last_active":"2024-03-29T04:10:05.910688+0000","last_peered":"2024-03-29T04:10:05.910688+0000","last_clean":"2024-03-29T04:10:05.910688+0000","last_became_active":"2024-03-29T04:09:59.537365+0000","last_became_peered":"2024-03-29T04:09:59.537365+0000","last_unstale":"2024-03-29T04:10:05.910688+0000","last_undegraded":"2024-03-29T04:10:05.910688+0000","last_fullsized":"2024-03-29T04:10:05.910688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:33:50.904931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911456+0000","last_change":"2024-03-29T04:10:00.532121+0000","last_active":"2024-03-29T04:10:05.911456+0000","last_peered":"2024-03-29T04:10:05.911456+0000","last_clean":"2024-03-29T04:10:05.911456+0000","last_became_active":"2024-03-29T04:10:00.531664+0000","last_became_peered":"2024-03-29T04:10:00.531664+0000","last_unstale":"2024-03-29T04:10:05.911456+0000","last_undegraded":"2024-03-29T04:10:05.911456+0000","last_fullsized":"2024-03-29T04:10:05.911456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:13:32.378332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.560130+0000","last_change":"2024-03-29T04:10:04.574558+0000","last_active":"2024-03-29T04:10:05.560130+0000","last_peered":"2024-03-29T04:10:05.560130+0000","last_clean":"2024-03-29T04:10:05.560130+0000","last_became_active":"2024-03-29T04:10:04.574156+0000","last_became_peered":"2024-03-29T04:10:04.574156+0000","last_unstale":"2024-03-29T04:10:05.560130+0000","last_undegraded":"2024-03-29T04:10:05.560130+0000","last_fullsized":"2024-03-29T04:10:05.560130+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:02:36.577244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910098+0000","last_change":"2024-03-29T04:09:59.536444+0000","last_active":"2024-03-29T04:10:05.910098+0000","last_peered":"2024-03-29T04:10:05.910098+0000","last_clean":"2024-03-29T04:10:05.910098+0000","last_became_active":"2024-03-29T04:09:59.536165+0000","last_became_peered":"2024-03-29T04:09:59.536165+0000","last_unstale":"2024-03-29T04:10:05.910098+0000","last_undegraded":"2024-03-29T04:10:05.910098+0000","last_fullsized":"2024-03-29T04:10:05.910098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:58.510388+0000","last_clean_scrub_stamp":"2024-03-29T04:09:58.510388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:37:46.716233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.558124+0000","last_change":"2024-03-29T04:10:00.544569+0000","last_active":"2024-03-29T04:10:05.558124+0000","last_peered":"2024-03-29T04:10:05.558124+0000","last_clean":"2024-03-29T04:10:05.558124+0000","last_became_active":"2024-03-29T04:10:00.544290+0000","last_became_peered":"2024-03-29T04:10:00.544290+0000","last_unstale":"2024-03-29T04:10:05.558124+0000","last_undegraded":"2024-03-29T04:10:05.558124+0000","last_fullsized":"2024-03-29T04:10:05.558124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:02.929633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910585+0000","last_change":"2024-03-29T04:10:04.580538+0000","last_active":"2024-03-29T04:10:05.910585+0000","last_peered":"2024-03-29T04:10:05.910585+0000","last_clean":"2024-03-29T04:10:05.910585+0000","last_became_active":"2024-03-29T04:10:04.580221+0000","last_became_peered":"2024-03-29T04:10:04.580221+0000","last_unstale":"2024-03-29T04:10:05.910585+0000","last_undegraded":"2024-03-29T04:10:05.910585+0000","last_fullsized":"2024-03-29T04:10:05.910585+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:03.547274+0000","last_clean_scrub_stamp":"2024-03-29T04:10:03.547274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:01:23.958242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910087+0000","last_change":"2024-03-29T04:10:00.546325+0000","last_active":"2024-03-29T04:10:05.910087+0000","last_peered":"2024-03-29T04:10:05.910087+0000","last_clean":"2024-03-29T04:10:05.910087+0000","last_became_active":"2024-03-29T04:10:00.545988+0000","last_became_peered":"2024-03-29T04:10:00.545988+0000","last_unstale":"2024-03-29T04:10:05.910087+0000","last_undegraded":"2024-03-29T04:10:05.910087+0000","last_fullsized":"2024-03-29T04:10:05.910087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:16:02.331457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.559836+0000","last_change":"2024-03-29T04:10:00.541221+0000","last_active":"2024-03-29T04:10:05.559836+0000","last_peered":"2024-03-29T04:10:05.559836+0000","last_clean":"2024-03-29T04:10:05.559836+0000","last_became_active":"2024-03-29T04:10:00.540946+0000","last_became_peered":"2024-03-29T04:10:00.540946+0000","last_unstale":"2024-03-29T04:10:05.559836+0000","last_undegraded":"2024-03-29T04:10:05.559836+0000","last_fullsized":"2024-03-29T04:10:05.559836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:35:02.488999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557358+0000","last_change":"2024-03-29T04:10:00.546304+0000","last_active":"2024-03-29T04:10:05.557358+0000","last_peered":"2024-03-29T04:10:05.557358+0000","last_clean":"2024-03-29T04:10:05.557358+0000","last_became_active":"2024-03-29T04:10:00.545529+0000","last_became_peered":"2024-03-29T04:10:00.545529+0000","last_unstale":"2024-03-29T04:10:05.557358+0000","last_undegraded":"2024-03-29T04:10:05.557358+0000","last_fullsized":"2024-03-29T04:10:05.557358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:44:13.128618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911371+0000","last_change":"2024-03-29T04:10:00.542033+0000","last_active":"2024-03-29T04:10:05.911371+0000","last_peered":"2024-03-29T04:10:05.911371+0000","last_clean":"2024-03-29T04:10:05.911371+0000","last_became_active":"2024-03-29T04:10:00.541764+0000","last_became_peered":"2024-03-29T04:10:00.541764+0000","last_unstale":"2024-03-29T04:10:05.911371+0000","last_undegraded":"2024-03-29T04:10:05.911371+0000","last_fullsized":"2024-03-29T04:10:05.911371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:46.707085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911616+0000","last_change":"2024-03-29T04:10:00.543634+0000","last_active":"2024-03-29T04:10:05.911616+0000","last_peered":"2024-03-29T04:10:05.911616+0000","last_clean":"2024-03-29T04:10:05.911616+0000","last_became_active":"2024-03-29T04:10:00.543343+0000","last_became_peered":"2024-03-29T04:10:00.543343+0000","last_unstale":"2024-03-29T04:10:05.911616+0000","last_undegraded":"2024-03-29T04:10:05.911616+0000","last_fullsized":"2024-03-29T04:10:05.911616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:35:00.452782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911632+0000","last_change":"2024-03-29T04:10:00.554863+0000","last_active":"2024-03-29T04:10:05.911632+0000","last_peered":"2024-03-29T04:10:05.911632+0000","last_clean":"2024-03-29T04:10:05.911632+0000","last_became_active":"2024-03-29T04:10:00.554560+0000","last_became_peered":"2024-03-29T04:10:00.554560+0000","last_unstale":"2024-03-29T04:10:05.911632+0000","last_undegraded":"2024-03-29T04:10:05.911632+0000","last_fullsized":"2024-03-29T04:10:05.911632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:29.233534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910973+0000","last_change":"2024-03-29T04:10:00.541999+0000","last_active":"2024-03-29T04:10:05.910973+0000","last_peered":"2024-03-29T04:10:05.910973+0000","last_clean":"2024-03-29T04:10:05.910973+0000","last_became_active":"2024-03-29T04:10:00.541699+0000","last_became_peered":"2024-03-29T04:10:00.541699+0000","last_unstale":"2024-03-29T04:10:05.910973+0000","last_undegraded":"2024-03-29T04:10:05.910973+0000","last_fullsized":"2024-03-29T04:10:05.910973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:48.213845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911525+0000","last_change":"2024-03-29T04:10:00.537506+0000","last_active":"2024-03-29T04:10:05.911525+0000","last_peered":"2024-03-29T04:10:05.911525+0000","last_clean":"2024-03-29T04:10:05.911525+0000","last_became_active":"2024-03-29T04:10:00.537239+0000","last_became_peered":"2024-03-29T04:10:00.537239+0000","last_unstale":"2024-03-29T04:10:05.911525+0000","last_undegraded":"2024-03-29T04:10:05.911525+0000","last_fullsized":"2024-03-29T04:10:05.911525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:57:30.096622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556535+0000","last_change":"2024-03-29T04:10:00.544443+0000","last_active":"2024-03-29T04:10:05.556535+0000","last_peered":"2024-03-29T04:10:05.556535+0000","last_clean":"2024-03-29T04:10:05.556535+0000","last_became_active":"2024-03-29T04:10:00.543427+0000","last_became_peered":"2024-03-29T04:10:00.543427+0000","last_unstale":"2024-03-29T04:10:05.556535+0000","last_undegraded":"2024-03-29T04:10:05.556535+0000","last_fullsized":"2024-03-29T04:10:05.556535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:48.213845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911292+0000","last_change":"2024-03-29T04:10:00.538050+0000","last_active":"2024-03-29T04:10:05.911292+0000","last_peered":"2024-03-29T04:10:05.911292+0000","last_clean":"2024-03-29T04:10:05.911292+0000","last_became_active":"2024-03-29T04:10:00.537771+0000","last_became_peered":"2024-03-29T04:10:00.537771+0000","last_unstale":"2024-03-29T04:10:05.911292+0000","last_undegraded":"2024-03-29T04:10:05.911292+0000","last_fullsized":"2024-03-29T04:10:05.911292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:21:11.329145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557260+0000","last_change":"2024-03-29T04:10:00.546534+0000","last_active":"2024-03-29T04:10:05.557260+0000","last_peered":"2024-03-29T04:10:05.557260+0000","last_clean":"2024-03-29T04:10:05.557260+0000","last_became_active":"2024-03-29T04:10:00.546028+0000","last_became_peered":"2024-03-29T04:10:00.546028+0000","last_unstale":"2024-03-29T04:10:05.557260+0000","last_undegraded":"2024-03-29T04:10:05.557260+0000","last_fullsized":"2024-03-29T04:10:05.557260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:23:46.996981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911142+0000","last_change":"2024-03-29T04:10:00.539866+0000","last_active":"2024-03-29T04:10:05.911142+0000","last_peered":"2024-03-29T04:10:05.911142+0000","last_clean":"2024-03-29T04:10:05.911142+0000","last_became_active":"2024-03-29T04:10:00.539562+0000","last_became_peered":"2024-03-29T04:10:00.539562+0000","last_unstale":"2024-03-29T04:10:05.911142+0000","last_undegraded":"2024-03-29T04:10:05.911142+0000","last_fullsized":"2024-03-29T04:10:05.911142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:52:12.957170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910850+0000","last_change":"2024-03-29T04:10:00.542361+0000","last_active":"2024-03-29T04:10:05.910850+0000","last_peered":"2024-03-29T04:10:05.910850+0000","last_clean":"2024-03-29T04:10:05.910850+0000","last_became_active":"2024-03-29T04:10:00.541625+0000","last_became_peered":"2024-03-29T04:10:00.541625+0000","last_unstale":"2024-03-29T04:10:05.910850+0000","last_undegraded":"2024-03-29T04:10:05.910850+0000","last_fullsized":"2024-03-29T04:10:05.910850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:46.707085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911557+0000","last_change":"2024-03-29T04:10:00.539117+0000","last_active":"2024-03-29T04:10:05.911557+0000","last_peered":"2024-03-29T04:10:05.911557+0000","last_clean":"2024-03-29T04:10:05.911557+0000","last_became_active":"2024-03-29T04:10:00.538833+0000","last_became_peered":"2024-03-29T04:10:00.538833+0000","last_unstale":"2024-03-29T04:10:05.911557+0000","last_undegraded":"2024-03-29T04:10:05.911557+0000","last_fullsized":"2024-03-29T04:10:05.911557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:13.002495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.560038+0000","last_change":"2024-03-29T04:10:00.551402+0000","last_active":"2024-03-29T04:10:05.560038+0000","last_peered":"2024-03-29T04:10:05.560038+0000","last_clean":"2024-03-29T04:10:05.560038+0000","last_became_active":"2024-03-29T04:10:00.551092+0000","last_became_peered":"2024-03-29T04:10:00.551092+0000","last_unstale":"2024-03-29T04:10:05.560038+0000","last_undegraded":"2024-03-29T04:10:05.560038+0000","last_fullsized":"2024-03-29T04:10:05.560038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:58:11.125964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.558449+0000","last_change":"2024-03-29T04:10:00.542777+0000","last_active":"2024-03-29T04:10:05.558449+0000","last_peered":"2024-03-29T04:10:05.558449+0000","last_clean":"2024-03-29T04:10:05.558449+0000","last_became_active":"2024-03-29T04:10:00.542241+0000","last_became_peered":"2024-03-29T04:10:00.542241+0000","last_unstale":"2024-03-29T04:10:05.558449+0000","last_undegraded":"2024-03-29T04:10:05.558449+0000","last_fullsized":"2024-03-29T04:10:05.558449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:46.804154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556402+0000","last_change":"2024-03-29T04:10:00.545811+0000","last_active":"2024-03-29T04:10:05.556402+0000","last_peered":"2024-03-29T04:10:05.556402+0000","last_clean":"2024-03-29T04:10:05.556402+0000","last_became_active":"2024-03-29T04:10:00.545501+0000","last_became_peered":"2024-03-29T04:10:00.545501+0000","last_unstale":"2024-03-29T04:10:05.556402+0000","last_undegraded":"2024-03-29T04:10:05.556402+0000","last_fullsized":"2024-03-29T04:10:05.556402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:29.233534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.909960+0000","last_change":"2024-03-29T04:10:00.545471+0000","last_active":"2024-03-29T04:10:05.909960+0000","last_peered":"2024-03-29T04:10:05.909960+0000","last_clean":"2024-03-29T04:10:05.909960+0000","last_became_active":"2024-03-29T04:10:00.545173+0000","last_became_peered":"2024-03-29T04:10:00.545173+0000","last_unstale":"2024-03-29T04:10:05.909960+0000","last_undegraded":"2024-03-29T04:10:05.909960+0000","last_fullsized":"2024-03-29T04:10:05.909960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:13:32.378332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910520+0000","last_change":"2024-03-29T04:10:00.545485+0000","last_active":"2024-03-29T04:10:05.910520+0000","last_peered":"2024-03-29T04:10:05.910520+0000","last_clean":"2024-03-29T04:10:05.910520+0000","last_became_active":"2024-03-29T04:10:00.545139+0000","last_became_peered":"2024-03-29T04:10:00.545139+0000","last_unstale":"2024-03-29T04:10:05.910520+0000","last_undegraded":"2024-03-29T04:10:05.910520+0000","last_fullsized":"2024-03-29T04:10:05.910520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:14:59.937144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.909924+0000","last_change":"2024-03-29T04:10:00.550441+0000","last_active":"2024-03-29T04:10:05.909924+0000","last_peered":"2024-03-29T04:10:05.909924+0000","last_clean":"2024-03-29T04:10:05.909924+0000","last_became_active":"2024-03-29T04:10:00.550150+0000","last_became_peered":"2024-03-29T04:10:00.550150+0000","last_unstale":"2024-03-29T04:10:05.909924+0000","last_undegraded":"2024-03-29T04:10:05.909924+0000","last_fullsized":"2024-03-29T04:10:05.909924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:52:12.957170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.558164+0000","last_change":"2024-03-29T04:10:00.540368+0000","last_active":"2024-03-29T04:10:05.558164+0000","last_peered":"2024-03-29T04:10:05.558164+0000","last_clean":"2024-03-29T04:10:05.558164+0000","last_became_active":"2024-03-29T04:10:00.538360+0000","last_became_peered":"2024-03-29T04:10:00.538360+0000","last_unstale":"2024-03-29T04:10:05.558164+0000","last_undegraded":"2024-03-29T04:10:05.558164+0000","last_fullsized":"2024-03-29T04:10:05.558164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:35:02.488999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911011+0000","last_change":"2024-03-29T04:10:00.550531+0000","last_active":"2024-03-29T04:10:05.911011+0000","last_peered":"2024-03-29T04:10:05.911011+0000","last_clean":"2024-03-29T04:10:05.911011+0000","last_became_active":"2024-03-29T04:10:00.550079+0000","last_became_peered":"2024-03-29T04:10:00.550079+0000","last_unstale":"2024-03-29T04:10:05.911011+0000","last_undegraded":"2024-03-29T04:10:05.911011+0000","last_fullsized":"2024-03-29T04:10:05.911011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:13:32.378332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910949+0000","last_change":"2024-03-29T04:10:00.550387+0000","last_active":"2024-03-29T04:10:05.910949+0000","last_peered":"2024-03-29T04:10:05.910949+0000","last_clean":"2024-03-29T04:10:05.910949+0000","last_became_active":"2024-03-29T04:10:00.550063+0000","last_became_peered":"2024-03-29T04:10:00.550063+0000","last_unstale":"2024-03-29T04:10:05.910949+0000","last_undegraded":"2024-03-29T04:10:05.910949+0000","last_fullsized":"2024-03-29T04:10:05.910949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:29.233534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911804+0000","last_change":"2024-03-29T04:10:00.541680+0000","last_active":"2024-03-29T04:10:05.911804+0000","last_peered":"2024-03-29T04:10:05.911804+0000","last_clean":"2024-03-29T04:10:05.911804+0000","last_became_active":"2024-03-29T04:10:00.541354+0000","last_became_peered":"2024-03-29T04:10:00.541354+0000","last_unstale":"2024-03-29T04:10:05.911804+0000","last_undegraded":"2024-03-29T04:10:05.911804+0000","last_fullsized":"2024-03-29T04:10:05.911804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:44:13.128618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910384+0000","last_change":"2024-03-29T04:10:00.544081+0000","last_active":"2024-03-29T04:10:05.910384+0000","last_peered":"2024-03-29T04:10:05.910384+0000","last_clean":"2024-03-29T04:10:05.910384+0000","last_became_active":"2024-03-29T04:10:00.543353+0000","last_became_peered":"2024-03-29T04:10:00.543353+0000","last_unstale":"2024-03-29T04:10:05.910384+0000","last_undegraded":"2024-03-29T04:10:05.910384+0000","last_fullsized":"2024-03-29T04:10:05.910384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:13:32.378332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910548+0000","last_change":"2024-03-29T04:10:00.543517+0000","last_active":"2024-03-29T04:10:05.910548+0000","last_peered":"2024-03-29T04:10:05.910548+0000","last_clean":"2024-03-29T04:10:05.910548+0000","last_became_active":"2024-03-29T04:10:00.542854+0000","last_became_peered":"2024-03-29T04:10:00.542854+0000","last_unstale":"2024-03-29T04:10:05.910548+0000","last_undegraded":"2024-03-29T04:10:05.910548+0000","last_fullsized":"2024-03-29T04:10:05.910548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:35:00.452782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.557167+0000","last_change":"2024-03-29T04:10:00.546548+0000","last_active":"2024-03-29T04:10:05.557167+0000","last_peered":"2024-03-29T04:10:05.557167+0000","last_clean":"2024-03-29T04:10:05.557167+0000","last_became_active":"2024-03-29T04:10:00.546267+0000","last_became_peered":"2024-03-29T04:10:00.546267+0000","last_unstale":"2024-03-29T04:10:05.557167+0000","last_undegraded":"2024-03-29T04:10:05.557167+0000","last_fullsized":"2024-03-29T04:10:05.557167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:26:10.380318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911101+0000","last_change":"2024-03-29T04:10:00.539796+0000","last_active":"2024-03-29T04:10:05.911101+0000","last_peered":"2024-03-29T04:10:05.911101+0000","last_clean":"2024-03-29T04:10:05.911101+0000","last_became_active":"2024-03-29T04:10:00.539532+0000","last_became_peered":"2024-03-29T04:10:00.539532+0000","last_unstale":"2024-03-29T04:10:05.911101+0000","last_undegraded":"2024-03-29T04:10:05.911101+0000","last_fullsized":"2024-03-29T04:10:05.911101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:26:10.380318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.556840+0000","last_change":"2024-03-29T04:10:00.549077+0000","last_active":"2024-03-29T04:10:05.556840+0000","last_peered":"2024-03-29T04:10:05.556840+0000","last_clean":"2024-03-29T04:10:05.556840+0000","last_became_active":"2024-03-29T04:10:00.548266+0000","last_became_peered":"2024-03-29T04:10:00.548266+0000","last_unstale":"2024-03-29T04:10:05.556840+0000","last_undegraded":"2024-03-29T04:10:05.556840+0000","last_fullsized":"2024-03-29T04:10:05.556840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:35:00.452782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.910792+0000","last_change":"2024-03-29T04:10:00.535635+0000","last_active":"2024-03-29T04:10:05.910792+0000","last_peered":"2024-03-29T04:10:05.910792+0000","last_clean":"2024-03-29T04:10:05.910792+0000","last_became_active":"2024-03-29T04:10:00.535357+0000","last_became_peered":"2024-03-29T04:10:00.535357+0000","last_unstale":"2024-03-29T04:10:05.910792+0000","last_undegraded":"2024-03-29T04:10:05.910792+0000","last_fullsized":"2024-03-29T04:10:05.910792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:41.517875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911835+0000","last_change":"2024-03-29T04:10:00.547651+0000","last_active":"2024-03-29T04:10:05.911835+0000","last_peered":"2024-03-29T04:10:05.911835+0000","last_clean":"2024-03-29T04:10:05.911835+0000","last_became_active":"2024-03-29T04:10:00.547369+0000","last_became_peered":"2024-03-29T04:10:00.547369+0000","last_unstale":"2024-03-29T04:10:05.911835+0000","last_undegraded":"2024-03-29T04:10:05.911835+0000","last_fullsized":"2024-03-29T04:10:05.911835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:46.707085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T04:10:05.911744+0000","last_change":"2024-03-29T04:10:00.545663+0000","last_active":"2024-03-29T04:10:05.911744+0000","last_peered":"2024-03-29T04:10:05.911744+0000","last_clean":"2024-03-29T04:10:05.911744+0000","last_became_active":"2024-03-29T04:10:00.545394+0000","last_became_peered":"2024-03-29T04:10:00.545394+0000","last_unstale":"2024-03-29T04:10:05.911744+0000","last_undegraded":"2024-03-29T04:10:05.911744+0000","last_fullsized":"2024-03-29T04:10:05.911744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:59.514798+0000","last_clean_scrub_stamp":"2024-03-29T04:09:59.514798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:23:46.996981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:10:29.536 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-29T04:10:29.536 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-29T04:10:29.536 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-29T04:10:29.537 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-29T04:10:29.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.697+0000 7f6d1b69a640 1 Processor -- start 2024-03-29T04:10:29.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.697+0000 7f6d1b69a640 1 -- start start 2024-03-29T04:10:29.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.698+0000 7f6d1b69a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6d1406b5b0 0x7f6d140ce710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:29.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.698+0000 7f6d1b69a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d1406dc80 con 0x7f6d14072c30 2024-03-29T04:10:29.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.698+0000 7f6d1b69a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d140b5ec0 con 0x7f6d1415e770 2024-03-29T04:10:29.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.698+0000 7f6d1b69a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f6d140b4090 con 0x7f6d1406b5b0 2024-03-29T04:10:29.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.698+0000 7f6d19c37640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6d1406b5b0 0x7f6d140ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:29.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.698+0000 7f6d19c37640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6d1406b5b0 0x7f6d140ce710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48962/0 (socket says 172.21.15.148:48962) 2024-03-29T04:10:29.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.698+0000 7f6d18c35640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6d14072c30 0x7f6d140cf860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47344/0 (socket says 172.21.15.148:47344) 2024-03-29T04:10:29.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.698+0000 7f6d19c37640 1 -- 172.21.15.148:0/2889095324 learned_addr learned my addr 172.21.15.148:0/2889095324 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:29.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.699+0000 7f6d19c37640 1 -- 172.21.15.148:0/2889095324 >> v1:172.21.15.157:6789/0 conn(0x7f6d1415e770 legacy=0x7f6d14072520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:29.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.699+0000 7f6d19c37640 1 -- 172.21.15.148:0/2889095324 >> v1:172.21.15.148:6789/0 conn(0x7f6d14072c30 legacy=0x7f6d140cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:29.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.699+0000 7f6d19c37640 1 -- 172.21.15.148:0/2889095324 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d140b61a0 con 0x7f6d1406b5b0 2024-03-29T04:10:29.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.699+0000 7f6d13fff640 1 -- 172.21.15.148:0/2889095324 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4292069013 0 0) 0x7f6d1406dc80 con 0x7f6d14072c30 2024-03-29T04:10:29.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.699+0000 7f6d19c37640 1 --2- 172.21.15.148:0/2889095324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d1406b5b0 0x7f6d140ce710 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6d04009870 tx=0x7f6d0402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982a3bc598cdbf37 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:29.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.700+0000 7f6d13fff640 1 -- 172.21.15.148:0/2889095324 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0402f840 con 0x7f6d1406b5b0 2024-03-29T04:10:29.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.700+0000 7f6d13fff640 1 -- 172.21.15.148:0/2889095324 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d0402f9e0 con 0x7f6d1406b5b0 2024-03-29T04:10:29.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.700+0000 7f6d13fff640 1 -- 172.21.15.148:0/2889095324 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0402fdb0 con 0x7f6d1406b5b0 2024-03-29T04:10:29.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.700+0000 7f6d1b69a640 1 -- 172.21.15.148:0/2889095324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d1406b5b0 msgr2=0x7f6d140ce710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:29.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.700+0000 7f6d1b69a640 1 --2- 172.21.15.148:0/2889095324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d1406b5b0 0x7f6d140ce710 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6d04009870 tx=0x7f6d0402ba20 comp rx=0 tx=0).stop 2024-03-29T04:10:29.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.700+0000 7f6d1b69a640 1 -- 172.21.15.148:0/2889095324 shutdown_connections 2024-03-29T04:10:29.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.700+0000 7f6d1b69a640 1 --2- 172.21.15.148:0/2889095324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d1406b5b0 0x7f6d140ce710 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:29.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.700+0000 7f6d1b69a640 1 -- 172.21.15.148:0/2889095324 >> 172.21.15.148:0/2889095324 conn(0x7f6d140d0f50 msgr2=0x7f6d140d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:29.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.700+0000 7f6d1b69a640 1 -- 172.21.15.148:0/2889095324 shutdown_connections 2024-03-29T04:10:29.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.700+0000 7f6d1b69a640 1 -- 172.21.15.148:0/2889095324 wait complete. 2024-03-29T04:10:29.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.702+0000 7f6d1b69a640 1 Processor -- start 2024-03-29T04:10:29.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.702+0000 7f6d1b69a640 1 -- start start 2024-03-29T04:10:29.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.703+0000 7f6d1b69a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d1406b5b0 0x7f6d1415d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:29.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.703+0000 7f6d1b69a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d14072c30 0x7f6d14145ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:29.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.703+0000 7f6d1b69a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d1415e770 0x7f6d1415b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:29.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.703+0000 7f6d1b69a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1406da20 con 0x7f6d14072c30 2024-03-29T04:10:29.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.703+0000 7f6d1b69a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1406d8a0 con 0x7f6d1406b5b0 2024-03-29T04:10:29.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.703+0000 7f6d1b69a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6d140b5ae0 con 0x7f6d1415e770 2024-03-29T04:10:29.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.704+0000 7f6d18c35640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d14072c30 0x7f6d14145ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:29.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.704+0000 7f6d18c35640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d14072c30 0x7f6d14145ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48974/0 (socket says 172.21.15.148:48974) 2024-03-29T04:10:29.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.704+0000 7f6d18c35640 1 -- 172.21.15.148:0/4142828028 learned_addr learned my addr 172.21.15.148:0/4142828028 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:29.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.704+0000 7f6d19c37640 1 --2- 172.21.15.148:0/4142828028 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d1415e770 0x7f6d1415b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:29.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.704+0000 7f6d19436640 1 --2- 172.21.15.148:0/4142828028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d1406b5b0 0x7f6d1415d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:29.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.704+0000 7f6d18c35640 1 -- 172.21.15.148:0/4142828028 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d1415e770 msgr2=0x7f6d1415b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:29.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.704+0000 7f6d18c35640 1 --2- 172.21.15.148:0/4142828028 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d1415e770 0x7f6d1415b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:29.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.704+0000 7f6d18c35640 1 -- 172.21.15.148:0/4142828028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d1406b5b0 msgr2=0x7f6d1415d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:29.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.704+0000 7f6d18c35640 1 --2- 172.21.15.148:0/4142828028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d1406b5b0 0x7f6d1415d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:29.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.704+0000 7f6d18c35640 1 -- 172.21.15.148:0/4142828028 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d1415bfb0 con 0x7f6d14072c30 2024-03-29T04:10:29.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.704+0000 7f6d18c35640 1 --2- 172.21.15.148:0/4142828028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d14072c30 0x7f6d14145ac0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800c4a0 tx=0x7f6d0800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:29.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.704+0000 7f6d11ffb640 1 -- 172.21.15.148:0/4142828028 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d08018070 con 0x7f6d14072c30 2024-03-29T04:10:29.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.704+0000 7f6d11ffb640 1 -- 172.21.15.148:0/4142828028 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d080074e0 con 0x7f6d14072c30 2024-03-29T04:10:29.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.704+0000 7f6d1b69a640 1 -- 172.21.15.148:0/4142828028 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d1415c290 con 0x7f6d14072c30 2024-03-29T04:10:29.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.704+0000 7f6d11ffb640 1 -- 172.21.15.148:0/4142828028 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d08004a20 con 0x7f6d14072c30 2024-03-29T04:10:29.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.704+0000 7f6d1b69a640 1 -- 172.21.15.148:0/4142828028 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d1415c850 con 0x7f6d14072c30 2024-03-29T04:10:29.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.706+0000 7f6d1b69a640 1 -- 172.21.15.148:0/4142828028 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d140687e0 con 0x7f6d14072c30 2024-03-29T04:10:29.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.706+0000 7f6d11ffb640 1 -- 172.21.15.148:0/4142828028 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6d0801b070 con 0x7f6d14072c30 2024-03-29T04:10:29.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.706+0000 7f6d11ffb640 1 --2- 172.21.15.148:0/4142828028 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6d0006c530 0x7f6d0006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:29.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.706+0000 7f6d11ffb640 1 -- 172.21.15.148:0/4142828028 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6d0808acc0 con 0x7f6d14072c30 2024-03-29T04:10:29.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.706+0000 7f6d19436640 1 --2- 172.21.15.148:0/4142828028 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6d0006c530 0x7f6d0006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:29.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.707+0000 7f6d19436640 1 --2- 172.21.15.148:0/4142828028 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6d0006c530 0x7f6d0006e9f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6d140cf760 tx=0x7f6d0c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:29.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:29.711+0000 7f6d11ffb640 1 -- 172.21.15.148:0/4142828028 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6d08058950 con 0x7f6d14072c30 2024-03-29T04:10:30.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.098+0000 7f6d1b69a640 1 -- 172.21.15.148:0/4142828028 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6d140cf4b0 con 0x7f6d14072c30 2024-03-29T04:10:30.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.099+0000 7f6d11ffb640 1 -- 172.21.15.148:0/4142828028 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f6d0805cc50 con 0x7f6d14072c30 2024-03-29T04:10:30.078 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:30.078 INFO:teuthology.orchestra.run.smithi148.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-29T04:10:30.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.101+0000 7f6d1b69a640 1 -- 172.21.15.148:0/4142828028 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6d0006c530 msgr2=0x7f6d0006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:30.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.101+0000 7f6d1b69a640 1 --2- 172.21.15.148:0/4142828028 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6d0006c530 0x7f6d0006e9f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6d140cf760 tx=0x7f6d0c009040 comp rx=0 tx=0).stop 2024-03-29T04:10:30.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.101+0000 7f6d1b69a640 1 -- 172.21.15.148:0/4142828028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d14072c30 msgr2=0x7f6d14145ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:30.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.101+0000 7f6d1b69a640 1 --2- 172.21.15.148:0/4142828028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d14072c30 0x7f6d14145ac0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800c4a0 tx=0x7f6d0800c960 comp rx=0 tx=0).stop 2024-03-29T04:10:30.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.102+0000 7f6d1b69a640 1 -- 172.21.15.148:0/4142828028 shutdown_connections 2024-03-29T04:10:30.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.102+0000 7f6d1b69a640 1 --2- 172.21.15.148:0/4142828028 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6d0006c530 0x7f6d0006e9f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.102+0000 7f6d1b69a640 1 --2- 172.21.15.148:0/4142828028 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d1415e770 0x7f6d1415b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.102+0000 7f6d1b69a640 1 --2- 172.21.15.148:0/4142828028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d14072c30 0x7f6d14145ac0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.102+0000 7f6d1b69a640 1 --2- 172.21.15.148:0/4142828028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d1406b5b0 0x7f6d1415d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.102+0000 7f6d1b69a640 1 -- 172.21.15.148:0/4142828028 >> 172.21.15.148:0/4142828028 conn(0x7f6d140d0f50 msgr2=0x7f6d141613e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:30.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.102+0000 7f6d1b69a640 1 -- 172.21.15.148:0/4142828028 shutdown_connections 2024-03-29T04:10:30.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.102+0000 7f6d1b69a640 1 -- 172.21.15.148:0/4142828028 wait complete. 2024-03-29T04:10:30.091 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-29T04:10:30.092 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:30.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.216+0000 7fd256a28640 1 Processor -- start 2024-03-29T04:10:30.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.216+0000 7fd256a28640 1 -- start start 2024-03-29T04:10:30.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.216+0000 7fd256a28640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd2500ba030 0x7fd2500d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:30.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.216+0000 7fd256a28640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd2500d10d0 0x7fd2500d1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:30.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.216+0000 7fd256a28640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd25015e7c0 con 0x7fd25015e3e0 2024-03-29T04:10:30.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.216+0000 7fd256a28640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fd2500d1a70 con 0x7fd2500ba030 2024-03-29T04:10:30.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.216+0000 7fd256a28640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd2500d3840 con 0x7fd2500d10d0 2024-03-29T04:10:30.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.217+0000 7fd24f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd2500ba030 0x7fd2500d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:30.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.217+0000 7fd24f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd2500ba030 0x7fd2500d3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48988/0 (socket says 172.21.15.148:48988) 2024-03-29T04:10:30.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.217+0000 7fd24f7fe640 1 -- 172.21.15.148:0/1677688543 learned_addr learned my addr 172.21.15.148:0/1677688543 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:30.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.217+0000 7fd254fc5640 1 --2- 172.21.15.148:0/1677688543 >> v2:172.21.15.157:3300/0 conn(0x7fd2500d10d0 0x7fd2500d1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:30.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.217+0000 7fd24f7fe640 1 -- 172.21.15.148:0/1677688543 >> v2:172.21.15.157:3300/0 conn(0x7fd2500d10d0 msgr2=0x7fd2500d1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:30.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.217+0000 7fd24f7fe640 1 --2- 172.21.15.148:0/1677688543 >> v2:172.21.15.157:3300/0 conn(0x7fd2500d10d0 0x7fd2500d1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.217+0000 7fd24f7fe640 1 -- 172.21.15.148:0/1677688543 >> v1:172.21.15.157:6789/0 conn(0x7fd25015e3e0 legacy=0x7fd2500d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:30.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.217+0000 7fd24f7fe640 1 -- 172.21.15.148:0/1677688543 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2500d2c30 con 0x7fd2500ba030 2024-03-29T04:10:30.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.218+0000 7fd24f7fe640 1 --2- 172.21.15.148:0/1677688543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2500ba030 0x7fd2500d3cd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd2500c7c30 tx=0x7fd24002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55bcd050e72f08f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:30.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.218+0000 7fd24effd640 1 -- 172.21.15.148:0/1677688543 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24002f850 con 0x7fd2500ba030 2024-03-29T04:10:30.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.218+0000 7fd24effd640 1 -- 172.21.15.148:0/1677688543 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd24002f9f0 con 0x7fd2500ba030 2024-03-29T04:10:30.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.218+0000 7fd24effd640 1 -- 172.21.15.148:0/1677688543 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24002fdc0 con 0x7fd2500ba030 2024-03-29T04:10:30.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.218+0000 7fd256a28640 1 -- 172.21.15.148:0/1677688543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2500ba030 msgr2=0x7fd2500d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:30.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.218+0000 7fd256a28640 1 --2- 172.21.15.148:0/1677688543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2500ba030 0x7fd2500d3cd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd2500c7c30 tx=0x7fd24002ba30 comp rx=0 tx=0).stop 2024-03-29T04:10:30.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.218+0000 7fd256a28640 1 -- 172.21.15.148:0/1677688543 shutdown_connections 2024-03-29T04:10:30.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.218+0000 7fd256a28640 1 --2- 172.21.15.148:0/1677688543 >> v2:172.21.15.157:3300/0 conn(0x7fd2500d10d0 0x7fd2500d1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.218+0000 7fd256a28640 1 --2- 172.21.15.148:0/1677688543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2500ba030 0x7fd2500d3cd0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.218+0000 7fd256a28640 1 -- 172.21.15.148:0/1677688543 >> 172.21.15.148:0/1677688543 conn(0x7fd2500b6210 msgr2=0x7fd2500b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:30.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.218+0000 7fd256a28640 1 -- 172.21.15.148:0/1677688543 shutdown_connections 2024-03-29T04:10:30.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.219+0000 7fd256a28640 1 -- 172.21.15.148:0/1677688543 wait complete. 2024-03-29T04:10:30.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.220+0000 7fd256a28640 1 Processor -- start 2024-03-29T04:10:30.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.220+0000 7fd256a28640 1 -- start start 2024-03-29T04:10:30.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.221+0000 7fd256a28640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2500ba030 0x7fd2501f87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:30.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.221+0000 7fd256a28640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2500d10d0 0x7fd250151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:30.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.221+0000 7fd256a28640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd25015e3e0 0x7fd2501f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:30.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.221+0000 7fd256a28640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd2500d34d0 con 0x7fd25015e3e0 2024-03-29T04:10:30.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.221+0000 7fd256a28640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd2500d31f0 con 0x7fd2500d10d0 2024-03-29T04:10:30.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.221+0000 7fd256a28640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd250167f50 con 0x7fd2500ba030 2024-03-29T04:10:30.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.221+0000 7fd254fc5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd25015e3e0 0x7fd2501f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:30.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.221+0000 7fd254fc5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd25015e3e0 0x7fd2501f47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49000/0 (socket says 172.21.15.148:49000) 2024-03-29T04:10:30.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.221+0000 7fd254fc5640 1 -- 172.21.15.148:0/853951875 learned_addr learned my addr 172.21.15.148:0/853951875 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:30.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.221+0000 7fd24ffff640 1 --2- 172.21.15.148:0/853951875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2500ba030 0x7fd2501f87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:30.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.221+0000 7fd24f7fe640 1 --2- 172.21.15.148:0/853951875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2500d10d0 0x7fd250151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:30.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.221+0000 7fd254fc5640 1 -- 172.21.15.148:0/853951875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2500ba030 msgr2=0x7fd2501f87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:30.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.221+0000 7fd254fc5640 1 --2- 172.21.15.148:0/853951875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2500ba030 0x7fd2501f87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.221+0000 7fd254fc5640 1 -- 172.21.15.148:0/853951875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2500d10d0 msgr2=0x7fd250151ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:30.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.221+0000 7fd254fc5640 1 --2- 172.21.15.148:0/853951875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2500d10d0 0x7fd250151ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.221+0000 7fd254fc5640 1 -- 172.21.15.148:0/853951875 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2501f5320 con 0x7fd25015e3e0 2024-03-29T04:10:30.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.222+0000 7fd254fc5640 1 --2- 172.21.15.148:0/853951875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd25015e3e0 0x7fd2501f47f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd248007e40 tx=0x7fd24800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:30.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.222+0000 7fd24cff9640 1 -- 172.21.15.148:0/853951875 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd248014620 con 0x7fd25015e3e0 2024-03-29T04:10:30.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.222+0000 7fd24cff9640 1 -- 172.21.15.148:0/853951875 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd248019070 con 0x7fd25015e3e0 2024-03-29T04:10:30.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.222+0000 7fd256a28640 1 -- 172.21.15.148:0/853951875 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2501f5600 con 0x7fd25015e3e0 2024-03-29T04:10:30.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.222+0000 7fd24cff9640 1 -- 172.21.15.148:0/853951875 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd248014a20 con 0x7fd25015e3e0 2024-03-29T04:10:30.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.222+0000 7fd256a28640 1 -- 172.21.15.148:0/853951875 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2500cd580 con 0x7fd25015e3e0 2024-03-29T04:10:30.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.223+0000 7fd256a28640 1 -- 172.21.15.148:0/853951875 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2500687e0 con 0x7fd25015e3e0 2024-03-29T04:10:30.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.223+0000 7fd24cff9640 1 -- 172.21.15.148:0/853951875 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd248014bc0 con 0x7fd25015e3e0 2024-03-29T04:10:30.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.224+0000 7fd24cff9640 1 --2- 172.21.15.148:0/853951875 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd23006c530 0x7fd23006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:30.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.224+0000 7fd24ffff640 1 --2- 172.21.15.148:0/853951875 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd23006c530 0x7fd23006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:30.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.224+0000 7fd24cff9640 1 -- 172.21.15.148:0/853951875 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd24808c210 con 0x7fd25015e3e0 2024-03-29T04:10:30.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.224+0000 7fd24ffff640 1 --2- 172.21.15.148:0/853951875 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd23006c530 0x7fd23006e9f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd2380016e0 tx=0x7fd238009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:30.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.228+0000 7fd24cff9640 1 -- 172.21.15.148:0/853951875 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd248059eb0 con 0x7fd25015e3e0 2024-03-29T04:10:30.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.505+0000 7fd256a28640 1 -- 172.21.15.148:0/853951875 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2500b88d0 con 0x7fd25015e3e0 2024-03-29T04:10:30.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.507+0000 7fd24cff9640 1 -- 172.21.15.148:0/853951875 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7fd24805e1b0 con 0x7fd25015e3e0 2024-03-29T04:10:30.486 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:30.486 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:05.026481+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:30.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.509+0000 7fd256a28640 1 -- 172.21.15.148:0/853951875 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd23006c530 msgr2=0x7fd23006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:30.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.509+0000 7fd256a28640 1 --2- 172.21.15.148:0/853951875 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd23006c530 0x7fd23006e9f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd2380016e0 tx=0x7fd238009040 comp rx=0 tx=0).stop 2024-03-29T04:10:30.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.509+0000 7fd256a28640 1 -- 172.21.15.148:0/853951875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd25015e3e0 msgr2=0x7fd2501f47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:30.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.509+0000 7fd256a28640 1 --2- 172.21.15.148:0/853951875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd25015e3e0 0x7fd2501f47f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd248007e40 tx=0x7fd24800e4b0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.509+0000 7fd256a28640 1 -- 172.21.15.148:0/853951875 shutdown_connections 2024-03-29T04:10:30.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.509+0000 7fd256a28640 1 --2- 172.21.15.148:0/853951875 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd23006c530 0x7fd23006e9f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.509+0000 7fd256a28640 1 --2- 172.21.15.148:0/853951875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd25015e3e0 0x7fd2501f47f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.509+0000 7fd256a28640 1 --2- 172.21.15.148:0/853951875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2500d10d0 0x7fd250151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.509+0000 7fd256a28640 1 --2- 172.21.15.148:0/853951875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2500ba030 0x7fd2501f87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.509+0000 7fd256a28640 1 -- 172.21.15.148:0/853951875 >> 172.21.15.148:0/853951875 conn(0x7fd2500b6210 msgr2=0x7fd2500b7fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:30.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.509+0000 7fd256a28640 1 -- 172.21.15.148:0/853951875 shutdown_connections 2024-03-29T04:10:30.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.509+0000 7fd256a28640 1 -- 172.21.15.148:0/853951875 wait complete. 2024-03-29T04:10:30.499 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:47.898351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:57.922313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:58.829405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:10:02.551763+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:10:30.500 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:10:30.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.625+0000 7fda3443f640 1 Processor -- start 2024-03-29T04:10:30.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.625+0000 7fda3443f640 1 -- start start 2024-03-29T04:10:30.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.626+0000 7fda3443f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fda2c15e3c0 0x7fda2c161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:30.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.626+0000 7fda3443f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda2c0b2f00 0x7fda2c1624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:30.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.626+0000 7fda3443f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda2c0cf330 con 0x7fda2c0b3430 2024-03-29T04:10:30.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.626+0000 7fda3443f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fda2c0cf960 con 0x7fda2c15e3c0 2024-03-29T04:10:30.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.626+0000 7fda3443f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fda2c162cf0 con 0x7fda2c0b2f00 2024-03-29T04:10:30.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.626+0000 7fda321db640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fda2c15e3c0 0x7fda2c161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:30.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.626+0000 7fda321db640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fda2c15e3c0 0x7fda2c161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49002/0 (socket says 172.21.15.148:49002) 2024-03-29T04:10:30.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.626+0000 7fda321db640 1 -- 172.21.15.148:0/893641294 learned_addr learned my addr 172.21.15.148:0/893641294 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:30.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.626+0000 7fda319da640 1 --2- 172.21.15.148:0/893641294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda2c0b2f00 0x7fda2c1624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:30.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.626+0000 7fda321db640 1 -- 172.21.15.148:0/893641294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda2c0b2f00 msgr2=0x7fda2c1624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:30.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.626+0000 7fda321db640 1 --2- 172.21.15.148:0/893641294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda2c0b2f00 0x7fda2c1624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.626+0000 7fda321db640 1 -- 172.21.15.148:0/893641294 >> v1:172.21.15.157:6789/0 conn(0x7fda2c0b3430 legacy=0x7fda2c0d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:30.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.626+0000 7fda321db640 1 -- 172.21.15.148:0/893641294 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2c0cf5f0 con 0x7fda2c15e3c0 2024-03-29T04:10:30.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.627+0000 7fda321db640 1 --2- 172.21.15.148:0/893641294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda2c15e3c0 0x7fda2c161f70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fda28009870 tx=0x7fda2802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85612def48cef72 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:30.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.630+0000 7fda311d9640 1 -- 172.21.15.148:0/893641294 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28035020 con 0x7fda2c15e3c0 2024-03-29T04:10:30.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.631+0000 7fda311d9640 1 -- 172.21.15.148:0/893641294 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda28031640 con 0x7fda2c15e3c0 2024-03-29T04:10:30.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.631+0000 7fda311d9640 1 -- 172.21.15.148:0/893641294 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2802d040 con 0x7fda2c15e3c0 2024-03-29T04:10:30.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.631+0000 7fda3443f640 1 -- 172.21.15.148:0/893641294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda2c15e3c0 msgr2=0x7fda2c161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:30.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.631+0000 7fda3443f640 1 --2- 172.21.15.148:0/893641294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda2c15e3c0 0x7fda2c161f70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fda28009870 tx=0x7fda2802b8f0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.631+0000 7fda3443f640 1 -- 172.21.15.148:0/893641294 shutdown_connections 2024-03-29T04:10:30.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.631+0000 7fda3443f640 1 --2- 172.21.15.148:0/893641294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda2c0b2f00 0x7fda2c1624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.631+0000 7fda3443f640 1 --2- 172.21.15.148:0/893641294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda2c15e3c0 0x7fda2c161f70 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.631+0000 7fda3443f640 1 -- 172.21.15.148:0/893641294 >> 172.21.15.148:0/893641294 conn(0x7fda2c0b6430 msgr2=0x7fda2c0d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:30.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.631+0000 7fda3443f640 1 -- 172.21.15.148:0/893641294 shutdown_connections 2024-03-29T04:10:30.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.631+0000 7fda3443f640 1 -- 172.21.15.148:0/893641294 wait complete. 2024-03-29T04:10:30.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.632+0000 7fda3443f640 1 Processor -- start 2024-03-29T04:10:30.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.633+0000 7fda3443f640 1 -- start start 2024-03-29T04:10:30.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.633+0000 7fda3443f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda2c0b3430 0x7fda2c0cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:30.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.633+0000 7fda3443f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda2c15e3c0 0x7fda2c15f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:30.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.633+0000 7fda3443f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda2c161110 0x7fda2c14a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:30.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.633+0000 7fda3443f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c0b39a0 con 0x7fda2c0b3430 2024-03-29T04:10:30.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.633+0000 7fda3443f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c15e7a0 con 0x7fda2c161110 2024-03-29T04:10:30.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.633+0000 7fda3443f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fda2c0b3b20 con 0x7fda2c15e3c0 2024-03-29T04:10:30.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.633+0000 7fda321db640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda2c0b3430 0x7fda2c0cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:30.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.633+0000 7fda321db640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda2c0b3430 0x7fda2c0cacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49004/0 (socket says 172.21.15.148:49004) 2024-03-29T04:10:30.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.633+0000 7fda321db640 1 -- 172.21.15.148:0/791231275 learned_addr learned my addr 172.21.15.148:0/791231275 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:30.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.633+0000 7fda319da640 1 --2- 172.21.15.148:0/791231275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda2c15e3c0 0x7fda2c15f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:30.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.633+0000 7fda329dc640 1 --2- 172.21.15.148:0/791231275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda2c161110 0x7fda2c14a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:30.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.634+0000 7fda321db640 1 -- 172.21.15.148:0/791231275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda2c15e3c0 msgr2=0x7fda2c15f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:30.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.634+0000 7fda321db640 1 --2- 172.21.15.148:0/791231275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda2c15e3c0 0x7fda2c15f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.634+0000 7fda321db640 1 -- 172.21.15.148:0/791231275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda2c161110 msgr2=0x7fda2c14a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:30.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.634+0000 7fda321db640 1 --2- 172.21.15.148:0/791231275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda2c161110 0x7fda2c14a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.634+0000 7fda321db640 1 -- 172.21.15.148:0/791231275 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2c160070 con 0x7fda2c0b3430 2024-03-29T04:10:30.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.634+0000 7fda321db640 1 --2- 172.21.15.148:0/791231275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda2c0b3430 0x7fda2c0cacd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fda2802b930 tx=0x7fda280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:30.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.634+0000 7fda22ffd640 1 -- 172.21.15.148:0/791231275 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28040070 con 0x7fda2c0b3430 2024-03-29T04:10:30.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.634+0000 7fda22ffd640 1 -- 172.21.15.148:0/791231275 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda280399f0 con 0x7fda2c0b3430 2024-03-29T04:10:30.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.634+0000 7fda3443f640 1 -- 172.21.15.148:0/791231275 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda2c160350 con 0x7fda2c0b3430 2024-03-29T04:10:30.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.634+0000 7fda22ffd640 1 -- 172.21.15.148:0/791231275 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28035040 con 0x7fda2c0b3430 2024-03-29T04:10:30.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.634+0000 7fda3443f640 1 -- 172.21.15.148:0/791231275 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda2c160910 con 0x7fda2c0b3430 2024-03-29T04:10:30.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.635+0000 7fda3443f640 1 -- 172.21.15.148:0/791231275 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda2c068850 con 0x7fda2c0b3430 2024-03-29T04:10:30.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.636+0000 7fda22ffd640 1 -- 172.21.15.148:0/791231275 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fda2804b060 con 0x7fda2c0b3430 2024-03-29T04:10:30.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.636+0000 7fda22ffd640 1 --2- 172.21.15.148:0/791231275 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fda1006c560 0x7fda1006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:30.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.636+0000 7fda22ffd640 1 -- 172.21.15.148:0/791231275 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fda280ad100 con 0x7fda2c0b3430 2024-03-29T04:10:30.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.637+0000 7fda319da640 1 --2- 172.21.15.148:0/791231275 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fda1006c560 0x7fda1006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:30.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.637+0000 7fda319da640 1 --2- 172.21.15.148:0/791231275 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fda1006c560 0x7fda1006ea20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fda2c1f7a50 tx=0x7fda1c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:30.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.641+0000 7fda22ffd640 1 -- 172.21.15.148:0/791231275 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fda2807ae20 con 0x7fda2c0b3430 2024-03-29T04:10:30.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.816+0000 7fda3443f640 1 -- 172.21.15.148:0/791231275 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fda2c0d3c90 con 0x7fda2c0b3430 2024-03-29T04:10:30.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.816+0000 7fda22ffd640 1 -- 172.21.15.148:0/791231275 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fda2807f120 con 0x7fda2c0b3430 2024-03-29T04:10:30.796 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-29T04:10:30.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.818+0000 7fda3443f640 1 -- 172.21.15.148:0/791231275 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fda1006c560 msgr2=0x7fda1006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:30.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.818+0000 7fda3443f640 1 --2- 172.21.15.148:0/791231275 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fda1006c560 0x7fda1006ea20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fda2c1f7a50 tx=0x7fda1c008040 comp rx=0 tx=0).stop 2024-03-29T04:10:30.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.818+0000 7fda3443f640 1 -- 172.21.15.148:0/791231275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda2c0b3430 msgr2=0x7fda2c0cacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:30.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.818+0000 7fda3443f640 1 --2- 172.21.15.148:0/791231275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda2c0b3430 0x7fda2c0cacd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fda2802b930 tx=0x7fda280046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.818+0000 7fda3443f640 1 -- 172.21.15.148:0/791231275 shutdown_connections 2024-03-29T04:10:30.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.818+0000 7fda3443f640 1 --2- 172.21.15.148:0/791231275 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fda1006c560 0x7fda1006ea20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.818+0000 7fda3443f640 1 --2- 172.21.15.148:0/791231275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda2c161110 0x7fda2c14a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.818+0000 7fda3443f640 1 --2- 172.21.15.148:0/791231275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda2c15e3c0 0x7fda2c15f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.818+0000 7fda3443f640 1 --2- 172.21.15.148:0/791231275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda2c0b3430 0x7fda2c0cacd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.818+0000 7fda3443f640 1 -- 172.21.15.148:0/791231275 >> 172.21.15.148:0/791231275 conn(0x7fda2c0b6430 msgr2=0x7fda2c0b7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:30.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.818+0000 7fda3443f640 1 -- 172.21.15.148:0/791231275 shutdown_connections 2024-03-29T04:10:30.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.818+0000 7fda3443f640 1 -- 172.21.15.148:0/791231275 wait complete. 2024-03-29T04:10:30.809 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:10:30.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.951+0000 7ff532628640 1 Processor -- start 2024-03-29T04:10:30.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.951+0000 7ff532628640 1 -- start start 2024-03-29T04:10:30.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.951+0000 7ff532628640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff52c06dfd0 0x7ff52c06c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:30.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.951+0000 7ff532628640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff52c06cb00 0x7ff52c165e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:30.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.952+0000 7ff532628640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff52c06e480 con 0x7ff52c06dbf0 2024-03-29T04:10:30.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.952+0000 7ff532628640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7ff52c06cf70 con 0x7ff52c06cb00 2024-03-29T04:10:30.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.952+0000 7ff532628640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff52c13f5d0 con 0x7ff52c06dfd0 2024-03-29T04:10:30.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.952+0000 7ff52bfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff52c06dbf0 0x7ff52c15d120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47360/0 (socket says 172.21.15.148:47360) 2024-03-29T04:10:30.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.952+0000 7ff530bc5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff52c06cb00 0x7ff52c165e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:30.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.952+0000 7ff52bfff640 1 -- 172.21.15.148:0/1488230218 learned_addr learned my addr 172.21.15.148:0/1488230218 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:30.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.952+0000 7ff52b7fe640 1 --2- 172.21.15.148:0/1488230218 >> v2:172.21.15.157:3300/0 conn(0x7ff52c06dfd0 0x7ff52c06c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:30.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.952+0000 7ff530bc5640 1 -- 172.21.15.148:0/1488230218 >> v2:172.21.15.157:3300/0 conn(0x7ff52c06dfd0 msgr2=0x7ff52c06c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:30.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.952+0000 7ff530bc5640 1 --2- 172.21.15.148:0/1488230218 >> v2:172.21.15.157:3300/0 conn(0x7ff52c06dfd0 0x7ff52c06c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.952+0000 7ff530bc5640 1 -- 172.21.15.148:0/1488230218 >> v1:172.21.15.148:6789/0 conn(0x7ff52c06dbf0 legacy=0x7ff52c15d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:30.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.952+0000 7ff52affd640 1 -- 172.21.15.148:0/1488230218 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1399994547 0 0) 0x7ff52c06e480 con 0x7ff52c06dbf0 2024-03-29T04:10:30.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.952+0000 7ff530bc5640 1 -- 172.21.15.148:0/1488230218 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52c0ceb00 con 0x7ff52c06cb00 2024-03-29T04:10:30.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.953+0000 7ff530bc5640 1 --2- 172.21.15.148:0/1488230218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff52c06cb00 0x7ff52c165e40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff524009d90 tx=0x7ff52402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f305e63e448a54fc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:30.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.953+0000 7ff52affd640 1 -- 172.21.15.148:0/1488230218 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff524039070 con 0x7ff52c06cb00 2024-03-29T04:10:30.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.953+0000 7ff52affd640 1 -- 172.21.15.148:0/1488230218 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff52402cc60 con 0x7ff52c06cb00 2024-03-29T04:10:30.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.953+0000 7ff52affd640 1 -- 172.21.15.148:0/1488230218 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff524034660 con 0x7ff52c06cb00 2024-03-29T04:10:30.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.954+0000 7ff532628640 1 -- 172.21.15.148:0/1488230218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff52c06cb00 msgr2=0x7ff52c165e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:30.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.954+0000 7ff532628640 1 --2- 172.21.15.148:0/1488230218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff52c06cb00 0x7ff52c165e40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff524009d90 tx=0x7ff52402beb0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.954+0000 7ff532628640 1 -- 172.21.15.148:0/1488230218 shutdown_connections 2024-03-29T04:10:30.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.954+0000 7ff532628640 1 --2- 172.21.15.148:0/1488230218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff52c06cb00 0x7ff52c165e40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.954+0000 7ff532628640 1 --2- 172.21.15.148:0/1488230218 >> v2:172.21.15.157:3300/0 conn(0x7ff52c06dfd0 0x7ff52c06c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.954+0000 7ff532628640 1 -- 172.21.15.148:0/1488230218 >> 172.21.15.148:0/1488230218 conn(0x7ff52c0d00a0 msgr2=0x7ff52c0d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:30.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.954+0000 7ff532628640 1 -- 172.21.15.148:0/1488230218 shutdown_connections 2024-03-29T04:10:30.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.954+0000 7ff532628640 1 -- 172.21.15.148:0/1488230218 wait complete. 2024-03-29T04:10:30.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.955+0000 7ff532628640 1 Processor -- start 2024-03-29T04:10:30.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.955+0000 7ff532628640 1 -- start start 2024-03-29T04:10:30.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.956+0000 7ff532628640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff52c06cb00 0x7ff52c0b45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:30.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.956+0000 7ff532628640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff52c06dbf0 0x7ff52c1f74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:30.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.956+0000 7ff532628640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff52c06dfd0 0x7ff52c13acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:30.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.956+0000 7ff532628640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c0cf0c0 con 0x7ff52c06cb00 2024-03-29T04:10:30.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.956+0000 7ff532628640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c0cf3a0 con 0x7ff52c06dfd0 2024-03-29T04:10:30.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.956+0000 7ff532628640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff52c0cf680 con 0x7ff52c06dbf0 2024-03-29T04:10:30.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.956+0000 7ff52bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff52c06cb00 0x7ff52c0b45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:30.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.956+0000 7ff52bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff52c06cb00 0x7ff52c0b45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49014/0 (socket says 172.21.15.148:49014) 2024-03-29T04:10:30.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.956+0000 7ff52bfff640 1 -- 172.21.15.148:0/1752821935 learned_addr learned my addr 172.21.15.148:0/1752821935 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:30.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.956+0000 7ff530bc5640 1 --2- 172.21.15.148:0/1752821935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff52c06dfd0 0x7ff52c13acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:30.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.956+0000 7ff52b7fe640 1 --2- 172.21.15.148:0/1752821935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff52c06dbf0 0x7ff52c1f74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:30.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.956+0000 7ff52bfff640 1 -- 172.21.15.148:0/1752821935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff52c06dbf0 msgr2=0x7ff52c1f74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:30.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.956+0000 7ff52bfff640 1 --2- 172.21.15.148:0/1752821935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff52c06dbf0 0x7ff52c1f74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.956+0000 7ff52bfff640 1 -- 172.21.15.148:0/1752821935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff52c06dfd0 msgr2=0x7ff52c13acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:30.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.956+0000 7ff52bfff640 1 --2- 172.21.15.148:0/1752821935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff52c06dfd0 0x7ff52c13acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:30.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.956+0000 7ff52bfff640 1 -- 172.21.15.148:0/1752821935 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52c13b210 con 0x7ff52c06cb00 2024-03-29T04:10:30.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.956+0000 7ff530bc5640 1 --2- 172.21.15.148:0/1752821935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff52c06dfd0 0x7ff52c13acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:30.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.956+0000 7ff52bfff640 1 --2- 172.21.15.148:0/1752821935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff52c06cb00 0x7ff52c0b45a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00c4d0 tx=0x7ff51c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:30.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.957+0000 7ff528ff9640 1 -- 172.21.15.148:0/1752821935 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51c018070 con 0x7ff52c06cb00 2024-03-29T04:10:30.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.957+0000 7ff528ff9640 1 -- 172.21.15.148:0/1752821935 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff51c007500 con 0x7ff52c06cb00 2024-03-29T04:10:30.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.957+0000 7ff532628640 1 -- 172.21.15.148:0/1752821935 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff52c13b4b0 con 0x7ff52c06cb00 2024-03-29T04:10:30.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.957+0000 7ff528ff9640 1 -- 172.21.15.148:0/1752821935 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51c004a20 con 0x7ff52c06cb00 2024-03-29T04:10:30.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.957+0000 7ff532628640 1 -- 172.21.15.148:0/1752821935 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff52c13ba70 con 0x7ff52c06cb00 2024-03-29T04:10:30.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.958+0000 7ff532628640 1 -- 172.21.15.148:0/1752821935 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff52c06c440 con 0x7ff52c06cb00 2024-03-29T04:10:30.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.958+0000 7ff528ff9640 1 -- 172.21.15.148:0/1752821935 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff51c01b070 con 0x7ff52c06cb00 2024-03-29T04:10:30.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.958+0000 7ff528ff9640 1 --2- 172.21.15.148:0/1752821935 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff51806c530 0x7ff51806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:30.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.959+0000 7ff528ff9640 1 -- 172.21.15.148:0/1752821935 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff51c08acb0 con 0x7ff52c06cb00 2024-03-29T04:10:30.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.959+0000 7ff52b7fe640 1 --2- 172.21.15.148:0/1752821935 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff51806c530 0x7ff51806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:30.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.959+0000 7ff52b7fe640 1 --2- 172.21.15.148:0/1752821935 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff51806c530 0x7ff51806e9f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff520006fd0 tx=0x7ff520008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:30.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:30.963+0000 7ff528ff9640 1 -- 172.21.15.148:0/1752821935 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff52c06c440 con 0x7ff52c06cb00 2024-03-29T04:10:31.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.144+0000 7ff532628640 1 -- 172.21.15.148:0/1752821935 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff52c068850 con 0x7ff52c06cb00 2024-03-29T04:10:31.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.144+0000 7ff528ff9640 1 -- 172.21.15.148:0/1752821935 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7ff51c059020 con 0x7ff52c06cb00 2024-03-29T04:10:31.123 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:10:31.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.146+0000 7ff532628640 1 -- 172.21.15.148:0/1752821935 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff51806c530 msgr2=0x7ff51806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.146+0000 7ff532628640 1 --2- 172.21.15.148:0/1752821935 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff51806c530 0x7ff51806e9f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff520006fd0 tx=0x7ff520008040 comp rx=0 tx=0).stop 2024-03-29T04:10:31.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.146+0000 7ff532628640 1 -- 172.21.15.148:0/1752821935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff52c06cb00 msgr2=0x7ff52c0b45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.146+0000 7ff532628640 1 --2- 172.21.15.148:0/1752821935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff52c06cb00 0x7ff52c0b45a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00c4d0 tx=0x7ff51c00c990 comp rx=0 tx=0).stop 2024-03-29T04:10:31.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.146+0000 7ff532628640 1 -- 172.21.15.148:0/1752821935 shutdown_connections 2024-03-29T04:10:31.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.146+0000 7ff532628640 1 --2- 172.21.15.148:0/1752821935 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff51806c530 0x7ff51806e9f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.146+0000 7ff532628640 1 --2- 172.21.15.148:0/1752821935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff52c06dfd0 0x7ff52c13acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.146+0000 7ff532628640 1 --2- 172.21.15.148:0/1752821935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff52c06dbf0 0x7ff52c1f74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.146+0000 7ff532628640 1 --2- 172.21.15.148:0/1752821935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff52c06cb00 0x7ff52c0b45a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.146+0000 7ff532628640 1 -- 172.21.15.148:0/1752821935 >> 172.21.15.148:0/1752821935 conn(0x7ff52c0d00a0 msgr2=0x7ff52c15e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:31.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.146+0000 7ff532628640 1 -- 172.21.15.148:0/1752821935 shutdown_connections 2024-03-29T04:10:31.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.146+0000 7ff532628640 1 -- 172.21.15.148:0/1752821935 wait complete. 2024-03-29T04:10:31.141 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:10:31.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.270+0000 7f1634235640 1 Processor -- start 2024-03-29T04:10:31.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.270+0000 7f1634235640 1 -- start start 2024-03-29T04:10:31.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.271+0000 7f1634235640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f162c0b2f00 0x7f162c162680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.271+0000 7f1634235640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f162c0b3430 0x7f162c06b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.271+0000 7f1634235640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f162c0cf330 con 0x7f162c15e3c0 2024-03-29T04:10:31.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.271+0000 7f1634235640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f162c0cf960 con 0x7f162c0b2f00 2024-03-29T04:10:31.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.271+0000 7f1634235640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f162c162cf0 con 0x7f162c0b3430 2024-03-29T04:10:31.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.271+0000 7f16317d0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f162c0b2f00 0x7f162c162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.271+0000 7f16317d0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f162c0b2f00 0x7f162c162680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49020/0 (socket says 172.21.15.148:49020) 2024-03-29T04:10:31.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.271+0000 7f1631fd1640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f162c15e3c0 0x7f162c161f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47370/0 (socket says 172.21.15.148:47370) 2024-03-29T04:10:31.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.271+0000 7f16317d0640 1 -- 172.21.15.148:0/1008080406 learned_addr learned my addr 172.21.15.148:0/1008080406 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:31.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.271+0000 7f16327d2640 1 --2- 172.21.15.148:0/1008080406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f162c0b3430 0x7f162c06b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.271+0000 7f16317d0640 1 -- 172.21.15.148:0/1008080406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f162c0b3430 msgr2=0x7f162c06b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:31.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.271+0000 7f16317d0640 1 --2- 172.21.15.148:0/1008080406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f162c0b3430 0x7f162c06b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.271+0000 7f16317d0640 1 -- 172.21.15.148:0/1008080406 >> v1:172.21.15.148:6789/0 conn(0x7f162c15e3c0 legacy=0x7f162c161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.271+0000 7f1630fcf640 1 -- 172.21.15.148:0/1008080406 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1227999835 0 0) 0x7f162c0cf330 con 0x7f162c15e3c0 2024-03-29T04:10:31.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.271+0000 7f16317d0640 1 -- 172.21.15.148:0/1008080406 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162c0cf5f0 con 0x7f162c0b2f00 2024-03-29T04:10:31.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.272+0000 7f16317d0640 1 --2- 172.21.15.148:0/1008080406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f162c0b2f00 0x7f162c162680 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f161c009870 tx=0x7f161c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa476e20c60332bf server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:31.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.272+0000 7f1630fcf640 1 -- 172.21.15.148:0/1008080406 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c035020 con 0x7f162c0b2f00 2024-03-29T04:10:31.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.272+0000 7f1630fcf640 1 -- 172.21.15.148:0/1008080406 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f161c031640 con 0x7f162c0b2f00 2024-03-29T04:10:31.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.272+0000 7f1630fcf640 1 -- 172.21.15.148:0/1008080406 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c02d040 con 0x7f162c0b2f00 2024-03-29T04:10:31.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.273+0000 7f1634235640 1 -- 172.21.15.148:0/1008080406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f162c0b2f00 msgr2=0x7f162c162680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.273+0000 7f1634235640 1 --2- 172.21.15.148:0/1008080406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f162c0b2f00 0x7f162c162680 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f161c009870 tx=0x7f161c02b8f0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.273+0000 7f1634235640 1 -- 172.21.15.148:0/1008080406 shutdown_connections 2024-03-29T04:10:31.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.273+0000 7f1634235640 1 --2- 172.21.15.148:0/1008080406 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f162c0b3430 0x7f162c06b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.273+0000 7f1634235640 1 --2- 172.21.15.148:0/1008080406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f162c0b2f00 0x7f162c162680 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.273+0000 7f1634235640 1 -- 172.21.15.148:0/1008080406 >> 172.21.15.148:0/1008080406 conn(0x7f162c0b6430 msgr2=0x7f162c0d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:31.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.273+0000 7f1634235640 1 -- 172.21.15.148:0/1008080406 shutdown_connections 2024-03-29T04:10:31.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.273+0000 7f1634235640 1 -- 172.21.15.148:0/1008080406 wait complete. 2024-03-29T04:10:31.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.275+0000 7f1634235640 1 Processor -- start 2024-03-29T04:10:31.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.275+0000 7f1634235640 1 -- start start 2024-03-29T04:10:31.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.276+0000 7f1634235640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f162c0b3430 0x7f162c1ea570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.276+0000 7f1634235640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f162c15e3c0 0x7f162c1e8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.276+0000 7f1634235640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f162c0cdf50 0x7f162c0b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.276+0000 7f1634235640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f162c0b3b20 con 0x7f162c15e3c0 2024-03-29T04:10:31.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.276+0000 7f1634235640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f162c0b39a0 con 0x7f162c0cdf50 2024-03-29T04:10:31.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.276+0000 7f1634235640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f162c06bd30 con 0x7f162c0b3430 2024-03-29T04:10:31.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.276+0000 7f16317d0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f162c15e3c0 0x7f162c1e8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.276+0000 7f16317d0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f162c15e3c0 0x7f162c1e8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49024/0 (socket says 172.21.15.148:49024) 2024-03-29T04:10:31.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.276+0000 7f16317d0640 1 -- 172.21.15.148:0/1518945130 learned_addr learned my addr 172.21.15.148:0/1518945130 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:31.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.276+0000 7f1631fd1640 1 --2- 172.21.15.148:0/1518945130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f162c0b3430 0x7f162c1ea570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.276+0000 7f16327d2640 1 --2- 172.21.15.148:0/1518945130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f162c0cdf50 0x7f162c0b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.276+0000 7f16317d0640 1 -- 172.21.15.148:0/1518945130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f162c0b3430 msgr2=0x7f162c1ea570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.276+0000 7f16317d0640 1 --2- 172.21.15.148:0/1518945130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f162c0b3430 0x7f162c1ea570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.276+0000 7f16317d0640 1 -- 172.21.15.148:0/1518945130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f162c0cdf50 msgr2=0x7f162c0b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.276+0000 7f16317d0640 1 --2- 172.21.15.148:0/1518945130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f162c0cdf50 0x7f162c0b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.276+0000 7f16317d0640 1 -- 172.21.15.148:0/1518945130 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162c1eaab0 con 0x7f162c15e3c0 2024-03-29T04:10:31.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.277+0000 7f16317d0640 1 --2- 172.21.15.148:0/1518945130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f162c15e3c0 0x7f162c1e8780 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f161c000c00 tx=0x7f161c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:31.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.277+0000 7f16227fc640 1 -- 172.21.15.148:0/1518945130 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c040070 con 0x7f162c15e3c0 2024-03-29T04:10:31.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.277+0000 7f16227fc640 1 -- 172.21.15.148:0/1518945130 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f161c0399f0 con 0x7f162c15e3c0 2024-03-29T04:10:31.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.277+0000 7f1634235640 1 -- 172.21.15.148:0/1518945130 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f162c1e90c0 con 0x7f162c15e3c0 2024-03-29T04:10:31.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.277+0000 7f16227fc640 1 -- 172.21.15.148:0/1518945130 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c035040 con 0x7f162c15e3c0 2024-03-29T04:10:31.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.277+0000 7f1634235640 1 -- 172.21.15.148:0/1518945130 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f162c1e9d70 con 0x7f162c15e3c0 2024-03-29T04:10:31.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.278+0000 7f1634235640 1 -- 172.21.15.148:0/1518945130 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f162c068850 con 0x7f162c15e3c0 2024-03-29T04:10:31.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.278+0000 7f16227fc640 1 -- 172.21.15.148:0/1518945130 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f161c04b060 con 0x7f162c15e3c0 2024-03-29T04:10:31.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.279+0000 7f16227fc640 1 --2- 172.21.15.148:0/1518945130 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f161006c530 0x7f161006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.279+0000 7f16227fc640 1 -- 172.21.15.148:0/1518945130 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f161c0ad0e0 con 0x7f162c15e3c0 2024-03-29T04:10:31.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.279+0000 7f1631fd1640 1 --2- 172.21.15.148:0/1518945130 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f161006c530 0x7f161006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.279+0000 7f1631fd1640 1 --2- 172.21.15.148:0/1518945130 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f161006c530 0x7f161006e9f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1628007fd0 tx=0x7f1628007de0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:31.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.283+0000 7f16227fc640 1 -- 172.21.15.148:0/1518945130 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f161c07ae00 con 0x7f162c15e3c0 2024-03-29T04:10:31.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.460+0000 7f1634235640 1 -- 172.21.15.148:0/1518945130 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f162c0cf110 con 0x7f162c15e3c0 2024-03-29T04:10:31.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.460+0000 7f16227fc640 1 -- 172.21.15.148:0/1518945130 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f161c07f100 con 0x7f162c15e3c0 2024-03-29T04:10:31.440 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-29T04:10:31.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.463+0000 7f1634235640 1 -- 172.21.15.148:0/1518945130 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f161006c530 msgr2=0x7f161006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.463+0000 7f1634235640 1 --2- 172.21.15.148:0/1518945130 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f161006c530 0x7f161006e9f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1628007fd0 tx=0x7f1628007de0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.463+0000 7f1634235640 1 -- 172.21.15.148:0/1518945130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f162c15e3c0 msgr2=0x7f162c1e8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.463+0000 7f1634235640 1 --2- 172.21.15.148:0/1518945130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f162c15e3c0 0x7f162c1e8780 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f161c000c00 tx=0x7f161c0046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.463+0000 7f1634235640 1 -- 172.21.15.148:0/1518945130 shutdown_connections 2024-03-29T04:10:31.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.463+0000 7f1634235640 1 --2- 172.21.15.148:0/1518945130 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f161006c530 0x7f161006e9f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.463+0000 7f1634235640 1 --2- 172.21.15.148:0/1518945130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f162c0cdf50 0x7f162c0b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.463+0000 7f1634235640 1 --2- 172.21.15.148:0/1518945130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f162c15e3c0 0x7f162c1e8780 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.463+0000 7f1634235640 1 --2- 172.21.15.148:0/1518945130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f162c0b3430 0x7f162c1ea570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.463+0000 7f1634235640 1 -- 172.21.15.148:0/1518945130 >> 172.21.15.148:0/1518945130 conn(0x7f162c0b6430 msgr2=0x7f162c0d3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:31.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.463+0000 7f1634235640 1 -- 172.21.15.148:0/1518945130 shutdown_connections 2024-03-29T04:10:31.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.463+0000 7f1634235640 1 -- 172.21.15.148:0/1518945130 wait complete. 2024-03-29T04:10:31.453 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:10:31.469 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:10:31.498 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:10:31.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.583+0000 7f446623c640 1 Processor -- start 2024-03-29T04:10:31.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.583+0000 7f446623c640 1 -- start start 2024-03-29T04:10:31.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.583+0000 7f446623c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f44600cc0f0 0x7f44600cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.583+0000 7f446623c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44600cfbd0 0x7f446006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.583+0000 7f446623c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44600cc660 con 0x7f4460144980 2024-03-29T04:10:31.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.583+0000 7f446623c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f44600701c0 con 0x7f44600cc0f0 2024-03-29T04:10:31.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.583+0000 7f446623c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f44600cd270 con 0x7f44600cfbd0 2024-03-29T04:10:31.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.584+0000 7f445effd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f44600cc0f0 0x7f44600cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.584+0000 7f445effd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f44600cc0f0 0x7f44600cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49036/0 (socket says 172.21.15.148:49036) 2024-03-29T04:10:31.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.584+0000 7f445effd640 1 -- 172.21.15.148:0/949440013 learned_addr learned my addr 172.21.15.148:0/949440013 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:31.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.584+0000 7f445ffff640 1 --2- 172.21.15.148:0/949440013 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44600cfbd0 0x7f446006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.584+0000 7f445effd640 1 -- 172.21.15.148:0/949440013 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44600cfbd0 msgr2=0x7f446006c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:31.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.584+0000 7f445effd640 1 --2- 172.21.15.148:0/949440013 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44600cfbd0 0x7f446006c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.584+0000 7f445effd640 1 -- 172.21.15.148:0/949440013 >> v1:172.21.15.157:6789/0 conn(0x7f4460144980 legacy=0x7f446006fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:31.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.584+0000 7f445effd640 1 -- 172.21.15.148:0/949440013 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44600cc940 con 0x7f44600cc0f0 2024-03-29T04:10:31.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.585+0000 7f445effd640 1 --2- 172.21.15.148:0/949440013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44600cc0f0 0x7f44600cf690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4454009e50 tx=0x7f445402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20121b408959dcf server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:31.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.585+0000 7f445e7fc640 1 -- 172.21.15.148:0/949440013 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445402ca70 con 0x7f44600cc0f0 2024-03-29T04:10:31.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.585+0000 7f445e7fc640 1 -- 172.21.15.148:0/949440013 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f445402cc10 con 0x7f44600cc0f0 2024-03-29T04:10:31.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.585+0000 7f445e7fc640 1 -- 172.21.15.148:0/949440013 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44540343f0 con 0x7f44600cc0f0 2024-03-29T04:10:31.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.585+0000 7f446623c640 1 -- 172.21.15.148:0/949440013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44600cc0f0 msgr2=0x7f44600cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.585+0000 7f446623c640 1 --2- 172.21.15.148:0/949440013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44600cc0f0 0x7f44600cf690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4454009e50 tx=0x7f445402bed0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.585+0000 7f446623c640 1 -- 172.21.15.148:0/949440013 shutdown_connections 2024-03-29T04:10:31.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.585+0000 7f446623c640 1 --2- 172.21.15.148:0/949440013 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44600cfbd0 0x7f446006c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.585+0000 7f446623c640 1 --2- 172.21.15.148:0/949440013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44600cc0f0 0x7f44600cf690 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.585+0000 7f446623c640 1 -- 172.21.15.148:0/949440013 >> 172.21.15.148:0/949440013 conn(0x7f44600b2c60 msgr2=0x7f44600b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:31.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.585+0000 7f446623c640 1 -- 172.21.15.148:0/949440013 shutdown_connections 2024-03-29T04:10:31.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.585+0000 7f446623c640 1 -- 172.21.15.148:0/949440013 wait complete. 2024-03-29T04:10:31.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.587+0000 7f446623c640 1 Processor -- start 2024-03-29T04:10:31.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.587+0000 7f446623c640 1 -- start start 2024-03-29T04:10:31.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.587+0000 7f446623c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44600cc0f0 0x7f44601f1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.587+0000 7f446623c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44600cfbd0 0x7f446015ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.587+0000 7f446623c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4460144980 0x7f446013c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.587+0000 7f446623c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4460144d60 con 0x7f4460144980 2024-03-29T04:10:31.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.587+0000 7f446623c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f44600ccf00 con 0x7f44600cc0f0 2024-03-29T04:10:31.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.587+0000 7f446623c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f44600cd700 con 0x7f44600cfbd0 2024-03-29T04:10:31.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.588+0000 7f445ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4460144980 0x7f446013c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.588+0000 7f445ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4460144980 0x7f446013c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49038/0 (socket says 172.21.15.148:49038) 2024-03-29T04:10:31.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.588+0000 7f445ffff640 1 -- 172.21.15.148:0/3422319499 learned_addr learned my addr 172.21.15.148:0/3422319499 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:31.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.588+0000 7f445effd640 1 --2- 172.21.15.148:0/3422319499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44600cfbd0 0x7f446015ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.588+0000 7f445f7fe640 1 --2- 172.21.15.148:0/3422319499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44600cc0f0 0x7f44601f1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.588+0000 7f445ffff640 1 -- 172.21.15.148:0/3422319499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44600cfbd0 msgr2=0x7f446015ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.588+0000 7f445ffff640 1 --2- 172.21.15.148:0/3422319499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44600cfbd0 0x7f446015ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.588+0000 7f445ffff640 1 -- 172.21.15.148:0/3422319499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44600cc0f0 msgr2=0x7f44601f1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.588+0000 7f445ffff640 1 --2- 172.21.15.148:0/3422319499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44600cc0f0 0x7f44601f1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.588+0000 7f445ffff640 1 -- 172.21.15.148:0/3422319499 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44601f2530 con 0x7f4460144980 2024-03-29T04:10:31.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.588+0000 7f445ffff640 1 --2- 172.21.15.148:0/3422319499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4460144980 0x7f446013c480 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f445800c820 tx=0x7f445800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:31.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.588+0000 7f4447fff640 1 -- 172.21.15.148:0/3422319499 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4458015070 con 0x7f4460144980 2024-03-29T04:10:31.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.588+0000 7f4447fff640 1 -- 172.21.15.148:0/3422319499 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4458004020 con 0x7f4460144980 2024-03-29T04:10:31.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.588+0000 7f4447fff640 1 -- 172.21.15.148:0/3422319499 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4458004410 con 0x7f4460144980 2024-03-29T04:10:31.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.588+0000 7f446623c640 1 -- 172.21.15.148:0/3422319499 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44601f2810 con 0x7f4460144980 2024-03-29T04:10:31.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.588+0000 7f446623c640 1 -- 172.21.15.148:0/3422319499 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44601f2dd0 con 0x7f4460144980 2024-03-29T04:10:31.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.589+0000 7f446623c640 1 -- 172.21.15.148:0/3422319499 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4460068850 con 0x7f4460144980 2024-03-29T04:10:31.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.590+0000 7f4447fff640 1 -- 172.21.15.148:0/3422319499 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4458007830 con 0x7f4460144980 2024-03-29T04:10:31.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.590+0000 7f4447fff640 1 --2- 172.21.15.148:0/3422319499 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f444c06c530 0x7f444c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.590+0000 7f445f7fe640 1 --2- 172.21.15.148:0/3422319499 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f444c06c530 0x7f444c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.591+0000 7f4447fff640 1 -- 172.21.15.148:0/3422319499 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f445808b810 con 0x7f4460144980 2024-03-29T04:10:31.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.591+0000 7f445f7fe640 1 --2- 172.21.15.148:0/3422319499 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f444c06c530 0x7f444c06e9f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f446006d900 tx=0x7f4450009060 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:31.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.595+0000 7f4447fff640 1 -- 172.21.15.148:0/3422319499 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f44580594a0 con 0x7f4460144980 2024-03-29T04:10:31.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.776+0000 7f446623c640 1 -- 172.21.15.148:0/3422319499 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f446006eb30 con 0x7f4460144980 2024-03-29T04:10:31.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.776+0000 7f4447fff640 1 -- 172.21.15.148:0/3422319499 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f4458089e20 con 0x7f4460144980 2024-03-29T04:10:31.755 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:10:31.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.778+0000 7f446623c640 1 -- 172.21.15.148:0/3422319499 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f444c06c530 msgr2=0x7f444c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.778+0000 7f446623c640 1 --2- 172.21.15.148:0/3422319499 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f444c06c530 0x7f444c06e9f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f446006d900 tx=0x7f4450009060 comp rx=0 tx=0).stop 2024-03-29T04:10:31.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.778+0000 7f446623c640 1 -- 172.21.15.148:0/3422319499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4460144980 msgr2=0x7f446013c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.778+0000 7f446623c640 1 --2- 172.21.15.148:0/3422319499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4460144980 0x7f446013c480 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f445800c820 tx=0x7f445800cce0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.778+0000 7f446623c640 1 -- 172.21.15.148:0/3422319499 shutdown_connections 2024-03-29T04:10:31.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.778+0000 7f446623c640 1 --2- 172.21.15.148:0/3422319499 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f444c06c530 0x7f444c06e9f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.778+0000 7f446623c640 1 --2- 172.21.15.148:0/3422319499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4460144980 0x7f446013c480 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.778+0000 7f446623c640 1 --2- 172.21.15.148:0/3422319499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44600cfbd0 0x7f446015ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.778+0000 7f446623c640 1 --2- 172.21.15.148:0/3422319499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44600cc0f0 0x7f44601f1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.778+0000 7f446623c640 1 -- 172.21.15.148:0/3422319499 >> 172.21.15.148:0/3422319499 conn(0x7f44600b2c60 msgr2=0x7f44600722c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:31.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.778+0000 7f446623c640 1 -- 172.21.15.148:0/3422319499 shutdown_connections 2024-03-29T04:10:31.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.778+0000 7f446623c640 1 -- 172.21.15.148:0/3422319499 wait complete. 2024-03-29T04:10:31.768 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:10:31.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.937+0000 7f4d59cd9640 1 Processor -- start 2024-03-29T04:10:31.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.937+0000 7f4d59cd9640 1 -- start start 2024-03-29T04:10:31.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.938+0000 7f4d59cd9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d54072c30 0x7f4d540ce710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.938+0000 7f4d59cd9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4d5406b5b0 0x7f4d540cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.938+0000 7f4d59cd9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d5406dc80 con 0x7f4d5415e770 2024-03-29T04:10:31.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.938+0000 7f4d59cd9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f4d540b4090 con 0x7f4d5406b5b0 2024-03-29T04:10:31.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.938+0000 7f4d59cd9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4d540b5ec0 con 0x7f4d54072c30 2024-03-29T04:10:31.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.938+0000 7f4d537fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4d5415e770 0x7f4d54072520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47386/0 (socket says 172.21.15.148:47386) 2024-03-29T04:10:31.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.938+0000 7f4d537fe640 1 -- 172.21.15.148:0/847992582 learned_addr learned my addr 172.21.15.148:0/847992582 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:31.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.938+0000 7f4d52ffd640 1 --2- 172.21.15.148:0/847992582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d54072c30 0x7f4d540ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.938+0000 7f4d53fff640 1 --2- 172.21.15.148:0/847992582 >> v2:172.21.15.157:3300/0 conn(0x7f4d5406b5b0 0x7f4d540cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.938+0000 7f4d527fc640 1 -- 172.21.15.148:0/847992582 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2701412888 0 0) 0x7f4d5406dc80 con 0x7f4d5415e770 2024-03-29T04:10:31.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.939+0000 7f4d527fc640 1 -- 172.21.15.148:0/847992582 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d3c003520 con 0x7f4d5415e770 2024-03-29T04:10:31.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.939+0000 7f4d53fff640 1 -- 172.21.15.148:0/847992582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d54072c30 msgr2=0x7f4d540ce710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:31.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.939+0000 7f4d53fff640 1 --2- 172.21.15.148:0/847992582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d54072c30 0x7f4d540ce710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.939+0000 7f4d53fff640 1 -- 172.21.15.148:0/847992582 >> v1:172.21.15.148:6789/0 conn(0x7f4d5415e770 legacy=0x7f4d54072520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.939+0000 7f4d53fff640 1 -- 172.21.15.148:0/847992582 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d540b61a0 con 0x7f4d5406b5b0 2024-03-29T04:10:31.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.940+0000 7f4d53fff640 1 --2- 172.21.15.148:0/847992582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d5406b5b0 0x7f4d540cf860 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c0097a0 tx=0x7f4d4c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37b428864ad146d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:31.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.940+0000 7f4d527fc640 1 -- 172.21.15.148:0/847992582 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c035020 con 0x7f4d5406b5b0 2024-03-29T04:10:31.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.940+0000 7f4d527fc640 1 -- 172.21.15.148:0/847992582 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d4c031830 con 0x7f4d5406b5b0 2024-03-29T04:10:31.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.940+0000 7f4d527fc640 1 -- 172.21.15.148:0/847992582 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c031be0 con 0x7f4d5406b5b0 2024-03-29T04:10:31.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.941+0000 7f4d59cd9640 1 -- 172.21.15.148:0/847992582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d5406b5b0 msgr2=0x7f4d540cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.941+0000 7f4d59cd9640 1 --2- 172.21.15.148:0/847992582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d5406b5b0 0x7f4d540cf860 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c0097a0 tx=0x7f4d4c02b8d0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.941+0000 7f4d59cd9640 1 -- 172.21.15.148:0/847992582 shutdown_connections 2024-03-29T04:10:31.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.941+0000 7f4d59cd9640 1 --2- 172.21.15.148:0/847992582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d5406b5b0 0x7f4d540cf860 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.941+0000 7f4d59cd9640 1 --2- 172.21.15.148:0/847992582 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d54072c30 0x7f4d540ce710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.941+0000 7f4d59cd9640 1 -- 172.21.15.148:0/847992582 >> 172.21.15.148:0/847992582 conn(0x7f4d540d0f50 msgr2=0x7f4d540d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:31.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.941+0000 7f4d59cd9640 1 -- 172.21.15.148:0/847992582 shutdown_connections 2024-03-29T04:10:31.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.941+0000 7f4d59cd9640 1 -- 172.21.15.148:0/847992582 wait complete. 2024-03-29T04:10:31.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.942+0000 7f4d59cd9640 1 Processor -- start 2024-03-29T04:10:31.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.942+0000 7f4d59cd9640 1 -- start start 2024-03-29T04:10:31.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.943+0000 7f4d59cd9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d5406b5b0 0x7f4d5414cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.943+0000 7f4d59cd9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d54072c30 0x7f4d541e81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.943+0000 7f4d59cd9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d5415e770 0x7f4d541e8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.943+0000 7f4d59cd9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5406da20 con 0x7f4d54072c30 2024-03-29T04:10:31.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.943+0000 7f4d59cd9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5406d8a0 con 0x7f4d5415e770 2024-03-29T04:10:31.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.943+0000 7f4d59cd9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4d540b5ae0 con 0x7f4d5406b5b0 2024-03-29T04:10:31.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.943+0000 7f4d52ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d54072c30 0x7f4d541e81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.943+0000 7f4d52ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d54072c30 0x7f4d541e81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49044/0 (socket says 172.21.15.148:49044) 2024-03-29T04:10:31.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.943+0000 7f4d52ffd640 1 -- 172.21.15.148:0/2975344637 learned_addr learned my addr 172.21.15.148:0/2975344637 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:31.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.943+0000 7f4d53fff640 1 --2- 172.21.15.148:0/2975344637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d5415e770 0x7f4d541e8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.943+0000 7f4d537fe640 1 --2- 172.21.15.148:0/2975344637 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d5406b5b0 0x7f4d5414cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.943+0000 7f4d52ffd640 1 -- 172.21.15.148:0/2975344637 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d5406b5b0 msgr2=0x7f4d5414cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.943+0000 7f4d52ffd640 1 --2- 172.21.15.148:0/2975344637 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d5406b5b0 0x7f4d5414cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.943+0000 7f4d52ffd640 1 -- 172.21.15.148:0/2975344637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d5415e770 msgr2=0x7f4d541e8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.943+0000 7f4d52ffd640 1 --2- 172.21.15.148:0/2975344637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d5415e770 0x7f4d541e8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.943+0000 7f4d52ffd640 1 -- 172.21.15.148:0/2975344637 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d5415a490 con 0x7f4d54072c30 2024-03-29T04:10:31.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.943+0000 7f4d537fe640 1 --2- 172.21.15.148:0/2975344637 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d5406b5b0 0x7f4d5414cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:31.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.943+0000 7f4d52ffd640 1 --2- 172.21.15.148:0/2975344637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d54072c30 0x7f4d541e81d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4d48009db0 tx=0x7f4d4800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:31.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.944+0000 7f4d3bfff640 1 -- 172.21.15.148:0/2975344637 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d48018020 con 0x7f4d54072c30 2024-03-29T04:10:31.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.944+0000 7f4d3bfff640 1 -- 172.21.15.148:0/2975344637 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d48014480 con 0x7f4d54072c30 2024-03-29T04:10:31.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.944+0000 7f4d59cd9640 1 -- 172.21.15.148:0/2975344637 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d54154100 con 0x7f4d54072c30 2024-03-29T04:10:31.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.944+0000 7f4d3bfff640 1 -- 172.21.15.148:0/2975344637 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d48014850 con 0x7f4d54072c30 2024-03-29T04:10:31.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.944+0000 7f4d59cd9640 1 -- 172.21.15.148:0/2975344637 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d541546c0 con 0x7f4d54072c30 2024-03-29T04:10:31.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.945+0000 7f4d59cd9640 1 -- 172.21.15.148:0/2975344637 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d540687e0 con 0x7f4d54072c30 2024-03-29T04:10:31.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.945+0000 7f4d3bfff640 1 -- 172.21.15.148:0/2975344637 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4d48005d10 con 0x7f4d54072c30 2024-03-29T04:10:31.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.945+0000 7f4d3bfff640 1 --2- 172.21.15.148:0/2975344637 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4d4006c530 0x7f4d4006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.946+0000 7f4d537fe640 1 --2- 172.21.15.148:0/2975344637 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4d4006c530 0x7f4d4006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.946+0000 7f4d3bfff640 1 -- 172.21.15.148:0/2975344637 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4d4808c710 con 0x7f4d54072c30 2024-03-29T04:10:31.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.946+0000 7f4d537fe640 1 --2- 172.21.15.148:0/2975344637 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4d4006c530 0x7f4d4006e9f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4d44007fd0 tx=0x7f4d44009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:31.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:31.950+0000 7f4d3bfff640 1 -- 172.21.15.148:0/2975344637 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4d48022030 con 0x7f4d54072c30 2024-03-29T04:10:32.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.367+0000 7f4d59cd9640 1 -- 172.21.15.148:0/2975344637 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d54073870 con 0x7f4d54072c30 2024-03-29T04:10:32.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.368+0000 7f4d3bfff640 1 -- 172.21.15.148:0/2975344637 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f4d48010040 con 0x7f4d54072c30 2024-03-29T04:10:32.347 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:32.347 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/36660496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":36660496},{"type":"v1","addr":"172.21.15.157:6834","nonce":36660496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2674481055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2674481055},{"type":"v1","addr":"172.21.15.157:6835","nonce":2674481055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3356449643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3356449643},{"type":"v1","addr":"172.21.15.148:6836","nonce":3356449643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:09:59.886367+0000","modified":"2024-03-29T04:10:22.957623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/2453181817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2453181817},{"type":"v1","addr":"172.21.15.148:6837","nonce":2453181817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-03-29T04:10:32.347 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 9 2024-03-29T04:10:32.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.370+0000 7f4d59cd9640 1 -- 172.21.15.148:0/2975344637 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4d4006c530 msgr2=0x7f4d4006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.370+0000 7f4d59cd9640 1 --2- 172.21.15.148:0/2975344637 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4d4006c530 0x7f4d4006e9f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4d44007fd0 tx=0x7f4d44009040 comp rx=0 tx=0).stop 2024-03-29T04:10:32.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.370+0000 7f4d59cd9640 1 -- 172.21.15.148:0/2975344637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d54072c30 msgr2=0x7f4d541e81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.370+0000 7f4d59cd9640 1 --2- 172.21.15.148:0/2975344637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d54072c30 0x7f4d541e81d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4d48009db0 tx=0x7f4d4800c490 comp rx=0 tx=0).stop 2024-03-29T04:10:32.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.370+0000 7f4d59cd9640 1 -- 172.21.15.148:0/2975344637 shutdown_connections 2024-03-29T04:10:32.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.370+0000 7f4d59cd9640 1 --2- 172.21.15.148:0/2975344637 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4d4006c530 0x7f4d4006e9f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.370+0000 7f4d59cd9640 1 --2- 172.21.15.148:0/2975344637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d5415e770 0x7f4d541e8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.370+0000 7f4d59cd9640 1 --2- 172.21.15.148:0/2975344637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d54072c30 0x7f4d541e81d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.370+0000 7f4d59cd9640 1 --2- 172.21.15.148:0/2975344637 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d5406b5b0 0x7f4d5414cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.370+0000 7f4d59cd9640 1 -- 172.21.15.148:0/2975344637 >> 172.21.15.148:0/2975344637 conn(0x7f4d540d0f50 msgr2=0x7f4d540cfcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:32.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.370+0000 7f4d59cd9640 1 -- 172.21.15.148:0/2975344637 shutdown_connections 2024-03-29T04:10:32.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.370+0000 7f4d59cd9640 1 -- 172.21.15.148:0/2975344637 wait complete. 2024-03-29T04:10:32.364 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:32.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.487+0000 7fd02bf52640 1 Processor -- start 2024-03-29T04:10:32.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.487+0000 7fd02bf52640 1 -- start start 2024-03-29T04:10:32.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.488+0000 7fd02bf52640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd02415e3e0 0x7fd0240d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.488+0000 7fd02bf52640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0240ba030 0x7fd0240d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.488+0000 7fd02bf52640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd02415e7c0 con 0x7fd0240d10d0 2024-03-29T04:10:32.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.488+0000 7fd02bf52640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd0240d3840 con 0x7fd02415e3e0 2024-03-29T04:10:32.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.488+0000 7fd02bf52640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd0240d2c30 con 0x7fd0240ba030 2024-03-29T04:10:32.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.488+0000 7fd02a4ef640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd0240d10d0 0x7fd0240b72d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47390/0 (socket says 172.21.15.148:47390) 2024-03-29T04:10:32.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.488+0000 7fd02a4ef640 1 -- 172.21.15.148:0/1576759047 learned_addr learned my addr 172.21.15.148:0/1576759047 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:32.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.488+0000 7fd0294ed640 1 --2- 172.21.15.148:0/1576759047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0240ba030 0x7fd0240d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.488+0000 7fd029cee640 1 --2- 172.21.15.148:0/1576759047 >> v2:172.21.15.157:3300/0 conn(0x7fd02415e3e0 0x7fd0240d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.489+0000 7fd028cec640 1 -- 172.21.15.148:0/1576759047 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 808897040 0 0) 0x7fd02415e7c0 con 0x7fd0240d10d0 2024-03-29T04:10:32.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.489+0000 7fd028cec640 1 -- 172.21.15.148:0/1576759047 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd010003520 con 0x7fd0240d10d0 2024-03-29T04:10:32.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.489+0000 7fd029cee640 1 -- 172.21.15.148:0/1576759047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0240ba030 msgr2=0x7fd0240d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:32.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.489+0000 7fd029cee640 1 --2- 172.21.15.148:0/1576759047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0240ba030 0x7fd0240d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.489+0000 7fd029cee640 1 -- 172.21.15.148:0/1576759047 >> v1:172.21.15.148:6789/0 conn(0x7fd0240d10d0 legacy=0x7fd0240b72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.489+0000 7fd029cee640 1 -- 172.21.15.148:0/1576759047 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0240d2f10 con 0x7fd02415e3e0 2024-03-29T04:10:32.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.489+0000 7fd0294ed640 1 --2- 172.21.15.148:0/1576759047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0240ba030 0x7fd0240d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:32.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.490+0000 7fd029cee640 1 --2- 172.21.15.148:0/1576759047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd02415e3e0 0x7fd0240d3cd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd0140097a0 tx=0x7fd01402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99f7b923c55b59e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:32.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.490+0000 7fd028cec640 1 -- 172.21.15.148:0/1576759047 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd014034070 con 0x7fd02415e3e0 2024-03-29T04:10:32.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.490+0000 7fd028cec640 1 -- 172.21.15.148:0/1576759047 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd014030610 con 0x7fd02415e3e0 2024-03-29T04:10:32.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.490+0000 7fd028cec640 1 -- 172.21.15.148:0/1576759047 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0140309c0 con 0x7fd02415e3e0 2024-03-29T04:10:32.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.490+0000 7fd02bf52640 1 -- 172.21.15.148:0/1576759047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd02415e3e0 msgr2=0x7fd0240d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.490+0000 7fd02bf52640 1 --2- 172.21.15.148:0/1576759047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd02415e3e0 0x7fd0240d3cd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd0140097a0 tx=0x7fd01402b960 comp rx=0 tx=0).stop 2024-03-29T04:10:32.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.490+0000 7fd02bf52640 1 -- 172.21.15.148:0/1576759047 shutdown_connections 2024-03-29T04:10:32.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.490+0000 7fd02bf52640 1 --2- 172.21.15.148:0/1576759047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0240ba030 0x7fd0240d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.490+0000 7fd02bf52640 1 --2- 172.21.15.148:0/1576759047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd02415e3e0 0x7fd0240d3cd0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.490+0000 7fd02bf52640 1 -- 172.21.15.148:0/1576759047 >> 172.21.15.148:0/1576759047 conn(0x7fd0240b6210 msgr2=0x7fd0240b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:32.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.490+0000 7fd02bf52640 1 -- 172.21.15.148:0/1576759047 shutdown_connections 2024-03-29T04:10:32.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.490+0000 7fd02bf52640 1 -- 172.21.15.148:0/1576759047 wait complete. 2024-03-29T04:10:32.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.492+0000 7fd02bf52640 1 Processor -- start 2024-03-29T04:10:32.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.492+0000 7fd02bf52640 1 -- start start 2024-03-29T04:10:32.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.492+0000 7fd02bf52640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0240ba030 0x7fd0241f3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.492+0000 7fd02bf52640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0240d10d0 0x7fd0241482b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.492+0000 7fd02bf52640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd02415e3e0 0x7fd024157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.492+0000 7fd02bf52640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd0240d2870 con 0x7fd02415e3e0 2024-03-29T04:10:32.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.492+0000 7fd02bf52640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd0240d34d0 con 0x7fd0240ba030 2024-03-29T04:10:32.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.492+0000 7fd02bf52640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd0240d29f0 con 0x7fd0240d10d0 2024-03-29T04:10:32.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.492+0000 7fd02a4ef640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd02415e3e0 0x7fd024157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.492+0000 7fd02a4ef640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd02415e3e0 0x7fd024157410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49046/0 (socket says 172.21.15.148:49046) 2024-03-29T04:10:32.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.492+0000 7fd02a4ef640 1 -- 172.21.15.148:0/330484912 learned_addr learned my addr 172.21.15.148:0/330484912 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:32.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.492+0000 7fd029cee640 1 --2- 172.21.15.148:0/330484912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0240ba030 0x7fd0241f3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.492+0000 7fd0294ed640 1 --2- 172.21.15.148:0/330484912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0240d10d0 0x7fd0241482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.493+0000 7fd029cee640 1 -- 172.21.15.148:0/330484912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0240d10d0 msgr2=0x7fd0241482b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.493+0000 7fd029cee640 1 --2- 172.21.15.148:0/330484912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0240d10d0 0x7fd0241482b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.493+0000 7fd029cee640 1 -- 172.21.15.148:0/330484912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd02415e3e0 msgr2=0x7fd024157410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.493+0000 7fd029cee640 1 --2- 172.21.15.148:0/330484912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd02415e3e0 0x7fd024157410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.493+0000 7fd029cee640 1 -- 172.21.15.148:0/330484912 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0241f41d0 con 0x7fd0240ba030 2024-03-29T04:10:32.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.493+0000 7fd02a4ef640 1 --2- 172.21.15.148:0/330484912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd02415e3e0 0x7fd024157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:32.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.493+0000 7fd029cee640 1 --2- 172.21.15.148:0/330484912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0240ba030 0x7fd0241f3c90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd014033000 tx=0x7fd0140046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:32.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.493+0000 7fd01a7fc640 1 -- 172.21.15.148:0/330484912 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd014034040 con 0x7fd0240ba030 2024-03-29T04:10:32.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.493+0000 7fd01a7fc640 1 -- 172.21.15.148:0/330484912 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd01403aaa0 con 0x7fd0240ba030 2024-03-29T04:10:32.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.493+0000 7fd01a7fc640 1 -- 172.21.15.148:0/330484912 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd014006e10 con 0x7fd0240ba030 2024-03-29T04:10:32.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.493+0000 7fd02bf52640 1 -- 172.21.15.148:0/330484912 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0240cc9d0 con 0x7fd0240ba030 2024-03-29T04:10:32.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.493+0000 7fd02bf52640 1 -- 172.21.15.148:0/330484912 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0240ccf90 con 0x7fd0240ba030 2024-03-29T04:10:32.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.494+0000 7fd02bf52640 1 -- 172.21.15.148:0/330484912 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd02415df20 con 0x7fd0240ba030 2024-03-29T04:10:32.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.495+0000 7fd01a7fc640 1 -- 172.21.15.148:0/330484912 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd01404b060 con 0x7fd0240ba030 2024-03-29T04:10:32.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.496+0000 7fd01a7fc640 1 --2- 172.21.15.148:0/330484912 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd00806c530 0x7fd00806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.496+0000 7fd01a7fc640 1 -- 172.21.15.148:0/330484912 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd0140ad7e0 con 0x7fd0240ba030 2024-03-29T04:10:32.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.496+0000 7fd0294ed640 1 --2- 172.21.15.148:0/330484912 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd00806c530 0x7fd00806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.496+0000 7fd0294ed640 1 --2- 172.21.15.148:0/330484912 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd00806c530 0x7fd00806e9f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd020009560 tx=0x7fd020009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:32.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.499+0000 7fd01a7fc640 1 -- 172.21.15.148:0/330484912 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd01403f070 con 0x7fd0240ba030 2024-03-29T04:10:32.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.686+0000 7fd02bf52640 1 -- 172.21.15.148:0/330484912 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0240687e0 con 0x7fd0240ba030 2024-03-29T04:10:32.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.688+0000 7fd01a7fc640 1 -- 172.21.15.148:0/330484912 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7fd01407b700 con 0x7fd0240ba030 2024-03-29T04:10:32.668 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:32.668 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:05.026481+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:32.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.690+0000 7fd02bf52640 1 -- 172.21.15.148:0/330484912 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd00806c530 msgr2=0x7fd00806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.690+0000 7fd02bf52640 1 --2- 172.21.15.148:0/330484912 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd00806c530 0x7fd00806e9f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd020009560 tx=0x7fd020009190 comp rx=0 tx=0).stop 2024-03-29T04:10:32.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.690+0000 7fd02bf52640 1 -- 172.21.15.148:0/330484912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0240ba030 msgr2=0x7fd0241f3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.690+0000 7fd02bf52640 1 --2- 172.21.15.148:0/330484912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0240ba030 0x7fd0241f3c90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd014033000 tx=0x7fd0140046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.690+0000 7fd02bf52640 1 -- 172.21.15.148:0/330484912 shutdown_connections 2024-03-29T04:10:32.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.690+0000 7fd02bf52640 1 --2- 172.21.15.148:0/330484912 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd00806c530 0x7fd00806e9f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.690+0000 7fd02bf52640 1 --2- 172.21.15.148:0/330484912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd02415e3e0 0x7fd024157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.690+0000 7fd02bf52640 1 --2- 172.21.15.148:0/330484912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0240d10d0 0x7fd0241482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.690+0000 7fd02bf52640 1 --2- 172.21.15.148:0/330484912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0240ba030 0x7fd0241f3c90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.690+0000 7fd02bf52640 1 -- 172.21.15.148:0/330484912 >> 172.21.15.148:0/330484912 conn(0x7fd0240b6210 msgr2=0x7fd0240b77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:32.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.690+0000 7fd02bf52640 1 -- 172.21.15.148:0/330484912 shutdown_connections 2024-03-29T04:10:32.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.690+0000 7fd02bf52640 1 -- 172.21.15.148:0/330484912 wait complete. 2024-03-29T04:10:32.682 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:47.898351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:57.922313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:58.829405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:10:02.551763+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:10:32.682 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:10:32.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.807+0000 7f146347b640 1 Processor -- start 2024-03-29T04:10:32.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.807+0000 7f146347b640 1 -- start start 2024-03-29T04:10:32.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.808+0000 7f146347b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f145c0b2f00 0x7f145c162680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.808+0000 7f146347b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f145c0b3430 0x7f145c06b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.808+0000 7f146347b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f145c0cf330 con 0x7f145c15e3c0 2024-03-29T04:10:32.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.808+0000 7f146347b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f145c0cf960 con 0x7f145c0b3430 2024-03-29T04:10:32.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.808+0000 7f146347b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f145c162cf0 con 0x7f145c0b2f00 2024-03-29T04:10:32.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.808+0000 7f1461a18640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f145c0b3430 0x7f145c06b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.808+0000 7f1461a18640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f145c0b3430 0x7f145c06b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49058/0 (socket says 172.21.15.148:49058) 2024-03-29T04:10:32.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.808+0000 7f1461a18640 1 -- 172.21.15.148:0/2019968387 learned_addr learned my addr 172.21.15.148:0/2019968387 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:32.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.809+0000 7f1460a16640 1 --2- 172.21.15.148:0/2019968387 >> v2:172.21.15.157:3300/0 conn(0x7f145c0b2f00 0x7f145c162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.809+0000 7f1461a18640 1 -- 172.21.15.148:0/2019968387 >> v2:172.21.15.157:3300/0 conn(0x7f145c0b2f00 msgr2=0x7f145c162680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:32.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.809+0000 7f1461a18640 1 --2- 172.21.15.148:0/2019968387 >> v2:172.21.15.157:3300/0 conn(0x7f145c0b2f00 0x7f145c162680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.809+0000 7f1461a18640 1 -- 172.21.15.148:0/2019968387 >> v1:172.21.15.157:6789/0 conn(0x7f145c15e3c0 legacy=0x7f145c161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:32.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.809+0000 7f1461a18640 1 -- 172.21.15.148:0/2019968387 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145c0cf5f0 con 0x7f145c0b3430 2024-03-29T04:10:32.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.809+0000 7f1461a18640 1 --2- 172.21.15.148:0/2019968387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f145c0b3430 0x7f145c06b7f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f145c0c7be0 tx=0x7f145402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c431292dde5ec51 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:32.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.810+0000 7f1453fff640 1 -- 172.21.15.148:0/2019968387 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145402e070 con 0x7f145c0b3430 2024-03-29T04:10:32.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.810+0000 7f1453fff640 1 -- 172.21.15.148:0/2019968387 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1454037070 con 0x7f145c0b3430 2024-03-29T04:10:32.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.810+0000 7f1453fff640 1 -- 172.21.15.148:0/2019968387 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1454031a00 con 0x7f145c0b3430 2024-03-29T04:10:32.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.810+0000 7f146347b640 1 -- 172.21.15.148:0/2019968387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f145c0b3430 msgr2=0x7f145c06b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.810+0000 7f146347b640 1 --2- 172.21.15.148:0/2019968387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f145c0b3430 0x7f145c06b7f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f145c0c7be0 tx=0x7f145402b880 comp rx=0 tx=0).stop 2024-03-29T04:10:32.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.810+0000 7f146347b640 1 -- 172.21.15.148:0/2019968387 shutdown_connections 2024-03-29T04:10:32.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.810+0000 7f146347b640 1 --2- 172.21.15.148:0/2019968387 >> v2:172.21.15.157:3300/0 conn(0x7f145c0b2f00 0x7f145c162680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.810+0000 7f146347b640 1 --2- 172.21.15.148:0/2019968387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f145c0b3430 0x7f145c06b7f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.810+0000 7f146347b640 1 -- 172.21.15.148:0/2019968387 >> 172.21.15.148:0/2019968387 conn(0x7f145c0b6430 msgr2=0x7f145c0d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:32.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.810+0000 7f146347b640 1 -- 172.21.15.148:0/2019968387 shutdown_connections 2024-03-29T04:10:32.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.810+0000 7f146347b640 1 -- 172.21.15.148:0/2019968387 wait complete. 2024-03-29T04:10:32.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.811+0000 7f146347b640 1 Processor -- start 2024-03-29T04:10:32.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.811+0000 7f146347b640 1 -- start start 2024-03-29T04:10:32.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.812+0000 7f146347b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f145c0b3430 0x7f145c0c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.812+0000 7f146347b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f145c15e3c0 0x7f145c0cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.812+0000 7f146347b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f145c0cb8e0 0x7f145c0ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.812+0000 7f146347b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f145c0b3b20 con 0x7f145c0b3430 2024-03-29T04:10:32.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.812+0000 7f146347b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f145c0b39a0 con 0x7f145c15e3c0 2024-03-29T04:10:32.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.812+0000 7f146347b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f145c06bd30 con 0x7f145c0cb8e0 2024-03-29T04:10:32.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.812+0000 7f1461217640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f145c0b3430 0x7f145c0c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.812+0000 7f1461217640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f145c0b3430 0x7f145c0c6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49070/0 (socket says 172.21.15.148:49070) 2024-03-29T04:10:32.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.812+0000 7f1461217640 1 -- 172.21.15.148:0/3374250921 learned_addr learned my addr 172.21.15.148:0/3374250921 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:32.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.812+0000 7f1460a16640 1 --2- 172.21.15.148:0/3374250921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f145c15e3c0 0x7f145c0cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.812+0000 7f1461a18640 1 --2- 172.21.15.148:0/3374250921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f145c0cb8e0 0x7f145c0ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.813+0000 7f1461217640 1 -- 172.21.15.148:0/3374250921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f145c0cb8e0 msgr2=0x7f145c0ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.813+0000 7f1461217640 1 --2- 172.21.15.148:0/3374250921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f145c0cb8e0 0x7f145c0ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.813+0000 7f1461217640 1 -- 172.21.15.148:0/3374250921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f145c15e3c0 msgr2=0x7f145c0cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.813+0000 7f1461217640 1 --2- 172.21.15.148:0/3374250921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f145c15e3c0 0x7f145c0cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.813+0000 7f1461217640 1 -- 172.21.15.148:0/3374250921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145c15f0d0 con 0x7f145c0b3430 2024-03-29T04:10:32.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.813+0000 7f1461a18640 1 --2- 172.21.15.148:0/3374250921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f145c0cb8e0 0x7f145c0ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:32.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.813+0000 7f1461217640 1 --2- 172.21.15.148:0/3374250921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f145c0b3430 0x7f145c0c6cf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f145800ba80 tx=0x7f145800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:32.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.813+0000 7f1451ffb640 1 -- 172.21.15.148:0/3374250921 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1458019070 con 0x7f145c0b3430 2024-03-29T04:10:32.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.813+0000 7f1451ffb640 1 -- 172.21.15.148:0/3374250921 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f145800cc90 con 0x7f145c0b3430 2024-03-29T04:10:32.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.813+0000 7f146347b640 1 -- 172.21.15.148:0/3374250921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f145c15f3b0 con 0x7f145c0b3430 2024-03-29T04:10:32.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.813+0000 7f1451ffb640 1 -- 172.21.15.148:0/3374250921 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1458014680 con 0x7f145c0b3430 2024-03-29T04:10:32.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.813+0000 7f146347b640 1 -- 172.21.15.148:0/3374250921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f145c15f970 con 0x7f145c0b3430 2024-03-29T04:10:32.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.814+0000 7f146347b640 1 -- 172.21.15.148:0/3374250921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f145c0d3ec0 con 0x7f145c0b3430 2024-03-29T04:10:32.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.815+0000 7f1451ffb640 1 -- 172.21.15.148:0/3374250921 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f145801c070 con 0x7f145c0b3430 2024-03-29T04:10:32.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.815+0000 7f1451ffb640 1 --2- 172.21.15.148:0/3374250921 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f144006c530 0x7f144006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.815+0000 7f1451ffb640 1 -- 172.21.15.148:0/3374250921 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f145808b860 con 0x7f145c0b3430 2024-03-29T04:10:32.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.815+0000 7f1460a16640 1 --2- 172.21.15.148:0/3374250921 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f144006c530 0x7f144006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.815+0000 7f1460a16640 1 --2- 172.21.15.148:0/3374250921 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f144006c530 0x7f144006e9f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f145c1ea900 tx=0x7f144c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:32.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.819+0000 7f1451ffb640 1 -- 172.21.15.148:0/3374250921 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1458059b40 con 0x7f145c0b3430 2024-03-29T04:10:32.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.995+0000 7f146347b640 1 -- 172.21.15.148:0/3374250921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f145c0d0510 con 0x7f145c0b3430 2024-03-29T04:10:32.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.995+0000 7f1451ffb640 1 -- 172.21.15.148:0/3374250921 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f1458059440 con 0x7f145c0b3430 2024-03-29T04:10:32.976 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-29T04:10:32.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.997+0000 7f146347b640 1 -- 172.21.15.148:0/3374250921 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f144006c530 msgr2=0x7f144006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.997+0000 7f146347b640 1 --2- 172.21.15.148:0/3374250921 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f144006c530 0x7f144006e9f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f145c1ea900 tx=0x7f144c008040 comp rx=0 tx=0).stop 2024-03-29T04:10:32.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.997+0000 7f146347b640 1 -- 172.21.15.148:0/3374250921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f145c0b3430 msgr2=0x7f145c0c6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.997+0000 7f146347b640 1 --2- 172.21.15.148:0/3374250921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f145c0b3430 0x7f145c0c6cf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f145800ba80 tx=0x7f145800bf40 comp rx=0 tx=0).stop 2024-03-29T04:10:32.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.997+0000 7f146347b640 1 -- 172.21.15.148:0/3374250921 shutdown_connections 2024-03-29T04:10:32.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.997+0000 7f146347b640 1 --2- 172.21.15.148:0/3374250921 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f144006c530 0x7f144006e9f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.997+0000 7f146347b640 1 --2- 172.21.15.148:0/3374250921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f145c0cb8e0 0x7f145c0ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.997+0000 7f146347b640 1 --2- 172.21.15.148:0/3374250921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f145c15e3c0 0x7f145c0cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.997+0000 7f146347b640 1 --2- 172.21.15.148:0/3374250921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f145c0b3430 0x7f145c0c6cf0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.997+0000 7f146347b640 1 -- 172.21.15.148:0/3374250921 >> 172.21.15.148:0/3374250921 conn(0x7f145c0b6430 msgr2=0x7f145c0b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:32.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.997+0000 7f146347b640 1 -- 172.21.15.148:0/3374250921 shutdown_connections 2024-03-29T04:10:32.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:32.997+0000 7f146347b640 1 -- 172.21.15.148:0/3374250921 wait complete. 2024-03-29T04:10:32.988 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:10:33.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.112+0000 7f831b152640 1 Processor -- start 2024-03-29T04:10:33.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.113+0000 7f831b152640 1 -- start start 2024-03-29T04:10:33.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.113+0000 7f831b152640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f83140d1910 0x7f8314167220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:33.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.113+0000 7f831b152640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f831415c9b0 con 0x7f831406f470 2024-03-29T04:10:33.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.113+0000 7f831b152640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f831415cc90 con 0x7f83140cf1d0 2024-03-29T04:10:33.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.113+0000 7f831b152640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f83140ccec0 con 0x7f83140d1910 2024-03-29T04:10:33.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.113+0000 7f83196ef640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f83140d1910 0x7f8314167220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:33.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.113+0000 7f8313fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f831406f470 0x7f83140d1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47396/0 (socket says 172.21.15.148:47396) 2024-03-29T04:10:33.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.114+0000 7f8313fff640 1 -- 172.21.15.148:0/568587147 learned_addr learned my addr 172.21.15.148:0/568587147 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:33.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.114+0000 7f83196ef640 1 -- 172.21.15.148:0/568587147 >> v1:172.21.15.157:6789/0 conn(0x7f83140cf1d0 legacy=0x7f83140cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:33.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.114+0000 7f83196ef640 1 -- 172.21.15.148:0/568587147 >> v1:172.21.15.148:6789/0 conn(0x7f831406f470 legacy=0x7f83140d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:33.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.114+0000 7f83137fe640 1 -- 172.21.15.148:0/568587147 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3049375698 0 0) 0x7f831415c9b0 con 0x7f831406f470 2024-03-29T04:10:33.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.114+0000 7f83196ef640 1 -- 172.21.15.148:0/568587147 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831415cf70 con 0x7f83140d1910 2024-03-29T04:10:33.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.115+0000 7f83196ef640 1 --2- 172.21.15.148:0/568587147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83140d1910 0x7f8314167220 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0097a0 tx=0x7f82fc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9937540299fdf8ed server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:33.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.115+0000 7f83137fe640 1 -- 172.21.15.148:0/568587147 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc035020 con 0x7f83140d1910 2024-03-29T04:10:33.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.115+0000 7f83137fe640 1 -- 172.21.15.148:0/568587147 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82fc031830 con 0x7f83140d1910 2024-03-29T04:10:33.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.115+0000 7f83137fe640 1 -- 172.21.15.148:0/568587147 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc031c00 con 0x7f83140d1910 2024-03-29T04:10:33.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.115+0000 7f831b152640 1 -- 172.21.15.148:0/568587147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83140d1910 msgr2=0x7f8314167220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:33.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.115+0000 7f831b152640 1 --2- 172.21.15.148:0/568587147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83140d1910 0x7f8314167220 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0097a0 tx=0x7f82fc02b8d0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.115+0000 7f831b152640 1 -- 172.21.15.148:0/568587147 shutdown_connections 2024-03-29T04:10:33.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.115+0000 7f831b152640 1 --2- 172.21.15.148:0/568587147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83140d1910 0x7f8314167220 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.115+0000 7f831b152640 1 -- 172.21.15.148:0/568587147 >> 172.21.15.148:0/568587147 conn(0x7f83140b3c70 msgr2=0x7f83140b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:33.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.115+0000 7f831b152640 1 -- 172.21.15.148:0/568587147 shutdown_connections 2024-03-29T04:10:33.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.115+0000 7f831b152640 1 -- 172.21.15.148:0/568587147 wait complete. 2024-03-29T04:10:33.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.116+0000 7f831b152640 1 Processor -- start 2024-03-29T04:10:33.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.116+0000 7f831b152640 1 -- start start 2024-03-29T04:10:33.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.118+0000 7f831b152640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f831406f470 0x7f83141f29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:33.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.118+0000 7f831b152640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83140cf1d0 0x7f83140c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:33.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.118+0000 7f831b152640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83140d1910 0x7f83140cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:33.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.118+0000 7f831b152640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f83140ce530 con 0x7f831406f470 2024-03-29T04:10:33.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.118+0000 7f831b152640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f83140ce3b0 con 0x7f83140d1910 2024-03-29T04:10:33.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.118+0000 7f831b152640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f83140ce6b0 con 0x7f83140cf1d0 2024-03-29T04:10:33.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.118+0000 7f8318eee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f831406f470 0x7f83141f29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:33.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.118+0000 7f8318eee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f831406f470 0x7f83141f29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49082/0 (socket says 172.21.15.148:49082) 2024-03-29T04:10:33.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.118+0000 7f8318eee640 1 -- 172.21.15.148:0/2443994781 learned_addr learned my addr 172.21.15.148:0/2443994781 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:33.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.118+0000 7f8313fff640 1 --2- 172.21.15.148:0/2443994781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83140cf1d0 0x7f83140c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:33.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.118+0000 7f83196ef640 1 --2- 172.21.15.148:0/2443994781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83140d1910 0x7f83140cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:33.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.118+0000 7f8318eee640 1 -- 172.21.15.148:0/2443994781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83140cf1d0 msgr2=0x7f83140c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:33.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.118+0000 7f8318eee640 1 --2- 172.21.15.148:0/2443994781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83140cf1d0 0x7f83140c89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.118+0000 7f8318eee640 1 -- 172.21.15.148:0/2443994781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83140d1910 msgr2=0x7f83140cb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:33.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.118+0000 7f8318eee640 1 --2- 172.21.15.148:0/2443994781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83140d1910 0x7f83140cb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.118+0000 7f8318eee640 1 -- 172.21.15.148:0/2443994781 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83140ca4e0 con 0x7f831406f470 2024-03-29T04:10:33.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.118+0000 7f8318eee640 1 --2- 172.21.15.148:0/2443994781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f831406f470 0x7f83141f29b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f830c00ba50 tx=0x7f830c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:33.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.118+0000 7f83117fa640 1 -- 172.21.15.148:0/2443994781 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c019070 con 0x7f831406f470 2024-03-29T04:10:33.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.118+0000 7f83117fa640 1 -- 172.21.15.148:0/2443994781 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f830c00cc20 con 0x7f831406f470 2024-03-29T04:10:33.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.118+0000 7f83117fa640 1 -- 172.21.15.148:0/2443994781 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c014680 con 0x7f831406f470 2024-03-29T04:10:33.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.119+0000 7f831b152640 1 -- 172.21.15.148:0/2443994781 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83141f2ef0 con 0x7f831406f470 2024-03-29T04:10:33.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.119+0000 7f831b152640 1 -- 172.21.15.148:0/2443994781 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83140c9440 con 0x7f831406f470 2024-03-29T04:10:33.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.120+0000 7f83117fa640 1 -- 172.21.15.148:0/2443994781 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f830c01c070 con 0x7f831406f470 2024-03-29T04:10:33.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.120+0000 7f831b152640 1 -- 172.21.15.148:0/2443994781 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83140758d0 con 0x7f831406f470 2024-03-29T04:10:33.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.120+0000 7f83117fa640 1 --2- 172.21.15.148:0/2443994781 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f82f406c530 0x7f82f406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:33.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.120+0000 7f83117fa640 1 -- 172.21.15.148:0/2443994781 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f830c08b630 con 0x7f831406f470 2024-03-29T04:10:33.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.120+0000 7f8313fff640 1 --2- 172.21.15.148:0/2443994781 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f82f406c530 0x7f82f406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:33.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.121+0000 7f8313fff640 1 --2- 172.21.15.148:0/2443994781 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f82f406c530 0x7f82f406e9f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8314156da0 tx=0x7f8304007c00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:33.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.125+0000 7f83117fa640 1 -- 172.21.15.148:0/2443994781 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f830c059350 con 0x7f831406f470 2024-03-29T04:10:33.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.354+0000 7f831b152640 1 -- 172.21.15.148:0/2443994781 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f83140701f0 con 0x7f831406f470 2024-03-29T04:10:33.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.354+0000 7f83117fa640 1 -- 172.21.15.148:0/2443994781 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f830c05d650 con 0x7f831406f470 2024-03-29T04:10:33.335 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:10:33.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.356+0000 7f831b152640 1 -- 172.21.15.148:0/2443994781 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f82f406c530 msgr2=0x7f82f406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:33.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.356+0000 7f831b152640 1 --2- 172.21.15.148:0/2443994781 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f82f406c530 0x7f82f406e9f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8314156da0 tx=0x7f8304007c00 comp rx=0 tx=0).stop 2024-03-29T04:10:33.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.356+0000 7f831b152640 1 -- 172.21.15.148:0/2443994781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f831406f470 msgr2=0x7f83141f29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:33.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.356+0000 7f831b152640 1 --2- 172.21.15.148:0/2443994781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f831406f470 0x7f83141f29b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f830c00ba50 tx=0x7f830c00bf10 comp rx=0 tx=0).stop 2024-03-29T04:10:33.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.356+0000 7f831b152640 1 -- 172.21.15.148:0/2443994781 shutdown_connections 2024-03-29T04:10:33.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.356+0000 7f831b152640 1 --2- 172.21.15.148:0/2443994781 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f82f406c530 0x7f82f406e9f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.356+0000 7f831b152640 1 --2- 172.21.15.148:0/2443994781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83140d1910 0x7f83140cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.356+0000 7f831b152640 1 --2- 172.21.15.148:0/2443994781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83140cf1d0 0x7f83140c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.356+0000 7f831b152640 1 --2- 172.21.15.148:0/2443994781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f831406f470 0x7f83141f29b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.356+0000 7f831b152640 1 -- 172.21.15.148:0/2443994781 >> 172.21.15.148:0/2443994781 conn(0x7f83140b3c70 msgr2=0x7f83140d0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:33.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.357+0000 7f831b152640 1 -- 172.21.15.148:0/2443994781 shutdown_connections 2024-03-29T04:10:33.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.357+0000 7f831b152640 1 -- 172.21.15.148:0/2443994781 wait complete. 2024-03-29T04:10:33.347 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:10:33.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.472+0000 7f174f967640 1 Processor -- start 2024-03-29T04:10:33.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.472+0000 7f174f967640 1 -- start start 2024-03-29T04:10:33.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.473+0000 7f174f967640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17480cff50 0x7f17480ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:33.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.473+0000 7f174f967640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f174815f560 0x7f17480b7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:33.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.473+0000 7f174f967640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17480b5030 con 0x7f17480cc6a0 2024-03-29T04:10:33.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.473+0000 7f174f967640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f17480ce7d0 con 0x7f174815f560 2024-03-29T04:10:33.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.473+0000 7f174f967640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f17480b6d90 con 0x7f17480cff50 2024-03-29T04:10:33.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.473+0000 7f174df04640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f174815f560 0x7f17480b7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:33.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.473+0000 7f174df04640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f174815f560 0x7f17480b7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49086/0 (socket says 172.21.15.148:49086) 2024-03-29T04:10:33.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.473+0000 7f174df04640 1 -- 172.21.15.148:0/3494870826 learned_addr learned my addr 172.21.15.148:0/3494870826 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:33.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.473+0000 7f174cf02640 1 --2- 172.21.15.148:0/3494870826 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17480cff50 0x7f17480ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:33.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.473+0000 7f174df04640 1 -- 172.21.15.148:0/3494870826 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17480cff50 msgr2=0x7f17480ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:33.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.473+0000 7f174df04640 1 --2- 172.21.15.148:0/3494870826 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17480cff50 0x7f17480ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.473+0000 7f174df04640 1 -- 172.21.15.148:0/3494870826 >> v1:172.21.15.157:6789/0 conn(0x7f17480cc6a0 legacy=0x7f17480cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:33.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.473+0000 7f174df04640 1 -- 172.21.15.148:0/3494870826 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17480b5310 con 0x7f174815f560 2024-03-29T04:10:33.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.474+0000 7f174df04640 1 --2- 172.21.15.148:0/3494870826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f174815f560 0x7f17480b7220 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1740009e50 tx=0x7f174002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73567443947de490 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:33.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.474+0000 7f173ffff640 1 -- 172.21.15.148:0/3494870826 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174002ca70 con 0x7f174815f560 2024-03-29T04:10:33.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.474+0000 7f173ffff640 1 -- 172.21.15.148:0/3494870826 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f174002cc10 con 0x7f174815f560 2024-03-29T04:10:33.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.474+0000 7f173ffff640 1 -- 172.21.15.148:0/3494870826 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17400343f0 con 0x7f174815f560 2024-03-29T04:10:33.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.474+0000 7f174f967640 1 -- 172.21.15.148:0/3494870826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f174815f560 msgr2=0x7f17480b7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:33.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.474+0000 7f174f967640 1 --2- 172.21.15.148:0/3494870826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f174815f560 0x7f17480b7220 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1740009e50 tx=0x7f174002bed0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.474+0000 7f174f967640 1 -- 172.21.15.148:0/3494870826 shutdown_connections 2024-03-29T04:10:33.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.474+0000 7f174f967640 1 --2- 172.21.15.148:0/3494870826 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17480cff50 0x7f17480ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.474+0000 7f174f967640 1 --2- 172.21.15.148:0/3494870826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f174815f560 0x7f17480b7220 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.474+0000 7f174f967640 1 -- 172.21.15.148:0/3494870826 >> 172.21.15.148:0/3494870826 conn(0x7f17480cead0 msgr2=0x7f17480ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:33.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.475+0000 7f174f967640 1 -- 172.21.15.148:0/3494870826 shutdown_connections 2024-03-29T04:10:33.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.475+0000 7f174f967640 1 -- 172.21.15.148:0/3494870826 wait complete. 2024-03-29T04:10:33.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.476+0000 7f174f967640 1 Processor -- start 2024-03-29T04:10:33.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.476+0000 7f174f967640 1 -- start start 2024-03-29T04:10:33.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.476+0000 7f174f967640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17480cc6a0 0x7f17480c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:33.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.476+0000 7f174f967640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17480cff50 0x7f17480af990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:33.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.476+0000 7f174f967640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f174815f560 0x7f17480b0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:33.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.476+0000 7f174f967640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1748159060 con 0x7f174815f560 2024-03-29T04:10:33.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.476+0000 7f174f967640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1748158ee0 con 0x7f17480cff50 2024-03-29T04:10:33.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.476+0000 7f174f967640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f174815f9b0 con 0x7f17480cc6a0 2024-03-29T04:10:33.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.477+0000 7f174df04640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f174815f560 0x7f17480b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:33.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.477+0000 7f174df04640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f174815f560 0x7f17480b0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49098/0 (socket says 172.21.15.148:49098) 2024-03-29T04:10:33.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.477+0000 7f174df04640 1 -- 172.21.15.148:0/2524736562 learned_addr learned my addr 172.21.15.148:0/2524736562 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:33.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.477+0000 7f174d703640 1 --2- 172.21.15.148:0/2524736562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17480cc6a0 0x7f17480c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:33.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.477+0000 7f174cf02640 1 --2- 172.21.15.148:0/2524736562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17480cff50 0x7f17480af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:33.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.477+0000 7f174df04640 1 -- 172.21.15.148:0/2524736562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17480cc6a0 msgr2=0x7f17480c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:33.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.477+0000 7f174df04640 1 --2- 172.21.15.148:0/2524736562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17480cc6a0 0x7f17480c8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.477+0000 7f174df04640 1 -- 172.21.15.148:0/2524736562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17480cff50 msgr2=0x7f17480af990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:33.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.477+0000 7f174df04640 1 --2- 172.21.15.148:0/2524736562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17480cff50 0x7f17480af990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.477+0000 7f174df04640 1 -- 172.21.15.148:0/2524736562 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17480ae8f0 con 0x7f174815f560 2024-03-29T04:10:33.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.477+0000 7f174df04640 1 --2- 172.21.15.148:0/2524736562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f174815f560 0x7f17480b0310 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1740009e50 tx=0x7f1740004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:33.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.477+0000 7f173dffb640 1 -- 172.21.15.148:0/2524736562 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1740043020 con 0x7f174815f560 2024-03-29T04:10:33.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.477+0000 7f173dffb640 1 -- 172.21.15.148:0/2524736562 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f174003e450 con 0x7f174815f560 2024-03-29T04:10:33.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.477+0000 7f174f967640 1 -- 172.21.15.148:0/2524736562 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17480aebd0 con 0x7f174815f560 2024-03-29T04:10:33.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.477+0000 7f173dffb640 1 -- 172.21.15.148:0/2524736562 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1740009510 con 0x7f174815f560 2024-03-29T04:10:33.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.477+0000 7f174f967640 1 -- 172.21.15.148:0/2524736562 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17480af190 con 0x7f174815f560 2024-03-29T04:10:33.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.479+0000 7f174f967640 1 -- 172.21.15.148:0/2524736562 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1748068850 con 0x7f174815f560 2024-03-29T04:10:33.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.479+0000 7f173dffb640 1 -- 172.21.15.148:0/2524736562 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1740046020 con 0x7f174815f560 2024-03-29T04:10:33.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.479+0000 7f173dffb640 1 --2- 172.21.15.148:0/2524736562 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f172c06c560 0x7f172c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:33.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.479+0000 7f174d703640 1 --2- 172.21.15.148:0/2524736562 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f172c06c560 0x7f172c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:33.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.479+0000 7f173dffb640 1 -- 172.21.15.148:0/2524736562 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f17400ae080 con 0x7f174815f560 2024-03-29T04:10:33.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.480+0000 7f174d703640 1 --2- 172.21.15.148:0/2524736562 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f172c06c560 0x7f172c06ea20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f17440016e0 tx=0x7f1744009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:33.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.483+0000 7f173dffb640 1 -- 172.21.15.148:0/2524736562 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f174004d030 con 0x7f174815f560 2024-03-29T04:10:33.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.663+0000 7f174f967640 1 -- 172.21.15.148:0/2524736562 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f17480cf8a0 con 0x7f174815f560 2024-03-29T04:10:33.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.664+0000 7f173dffb640 1 -- 172.21.15.148:0/2524736562 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f174007bc90 con 0x7f174815f560 2024-03-29T04:10:33.644 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-29T04:10:33.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.666+0000 7f174f967640 1 -- 172.21.15.148:0/2524736562 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f172c06c560 msgr2=0x7f172c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:33.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.666+0000 7f174f967640 1 --2- 172.21.15.148:0/2524736562 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f172c06c560 0x7f172c06ea20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f17440016e0 tx=0x7f1744009040 comp rx=0 tx=0).stop 2024-03-29T04:10:33.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.666+0000 7f174f967640 1 -- 172.21.15.148:0/2524736562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f174815f560 msgr2=0x7f17480b0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:33.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.666+0000 7f174f967640 1 --2- 172.21.15.148:0/2524736562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f174815f560 0x7f17480b0310 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1740009e50 tx=0x7f1740004a40 comp rx=0 tx=0).stop 2024-03-29T04:10:33.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.666+0000 7f174f967640 1 -- 172.21.15.148:0/2524736562 shutdown_connections 2024-03-29T04:10:33.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.666+0000 7f174f967640 1 --2- 172.21.15.148:0/2524736562 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f172c06c560 0x7f172c06ea20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.666+0000 7f174f967640 1 --2- 172.21.15.148:0/2524736562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f174815f560 0x7f17480b0310 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.666+0000 7f174f967640 1 --2- 172.21.15.148:0/2524736562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17480cff50 0x7f17480af990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.666+0000 7f174f967640 1 --2- 172.21.15.148:0/2524736562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17480cc6a0 0x7f17480c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.666+0000 7f174f967640 1 -- 172.21.15.148:0/2524736562 >> 172.21.15.148:0/2524736562 conn(0x7f17480cead0 msgr2=0x7f1748160240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:33.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.666+0000 7f174f967640 1 -- 172.21.15.148:0/2524736562 shutdown_connections 2024-03-29T04:10:33.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.666+0000 7f174f967640 1 -- 172.21.15.148:0/2524736562 wait complete. 2024-03-29T04:10:33.657 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:10:33.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.781+0000 7f9898582640 1 Processor -- start 2024-03-29T04:10:33.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.781+0000 7f9898582640 1 -- start start 2024-03-29T04:10:33.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.782+0000 7f9898582640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f989015e3c0 0x7f9890161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:33.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.782+0000 7f9898582640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98900b2f00 0x7f98901624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:33.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.782+0000 7f9898582640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f98900b3430 0x7f98900d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:33.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.782+0000 7f9898582640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f98900cf960 con 0x7f989015e3c0 2024-03-29T04:10:33.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.782+0000 7f9898582640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f98900cf330 con 0x7f98900b3430 2024-03-29T04:10:33.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.782+0000 7f9898582640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9890162cf0 con 0x7f98900b2f00 2024-03-29T04:10:33.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.782+0000 7f989631e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f989015e3c0 0x7f9890161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:33.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.782+0000 7f989631e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f989015e3c0 0x7f9890161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49104/0 (socket says 172.21.15.148:49104) 2024-03-29T04:10:33.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.782+0000 7f989631e640 1 -- 172.21.15.148:0/4266847609 learned_addr learned my addr 172.21.15.148:0/4266847609 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:33.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.782+0000 7f9895b1d640 1 --2- 172.21.15.148:0/4266847609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98900b2f00 0x7f98901624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:33.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.782+0000 7f9896b1f640 1 --2- 172.21.15.148:0/4266847609 >> v2:172.21.15.157:3300/0 conn(0x7f98900b3430 0x7f98900d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:33.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.782+0000 7f989631e640 1 -- 172.21.15.148:0/4266847609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98900b2f00 msgr2=0x7f98901624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:33.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.782+0000 7f989631e640 1 --2- 172.21.15.148:0/4266847609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98900b2f00 0x7f98901624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.782+0000 7f989631e640 1 -- 172.21.15.148:0/4266847609 >> v2:172.21.15.157:3300/0 conn(0x7f98900b3430 msgr2=0x7f98900d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:33.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.782+0000 7f989631e640 1 --2- 172.21.15.148:0/4266847609 >> v2:172.21.15.157:3300/0 conn(0x7f98900b3430 0x7f98900d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.782+0000 7f989631e640 1 -- 172.21.15.148:0/4266847609 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98900cf5f0 con 0x7f989015e3c0 2024-03-29T04:10:33.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.783+0000 7f989631e640 1 --2- 172.21.15.148:0/4266847609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f989015e3c0 0x7f9890161f70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f98900cf0c0 tx=0x7f988002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe90721f002ab137 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:33.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.783+0000 7f989531c640 1 -- 172.21.15.148:0/4266847609 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880035020 con 0x7f989015e3c0 2024-03-29T04:10:33.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.783+0000 7f989531c640 1 -- 172.21.15.148:0/4266847609 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9880031870 con 0x7f989015e3c0 2024-03-29T04:10:33.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.783+0000 7f989531c640 1 -- 172.21.15.148:0/4266847609 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880031c40 con 0x7f989015e3c0 2024-03-29T04:10:33.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.784+0000 7f9898582640 1 -- 172.21.15.148:0/4266847609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f989015e3c0 msgr2=0x7f9890161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:33.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.784+0000 7f9898582640 1 --2- 172.21.15.148:0/4266847609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f989015e3c0 0x7f9890161f70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f98900cf0c0 tx=0x7f988002b910 comp rx=0 tx=0).stop 2024-03-29T04:10:33.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.784+0000 7f9898582640 1 -- 172.21.15.148:0/4266847609 shutdown_connections 2024-03-29T04:10:33.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.784+0000 7f9898582640 1 --2- 172.21.15.148:0/4266847609 >> v2:172.21.15.157:3300/0 conn(0x7f98900b3430 0x7f98900d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.784+0000 7f9898582640 1 --2- 172.21.15.148:0/4266847609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98900b2f00 0x7f98901624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.784+0000 7f9898582640 1 --2- 172.21.15.148:0/4266847609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f989015e3c0 0x7f9890161f70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.784+0000 7f9898582640 1 -- 172.21.15.148:0/4266847609 >> 172.21.15.148:0/4266847609 conn(0x7f98900b6430 msgr2=0x7f98900d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:33.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.784+0000 7f9898582640 1 -- 172.21.15.148:0/4266847609 shutdown_connections 2024-03-29T04:10:33.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.784+0000 7f9898582640 1 -- 172.21.15.148:0/4266847609 wait complete. 2024-03-29T04:10:33.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.785+0000 7f9898582640 1 Processor -- start 2024-03-29T04:10:33.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.785+0000 7f9898582640 1 -- start start 2024-03-29T04:10:33.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.786+0000 7f9898582640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98900b3430 0x7f98900cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:33.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.786+0000 7f9898582640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f989015e3c0 0x7f98901e7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:33.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.786+0000 7f9898582640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98900cd670 0x7f9890160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:33.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.786+0000 7f9898582640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f98900b39a0 con 0x7f98900cd670 2024-03-29T04:10:33.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.786+0000 7f9898582640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f989015e7a0 con 0x7f989015e3c0 2024-03-29T04:10:33.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.786+0000 7f9898582640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f98900b3b20 con 0x7f98900b3430 2024-03-29T04:10:33.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.786+0000 7f9896b1f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98900cd670 0x7f9890160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:33.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.786+0000 7f9896b1f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98900cd670 0x7f9890160eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49110/0 (socket says 172.21.15.148:49110) 2024-03-29T04:10:33.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.786+0000 7f9896b1f640 1 -- 172.21.15.148:0/2862876742 learned_addr learned my addr 172.21.15.148:0/2862876742 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:33.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.786+0000 7f989631e640 1 --2- 172.21.15.148:0/2862876742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98900b3430 0x7f98900cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:33.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.786+0000 7f9895b1d640 1 --2- 172.21.15.148:0/2862876742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f989015e3c0 0x7f98901e7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:33.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.786+0000 7f9896b1f640 1 -- 172.21.15.148:0/2862876742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98900b3430 msgr2=0x7f98900cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:33.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.786+0000 7f9896b1f640 1 --2- 172.21.15.148:0/2862876742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98900b3430 0x7f98900cd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.786+0000 7f9896b1f640 1 -- 172.21.15.148:0/2862876742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f989015e3c0 msgr2=0x7f98901e7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:33.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.786+0000 7f9896b1f640 1 --2- 172.21.15.148:0/2862876742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f989015e3c0 0x7f98901e7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.786+0000 7f9896b1f640 1 -- 172.21.15.148:0/2862876742 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98901600f0 con 0x7f98900cd670 2024-03-29T04:10:33.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.786+0000 7f9896b1f640 1 --2- 172.21.15.148:0/2862876742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98900cd670 0x7f9890160eb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f988800efc0 tx=0x7f988800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:33.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.786+0000 7f9886ffd640 1 -- 172.21.15.148:0/2862876742 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888018020 con 0x7f98900cd670 2024-03-29T04:10:33.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.786+0000 7f9886ffd640 1 -- 172.21.15.148:0/2862876742 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9888014480 con 0x7f98900cd670 2024-03-29T04:10:33.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.787+0000 7f9898582640 1 -- 172.21.15.148:0/2862876742 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98901603d0 con 0x7f98900cd670 2024-03-29T04:10:33.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.787+0000 7f9886ffd640 1 -- 172.21.15.148:0/2862876742 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888014870 con 0x7f98900cd670 2024-03-29T04:10:33.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.787+0000 7f9898582640 1 -- 172.21.15.148:0/2862876742 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9890160990 con 0x7f98900cd670 2024-03-29T04:10:33.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.788+0000 7f9898582640 1 -- 172.21.15.148:0/2862876742 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98900d0200 con 0x7f98900cd670 2024-03-29T04:10:33.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.788+0000 7f9886ffd640 1 -- 172.21.15.148:0/2862876742 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9888005d10 con 0x7f98900cd670 2024-03-29T04:10:33.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.788+0000 7f9886ffd640 1 --2- 172.21.15.148:0/2862876742 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f987406c530 0x7f987406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:33.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.788+0000 7f9886ffd640 1 -- 172.21.15.148:0/2862876742 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f988808c570 con 0x7f98900cd670 2024-03-29T04:10:33.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.788+0000 7f989631e640 1 --2- 172.21.15.148:0/2862876742 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f987406c530 0x7f987406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:33.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.789+0000 7f989631e640 1 --2- 172.21.15.148:0/2862876742 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f987406c530 0x7f987406e9f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9880036000 tx=0x7f988003c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:33.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.793+0000 7f9886ffd640 1 -- 172.21.15.148:0/2862876742 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f98900d0200 con 0x7f98900cd670 2024-03-29T04:10:33.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.974+0000 7f9898582640 1 -- 172.21.15.148:0/2862876742 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f98900c75d0 con 0x7f98900cd670 2024-03-29T04:10:33.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.975+0000 7f9886ffd640 1 -- 172.21.15.148:0/2862876742 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f9888022070 con 0x7f98900cd670 2024-03-29T04:10:33.955 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:10:33.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.977+0000 7f9898582640 1 -- 172.21.15.148:0/2862876742 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f987406c530 msgr2=0x7f987406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:33.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.977+0000 7f9898582640 1 --2- 172.21.15.148:0/2862876742 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f987406c530 0x7f987406e9f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9880036000 tx=0x7f988003c040 comp rx=0 tx=0).stop 2024-03-29T04:10:33.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.977+0000 7f9898582640 1 -- 172.21.15.148:0/2862876742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98900cd670 msgr2=0x7f9890160eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:33.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.977+0000 7f9898582640 1 --2- 172.21.15.148:0/2862876742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98900cd670 0x7f9890160eb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f988800efc0 tx=0x7f988800c490 comp rx=0 tx=0).stop 2024-03-29T04:10:33.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.977+0000 7f9898582640 1 -- 172.21.15.148:0/2862876742 shutdown_connections 2024-03-29T04:10:33.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.977+0000 7f9898582640 1 --2- 172.21.15.148:0/2862876742 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f987406c530 0x7f987406e9f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.977+0000 7f9898582640 1 --2- 172.21.15.148:0/2862876742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98900cd670 0x7f9890160eb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.977+0000 7f9898582640 1 --2- 172.21.15.148:0/2862876742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f989015e3c0 0x7f98901e7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.977+0000 7f9898582640 1 --2- 172.21.15.148:0/2862876742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98900b3430 0x7f98900cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.977+0000 7f9898582640 1 -- 172.21.15.148:0/2862876742 >> 172.21.15.148:0/2862876742 conn(0x7f98900b6430 msgr2=0x7f98900d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:33.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.977+0000 7f9898582640 1 -- 172.21.15.148:0/2862876742 shutdown_connections 2024-03-29T04:10:33.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:33.977+0000 7f9898582640 1 -- 172.21.15.148:0/2862876742 wait complete. 2024-03-29T04:10:33.968 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:10:34.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.089+0000 7f0f84b1c640 1 Processor -- start 2024-03-29T04:10:34.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.089+0000 7f0f84b1c640 1 -- start start 2024-03-29T04:10:34.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.090+0000 7f0f84b1c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f800ceb50 0x7f0f800cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:34.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.090+0000 7f0f84b1c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0f801608b0 0x7f0f8015cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:34.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.090+0000 7f0f84b1c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f800d0250 con 0x7f0f8015c5f0 2024-03-29T04:10:34.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.090+0000 7f0f84b1c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f0f800d19d0 con 0x7f0f801608b0 2024-03-29T04:10:34.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.090+0000 7f0f84b1c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0f800ccde0 con 0x7f0f800ceb50 2024-03-29T04:10:34.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.090+0000 7f0f7e59c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f800ceb50 0x7f0f800cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:34.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.090+0000 7f0f7ed9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0f801608b0 0x7f0f8015cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:34.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.090+0000 7f0f7e59c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f800ceb50 0x7f0f800cfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.148:44904/0 (socket says 172.21.15.148:44904) 2024-03-29T04:10:34.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.090+0000 7f0f7e59c640 1 -- 172.21.15.148:0/2372384193 learned_addr learned my addr 172.21.15.148:0/2372384193 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:34.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.091+0000 7f0f7e59c640 1 -- 172.21.15.148:0/2372384193 >> v2:172.21.15.157:3300/0 conn(0x7f0f801608b0 msgr2=0x7f0f8015cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:34.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.091+0000 7f0f7e59c640 1 --2- 172.21.15.148:0/2372384193 >> v2:172.21.15.157:3300/0 conn(0x7f0f801608b0 0x7f0f8015cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.091+0000 7f0f7e59c640 1 -- 172.21.15.148:0/2372384193 >> v1:172.21.15.157:6789/0 conn(0x7f0f8015c5f0 legacy=0x7f0f800cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:34.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.091+0000 7f0f7e59c640 1 -- 172.21.15.148:0/2372384193 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f800ccb00 con 0x7f0f800ceb50 2024-03-29T04:10:34.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.091+0000 7f0f7dd9b640 1 -- 172.21.15.148:0/2372384193 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3795157804 0 0) 0x7f0f800d0250 con 0x7f0f8015c5f0 2024-03-29T04:10:34.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.091+0000 7f0f7ed9d640 1 --2- 172.21.15.148:0/2372384193 >> v2:172.21.15.157:3300/0 conn(0x7f0f801608b0 0x7f0f8015cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:34.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.091+0000 7f0f7e59c640 1 --2- 172.21.15.148:0/2372384193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f800ceb50 0x7f0f800cfd10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009870 tx=0x7f0f7002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14386b51373a1074 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:34.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.092+0000 7f0f7dd9b640 1 -- 172.21.15.148:0/2372384193 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f70034070 con 0x7f0f800ceb50 2024-03-29T04:10:34.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.092+0000 7f0f7dd9b640 1 -- 172.21.15.148:0/2372384193 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f70030840 con 0x7f0f800ceb50 2024-03-29T04:10:34.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.092+0000 7f0f7dd9b640 1 -- 172.21.15.148:0/2372384193 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f70030c10 con 0x7f0f800ceb50 2024-03-29T04:10:34.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.092+0000 7f0f84b1c640 1 -- 172.21.15.148:0/2372384193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f800ceb50 msgr2=0x7f0f800cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:34.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.092+0000 7f0f84b1c640 1 --2- 172.21.15.148:0/2372384193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f800ceb50 0x7f0f800cfd10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009870 tx=0x7f0f7002b980 comp rx=0 tx=0).stop 2024-03-29T04:10:34.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.092+0000 7f0f84b1c640 1 -- 172.21.15.148:0/2372384193 shutdown_connections 2024-03-29T04:10:34.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.092+0000 7f0f84b1c640 1 --2- 172.21.15.148:0/2372384193 >> v2:172.21.15.157:3300/0 conn(0x7f0f801608b0 0x7f0f8015cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.092+0000 7f0f84b1c640 1 --2- 172.21.15.148:0/2372384193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f800ceb50 0x7f0f800cfd10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.092+0000 7f0f84b1c640 1 -- 172.21.15.148:0/2372384193 >> 172.21.15.148:0/2372384193 conn(0x7f0f800b3c70 msgr2=0x7f0f800b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:34.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.092+0000 7f0f84b1c640 1 -- 172.21.15.148:0/2372384193 shutdown_connections 2024-03-29T04:10:34.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.092+0000 7f0f84b1c640 1 -- 172.21.15.148:0/2372384193 wait complete. 2024-03-29T04:10:34.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.093+0000 7f0f84b1c640 1 Processor -- start 2024-03-29T04:10:34.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.094+0000 7f0f84b1c640 1 -- start start 2024-03-29T04:10:34.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.094+0000 7f0f84b1c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f800ceb50 0x7f0f800cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:34.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.094+0000 7f0f84b1c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f801608b0 0x7f0f800c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:34.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.094+0000 7f0f84b1c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f800cbe70 0x7f0f800c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:34.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.094+0000 7f0f84b1c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0f800ce540 con 0x7f0f801608b0 2024-03-29T04:10:34.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.094+0000 7f0f84b1c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0f800cc790 con 0x7f0f800ceb50 2024-03-29T04:10:34.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.094+0000 7f0f84b1c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0f800ce6c0 con 0x7f0f800cbe70 2024-03-29T04:10:34.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.094+0000 7f0f77fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f801608b0 0x7f0f800c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:34.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.094+0000 7f0f77fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f801608b0 0x7f0f800c7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49124/0 (socket says 172.21.15.148:49124) 2024-03-29T04:10:34.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.094+0000 7f0f77fff640 1 -- 172.21.15.148:0/500216881 learned_addr learned my addr 172.21.15.148:0/500216881 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:34.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.094+0000 7f0f7ed9d640 1 --2- 172.21.15.148:0/500216881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f800cbe70 0x7f0f800c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:34.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.094+0000 7f0f7e59c640 1 --2- 172.21.15.148:0/500216881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f800ceb50 0x7f0f800cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:34.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.095+0000 7f0f77fff640 1 -- 172.21.15.148:0/500216881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f800cbe70 msgr2=0x7f0f800c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:34.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.095+0000 7f0f77fff640 1 --2- 172.21.15.148:0/500216881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f800cbe70 0x7f0f800c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.095+0000 7f0f77fff640 1 -- 172.21.15.148:0/500216881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f800ceb50 msgr2=0x7f0f800cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:34.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.095+0000 7f0f77fff640 1 --2- 172.21.15.148:0/500216881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f800ceb50 0x7f0f800cb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.095+0000 7f0f77fff640 1 -- 172.21.15.148:0/500216881 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f800c8750 con 0x7f0f801608b0 2024-03-29T04:10:34.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.095+0000 7f0f77fff640 1 --2- 172.21.15.148:0/500216881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f801608b0 0x7f0f800c7ae0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800c4e0 tx=0x7f0f6800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:34.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.095+0000 7f0f76ffd640 1 -- 172.21.15.148:0/500216881 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f68017070 con 0x7f0f801608b0 2024-03-29T04:10:34.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.095+0000 7f0f76ffd640 1 -- 172.21.15.148:0/500216881 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f680035f0 con 0x7f0f801608b0 2024-03-29T04:10:34.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.095+0000 7f0f84b1c640 1 -- 172.21.15.148:0/500216881 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f800c8a30 con 0x7f0f801608b0 2024-03-29T04:10:34.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.095+0000 7f0f76ffd640 1 -- 172.21.15.148:0/500216881 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f680039e0 con 0x7f0f801608b0 2024-03-29T04:10:34.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.095+0000 7f0f84b1c640 1 -- 172.21.15.148:0/500216881 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f80145300 con 0x7f0f801608b0 2024-03-29T04:10:34.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.096+0000 7f0f84b1c640 1 -- 172.21.15.148:0/500216881 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f800687e0 con 0x7f0f801608b0 2024-03-29T04:10:34.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.096+0000 7f0f76ffd640 1 -- 172.21.15.148:0/500216881 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0f6801a070 con 0x7f0f801608b0 2024-03-29T04:10:34.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.097+0000 7f0f76ffd640 1 --2- 172.21.15.148:0/500216881 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f0f6006c530 0x7f0f6006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:34.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.097+0000 7f0f76ffd640 1 -- 172.21.15.148:0/500216881 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f0f6808aca0 con 0x7f0f801608b0 2024-03-29T04:10:34.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.097+0000 7f0f7e59c640 1 --2- 172.21.15.148:0/500216881 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f0f6006c530 0x7f0f6006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:34.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.097+0000 7f0f7e59c640 1 --2- 172.21.15.148:0/500216881 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f0f6006c530 0x7f0f6006e9f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0f70036000 tx=0x7f0f70008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:34.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.101+0000 7f0f76ffd640 1 -- 172.21.15.148:0/500216881 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0f680589c0 con 0x7f0f801608b0 2024-03-29T04:10:34.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.322+0000 7f0f84b1c640 1 -- 172.21.15.148:0/500216881 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f800701d0 con 0x7f0f801608b0 2024-03-29T04:10:34.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.323+0000 7f0f76ffd640 1 -- 172.21.15.148:0/500216881 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f0f6805ccc0 con 0x7f0f801608b0 2024-03-29T04:10:34.303 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:34.303 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/36660496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":36660496},{"type":"v1","addr":"172.21.15.157:6834","nonce":36660496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2674481055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2674481055},{"type":"v1","addr":"172.21.15.157:6835","nonce":2674481055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3356449643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3356449643},{"type":"v1","addr":"172.21.15.148:6836","nonce":3356449643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:09:59.886367+0000","modified":"2024-03-29T04:10:22.957623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/2453181817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2453181817},{"type":"v1","addr":"172.21.15.148:6837","nonce":2453181817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-03-29T04:10:34.303 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 9 2024-03-29T04:10:34.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.325+0000 7f0f84b1c640 1 -- 172.21.15.148:0/500216881 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f0f6006c530 msgr2=0x7f0f6006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:34.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.325+0000 7f0f84b1c640 1 --2- 172.21.15.148:0/500216881 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f0f6006c530 0x7f0f6006e9f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0f70036000 tx=0x7f0f70008030 comp rx=0 tx=0).stop 2024-03-29T04:10:34.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.325+0000 7f0f84b1c640 1 -- 172.21.15.148:0/500216881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f801608b0 msgr2=0x7f0f800c7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:34.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.325+0000 7f0f84b1c640 1 --2- 172.21.15.148:0/500216881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f801608b0 0x7f0f800c7ae0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800c4e0 tx=0x7f0f6800c8e0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.325+0000 7f0f84b1c640 1 -- 172.21.15.148:0/500216881 shutdown_connections 2024-03-29T04:10:34.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.325+0000 7f0f84b1c640 1 --2- 172.21.15.148:0/500216881 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f0f6006c530 0x7f0f6006e9f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.325+0000 7f0f84b1c640 1 --2- 172.21.15.148:0/500216881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f800cbe70 0x7f0f800c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.325+0000 7f0f84b1c640 1 --2- 172.21.15.148:0/500216881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f801608b0 0x7f0f800c7ae0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.325+0000 7f0f84b1c640 1 --2- 172.21.15.148:0/500216881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f800ceb50 0x7f0f800cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.325+0000 7f0f84b1c640 1 -- 172.21.15.148:0/500216881 >> 172.21.15.148:0/500216881 conn(0x7f0f800b3c70 msgr2=0x7f0f800b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:34.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.325+0000 7f0f84b1c640 1 -- 172.21.15.148:0/500216881 shutdown_connections 2024-03-29T04:10:34.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.325+0000 7f0f84b1c640 1 -- 172.21.15.148:0/500216881 wait complete. 2024-03-29T04:10:34.317 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:34.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.440+0000 7f65f52d3640 1 Processor -- start 2024-03-29T04:10:34.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.440+0000 7f65f52d3640 1 -- start start 2024-03-29T04:10:34.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.441+0000 7f65f52d3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f015c5f0 0x7f65f00cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:34.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.441+0000 7f65f52d3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65f00d19d0 con 0x7f65f00ceb50 2024-03-29T04:10:34.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.441+0000 7f65f52d3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65f00cf470 con 0x7f65f01608b0 2024-03-29T04:10:34.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.441+0000 7f65f52d3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f65f00ccde0 con 0x7f65f015c5f0 2024-03-29T04:10:34.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.441+0000 7f65eeffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f65f00ceb50 0x7f65f00cfd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47400/0 (socket says 172.21.15.148:47400) 2024-03-29T04:10:34.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.441+0000 7f65eeffd640 1 -- 172.21.15.148:0/2561859568 learned_addr learned my addr 172.21.15.148:0/2561859568 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:34.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.441+0000 7f65ee7fc640 1 --2- 172.21.15.148:0/2561859568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f015c5f0 0x7f65f00cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:34.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.441+0000 7f65edffb640 1 -- 172.21.15.148:0/2561859568 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3434222026 0 0) 0x7f65f00d19d0 con 0x7f65f00ceb50 2024-03-29T04:10:34.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.441+0000 7f65edffb640 1 -- 172.21.15.148:0/2561859568 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65d8003520 con 0x7f65f00ceb50 2024-03-29T04:10:34.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.442+0000 7f65edffb640 1 -- 172.21.15.148:0/2561859568 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2870872515 0 0) 0x7f65d8003520 con 0x7f65f00ceb50 2024-03-29T04:10:34.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.442+0000 7f65edffb640 1 -- 172.21.15.148:0/2561859568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f015c5f0 msgr2=0x7f65f00cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:34.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.442+0000 7f65edffb640 1 --2- 172.21.15.148:0/2561859568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f015c5f0 0x7f65f00cef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.442+0000 7f65edffb640 1 -- 172.21.15.148:0/2561859568 >> v1:172.21.15.157:6789/0 conn(0x7f65f01608b0 legacy=0x7f65f015cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:34.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.442+0000 7f65edffb640 1 -- 172.21.15.148:0/2561859568 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f00ccb00 con 0x7f65f00ceb50 2024-03-29T04:10:34.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.442+0000 7f65edffb640 1 -- 172.21.15.148:0/2561859568 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f65e0001690 con 0x7f65f00ceb50 2024-03-29T04:10:34.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.442+0000 7f65edffb640 1 -- 172.21.15.148:0/2561859568 >> v1:172.21.15.148:6789/0 conn(0x7f65f00ceb50 legacy=0x7f65f00cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:34.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.442+0000 7f65edffb640 1 --2- 172.21.15.148:0/2561859568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65d80039e0 0x7f65d8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:34.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.442+0000 7f65edffb640 1 --2- 172.21.15.148:0/2561859568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65d8004340 0x7f65d8004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:34.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.442+0000 7f65edffb640 1 --2- 172.21.15.148:0/2561859568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65d8005020 0x7f65d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:34.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.442+0000 7f65edffb640 1 -- 172.21.15.148:0/2561859568 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f65f00ccde0 con 0x7f65d8004340 2024-03-29T04:10:34.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.442+0000 7f65edffb640 1 -- 172.21.15.148:0/2561859568 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f65d800bf00 con 0x7f65d8005020 2024-03-29T04:10:34.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.442+0000 7f65edffb640 1 -- 172.21.15.148:0/2561859568 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f65d800c0b0 con 0x7f65d80039e0 2024-03-29T04:10:34.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.442+0000 7f65ee7fc640 1 --2- 172.21.15.148:0/2561859568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65d8004340 0x7f65d8004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:34.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.442+0000 7f65eeffd640 1 --2- 172.21.15.148:0/2561859568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65d80039e0 0x7f65d8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:34.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.442+0000 7f65ef7fe640 1 --2- 172.21.15.148:0/2561859568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65d8005020 0x7f65d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:34.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.442+0000 7f65ee7fc640 1 -- 172.21.15.148:0/2561859568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65d80039e0 msgr2=0x7f65d8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:34.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.442+0000 7f65ee7fc640 1 --2- 172.21.15.148:0/2561859568 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65d80039e0 0x7f65d8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.442+0000 7f65ee7fc640 1 -- 172.21.15.148:0/2561859568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65d8005020 msgr2=0x7f65d800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:34.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.442+0000 7f65ee7fc640 1 --2- 172.21.15.148:0/2561859568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65d8005020 0x7f65d800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.442+0000 7f65ee7fc640 1 -- 172.21.15.148:0/2561859568 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d8003520 con 0x7f65d8004340 2024-03-29T04:10:34.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.443+0000 7f65ee7fc640 1 --2- 172.21.15.148:0/2561859568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65d8004340 0x7f65d8004820 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f65e4009ac0 tx=0x7f65d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f3823bb40913fc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:34.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.443+0000 7f65edffb640 1 -- 172.21.15.148:0/2561859568 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4039070 con 0x7f65d8004340 2024-03-29T04:10:34.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.443+0000 7f65edffb640 1 -- 172.21.15.148:0/2561859568 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65e402c6b0 con 0x7f65d8004340 2024-03-29T04:10:34.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.443+0000 7f65edffb640 1 -- 172.21.15.148:0/2561859568 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4009690 con 0x7f65d8004340 2024-03-29T04:10:34.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.443+0000 7f65f52d3640 1 -- 172.21.15.148:0/2561859568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65d8004340 msgr2=0x7f65d8004820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:34.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.443+0000 7f65f52d3640 1 --2- 172.21.15.148:0/2561859568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65d8004340 0x7f65d8004820 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f65e4009ac0 tx=0x7f65d8005830 comp rx=0 tx=0).stop 2024-03-29T04:10:34.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.443+0000 7f65ef7fe640 1 -- 172.21.15.148:0/2561859568 reap_dead start 2024-03-29T04:10:34.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.443+0000 7f65f52d3640 1 -- 172.21.15.148:0/2561859568 shutdown_connections 2024-03-29T04:10:34.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.444+0000 7f65f52d3640 1 -- 172.21.15.148:0/2561859568 >> 172.21.15.148:0/2561859568 conn(0x7f65f00b3c70 msgr2=0x7f65f00b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:34.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.444+0000 7f65f52d3640 1 -- 172.21.15.148:0/2561859568 shutdown_connections 2024-03-29T04:10:34.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.444+0000 7f65f52d3640 1 -- 172.21.15.148:0/2561859568 wait complete. 2024-03-29T04:10:34.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.445+0000 7f65f52d3640 1 Processor -- start 2024-03-29T04:10:34.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.445+0000 7f65f52d3640 1 -- start start 2024-03-29T04:10:34.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.446+0000 7f65f52d3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65d8004340 0x7f65f015e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:34.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.446+0000 7f65f52d3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65f00cbfc0 0x7f65f014dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:34.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.446+0000 7f65f52d3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f014e520 0x7f65f00ca350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:34.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.446+0000 7f65f52d3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f65f00ce540 con 0x7f65f00cbfc0 2024-03-29T04:10:34.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.446+0000 7f65f52d3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f65f00cc790 con 0x7f65d8004340 2024-03-29T04:10:34.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.446+0000 7f65f52d3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f65f00ce6c0 con 0x7f65f014e520 2024-03-29T04:10:34.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.446+0000 7f65ee7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65f00cbfc0 0x7f65f014dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:34.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.446+0000 7f65ee7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65f00cbfc0 0x7f65f014dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49150/0 (socket says 172.21.15.148:49150) 2024-03-29T04:10:34.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.446+0000 7f65ee7fc640 1 -- 172.21.15.148:0/3786951495 learned_addr learned my addr 172.21.15.148:0/3786951495 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:34.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.446+0000 7f65eeffd640 1 --2- 172.21.15.148:0/3786951495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65d8004340 0x7f65f015e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:34.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.446+0000 7f65ef7fe640 1 --2- 172.21.15.148:0/3786951495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f014e520 0x7f65f00ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:34.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.446+0000 7f65ee7fc640 1 -- 172.21.15.148:0/3786951495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f014e520 msgr2=0x7f65f00ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:34.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.446+0000 7f65ee7fc640 1 --2- 172.21.15.148:0/3786951495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f014e520 0x7f65f00ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.446+0000 7f65ee7fc640 1 -- 172.21.15.148:0/3786951495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65d8004340 msgr2=0x7f65f015e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:34.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.446+0000 7f65ee7fc640 1 --2- 172.21.15.148:0/3786951495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65d8004340 0x7f65f015e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.446+0000 7f65ee7fc640 1 -- 172.21.15.148:0/3786951495 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f00c92b0 con 0x7f65f00cbfc0 2024-03-29T04:10:34.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.446+0000 7f65eeffd640 1 --2- 172.21.15.148:0/3786951495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65d8004340 0x7f65f015e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:34.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.446+0000 7f65ee7fc640 1 --2- 172.21.15.148:0/3786951495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65f00cbfc0 0x7f65f014dfe0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f65e4000c00 tx=0x7f65e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:34.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.446+0000 7f65d77fe640 1 -- 172.21.15.148:0/3786951495 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4042020 con 0x7f65f00cbfc0 2024-03-29T04:10:34.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.446+0000 7f65d77fe640 1 -- 172.21.15.148:0/3786951495 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65e403a040 con 0x7f65f00cbfc0 2024-03-29T04:10:34.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.446+0000 7f65d77fe640 1 -- 172.21.15.148:0/3786951495 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4039070 con 0x7f65f00cbfc0 2024-03-29T04:10:34.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.446+0000 7f65f52d3640 1 -- 172.21.15.148:0/3786951495 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f00c9590 con 0x7f65f00cbfc0 2024-03-29T04:10:34.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.446+0000 7f65f52d3640 1 -- 172.21.15.148:0/3786951495 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f00c9b50 con 0x7f65f00cbfc0 2024-03-29T04:10:34.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.447+0000 7f65f52d3640 1 -- 172.21.15.148:0/3786951495 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65f00d0f40 con 0x7f65f00cbfc0 2024-03-29T04:10:34.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.448+0000 7f65d77fe640 1 -- 172.21.15.148:0/3786951495 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f65e4045020 con 0x7f65f00cbfc0 2024-03-29T04:10:34.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.448+0000 7f65d77fe640 1 --2- 172.21.15.148:0/3786951495 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f65dc06c560 0x7f65dc06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:34.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.448+0000 7f65eeffd640 1 --2- 172.21.15.148:0/3786951495 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f65dc06c560 0x7f65dc06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:34.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.448+0000 7f65d77fe640 1 -- 172.21.15.148:0/3786951495 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f65e40ad1c0 con 0x7f65f00cbfc0 2024-03-29T04:10:34.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.449+0000 7f65eeffd640 1 --2- 172.21.15.148:0/3786951495 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f65dc06c560 0x7f65dc06ea20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f65f015ba90 tx=0x7f65e0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:34.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.453+0000 7f65d77fe640 1 -- 172.21.15.148:0/3786951495 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f65f00d0f40 con 0x7f65f00cbfc0 2024-03-29T04:10:34.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.636+0000 7f65f52d3640 1 -- 172.21.15.148:0/3786951495 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f65f015bbe0 con 0x7f65f00cbfc0 2024-03-29T04:10:34.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.640+0000 7f65d77fe640 1 -- 172.21.15.148:0/3786951495 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18526 (secure 0 0 0) 0x7f65e404c030 con 0x7f65f00cbfc0 2024-03-29T04:10:34.621 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:34.621 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:05.026481+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:34.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.642+0000 7f65f52d3640 1 -- 172.21.15.148:0/3786951495 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f65dc06c560 msgr2=0x7f65dc06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:34.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.642+0000 7f65f52d3640 1 --2- 172.21.15.148:0/3786951495 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f65dc06c560 0x7f65dc06ea20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f65f015ba90 tx=0x7f65e0009040 comp rx=0 tx=0).stop 2024-03-29T04:10:34.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.642+0000 7f65f52d3640 1 -- 172.21.15.148:0/3786951495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65f00cbfc0 msgr2=0x7f65f014dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:34.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.642+0000 7f65f52d3640 1 --2- 172.21.15.148:0/3786951495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65f00cbfc0 0x7f65f014dfe0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f65e4000c00 tx=0x7f65e4004730 comp rx=0 tx=0).stop 2024-03-29T04:10:34.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.642+0000 7f65f52d3640 1 -- 172.21.15.148:0/3786951495 shutdown_connections 2024-03-29T04:10:34.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.642+0000 7f65f52d3640 1 --2- 172.21.15.148:0/3786951495 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f65dc06c560 0x7f65dc06ea20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.642+0000 7f65f52d3640 1 --2- 172.21.15.148:0/3786951495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65f014e520 0x7f65f00ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.642+0000 7f65f52d3640 1 --2- 172.21.15.148:0/3786951495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65f00cbfc0 0x7f65f014dfe0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.642+0000 7f65f52d3640 1 --2- 172.21.15.148:0/3786951495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65d8004340 0x7f65f015e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.642+0000 7f65f52d3640 1 -- 172.21.15.148:0/3786951495 >> 172.21.15.148:0/3786951495 conn(0x7f65f00b3c70 msgr2=0x7f65f0160100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:34.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.643+0000 7f65f52d3640 1 -- 172.21.15.148:0/3786951495 shutdown_connections 2024-03-29T04:10:34.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.643+0000 7f65f52d3640 1 -- 172.21.15.148:0/3786951495 wait complete. 2024-03-29T04:10:34.635 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:10:34.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.768+0000 7f6c06b9b640 1 Processor -- start 2024-03-29T04:10:34.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.768+0000 7f6c06b9b640 1 -- start start 2024-03-29T04:10:34.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.768+0000 7f6c06b9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6c0015e3e0 0x7f6c000d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:34.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.768+0000 7f6c06b9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6c000ba030 0x7f6c000d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:34.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.768+0000 7f6c06b9b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c0015e7c0 con 0x7f6c000d10d0 2024-03-29T04:10:34.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.768+0000 7f6c06b9b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f6c000d3840 con 0x7f6c0015e3e0 2024-03-29T04:10:34.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.768+0000 7f6c06b9b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f6c000d2c30 con 0x7f6c000ba030 2024-03-29T04:10:34.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.769+0000 7f6c04937640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6c0015e3e0 0x7f6c000d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:34.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.769+0000 7f6c04937640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6c0015e3e0 0x7f6c000d3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49160/0 (socket says 172.21.15.148:49160) 2024-03-29T04:10:34.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.769+0000 7f6c04937640 1 -- 172.21.15.148:0/601548929 learned_addr learned my addr 172.21.15.148:0/601548929 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:34.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.769+0000 7f6c05138640 1 --1- 172.21.15.148:0/601548929 >> v1:172.21.15.148:6789/0 conn(0x7f6c000d10d0 0x7f6c000b72d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47408/0 (socket says 172.21.15.148:47408) 2024-03-29T04:10:34.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.769+0000 7f6bfffff640 1 --2- 172.21.15.148:0/601548929 >> v2:172.21.15.157:3300/0 conn(0x7f6c000ba030 0x7f6c000d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:34.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.769+0000 7f6c04937640 1 -- 172.21.15.148:0/601548929 >> v2:172.21.15.157:3300/0 conn(0x7f6c000ba030 msgr2=0x7f6c000d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:34.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.769+0000 7f6c04937640 1 --2- 172.21.15.148:0/601548929 >> v2:172.21.15.157:3300/0 conn(0x7f6c000ba030 0x7f6c000d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.769+0000 7f6c04937640 1 -- 172.21.15.148:0/601548929 >> v1:172.21.15.148:6789/0 conn(0x7f6c000d10d0 legacy=0x7f6c000b72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:34.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.769+0000 7f6c04937640 1 -- 172.21.15.148:0/601548929 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c000d2f10 con 0x7f6c0015e3e0 2024-03-29T04:10:34.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.769+0000 7f6bff7fe640 1 -- 172.21.15.148:0/601548929 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3864915511 0 0) 0x7f6c0015e7c0 con 0x7f6c000d10d0 2024-03-29T04:10:34.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.769+0000 7f6bfffff640 1 --2- 172.21.15.148:0/601548929 >> v2:172.21.15.157:3300/0 conn(0x7f6c000ba030 0x7f6c000d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:34.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.770+0000 7f6c04937640 1 --2- 172.21.15.148:0/601548929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c0015e3e0 0x7f6c000d3cd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6c000d05d0 tx=0x7f6bf802c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ca5acf86fa2732 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:34.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.770+0000 7f6bff7fe640 1 -- 172.21.15.148:0/601548929 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf802cbb0 con 0x7f6c0015e3e0 2024-03-29T04:10:34.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.770+0000 7f6bff7fe640 1 -- 172.21.15.148:0/601548929 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bf802cd50 con 0x7f6c0015e3e0 2024-03-29T04:10:34.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.770+0000 7f6bff7fe640 1 -- 172.21.15.148:0/601548929 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8034610 con 0x7f6c0015e3e0 2024-03-29T04:10:34.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.770+0000 7f6c06b9b640 1 -- 172.21.15.148:0/601548929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c0015e3e0 msgr2=0x7f6c000d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:34.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.770+0000 7f6c06b9b640 1 --2- 172.21.15.148:0/601548929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c0015e3e0 0x7f6c000d3cd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6c000d05d0 tx=0x7f6bf802c010 comp rx=0 tx=0).stop 2024-03-29T04:10:34.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.770+0000 7f6c06b9b640 1 -- 172.21.15.148:0/601548929 shutdown_connections 2024-03-29T04:10:34.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.770+0000 7f6c06b9b640 1 --2- 172.21.15.148:0/601548929 >> v2:172.21.15.157:3300/0 conn(0x7f6c000ba030 0x7f6c000d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.770+0000 7f6c06b9b640 1 --2- 172.21.15.148:0/601548929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c0015e3e0 0x7f6c000d3cd0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.770+0000 7f6c06b9b640 1 -- 172.21.15.148:0/601548929 >> 172.21.15.148:0/601548929 conn(0x7f6c000b6210 msgr2=0x7f6c000b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:34.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.770+0000 7f6c06b9b640 1 -- 172.21.15.148:0/601548929 shutdown_connections 2024-03-29T04:10:34.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.771+0000 7f6c06b9b640 1 -- 172.21.15.148:0/601548929 wait complete. 2024-03-29T04:10:34.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.772+0000 7f6c06b9b640 1 Processor -- start 2024-03-29T04:10:34.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.772+0000 7f6c06b9b640 1 -- start start 2024-03-29T04:10:34.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.773+0000 7f6c06b9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c000ba030 0x7f6c0006bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:34.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.773+0000 7f6c06b9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c000d10d0 0x7f6c000b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:34.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.773+0000 7f6c06b9b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c0015e3e0 0x7f6c0015e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:34.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.773+0000 7f6c06b9b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6c000d2870 con 0x7f6c000ba030 2024-03-29T04:10:34.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.773+0000 7f6c06b9b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6c000d34d0 con 0x7f6c000d10d0 2024-03-29T04:10:34.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.773+0000 7f6c06b9b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6c000d29f0 con 0x7f6c0015e3e0 2024-03-29T04:10:34.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.773+0000 7f6c04937640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c000ba030 0x7f6c0006bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:34.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.773+0000 7f6c04937640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c000ba030 0x7f6c0006bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49174/0 (socket says 172.21.15.148:49174) 2024-03-29T04:10:34.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.773+0000 7f6c04937640 1 -- 172.21.15.148:0/1316059173 learned_addr learned my addr 172.21.15.148:0/1316059173 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:34.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.773+0000 7f6bfffff640 1 --2- 172.21.15.148:0/1316059173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c000d10d0 0x7f6c000b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:34.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.773+0000 7f6c05138640 1 --2- 172.21.15.148:0/1316059173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c0015e3e0 0x7f6c0015e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:34.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.773+0000 7f6c04937640 1 -- 172.21.15.148:0/1316059173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c0015e3e0 msgr2=0x7f6c0015e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:34.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.773+0000 7f6c04937640 1 --2- 172.21.15.148:0/1316059173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c0015e3e0 0x7f6c0015e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.773+0000 7f6c04937640 1 -- 172.21.15.148:0/1316059173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c000d10d0 msgr2=0x7f6c000b8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:34.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.773+0000 7f6c04937640 1 --2- 172.21.15.148:0/1316059173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c000d10d0 0x7f6c000b8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.773+0000 7f6c04937640 1 -- 172.21.15.148:0/1316059173 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c0006c4b0 con 0x7f6c000ba030 2024-03-29T04:10:34.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.773+0000 7f6c05138640 1 --2- 172.21.15.148:0/1316059173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c0015e3e0 0x7f6c0015e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:34.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.773+0000 7f6c04937640 1 --2- 172.21.15.148:0/1316059173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c000ba030 0x7f6c0006bf70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8000f80 tx=0x7f6bf8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:34.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.773+0000 7f6bfd7fa640 1 -- 172.21.15.148:0/1316059173 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8043020 con 0x7f6c000ba030 2024-03-29T04:10:34.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.773+0000 7f6bfd7fa640 1 -- 172.21.15.148:0/1316059173 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bf803e450 con 0x7f6c000ba030 2024-03-29T04:10:34.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.773+0000 7f6bfd7fa640 1 -- 172.21.15.148:0/1316059173 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8009510 con 0x7f6c000ba030 2024-03-29T04:10:34.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.773+0000 7f6c06b9b640 1 -- 172.21.15.148:0/1316059173 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c0015ba70 con 0x7f6c000ba030 2024-03-29T04:10:34.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.773+0000 7f6c06b9b640 1 -- 172.21.15.148:0/1316059173 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c0015c030 con 0x7f6c000ba030 2024-03-29T04:10:34.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.774+0000 7f6c06b9b640 1 -- 172.21.15.148:0/1316059173 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c000687e0 con 0x7f6c000ba030 2024-03-29T04:10:34.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.775+0000 7f6bfd7fa640 1 -- 172.21.15.148:0/1316059173 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6bf8046020 con 0x7f6c000ba030 2024-03-29T04:10:34.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.775+0000 7f6bfd7fa640 1 --2- 172.21.15.148:0/1316059173 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6be006c530 0x7f6be006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:34.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.775+0000 7f6bfffff640 1 --2- 172.21.15.148:0/1316059173 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6be006c530 0x7f6be006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:34.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.775+0000 7f6bfd7fa640 1 -- 172.21.15.148:0/1316059173 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6bf80ae2a0 con 0x7f6c000ba030 2024-03-29T04:10:34.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.776+0000 7f6bfffff640 1 --2- 172.21.15.148:0/1316059173 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6be006c530 0x7f6be006e9f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6c0006d2c0 tx=0x7f6be8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:34.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:34.780+0000 7f6bfd7fa640 1 -- 172.21.15.148:0/1316059173 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6bf804d030 con 0x7f6c000ba030 2024-03-29T04:10:34.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.000+0000 7f6c06b9b640 1 -- 172.21.15.148:0/1316059173 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c000ca680 con 0x7f6c000ba030 2024-03-29T04:10:34.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.000+0000 7f6bfd7fa640 1 -- 172.21.15.148:0/1316059173 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f6bf807c580 con 0x7f6c000ba030 2024-03-29T04:10:34.981 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:34.981 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/36660496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":36660496},{"type":"v1","addr":"172.21.15.157:6834","nonce":36660496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2674481055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2674481055},{"type":"v1","addr":"172.21.15.157:6835","nonce":2674481055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3356449643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3356449643},{"type":"v1","addr":"172.21.15.148:6836","nonce":3356449643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:09:59.886367+0000","modified":"2024-03-29T04:10:22.957623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/2453181817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2453181817},{"type":"v1","addr":"172.21.15.148:6837","nonce":2453181817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-03-29T04:10:34.981 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 9 2024-03-29T04:10:34.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.003+0000 7f6c06b9b640 1 -- 172.21.15.148:0/1316059173 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6be006c530 msgr2=0x7f6be006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:34.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.003+0000 7f6c06b9b640 1 --2- 172.21.15.148:0/1316059173 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6be006c530 0x7f6be006e9f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6c0006d2c0 tx=0x7f6be8008040 comp rx=0 tx=0).stop 2024-03-29T04:10:34.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.003+0000 7f6c06b9b640 1 -- 172.21.15.148:0/1316059173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c000ba030 msgr2=0x7f6c0006bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:34.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.003+0000 7f6c06b9b640 1 --2- 172.21.15.148:0/1316059173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c000ba030 0x7f6c0006bf70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8000f80 tx=0x7f6bf8004a40 comp rx=0 tx=0).stop 2024-03-29T04:10:34.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.003+0000 7f6c06b9b640 1 -- 172.21.15.148:0/1316059173 shutdown_connections 2024-03-29T04:10:34.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.003+0000 7f6c06b9b640 1 --2- 172.21.15.148:0/1316059173 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6be006c530 0x7f6be006e9f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.003+0000 7f6c06b9b640 1 --2- 172.21.15.148:0/1316059173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c0015e3e0 0x7f6c0015e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.003+0000 7f6c06b9b640 1 --2- 172.21.15.148:0/1316059173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c000d10d0 0x7f6c000b8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.003+0000 7f6c06b9b640 1 --2- 172.21.15.148:0/1316059173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c000ba030 0x7f6c0006bf70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.003+0000 7f6c06b9b640 1 -- 172.21.15.148:0/1316059173 >> 172.21.15.148:0/1316059173 conn(0x7f6c000b6210 msgr2=0x7f6c000b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:34.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.003+0000 7f6c06b9b640 1 -- 172.21.15.148:0/1316059173 shutdown_connections 2024-03-29T04:10:34.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.003+0000 7f6c06b9b640 1 -- 172.21.15.148:0/1316059173 wait complete. 2024-03-29T04:10:35.398 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T04:09:59.886367+0000', 'modified': '2024-03-29T04:10:22.957623+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4290}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4290': {'gid': 4290, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6837/2453181817', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 2453181817}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 2453181817}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4290, 'qdb_cluster': [4290]} 2024-03-29T04:10:35.398 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T04:10:35.398 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-29T04:10:35.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.524+0000 7fbf18422640 1 Processor -- start 2024-03-29T04:10:35.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.524+0000 7fbf18422640 1 -- start start 2024-03-29T04:10:35.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.525+0000 7fbf18422640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbf100ba030 0x7fbf100d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:35.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.525+0000 7fbf18422640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbf100d10d0 0x7fbf100d1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:35.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.525+0000 7fbf18422640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf1015e7c0 con 0x7fbf1015e3e0 2024-03-29T04:10:35.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.525+0000 7fbf18422640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fbf100d1a70 con 0x7fbf100ba030 2024-03-29T04:10:35.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.525+0000 7fbf18422640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fbf100d3840 con 0x7fbf100d10d0 2024-03-29T04:10:35.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.525+0000 7fbf169bf640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbf100d10d0 0x7fbf100d1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:35.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.525+0000 7fbf159bd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbf100ba030 0x7fbf100d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:35.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.526+0000 7fbf169bf640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbf100d10d0 0x7fbf100d1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:39266/0 (socket says 172.21.15.148:39266) 2024-03-29T04:10:35.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.526+0000 7fbf159bd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbf100ba030 0x7fbf100d3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46108/0 (socket says 172.21.15.148:46108) 2024-03-29T04:10:35.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.526+0000 7fbf169bf640 1 -- 172.21.15.148:0/3276518111 learned_addr learned my addr 172.21.15.148:0/3276518111 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:35.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.526+0000 7fbf159bd640 1 -- 172.21.15.148:0/3276518111 >> v2:172.21.15.157:3300/0 conn(0x7fbf100d10d0 msgr2=0x7fbf100d1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:35.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.526+0000 7fbf159bd640 1 --2- 172.21.15.148:0/3276518111 >> v2:172.21.15.157:3300/0 conn(0x7fbf100d10d0 0x7fbf100d1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.526+0000 7fbf159bd640 1 -- 172.21.15.148:0/3276518111 >> v1:172.21.15.148:6789/0 conn(0x7fbf1015e3e0 legacy=0x7fbf100d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:35.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.526+0000 7fbf151bc640 1 -- 172.21.15.148:0/3276518111 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 225594795 0 0) 0x7fbf1015e7c0 con 0x7fbf1015e3e0 2024-03-29T04:10:35.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.526+0000 7fbf159bd640 1 -- 172.21.15.148:0/3276518111 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf100d2c30 con 0x7fbf100ba030 2024-03-29T04:10:35.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.526+0000 7fbf169bf640 1 --2- 172.21.15.148:0/3276518111 >> v2:172.21.15.157:3300/0 conn(0x7fbf100d10d0 0x7fbf100d1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:35.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.527+0000 7fbf159bd640 1 --2- 172.21.15.148:0/3276518111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf100ba030 0x7fbf100d3cd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbf100c7c30 tx=0x7fbf0c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde6f788b57312f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:35.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.527+0000 7fbf151bc640 1 -- 172.21.15.148:0/3276518111 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c034070 con 0x7fbf100ba030 2024-03-29T04:10:35.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.527+0000 7fbf151bc640 1 -- 172.21.15.148:0/3276518111 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf0c030610 con 0x7fbf100ba030 2024-03-29T04:10:35.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.527+0000 7fbf151bc640 1 -- 172.21.15.148:0/3276518111 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c0309e0 con 0x7fbf100ba030 2024-03-29T04:10:35.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.527+0000 7fbf18422640 1 -- 172.21.15.148:0/3276518111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf100ba030 msgr2=0x7fbf100d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:35.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.527+0000 7fbf18422640 1 --2- 172.21.15.148:0/3276518111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf100ba030 0x7fbf100d3cd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbf100c7c30 tx=0x7fbf0c02b960 comp rx=0 tx=0).stop 2024-03-29T04:10:35.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.528+0000 7fbf18422640 1 -- 172.21.15.148:0/3276518111 shutdown_connections 2024-03-29T04:10:35.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.528+0000 7fbf18422640 1 --2- 172.21.15.148:0/3276518111 >> v2:172.21.15.157:3300/0 conn(0x7fbf100d10d0 0x7fbf100d1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.528+0000 7fbf18422640 1 --2- 172.21.15.148:0/3276518111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf100ba030 0x7fbf100d3cd0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.528+0000 7fbf18422640 1 -- 172.21.15.148:0/3276518111 >> 172.21.15.148:0/3276518111 conn(0x7fbf100b6210 msgr2=0x7fbf100b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:35.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.528+0000 7fbf18422640 1 -- 172.21.15.148:0/3276518111 shutdown_connections 2024-03-29T04:10:35.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.528+0000 7fbf18422640 1 -- 172.21.15.148:0/3276518111 wait complete. 2024-03-29T04:10:35.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.529+0000 7fbf18422640 1 Processor -- start 2024-03-29T04:10:35.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.529+0000 7fbf18422640 1 -- start start 2024-03-29T04:10:35.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.530+0000 7fbf18422640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf100ba030 0x7fbf10154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:35.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.530+0000 7fbf18422640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf100d10d0 0x7fbf10156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:35.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.530+0000 7fbf18422640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf1015e3e0 0x7fbf10157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:35.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.530+0000 7fbf18422640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbf100d34d0 con 0x7fbf100d10d0 2024-03-29T04:10:35.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.530+0000 7fbf18422640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbf100d31f0 con 0x7fbf100ba030 2024-03-29T04:10:35.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.530+0000 7fbf18422640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbf10167f50 con 0x7fbf1015e3e0 2024-03-29T04:10:35.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.530+0000 7fbf159bd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf100d10d0 0x7fbf10156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:35.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.530+0000 7fbf159bd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf100d10d0 0x7fbf10156e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46112/0 (socket says 172.21.15.148:46112) 2024-03-29T04:10:35.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.530+0000 7fbf159bd640 1 -- 172.21.15.148:0/2972038829 learned_addr learned my addr 172.21.15.148:0/2972038829 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:35.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.530+0000 7fbf161be640 1 --2- 172.21.15.148:0/2972038829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf100ba030 0x7fbf10154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:35.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.530+0000 7fbf169bf640 1 --2- 172.21.15.148:0/2972038829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf1015e3e0 0x7fbf10157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:35.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.530+0000 7fbf159bd640 1 -- 172.21.15.148:0/2972038829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf1015e3e0 msgr2=0x7fbf10157890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:35.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.530+0000 7fbf159bd640 1 --2- 172.21.15.148:0/2972038829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf1015e3e0 0x7fbf10157890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.530+0000 7fbf159bd640 1 -- 172.21.15.148:0/2972038829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf100ba030 msgr2=0x7fbf10154360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:35.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.530+0000 7fbf159bd640 1 --2- 172.21.15.148:0/2972038829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf100ba030 0x7fbf10154360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.530+0000 7fbf159bd640 1 -- 172.21.15.148:0/2972038829 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf100c93d0 con 0x7fbf100d10d0 2024-03-29T04:10:35.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.530+0000 7fbf169bf640 1 --2- 172.21.15.148:0/2972038829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf1015e3e0 0x7fbf10157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:35.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.530+0000 7fbf159bd640 1 --2- 172.21.15.148:0/2972038829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf100d10d0 0x7fbf10156e00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c000c00 tx=0x7fbf0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:35.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.530+0000 7fbf06ffd640 1 -- 172.21.15.148:0/2972038829 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c034040 con 0x7fbf100d10d0 2024-03-29T04:10:35.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.530+0000 7fbf06ffd640 1 -- 172.21.15.148:0/2972038829 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf0c030410 con 0x7fbf100d10d0 2024-03-29T04:10:35.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.531+0000 7fbf18422640 1 -- 172.21.15.148:0/2972038829 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf100c96b0 con 0x7fbf100d10d0 2024-03-29T04:10:35.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.531+0000 7fbf06ffd640 1 -- 172.21.15.148:0/2972038829 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c039c50 con 0x7fbf100d10d0 2024-03-29T04:10:35.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.531+0000 7fbf18422640 1 -- 172.21.15.148:0/2972038829 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf100c9c70 con 0x7fbf100d10d0 2024-03-29T04:10:35.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.532+0000 7fbf06ffd640 1 -- 172.21.15.148:0/2972038829 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbf0c037070 con 0x7fbf100d10d0 2024-03-29T04:10:35.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.532+0000 7fbf06ffd640 1 --2- 172.21.15.148:0/2972038829 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbef406c530 0x7fbef406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:35.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.532+0000 7fbf06ffd640 1 -- 172.21.15.148:0/2972038829 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fbf0c0ad9e0 con 0x7fbf100d10d0 2024-03-29T04:10:35.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.533+0000 7fbf161be640 1 --2- 172.21.15.148:0/2972038829 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbef406c530 0x7fbef406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:35.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.533+0000 7fbf161be640 1 --2- 172.21.15.148:0/2972038829 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbef406c530 0x7fbef406e9f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbf00007fd0 tx=0x7fbf00009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:35.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.535+0000 7fbf1b010740 1 Processor -- start 2024-03-29T04:10:35.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.535+0000 7fbf1b010740 1 -- start start 2024-03-29T04:10:35.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.536+0000 7fbf1b010740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559360c76860 0x559360c49800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:35.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.536+0000 7fbf1b010740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559360c76c40 0x559360c795f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:35.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.536+0000 7fbf1b010740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559360c49d40 0x559360c79b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:35.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.536+0000 7fbf1b010740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559360bb3020 con 0x559360c76c40 2024-03-29T04:10:35.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.536+0000 7fbf1b010740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559360b5b5f0 con 0x559360c49d40 2024-03-29T04:10:35.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.536+0000 7fbf1b010740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x559360baefd0 con 0x559360c76860 2024-03-29T04:10:35.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.536+0000 7fbf159bd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559360c76c40 0x559360c795f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:35.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.536+0000 7fbf159bd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559360c76c40 0x559360c795f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46114/0 (socket says 172.21.15.148:46114) 2024-03-29T04:10:35.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.536+0000 7fbf159bd640 1 -- 172.21.15.148:0/736207744 learned_addr learned my addr 172.21.15.148:0/736207744 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:35.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.536+0000 7fbf161be640 1 --2- 172.21.15.148:0/736207744 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559360c76860 0x559360c49800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:35.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.536+0000 7fbf169bf640 1 --2- 172.21.15.148:0/736207744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559360c49d40 0x559360c79b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:35.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.537+0000 7fbf159bd640 1 -- 172.21.15.148:0/736207744 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559360c76860 msgr2=0x559360c49800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:35.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.537+0000 7fbf159bd640 1 --2- 172.21.15.148:0/736207744 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559360c76860 0x559360c49800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.537+0000 7fbf159bd640 1 -- 172.21.15.148:0/736207744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559360c49d40 msgr2=0x559360c79b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:35.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.537+0000 7fbf159bd640 1 --2- 172.21.15.148:0/736207744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559360c49d40 0x559360c79b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.537+0000 7fbf159bd640 1 -- 172.21.15.148:0/736207744 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593608c72d0 con 0x559360c76c40 2024-03-29T04:10:35.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.537+0000 7fbf161be640 1 --2- 172.21.15.148:0/736207744 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559360c76860 0x559360c49800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:35.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.537+0000 7fbf169bf640 1 --2- 172.21.15.148:0/736207744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559360c49d40 0x559360c79b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:35.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.537+0000 7fbf159bd640 1 --2- 172.21.15.148:0/736207744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559360c76c40 0x559360c795f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c0abb30 tx=0x7fbf0c0094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789774135c2f4c14 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:35.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.537+0000 7fbef3fff640 1 -- 172.21.15.148:0/736207744 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c07b5b0 con 0x559360c76c40 2024-03-29T04:10:35.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.537+0000 7fbef3fff640 1 -- 172.21.15.148:0/736207744 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf0c07f960 con 0x559360c76c40 2024-03-29T04:10:35.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.537+0000 7fbef3fff640 1 -- 172.21.15.148:0/736207744 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c084190 con 0x559360c76c40 2024-03-29T04:10:35.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.538+0000 7fbf1b010740 1 -- 172.21.15.148:0/736207744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559360c76c40 msgr2=0x559360c795f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:35.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.538+0000 7fbf1b010740 1 --2- 172.21.15.148:0/736207744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559360c76c40 0x559360c795f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c0abb30 tx=0x7fbf0c0094b0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.538+0000 7fbf1b010740 1 -- 172.21.15.148:0/736207744 shutdown_connections 2024-03-29T04:10:35.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.538+0000 7fbf1b010740 1 --2- 172.21.15.148:0/736207744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559360c49d40 0x559360c79b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.538+0000 7fbf1b010740 1 --2- 172.21.15.148:0/736207744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559360c76c40 0x559360c795f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.538+0000 7fbf1b010740 1 --2- 172.21.15.148:0/736207744 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559360c76860 0x559360c49800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.538+0000 7fbf1b010740 1 -- 172.21.15.148:0/736207744 >> 172.21.15.148:0/736207744 conn(0x559360c4ba70 msgr2=0x559360c945f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:35.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.538+0000 7fbf1b010740 1 -- 172.21.15.148:0/736207744 shutdown_connections 2024-03-29T04:10:35.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.538+0000 7fbf1b010740 1 -- 172.21.15.148:0/736207744 wait complete. 2024-03-29T04:10:35.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.539+0000 7fbf1b010740 1 -- start start 2024-03-29T04:10:35.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.539+0000 7fbf1b010740 1 Processor -- start 2024-03-29T04:10:35.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.541+0000 7fbf1b010740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559360c76860 0x559360c6c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:35.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.541+0000 7fbf1b010740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559360c76c40 0x559360c6e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:35.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.541+0000 7fbf1b010740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559360c60bc0 0x559360c7ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:35.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.541+0000 7fbf1b010740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559360c4e110 con 0x559360c76860 2024-03-29T04:10:35.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.541+0000 7fbf1b010740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559360b6e9c0 con 0x559360c60bc0 2024-03-29T04:10:35.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.541+0000 7fbf1b010740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x559360c50950 con 0x559360c76c40 2024-03-29T04:10:35.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.541+0000 7fbf161be640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559360c76860 0x559360c6c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:35.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.541+0000 7fbf161be640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559360c76860 0x559360c6c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46122/0 (socket says 172.21.15.148:46122) 2024-03-29T04:10:35.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.541+0000 7fbf161be640 1 -- 172.21.15.148:0/154373783 learned_addr learned my addr 172.21.15.148:0/154373783 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:35.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.541+0000 7fbf1b010740 10 client.0 _ll_register_callbacks cb 0x559360b52fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:10:35.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.541+0000 7fbf159bd640 1 --2- 172.21.15.148:0/154373783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559360c76c40 0x559360c6e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:35.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.541+0000 7fbf169bf640 1 --2- 172.21.15.148:0/154373783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559360c60bc0 0x559360c7ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:35.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.541+0000 7fbf161be640 1 -- 172.21.15.148:0/154373783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559360c76c40 msgr2=0x559360c6e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:35.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.541+0000 7fbf161be640 1 --2- 172.21.15.148:0/154373783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559360c76c40 0x559360c6e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.541+0000 7fbf161be640 1 -- 172.21.15.148:0/154373783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559360c60bc0 msgr2=0x559360c7ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:35.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.541+0000 7fbf161be640 1 --2- 172.21.15.148:0/154373783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559360c60bc0 0x559360c7ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.541+0000 7fbf161be640 1 -- 172.21.15.148:0/154373783 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593609916b0 con 0x559360c76860 2024-03-29T04:10:35.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.541+0000 7fbf169bf640 1 --2- 172.21.15.148:0/154373783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559360c60bc0 0x559360c7ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:35.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.542+0000 7fbf161be640 1 --2- 172.21.15.148:0/154373783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559360c76860 0x559360c6c890 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbf0000e890 tx=0x7fbf00000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa513fe36f31b529 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:35.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.542+0000 7fbef17fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-03-29T04:10:35.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.542+0000 7fbef17fa640 1 -- 172.21.15.148:0/154373783 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf00027060 con 0x559360c76860 2024-03-29T04:10:35.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.542+0000 7fbef17fa640 1 -- 172.21.15.148:0/154373783 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fbf00010040 con 0x559360c76860 2024-03-29T04:10:35.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.542+0000 7fbf1b010740 1 -- 172.21.15.148:0/154373783 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf0c07f960 con 0x559360c76860 2024-03-29T04:10:35.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.542+0000 7fbef17fa640 1 -- 172.21.15.148:0/154373783 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7fbf00003b70 con 0x559360c76860 2024-03-29T04:10:35.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.542+0000 7fbf1b010740 10 client.4890 fetch_fsmap learned FSMap version 9 2024-03-29T04:10:35.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.542+0000 7fbf1b010740 1 -- 172.21.15.148:0/154373783 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x559360c7d210 con 0x559360c76860 2024-03-29T04:10:35.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.542+0000 7fbef17fa640 1 -- 172.21.15.148:0/154373783 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf0c07f960 con 0x559360c76860 2024-03-29T04:10:35.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.542+0000 7fbef17fa640 1 -- 172.21.15.148:0/154373783 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7fbf00022430 con 0x559360c76860 2024-03-29T04:10:35.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.542+0000 7fbf1b010740 10 client.4890 fetch_fsmap finished waiting for FSMap version 9 2024-03-29T04:10:35.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.542+0000 7fbf1b010740 10 client.4890 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-29T04:10:35.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.542+0000 7fbf1b010740 20 client.4890 populate_metadata read hostname 'smithi148' 2024-03-29T04:10:35.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.542+0000 7fbf1b010740 1 --2- 172.21.15.148:0/154373783 >> [v2:172.21.15.148:6835/2453181817,v1:172.21.15.148:6837/2453181817] conn(0x559360c945d0 0x559360ca89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:35.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.542+0000 7fbf1b010740 4 client.4890 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T04:10:35.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.542+0000 7fbf1b010740 1 -- 172.21.15.148:0/154373783 --> [v2:172.21.15.148:6835/2453181817,v1:172.21.15.148:6837/2453181817] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559360b874e0 con 0x559360c945d0 2024-03-29T04:10:35.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.543+0000 7fbf159bd640 1 --2- 172.21.15.148:0/154373783 >> [v2:172.21.15.148:6835/2453181817,v1:172.21.15.148:6837/2453181817] conn(0x559360c945d0 0x559360ca89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:35.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.543+0000 7fbef17fa640 10 client.4890 ms_handle_connect on v2:172.21.15.148:6835/2453181817 2024-03-29T04:10:35.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.543+0000 7fbf159bd640 1 --2- 172.21.15.148:0/154373783 >> [v2:172.21.15.148:6835/2453181817,v1:172.21.15.148:6837/2453181817] conn(0x559360c945d0 0x559360ca89f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5138d1148d6b4756 server_cookie=5262b359524fff7a in_seq=0 out_seq=0 2024-03-29T04:10:35.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.544+0000 7fbef17fa640 1 -- 172.21.15.148:0/154373783 <== mds.0 v2:172.21.15.148:6835/2453181817 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x559360b874e0 con 0x559360c945d0 2024-03-29T04:10:35.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.544+0000 7fbef17fa640 10 client.4890 handle_command_reply: tid=0 2024-03-29T04:10:35.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.544+0000 7fbf1b010740 1 client.4890 shutdown 2024-03-29T04:10:35.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.544+0000 7fbf1b010740 1 -- 172.21.15.148:0/154373783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559360c76860 msgr2=0x559360c6c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:35.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.544+0000 7fbf1b010740 1 --2- 172.21.15.148:0/154373783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559360c76860 0x559360c6c890 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbf0000e890 tx=0x7fbf00000ed0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.544+0000 7fbf1b010740 1 -- 172.21.15.148:0/154373783 shutdown_connections 2024-03-29T04:10:35.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.544+0000 7fbf1b010740 1 --2- 172.21.15.148:0/154373783 >> [v2:172.21.15.148:6835/2453181817,v1:172.21.15.148:6837/2453181817] conn(0x559360c945d0 0x559360ca89f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.544+0000 7fbf1b010740 1 --2- 172.21.15.148:0/154373783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559360c60bc0 0x559360c7ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.544+0000 7fbf1b010740 1 --2- 172.21.15.148:0/154373783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559360c76c40 0x559360c6e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.544+0000 7fbf1b010740 1 --2- 172.21.15.148:0/154373783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559360c76860 0x559360c6c890 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.544+0000 7fbef17fa640 0 client.4890 ms_handle_reset on v2:172.21.15.148:6835/2453181817 2024-03-29T04:10:35.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.544+0000 7fbf1b010740 1 -- 172.21.15.148:0/154373783 >> 172.21.15.148:0/154373783 conn(0x559360c49d40 msgr2=0x559360c4d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:35.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.544+0000 7fbf1b010740 1 -- 172.21.15.148:0/154373783 shutdown_connections 2024-03-29T04:10:35.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.544+0000 7fbf1b010740 1 -- 172.21.15.148:0/154373783 wait complete. 2024-03-29T04:10:35.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.544+0000 7fbf1b010740 20 client.4890 trim_cache size 0 max 16384 2024-03-29T04:10:35.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.563+0000 7fbf1b010740 1 Processor -- start 2024-03-29T04:10:35.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.563+0000 7fbf1b010740 1 -- start start 2024-03-29T04:10:35.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.563+0000 7fbf1b010740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559360c60bc0 0x559360c75da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:35.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.563+0000 7fbf1b010740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559360c76860 0x559360c50530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:35.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.563+0000 7fbf1b010740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559360c76c40 0x559360c6b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:35.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.563+0000 7fbf1b010740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbf10200e00 con 0x559360c76860 2024-03-29T04:10:35.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.563+0000 7fbf1b010740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1015ff70 con 0x559360c60bc0 2024-03-29T04:10:35.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.563+0000 7fbf1b010740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbf102007b0 con 0x559360c76c40 2024-03-29T04:10:35.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.564+0000 7fbf159bd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559360c76860 0x559360c50530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:35.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.564+0000 7fbf159bd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559360c76860 0x559360c50530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46134/0 (socket says 172.21.15.148:46134) 2024-03-29T04:10:35.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.564+0000 7fbf159bd640 1 -- 172.21.15.148:0/2618452498 learned_addr learned my addr 172.21.15.148:0/2618452498 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:35.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.564+0000 7fbf161be640 1 --2- 172.21.15.148:0/2618452498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559360c60bc0 0x559360c75da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:35.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.564+0000 7fbf169bf640 1 --2- 172.21.15.148:0/2618452498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559360c76c40 0x559360c6b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:35.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.564+0000 7fbf159bd640 1 -- 172.21.15.148:0/2618452498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559360c76c40 msgr2=0x559360c6b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:35.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.564+0000 7fbf159bd640 1 --2- 172.21.15.148:0/2618452498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559360c76c40 0x559360c6b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.564+0000 7fbf159bd640 1 -- 172.21.15.148:0/2618452498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559360c60bc0 msgr2=0x559360c75da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:35.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.564+0000 7fbf159bd640 1 --2- 172.21.15.148:0/2618452498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559360c60bc0 0x559360c75da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.564+0000 7fbf159bd640 1 -- 172.21.15.148:0/2618452498 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559360baf150 con 0x559360c76860 2024-03-29T04:10:35.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.564+0000 7fbf159bd640 1 --2- 172.21.15.148:0/2618452498 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559360c76860 0x559360c50530 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c02bf70 tx=0x7fbf0c03b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c049ce455a0612 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:35.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.564+0000 7fbef37fe640 1 -- 172.21.15.148:0/2618452498 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c084190 con 0x559360c76860 2024-03-29T04:10:35.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.564+0000 7fbef37fe640 1 -- 172.21.15.148:0/2618452498 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf0c07b5b0 con 0x559360c76860 2024-03-29T04:10:35.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.564+0000 7fbef37fe640 1 -- 172.21.15.148:0/2618452498 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c07f960 con 0x559360c76860 2024-03-29T04:10:35.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.565+0000 7fbf1b010740 1 -- 172.21.15.148:0/2618452498 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559360c76860 msgr2=0x559360c50530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:35.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.565+0000 7fbf1b010740 1 --2- 172.21.15.148:0/2618452498 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559360c76860 0x559360c50530 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c02bf70 tx=0x7fbf0c03b000 comp rx=0 tx=0).stop 2024-03-29T04:10:35.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.565+0000 7fbf1b010740 1 -- 172.21.15.148:0/2618452498 shutdown_connections 2024-03-29T04:10:35.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.565+0000 7fbf1b010740 1 --2- 172.21.15.148:0/2618452498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559360c76c40 0x559360c6b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.565+0000 7fbf1b010740 1 --2- 172.21.15.148:0/2618452498 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559360c76860 0x559360c50530 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.565+0000 7fbf1b010740 1 --2- 172.21.15.148:0/2618452498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559360c60bc0 0x559360c75da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.565+0000 7fbf1b010740 1 -- 172.21.15.148:0/2618452498 >> 172.21.15.148:0/2618452498 conn(0x559360c49d40 msgr2=0x559360c93fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:35.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.565+0000 7fbf1b010740 1 -- 172.21.15.148:0/2618452498 shutdown_connections 2024-03-29T04:10:35.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.565+0000 7fbf1b010740 1 -- 172.21.15.148:0/2618452498 wait complete. 2024-03-29T04:10:35.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.566+0000 7fbf1b010740 1 -- start start 2024-03-29T04:10:35.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.566+0000 7fbf1b010740 1 Processor -- start 2024-03-29T04:10:35.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.567+0000 7fbf1b010740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559360c60bc0 0x559360d12f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:35.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.567+0000 7fbf1b010740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559360c76860 0x559360cae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:35.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.567+0000 7fbf1b010740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559360c76c40 0x559360c4df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:35.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.567+0000 7fbf1b010740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559360c785e0 con 0x559360c60bc0 2024-03-29T04:10:35.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.567+0000 7fbf1b010740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x559360c7c870 con 0x559360c76860 2024-03-29T04:10:35.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.567+0000 7fbf1b010740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x559360c949d0 con 0x559360c76c40 2024-03-29T04:10:35.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.567+0000 7fbf1b010740 10 client.0 _ll_register_callbacks cb 0x7fbf00005300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:10:35.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.567+0000 7fbf161be640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559360c60bc0 0x559360d12f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:35.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.567+0000 7fbf161be640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559360c60bc0 0x559360d12f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46148/0 (socket says 172.21.15.148:46148) 2024-03-29T04:10:35.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.567+0000 7fbf161be640 1 -- 172.21.15.148:0/2296948016 learned_addr learned my addr 172.21.15.148:0/2296948016 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:35.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.567+0000 7fbf159bd640 1 --2- 172.21.15.148:0/2296948016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559360c76860 0x559360cae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:35.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.567+0000 7fbf169bf640 1 --2- 172.21.15.148:0/2296948016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559360c76c40 0x559360c4df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:35.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.567+0000 7fbf161be640 1 -- 172.21.15.148:0/2296948016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559360c76c40 msgr2=0x559360c4df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:35.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.567+0000 7fbf161be640 1 --2- 172.21.15.148:0/2296948016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559360c76c40 0x559360c4df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.567+0000 7fbf161be640 1 -- 172.21.15.148:0/2296948016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559360c76860 msgr2=0x559360cae9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:35.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.567+0000 7fbf161be640 1 --2- 172.21.15.148:0/2296948016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559360c76860 0x559360cae9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.567+0000 7fbf161be640 1 -- 172.21.15.148:0/2296948016 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559360c4ed10 con 0x559360c60bc0 2024-03-29T04:10:35.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.567+0000 7fbf161be640 1 --2- 172.21.15.148:0/2296948016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559360c60bc0 0x559360d12f60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbf00001550 tx=0x7fbf00013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4834aa96d285a3aa server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:35.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.567+0000 7fbef17fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-03-29T04:10:35.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.567+0000 7fbef17fa640 1 -- 172.21.15.148:0/2296948016 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0000e580 con 0x559360c60bc0 2024-03-29T04:10:35.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.567+0000 7fbef17fa640 1 -- 172.21.15.148:0/2296948016 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fbf00010040 con 0x559360c60bc0 2024-03-29T04:10:35.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.567+0000 7fbf1b010740 1 -- 172.21.15.148:0/2296948016 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf0c07b5b0 con 0x559360c60bc0 2024-03-29T04:10:35.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.568+0000 7fbef17fa640 1 -- 172.21.15.148:0/2296948016 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7fbf00027030 con 0x559360c60bc0 2024-03-29T04:10:35.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.568+0000 7fbf1b010740 10 client.4896 fetch_fsmap learned FSMap version 9 2024-03-29T04:10:35.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.568+0000 7fbf1b010740 1 -- 172.21.15.148:0/2296948016 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x559360c6cb10 con 0x559360c60bc0 2024-03-29T04:10:35.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.568+0000 7fbef17fa640 1 -- 172.21.15.148:0/2296948016 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf0c07b5b0 con 0x559360c60bc0 2024-03-29T04:10:35.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.568+0000 7fbef17fa640 1 -- 172.21.15.148:0/2296948016 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7fbf000229d0 con 0x559360c60bc0 2024-03-29T04:10:35.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.568+0000 7fbf1b010740 10 client.4896 fetch_fsmap finished waiting for FSMap version 9 2024-03-29T04:10:35.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.568+0000 7fbf1b010740 10 client.4896 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-29T04:10:35.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.568+0000 7fbf1b010740 20 client.4896 populate_metadata read hostname 'smithi148' 2024-03-29T04:10:35.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.568+0000 7fbf1b010740 1 --2- 172.21.15.148:0/2296948016 >> [v2:172.21.15.148:6835/2453181817,v1:172.21.15.148:6837/2453181817] conn(0x559360c4e7e0 0x559360c5f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:35.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.568+0000 7fbf1b010740 4 client.4896 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T04:10:35.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.568+0000 7fbf1b010740 1 -- 172.21.15.148:0/2296948016 --> [v2:172.21.15.148:6835/2453181817,v1:172.21.15.148:6837/2453181817] -- command(tid 0: {"prefix": "status"}) -- 0x559360bc0560 con 0x559360c4e7e0 2024-03-29T04:10:35.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.568+0000 7fbf159bd640 1 --2- 172.21.15.148:0/2296948016 >> [v2:172.21.15.148:6835/2453181817,v1:172.21.15.148:6837/2453181817] conn(0x559360c4e7e0 0x559360c5f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:35.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.568+0000 7fbef17fa640 10 client.4896 ms_handle_connect on v2:172.21.15.148:6835/2453181817 2024-03-29T04:10:35.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.568+0000 7fbf159bd640 1 --2- 172.21.15.148:0/2296948016 >> [v2:172.21.15.148:6835/2453181817,v1:172.21.15.148:6837/2453181817] conn(0x559360c4e7e0 0x559360c5f0b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aec9d6b3cf38ec83 server_cookie=aad4416bfed7e9a3 in_seq=0 out_seq=0 2024-03-29T04:10:35.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.569+0000 7fbef17fa640 1 -- 172.21.15.148:0/2296948016 <== mds.0 v2:172.21.15.148:6835/2453181817 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x559360bc0560 con 0x559360c4e7e0 2024-03-29T04:10:35.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.569+0000 7fbef17fa640 10 client.4896 handle_command_reply: tid=0 2024-03-29T04:10:35.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.569+0000 7fbf1b010740 1 client.4896 shutdown 2024-03-29T04:10:35.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.569+0000 7fbf1b010740 1 -- 172.21.15.148:0/2296948016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559360c60bc0 msgr2=0x559360d12f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:35.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.569+0000 7fbf1b010740 1 --2- 172.21.15.148:0/2296948016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559360c60bc0 0x559360d12f60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbf00001550 tx=0x7fbf00013000 comp rx=0 tx=0).stop 2024-03-29T04:10:35.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.569+0000 7fbf1b010740 1 -- 172.21.15.148:0/2296948016 shutdown_connections 2024-03-29T04:10:35.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.569+0000 7fbf1b010740 1 --2- 172.21.15.148:0/2296948016 >> [v2:172.21.15.148:6835/2453181817,v1:172.21.15.148:6837/2453181817] conn(0x559360c4e7e0 0x559360c5f0b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.569+0000 7fbf1b010740 1 --2- 172.21.15.148:0/2296948016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x559360c76c40 0x559360c4df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.569+0000 7fbf1b010740 1 --2- 172.21.15.148:0/2296948016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559360c76860 0x559360cae9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.569+0000 7fbf1b010740 1 --2- 172.21.15.148:0/2296948016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559360c60bc0 0x559360d12f60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.569+0000 7fbef17fa640 0 client.4896 ms_handle_reset on v2:172.21.15.148:6835/2453181817 2024-03-29T04:10:35.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.569+0000 7fbf1b010740 1 -- 172.21.15.148:0/2296948016 >> 172.21.15.148:0/2296948016 conn(0x559360c49d40 msgr2=0x559360c77230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:35.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.569+0000 7fbf1b010740 1 -- 172.21.15.148:0/2296948016 shutdown_connections 2024-03-29T04:10:35.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.569+0000 7fbf1b010740 1 -- 172.21.15.148:0/2296948016 wait complete. 2024-03-29T04:10:35.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.569+0000 7fbf1b010740 20 client.4896 trim_cache size 0 max 16384 2024-03-29T04:10:35.556 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-03-29T04:10:35.556 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "56b659bb-6db8-45e8-b612-135349c3fd6b", 2024-03-29T04:10:35.556 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2024-03-29T04:10:35.556 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 4290, 2024-03-29T04:10:35.556 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2024-03-29T04:10:35.556 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2024-03-29T04:10:35.556 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2024-03-29T04:10:35.556 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 35.033310780999997, 2024-03-29T04:10:35.556 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 9, 2024-03-29T04:10:35.556 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 24, 2024-03-29T04:10:35.556 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 24, 2024-03-29T04:10:35.556 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 42.084809794999998 2024-03-29T04:10:35.556 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-03-29T04:10:35.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.570+0000 7fbf18422640 1 -- 172.21.15.148:0/2972038829 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbef406c530 msgr2=0x7fbef406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:35.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.570+0000 7fbf18422640 1 --2- 172.21.15.148:0/2972038829 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbef406c530 0x7fbef406e9f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbf00007fd0 tx=0x7fbf00009040 comp rx=0 tx=0).stop 2024-03-29T04:10:35.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.570+0000 7fbf18422640 1 -- 172.21.15.148:0/2972038829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf100d10d0 msgr2=0x7fbf10156e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:35.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.570+0000 7fbf18422640 1 --2- 172.21.15.148:0/2972038829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf100d10d0 0x7fbf10156e00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c000c00 tx=0x7fbf0c0046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.570+0000 7fbf18422640 1 -- 172.21.15.148:0/2972038829 shutdown_connections 2024-03-29T04:10:35.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.570+0000 7fbf18422640 1 --2- 172.21.15.148:0/2972038829 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbef406c530 0x7fbef406e9f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.570+0000 7fbf18422640 1 --2- 172.21.15.148:0/2972038829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf1015e3e0 0x7fbf10157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.570+0000 7fbf18422640 1 --2- 172.21.15.148:0/2972038829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf100d10d0 0x7fbf10156e00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.570+0000 7fbf18422640 1 --2- 172.21.15.148:0/2972038829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf100ba030 0x7fbf10154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.570+0000 7fbf18422640 1 -- 172.21.15.148:0/2972038829 >> 172.21.15.148:0/2972038829 conn(0x7fbf100b6210 msgr2=0x7fbf100b7080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:35.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.570+0000 7fbf18422640 1 -- 172.21.15.148:0/2972038829 shutdown_connections 2024-03-29T04:10:35.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:35.570+0000 7fbf18422640 1 -- 172.21.15.148:0/2972038829 wait complete. 2024-03-29T04:10:35.561 INFO:teuthology.run_tasks:Running task sequential... 2024-03-29T04:10:35.570 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-03-29T04:10:35.570 INFO:teuthology.task.sequential:In sequential, running task print... 2024-03-29T04:10:35.571 INFO:teuthology.task.print:Enabling mgr modules 2024-03-29T04:10:35.571 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-29T04:10:35.611 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-29T04:10:35.611 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-03-29T04:10:35.612 INFO:tasks.ceph_fuse:client.0 config is {} 2024-03-29T04:10:35.612 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T04:10:35.612 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T04:10:35.612 INFO:tasks.ceph_fuse:client.2 config is {} 2024-03-29T04:10:35.613 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T04:10:35.613 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-29T04:10:35.613 INFO:tasks.ceph_fuse:client.1 config is {} 2024-03-29T04:10:35.613 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T04:10:35.614 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-29T04:10:35.614 INFO:tasks.ceph_fuse:client.3 config is {} 2024-03-29T04:10:35.614 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T04:10:35.614 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-29T04:10:35.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:35.614 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-03-29T04:10:35.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:35.634 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2024-03-29T04:10:35.707 INFO:teuthology.orchestra.run.smithi157.stderr:Cannot find device "ceph-brx" 2024-03-29T04:10:35.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:10:35.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:35.709 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-03-29T04:10:35.724 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-29T04:10:35.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:35.724 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-29T04:10:35.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:35.792 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link delete ceph-brx 2024-03-29T04:10:35.905 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-29T04:10:35.905 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T04:10:36.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.033+0000 7f07b6c6a640 1 Processor -- start 2024-03-29T04:10:36.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.033+0000 7f07b6c6a640 1 -- start start 2024-03-29T04:10:36.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.034+0000 7f07b6c6a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f07b015b060 0x7f07b00b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:36.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.034+0000 7f07b6c6a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07b00b3630 con 0x7f07b006c5c0 2024-03-29T04:10:36.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.034+0000 7f07b6c6a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07b00b3910 con 0x7f07b015f6d0 2024-03-29T04:10:36.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.034+0000 7f07b6c6a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f07b00b3bf0 con 0x7f07b015b060 2024-03-29T04:10:36.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.034+0000 7f07b5207640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f07b006c5c0 0x7f07b00cd530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49348/0 (socket says 172.21.15.148:49348) 2024-03-29T04:10:36.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.034+0000 7f07b5207640 1 -- 172.21.15.148:0/2787807896 learned_addr learned my addr 172.21.15.148:0/2787807896 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:36.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.034+0000 7f07b4a06640 1 --2- 172.21.15.148:0/2787807896 >> v2:172.21.15.157:3300/0 conn(0x7f07b015b060 0x7f07b00b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:36.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.034+0000 7f07af7fe640 1 -- 172.21.15.148:0/2787807896 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1509435048 0 0) 0x7f07b00b3630 con 0x7f07b006c5c0 2024-03-29T04:10:36.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.034+0000 7f07af7fe640 1 -- 172.21.15.148:0/2787807896 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07a0003520 con 0x7f07b006c5c0 2024-03-29T04:10:36.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.035+0000 7f07af7fe640 1 -- 172.21.15.148:0/2787807896 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3230716224 0 0) 0x7f07a0003520 con 0x7f07b006c5c0 2024-03-29T04:10:36.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.035+0000 7f07af7fe640 1 -- 172.21.15.148:0/2787807896 >> v2:172.21.15.157:3300/0 conn(0x7f07b015b060 msgr2=0x7f07b00b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:36.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.035+0000 7f07af7fe640 1 --2- 172.21.15.148:0/2787807896 >> v2:172.21.15.157:3300/0 conn(0x7f07b015b060 0x7f07b00b2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:36.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.035+0000 7f07af7fe640 1 -- 172.21.15.148:0/2787807896 >> v1:172.21.15.157:6789/0 conn(0x7f07b015f6d0 legacy=0x7f07b00b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:36.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.035+0000 7f07af7fe640 1 -- 172.21.15.148:0/2787807896 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b00b3ed0 con 0x7f07b006c5c0 2024-03-29T04:10:36.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.035+0000 7f07af7fe640 1 -- 172.21.15.148:0/2787807896 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f07a80019f0 con 0x7f07b006c5c0 2024-03-29T04:10:36.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.035+0000 7f07af7fe640 1 -- 172.21.15.148:0/2787807896 >> v1:172.21.15.148:6789/0 conn(0x7f07b006c5c0 legacy=0x7f07b00cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:36.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.035+0000 7f07af7fe640 1 --2- 172.21.15.148:0/2787807896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07a00039b0 0x7f07a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:36.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.035+0000 7f07af7fe640 1 --2- 172.21.15.148:0/2787807896 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07a0004800 0x7f07a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:36.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.035+0000 7f07af7fe640 1 --2- 172.21.15.148:0/2787807896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07a00052e0 0x7f07a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:36.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.035+0000 7f07af7fe640 1 -- 172.21.15.148:0/2787807896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f07b00b3bf0 con 0x7f07a00039b0 2024-03-29T04:10:36.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.035+0000 7f07af7fe640 1 -- 172.21.15.148:0/2787807896 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f07a000c1b0 con 0x7f07a00052e0 2024-03-29T04:10:36.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.035+0000 7f07af7fe640 1 -- 172.21.15.148:0/2787807896 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f07a000c3b0 con 0x7f07a0004800 2024-03-29T04:10:36.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.035+0000 7f07b4a06640 1 --2- 172.21.15.148:0/2787807896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07a00039b0 0x7f07a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:36.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.035+0000 7f07affff640 1 --2- 172.21.15.148:0/2787807896 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07a0004800 0x7f07a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:36.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.035+0000 7f07b5207640 1 --2- 172.21.15.148:0/2787807896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07a00052e0 0x7f07a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:36.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.035+0000 7f07b4a06640 1 -- 172.21.15.148:0/2787807896 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07a0004800 msgr2=0x7f07a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:36.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.035+0000 7f07b4a06640 1 --2- 172.21.15.148:0/2787807896 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07a0004800 0x7f07a0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:36.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.035+0000 7f07b4a06640 1 -- 172.21.15.148:0/2787807896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07a00052e0 msgr2=0x7f07a000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:36.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.035+0000 7f07b4a06640 1 --2- 172.21.15.148:0/2787807896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07a00052e0 0x7f07a000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:36.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.035+0000 7f07b5207640 1 -- 172.21.15.148:0/2787807896 reap_dead start 2024-03-29T04:10:36.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.035+0000 7f07b4a06640 1 -- 172.21.15.148:0/2787807896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a0003520 con 0x7f07a00039b0 2024-03-29T04:10:36.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.035+0000 7f07affff640 1 --2- 172.21.15.148:0/2787807896 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07a0004800 0x7f07a0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:36.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.036+0000 7f07b4a06640 1 --2- 172.21.15.148:0/2787807896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07a00039b0 0x7f07a0003dd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f07a4009a90 tx=0x7f07a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f033445a2acf9889 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:36.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.036+0000 7f07af7fe640 1 -- 172.21.15.148:0/2787807896 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a4039070 con 0x7f07a00039b0 2024-03-29T04:10:36.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.036+0000 7f07af7fe640 1 -- 172.21.15.148:0/2787807896 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a402c680 con 0x7f07a00039b0 2024-03-29T04:10:36.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.036+0000 7f07af7fe640 1 -- 172.21.15.148:0/2787807896 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a4009690 con 0x7f07a00039b0 2024-03-29T04:10:36.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.036+0000 7f07b6c6a640 1 -- 172.21.15.148:0/2787807896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07a00039b0 msgr2=0x7f07a0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:36.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.036+0000 7f07b6c6a640 1 --2- 172.21.15.148:0/2787807896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07a00039b0 0x7f07a0003dd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f07a4009a90 tx=0x7f07a0005830 comp rx=0 tx=0).stop 2024-03-29T04:10:36.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.037+0000 7f07b6c6a640 1 -- 172.21.15.148:0/2787807896 shutdown_connections 2024-03-29T04:10:36.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.037+0000 7f07b6c6a640 1 --2- 172.21.15.148:0/2787807896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07a00039b0 0x7f07a0003dd0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:36.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.037+0000 7f07b6c6a640 1 -- 172.21.15.148:0/2787807896 >> 172.21.15.148:0/2787807896 conn(0x7f07b00cfed0 msgr2=0x7f07b00d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:36.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.037+0000 7f07b6c6a640 1 -- 172.21.15.148:0/2787807896 shutdown_connections 2024-03-29T04:10:36.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.037+0000 7f07b6c6a640 1 -- 172.21.15.148:0/2787807896 wait complete. 2024-03-29T04:10:36.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.038+0000 7f07b6c6a640 1 Processor -- start 2024-03-29T04:10:36.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.038+0000 7f07b6c6a640 1 -- start start 2024-03-29T04:10:36.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.039+0000 7f07b6c6a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07a00039b0 0x7f07b006d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:36.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.039+0000 7f07b6c6a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07b00c6b10 0x7f07b00c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:36.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.039+0000 7f07b6c6a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07b0145730 0x7f07b0145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:36.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.039+0000 7f07b6c6a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f07b015b440 con 0x7f07b00c6b10 2024-03-29T04:10:36.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.039+0000 7f07b6c6a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f07b015fb30 con 0x7f07a00039b0 2024-03-29T04:10:36.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.039+0000 7f07b6c6a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f07b006ca50 con 0x7f07b0145730 2024-03-29T04:10:36.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.039+0000 7f07affff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07b00c6b10 0x7f07b00c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.039+0000 7f07affff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07b00c6b10 0x7f07b00c6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46172/0 (socket says 172.21.15.148:46172) 2024-03-29T04:10:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.039+0000 7f07affff640 1 -- 172.21.15.148:0/4217071713 learned_addr learned my addr 172.21.15.148:0/4217071713 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.039+0000 7f07b4a06640 1 --2- 172.21.15.148:0/4217071713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07a00039b0 0x7f07b006d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.039+0000 7f07b5207640 1 --2- 172.21.15.148:0/4217071713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07b0145730 0x7f07b0145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.039+0000 7f07affff640 1 -- 172.21.15.148:0/4217071713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07b0145730 msgr2=0x7f07b0145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.039+0000 7f07affff640 1 --2- 172.21.15.148:0/4217071713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07b0145730 0x7f07b0145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.039+0000 7f07affff640 1 -- 172.21.15.148:0/4217071713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07a00039b0 msgr2=0x7f07b006d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.039+0000 7f07affff640 1 --2- 172.21.15.148:0/4217071713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07a00039b0 0x7f07b006d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.039+0000 7f07affff640 1 -- 172.21.15.148:0/4217071713 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b01554e0 con 0x7f07b00c6b10 2024-03-29T04:10:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.039+0000 7f07b5207640 1 --2- 172.21.15.148:0/4217071713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07b0145730 0x7f07b0145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.040+0000 7f07affff640 1 --2- 172.21.15.148:0/4217071713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07b00c6b10 0x7f07b00c6f90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f079c00e480 tx=0x7f079c00e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.040+0000 7f07ad7fa640 1 -- 172.21.15.148:0/4217071713 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c0052e0 con 0x7f07b00c6b10 2024-03-29T04:10:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.040+0000 7f07ad7fa640 1 -- 172.21.15.148:0/4217071713 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f079c007020 con 0x7f07b00c6b10 2024-03-29T04:10:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.040+0000 7f07ad7fa640 1 -- 172.21.15.148:0/4217071713 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c007210 con 0x7f07b00c6b10 2024-03-29T04:10:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.040+0000 7f07b6c6a640 1 -- 172.21.15.148:0/4217071713 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b01f5b30 con 0x7f07b00c6b10 2024-03-29T04:10:36.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.040+0000 7f07b6c6a640 1 -- 172.21.15.148:0/4217071713 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b01f60f0 con 0x7f07b00c6b10 2024-03-29T04:10:36.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.041+0000 7f07b6c6a640 1 -- 172.21.15.148:0/4217071713 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07b00687e0 con 0x7f07b00c6b10 2024-03-29T04:10:36.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.041+0000 7f07ad7fa640 1 -- 172.21.15.148:0/4217071713 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f079c01a020 con 0x7f07b00c6b10 2024-03-29T04:10:36.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.042+0000 7f07ad7fa640 1 --2- 172.21.15.148:0/4217071713 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f079406c530 0x7f079406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:36.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.042+0000 7f07b4a06640 1 --2- 172.21.15.148:0/4217071713 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f079406c530 0x7f079406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:36.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.042+0000 7f07ad7fa640 1 -- 172.21.15.148:0/4217071713 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f079c08ae50 con 0x7f07b00c6b10 2024-03-29T04:10:36.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.042+0000 7f07b4a06640 1 --2- 172.21.15.148:0/4217071713 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f079406c530 0x7f079406e9f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f07a4036000 tx=0x7f07a4030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:36.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.046+0000 7f07ad7fa640 1 -- 172.21.15.148:0/4217071713 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f07b01554e0 con 0x7f07b00c6b10 2024-03-29T04:10:36.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.289+0000 7f07b6c6a640 1 -- 172.21.15.148:0/4217071713 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f07b00721e0 con 0x7f07b00c6b10 2024-03-29T04:10:36.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.290+0000 7f07ad7fa640 1 -- 172.21.15.148:0/4217071713 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f079c0591c0 con 0x7f07b00c6b10 2024-03-29T04:10:36.270 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-29T04:10:36.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.293+0000 7f07b6c6a640 1 -- 172.21.15.148:0/4217071713 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f079406c530 msgr2=0x7f079406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:36.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.293+0000 7f07b6c6a640 1 --2- 172.21.15.148:0/4217071713 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f079406c530 0x7f079406e9f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f07a4036000 tx=0x7f07a4030000 comp rx=0 tx=0).stop 2024-03-29T04:10:36.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.293+0000 7f07b6c6a640 1 -- 172.21.15.148:0/4217071713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07b00c6b10 msgr2=0x7f07b00c6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:36.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.293+0000 7f07b6c6a640 1 --2- 172.21.15.148:0/4217071713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07b00c6b10 0x7f07b00c6f90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f079c00e480 tx=0x7f079c00e940 comp rx=0 tx=0).stop 2024-03-29T04:10:36.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.293+0000 7f07b6c6a640 1 -- 172.21.15.148:0/4217071713 shutdown_connections 2024-03-29T04:10:36.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.293+0000 7f07b6c6a640 1 --2- 172.21.15.148:0/4217071713 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f079406c530 0x7f079406e9f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:36.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.293+0000 7f07b6c6a640 1 --2- 172.21.15.148:0/4217071713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07b0145730 0x7f07b0145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:36.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.293+0000 7f07b6c6a640 1 --2- 172.21.15.148:0/4217071713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07b00c6b10 0x7f07b00c6f90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:36.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.293+0000 7f07b6c6a640 1 --2- 172.21.15.148:0/4217071713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07a00039b0 0x7f07b006d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:36.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.293+0000 7f07b6c6a640 1 -- 172.21.15.148:0/4217071713 >> 172.21.15.148:0/4217071713 conn(0x7f07b00cfed0 msgr2=0x7f07b015ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:36.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.293+0000 7f07b6c6a640 1 -- 172.21.15.148:0/4217071713 shutdown_connections 2024-03-29T04:10:36.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36.293+0000 7f07b6c6a640 1 -- 172.21.15.148:0/4217071713 wait complete. 2024-03-29T04:10:36.284 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T04:10:36.284 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:10:36.284 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-03-29T04:10:36.285 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T04:10:36.285 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T04:10:36.285 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T04:10:36.285 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T04:10:36.285 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:10:36.285 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T04:10:36.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:36.285 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-03-29T04:10:36.300 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:10:36.300 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:10:36.300 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:10:36.300 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:36.300 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-03-29T04:10:36.300 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:36.300 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:10:36.300 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:36.300 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-03-29T04:10:36.300 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:10:36.300 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:36.300 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-03-29T04:10:36.301 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:10:36.301 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:36.301 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:10:36.301 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:36.301 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T04:10:36.301 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42706sec preferred_lft 42706sec 2024-03-29T04:10:36.301 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link 2024-03-29T04:10:36.301 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:36.301 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-29T04:10:36.301 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:10:36.301 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-29T04:10:36.302 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add name ceph-brx type bridge 2024-03-29T04:10:36.302 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip addr flush dev ceph-brx 2024-03-29T04:10:36.302 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set ceph-brx up 2024-03-29T04:10:36.302 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-29T04:10:36.302 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-29T04:10:36.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:10:36.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:36.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:36.452 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:10:36.477 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-03-29T04:10:36.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:36.479 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-03-29T04:10:36.535 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-03-29T04:10:36.535 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:10:36.535 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T04:10:36.535 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T04:10:36.535 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:10:36.536 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:10:36.536 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-29T04:10:36.536 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T04:10:36.536 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T04:10:36.536 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T04:10:36.536 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-29T04:10:36.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:10:36.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:36.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:36.674 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-03-29T04:10:36.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:36.689 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list-id 2024-03-29T04:10:36.744 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:10:36.744 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-29T04:10:36.744 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:10:36.744 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-29T04:10:36.745 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-29T04:10:36.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:10:36.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:36.867 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-29T04:10:36.868 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:10:36.868 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-29T04:10:36.868 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-29T04:10:36.868 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-29T04:10:36.868 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-29T04:10:36.868 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-29T04:10:36.868 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-29T04:10:36.868 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-29T04:10:36.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:10:36.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:36.996 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:10:36.996 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-29T04:10:36.996 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2024-03-29T04:10:36.996 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set dev brx.0 master ceph-brx 2024-03-29T04:10:36.996 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-29T04:10:37.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:10:37.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:37.081 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T04:10:37.081 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:10:37.081 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T04:10:37.097 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T04:10:37.097 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:10:37.097 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T04:10:37.151 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2024-03-29T04:10:37.221 DEBUG:teuthology.orchestra.run.smithi148:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:10:37.277 INFO:teuthology.orchestra.run.smithi148.stdout:/proc 2024-03-29T04:10:37.277 INFO:teuthology.orchestra.run.smithi148.stdout:/sys 2024-03-29T04:10:37.277 INFO:teuthology.orchestra.run.smithi148.stdout:/dev 2024-03-29T04:10:37.277 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/security 2024-03-29T04:10:37.278 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/shm 2024-03-29T04:10:37.278 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/pts 2024-03-29T04:10:37.278 INFO:teuthology.orchestra.run.smithi148.stdout:/run 2024-03-29T04:10:37.278 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/cgroup 2024-03-29T04:10:37.278 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/pstore 2024-03-29T04:10:37.278 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/bpf 2024-03-29T04:10:37.278 INFO:teuthology.orchestra.run.smithi148.stdout:/ 2024-03-29T04:10:37.278 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/selinux 2024-03-29T04:10:37.278 INFO:teuthology.orchestra.run.smithi148.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:10:37.278 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/hugepages 2024-03-29T04:10:37.278 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/mqueue 2024-03-29T04:10:37.278 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/debug 2024-03-29T04:10:37.278 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/tracing 2024-03-29T04:10:37.278 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-29T04:10:37.278 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/fuse/connections 2024-03-29T04:10:37.279 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-sysctl.service 2024-03-29T04:10:37.279 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/config 2024-03-29T04:10:37.279 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-29T04:10:37.279 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/nfs/rpc_pipefs 2024-03-29T04:10:37.279 INFO:teuthology.orchestra.run.smithi148.stdout:/run/user/1000 2024-03-29T04:10:37.279 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph 2024-03-29T04:10:37.279 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-0 2024-03-29T04:10:37.279 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-1 2024-03-29T04:10:37.279 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-2 2024-03-29T04:10:37.279 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-3 2024-03-29T04:10:37.279 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns 2024-03-29T04:10:37.279 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:10:37.279 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:10:37.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:37.280 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2024-03-29T04:10:37.336 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T04:10:37.336 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T04:10:37.379 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2024-03-29T04:10:37.405 DEBUG:teuthology.orchestra.run.smithi148:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:10:37.461 INFO:teuthology.orchestra.run.smithi148.stdout:/proc 2024-03-29T04:10:37.462 INFO:teuthology.orchestra.run.smithi148.stdout:/sys 2024-03-29T04:10:37.462 INFO:teuthology.orchestra.run.smithi148.stdout:/dev 2024-03-29T04:10:37.462 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/security 2024-03-29T04:10:37.462 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/shm 2024-03-29T04:10:37.462 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/pts 2024-03-29T04:10:37.462 INFO:teuthology.orchestra.run.smithi148.stdout:/run 2024-03-29T04:10:37.462 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/cgroup 2024-03-29T04:10:37.462 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/pstore 2024-03-29T04:10:37.462 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/bpf 2024-03-29T04:10:37.462 INFO:teuthology.orchestra.run.smithi148.stdout:/ 2024-03-29T04:10:37.462 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/selinux 2024-03-29T04:10:37.462 INFO:teuthology.orchestra.run.smithi148.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:10:37.462 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/hugepages 2024-03-29T04:10:37.463 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/mqueue 2024-03-29T04:10:37.463 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/debug 2024-03-29T04:10:37.463 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/tracing 2024-03-29T04:10:37.463 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-29T04:10:37.463 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/fuse/connections 2024-03-29T04:10:37.463 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-sysctl.service 2024-03-29T04:10:37.463 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/config 2024-03-29T04:10:37.463 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-29T04:10:37.463 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/nfs/rpc_pipefs 2024-03-29T04:10:37.463 INFO:teuthology.orchestra.run.smithi148.stdout:/run/user/1000 2024-03-29T04:10:37.463 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph 2024-03-29T04:10:37.463 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-0 2024-03-29T04:10:37.463 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-1 2024-03-29T04:10:37.463 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-2 2024-03-29T04:10:37.463 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-3 2024-03-29T04:10:37.464 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns 2024-03-29T04:10:37.464 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:10:37.464 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:10:37.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:37.467 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2024-03-29T04:10:37.467 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi148.stderr:2024-03-29T04:10:37.486+0000 7fd81040c480 -1 init, newargv = 0x5609d6c614e0 newargc=15 2024-03-29T04:10:37.467 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi148.stderr:ceph-fuse[48180]: starting ceph client 2024-03-29T04:10:37.470 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi148.stderr:ceph-fuse[48180]: starting fuse 2024-03-29T04:10:37.485 INFO:teuthology.orchestra.run.smithi148.stdout:42 2024-03-29T04:10:37.485 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-03-29T04:10:37.485 DEBUG:teuthology.orchestra.run.smithi148:> sudo stdin-killer -- python3 -c ' 2024-03-29T04:10:37.485 DEBUG:teuthology.orchestra.run.smithi148:> import glob 2024-03-29T04:10:37.485 DEBUG:teuthology.orchestra.run.smithi148:> import re 2024-03-29T04:10:37.485 DEBUG:teuthology.orchestra.run.smithi148:> import os 2024-03-29T04:10:37.485 DEBUG:teuthology.orchestra.run.smithi148:> import subprocess 2024-03-29T04:10:37.485 DEBUG:teuthology.orchestra.run.smithi148:> 2024-03-29T04:10:37.485 DEBUG:teuthology.orchestra.run.smithi148:> def _find_admin_socket(client_name): 2024-03-29T04:10:37.485 DEBUG:teuthology.orchestra.run.smithi148:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T04:10:37.485 DEBUG:teuthology.orchestra.run.smithi148:> files = glob.glob(asok_path) 2024-03-29T04:10:37.486 DEBUG:teuthology.orchestra.run.smithi148:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T04:10:37.486 DEBUG:teuthology.orchestra.run.smithi148:> 2024-03-29T04:10:37.486 DEBUG:teuthology.orchestra.run.smithi148:> # Given a non-glob path, it better be there 2024-03-29T04:10:37.486 DEBUG:teuthology.orchestra.run.smithi148:> if "*" not in asok_path: 2024-03-29T04:10:37.486 DEBUG:teuthology.orchestra.run.smithi148:> assert(len(files) == 1) 2024-03-29T04:10:37.486 DEBUG:teuthology.orchestra.run.smithi148:> return files[0] 2024-03-29T04:10:37.486 DEBUG:teuthology.orchestra.run.smithi148:> 2024-03-29T04:10:37.486 DEBUG:teuthology.orchestra.run.smithi148:> for f in files: 2024-03-29T04:10:37.486 DEBUG:teuthology.orchestra.run.smithi148:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T04:10:37.486 DEBUG:teuthology.orchestra.run.smithi148:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T04:10:37.486 DEBUG:teuthology.orchestra.run.smithi148:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T04:10:37.486 DEBUG:teuthology.orchestra.run.smithi148:> contents = proc_f.read() 2024-03-29T04:10:37.486 DEBUG:teuthology.orchestra.run.smithi148:> if mountpoint in contents: 2024-03-29T04:10:37.486 DEBUG:teuthology.orchestra.run.smithi148:> return f 2024-03-29T04:10:37.486 DEBUG:teuthology.orchestra.run.smithi148:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T04:10:37.487 DEBUG:teuthology.orchestra.run.smithi148:> 2024-03-29T04:10:37.487 DEBUG:teuthology.orchestra.run.smithi148:> print(_find_admin_socket("client.0")) 2024-03-29T04:10:37.487 DEBUG:teuthology.orchestra.run.smithi148:> ' 2024-03-29T04:10:37.605 INFO:teuthology.orchestra.run.smithi148.stdout:/var/run/ceph/ceph-client.0.48180.asok 2024-03-29T04:10:37.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:37.617 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48180.asok 2024-03-29T04:10:37.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:37.618 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48180.asok status 2024-03-29T04:10:37.755 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-03-29T04:10:37.755 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": { 2024-03-29T04:10:37.755 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_sha1": "69e2cd73c52284dfd5096c66b60c6e6b26a3c447", 2024-03-29T04:10:37.755 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_version": "ceph version 19.0.0-2520-g69e2cd73 (69e2cd73c52284dfd5096c66b60c6e6b26a3c447) squid (dev)", 2024-03-29T04:10:37.755 INFO:teuthology.orchestra.run.smithi148.stdout: "entity_id": "0", 2024-03-29T04:10:37.755 INFO:teuthology.orchestra.run.smithi148.stdout: "hostname": "smithi148", 2024-03-29T04:10:37.755 INFO:teuthology.orchestra.run.smithi148.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T04:10:37.755 INFO:teuthology.orchestra.run.smithi148.stdout: "pid": "48180", 2024-03-29T04:10:37.755 INFO:teuthology.orchestra.run.smithi148.stdout: "root": "/" 2024-03-29T04:10:37.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:10:37.755 INFO:teuthology.orchestra.run.smithi148.stdout: "dentry_count": 0, 2024-03-29T04:10:37.755 INFO:teuthology.orchestra.run.smithi148.stdout: "dentry_pinned_count": 0, 2024-03-29T04:10:37.756 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 4908, 2024-03-29T04:10:37.756 INFO:teuthology.orchestra.run.smithi148.stdout: "inst": { 2024-03-29T04:10:37.756 INFO:teuthology.orchestra.run.smithi148.stdout: "name": { 2024-03-29T04:10:37.756 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client", 2024-03-29T04:10:37.756 INFO:teuthology.orchestra.run.smithi148.stdout: "num": 4908 2024-03-29T04:10:37.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:10:37.756 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": { 2024-03-29T04:10:37.756 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2024-03-29T04:10:37.756 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "192.168.0.1:0", 2024-03-29T04:10:37.756 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 2599435160 2024-03-29T04:10:37.756 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:10:37.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:10:37.756 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": { 2024-03-29T04:10:37.756 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2024-03-29T04:10:37.756 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "192.168.0.1:0", 2024-03-29T04:10:37.757 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 2599435160 2024-03-29T04:10:37.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:10:37.757 INFO:teuthology.orchestra.run.smithi148.stdout: "inst_str": "client.4908 192.168.0.1:0/2599435160", 2024-03-29T04:10:37.757 INFO:teuthology.orchestra.run.smithi148.stdout: "addr_str": "192.168.0.1:0/2599435160", 2024-03-29T04:10:37.757 INFO:teuthology.orchestra.run.smithi148.stdout: "inode_count": 1, 2024-03-29T04:10:37.757 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_epoch": 9, 2024-03-29T04:10:37.757 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_epoch": 24, 2024-03-29T04:10:37.757 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_epoch_barrier": 0, 2024-03-29T04:10:37.757 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklisted": false, 2024-03-29T04:10:37.757 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs" 2024-03-29T04:10:37.757 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-03-29T04:10:37.763 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T04:10:37.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.884+0000 7f1f23160640 1 Processor -- start 2024-03-29T04:10:37.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.884+0000 7f1f23160640 1 -- start start 2024-03-29T04:10:37.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.885+0000 7f1f23160640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1f1c0b2ff0 0x7f1f1c0b6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:37.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.885+0000 7f1f23160640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f1c0b50a0 con 0x7f1f1c0b3480 2024-03-29T04:10:37.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.885+0000 7f1f23160640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f1c0b5380 con 0x7f1f1c070e20 2024-03-29T04:10:37.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.885+0000 7f1f23160640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f1f1c0b5660 con 0x7f1f1c0b2ff0 2024-03-29T04:10:37.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.885+0000 7f1f1bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1f1c0b2ff0 0x7f1f1c0b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:37.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.885+0000 7f1f216fd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f1f1c0b3480 0x7f1f1c0b6680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49352/0 (socket says 172.21.15.148:49352) 2024-03-29T04:10:37.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.885+0000 7f1f216fd640 1 -- 172.21.15.148:0/3659261657 learned_addr learned my addr 172.21.15.148:0/3659261657 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:37.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.885+0000 7f1f1bfff640 1 -- 172.21.15.148:0/3659261657 >> v1:172.21.15.157:6789/0 conn(0x7f1f1c070e20 legacy=0x7f1f1c071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:37.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.885+0000 7f1f1bfff640 1 -- 172.21.15.148:0/3659261657 >> v1:172.21.15.148:6789/0 conn(0x7f1f1c0b3480 legacy=0x7f1f1c0b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:37.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.885+0000 7f1f1b7fe640 1 -- 172.21.15.148:0/3659261657 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 948203789 0 0) 0x7f1f1c0b50a0 con 0x7f1f1c0b3480 2024-03-29T04:10:37.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.885+0000 7f1f1bfff640 1 -- 172.21.15.148:0/3659261657 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f1c0b5940 con 0x7f1f1c0b2ff0 2024-03-29T04:10:37.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.886+0000 7f1f1bfff640 1 --2- 172.21.15.148:0/3659261657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f1c0b2ff0 0x7f1f1c0b6140 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1f080097a0 tx=0x7f1f0802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aae5b094efd3ecd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:37.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.886+0000 7f1f1b7fe640 1 -- 172.21.15.148:0/3659261657 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08035020 con 0x7f1f1c0b2ff0 2024-03-29T04:10:37.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.886+0000 7f1f1b7fe640 1 -- 172.21.15.148:0/3659261657 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f08031830 con 0x7f1f1c0b2ff0 2024-03-29T04:10:37.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.886+0000 7f1f1b7fe640 1 -- 172.21.15.148:0/3659261657 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08031c00 con 0x7f1f1c0b2ff0 2024-03-29T04:10:37.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.887+0000 7f1f23160640 1 -- 172.21.15.148:0/3659261657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f1c0b2ff0 msgr2=0x7f1f1c0b6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:37.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.887+0000 7f1f23160640 1 --2- 172.21.15.148:0/3659261657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f1c0b2ff0 0x7f1f1c0b6140 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1f080097a0 tx=0x7f1f0802b8d0 comp rx=0 tx=0).stop 2024-03-29T04:10:37.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.887+0000 7f1f23160640 1 -- 172.21.15.148:0/3659261657 shutdown_connections 2024-03-29T04:10:37.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.887+0000 7f1f23160640 1 --2- 172.21.15.148:0/3659261657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f1c0b2ff0 0x7f1f1c0b6140 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:37.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.887+0000 7f1f23160640 1 -- 172.21.15.148:0/3659261657 >> 172.21.15.148:0/3659261657 conn(0x7f1f1c0d2100 msgr2=0x7f1f1c0d2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:37.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.887+0000 7f1f23160640 1 -- 172.21.15.148:0/3659261657 shutdown_connections 2024-03-29T04:10:37.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.887+0000 7f1f23160640 1 -- 172.21.15.148:0/3659261657 wait complete. 2024-03-29T04:10:37.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.888+0000 7f1f23160640 1 Processor -- start 2024-03-29T04:10:37.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.888+0000 7f1f23160640 1 -- start start 2024-03-29T04:10:37.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.889+0000 7f1f23160640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f1c070e20 0x7f1f1c158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:37.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.889+0000 7f1f23160640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f1c0b2ff0 0x7f1f1c155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:37.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.889+0000 7f1f23160640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f1c0b3480 0x7f1f1c0cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:37.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.889+0000 7f1f23160640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c0ce6e0 con 0x7f1f1c0b2ff0 2024-03-29T04:10:37.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.889+0000 7f1f23160640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c0ce560 con 0x7f1f1c070e20 2024-03-29T04:10:37.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.889+0000 7f1f23160640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1f1c15c510 con 0x7f1f1c0b3480 2024-03-29T04:10:37.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.889+0000 7f1f1bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f1c0b2ff0 0x7f1f1c155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:37.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.889+0000 7f1f1bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f1c0b2ff0 0x7f1f1c155740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46182/0 (socket says 172.21.15.148:46182) 2024-03-29T04:10:37.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.889+0000 7f1f1bfff640 1 -- 172.21.15.148:0/2194207794 learned_addr learned my addr 172.21.15.148:0/2194207794 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:37.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.889+0000 7f1f216fd640 1 --2- 172.21.15.148:0/2194207794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f1c0b3480 0x7f1f1c0cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:37.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.889+0000 7f1f20efc640 1 --2- 172.21.15.148:0/2194207794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f1c070e20 0x7f1f1c158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:37.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.889+0000 7f1f1bfff640 1 -- 172.21.15.148:0/2194207794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f1c0b3480 msgr2=0x7f1f1c0cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:37.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.889+0000 7f1f1bfff640 1 --2- 172.21.15.148:0/2194207794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f1c0b3480 0x7f1f1c0cfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:37.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.890+0000 7f1f1bfff640 1 -- 172.21.15.148:0/2194207794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f1c070e20 msgr2=0x7f1f1c158d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:37.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.890+0000 7f1f1bfff640 1 --2- 172.21.15.148:0/2194207794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f1c070e20 0x7f1f1c158d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:37.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.890+0000 7f1f1bfff640 1 -- 172.21.15.148:0/2194207794 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f1c0c8010 con 0x7f1f1c0b2ff0 2024-03-29T04:10:37.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.890+0000 7f1f1bfff640 1 --2- 172.21.15.148:0/2194207794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f1c0b2ff0 0x7f1f1c155740 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1f08000c00 tx=0x7f1f080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:37.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.890+0000 7f1f197fa640 1 -- 172.21.15.148:0/2194207794 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08035050 con 0x7f1f1c0b2ff0 2024-03-29T04:10:37.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.890+0000 7f1f197fa640 1 -- 172.21.15.148:0/2194207794 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f08004230 con 0x7f1f1c0b2ff0 2024-03-29T04:10:37.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.890+0000 7f1f23160640 1 -- 172.21.15.148:0/2194207794 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f1c0c82f0 con 0x7f1f1c0b2ff0 2024-03-29T04:10:37.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.890+0000 7f1f197fa640 1 -- 172.21.15.148:0/2194207794 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0803b740 con 0x7f1f1c0b2ff0 2024-03-29T04:10:37.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.890+0000 7f1f23160640 1 -- 172.21.15.148:0/2194207794 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f1c0c88b0 con 0x7f1f1c0b2ff0 2024-03-29T04:10:37.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.891+0000 7f1f23160640 1 -- 172.21.15.148:0/2194207794 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f1c0687e0 con 0x7f1f1c0b2ff0 2024-03-29T04:10:37.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.891+0000 7f1f197fa640 1 -- 172.21.15.148:0/2194207794 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1f08038020 con 0x7f1f1c0b2ff0 2024-03-29T04:10:37.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.892+0000 7f1f197fa640 1 --2- 172.21.15.148:0/2194207794 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1f0006c530 0x7f1f0006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:37.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.892+0000 7f1f197fa640 1 -- 172.21.15.148:0/2194207794 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1f080ad860 con 0x7f1f1c0b2ff0 2024-03-29T04:10:37.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.892+0000 7f1f20efc640 1 --2- 172.21.15.148:0/2194207794 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1f0006c530 0x7f1f0006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:37.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.892+0000 7f1f20efc640 1 --2- 172.21.15.148:0/2194207794 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1f0006c530 0x7f1f0006e9f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c0d2650 tx=0x7f1f10009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:37.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:37.896+0000 7f1f197fa640 1 -- 172.21.15.148:0/2194207794 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1f08040070 con 0x7f1f1c0b2ff0 2024-03-29T04:10:38.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:38.115+0000 7f1f23160640 1 -- 172.21.15.148:0/2194207794 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1f1c071660 con 0x7f1f1c0b2ff0 2024-03-29T04:10:38.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:38.115+0000 7f1f197fa640 1 -- 172.21.15.148:0/2194207794 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f1f0807bb50 con 0x7f1f1c0b2ff0 2024-03-29T04:10:38.096 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-29T04:10:38.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:38.117+0000 7f1f23160640 1 -- 172.21.15.148:0/2194207794 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1f0006c530 msgr2=0x7f1f0006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:38.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:38.117+0000 7f1f23160640 1 --2- 172.21.15.148:0/2194207794 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1f0006c530 0x7f1f0006e9f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c0d2650 tx=0x7f1f10009040 comp rx=0 tx=0).stop 2024-03-29T04:10:38.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:38.117+0000 7f1f23160640 1 -- 172.21.15.148:0/2194207794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f1c0b2ff0 msgr2=0x7f1f1c155740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:38.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:38.117+0000 7f1f23160640 1 --2- 172.21.15.148:0/2194207794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f1c0b2ff0 0x7f1f1c155740 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1f08000c00 tx=0x7f1f080046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:38.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:38.117+0000 7f1f23160640 1 -- 172.21.15.148:0/2194207794 shutdown_connections 2024-03-29T04:10:38.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:38.117+0000 7f1f23160640 1 --2- 172.21.15.148:0/2194207794 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1f0006c530 0x7f1f0006e9f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:38.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:38.117+0000 7f1f23160640 1 --2- 172.21.15.148:0/2194207794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f1c0b3480 0x7f1f1c0cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:38.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:38.117+0000 7f1f23160640 1 --2- 172.21.15.148:0/2194207794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f1c0b2ff0 0x7f1f1c155740 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:38.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:38.117+0000 7f1f23160640 1 --2- 172.21.15.148:0/2194207794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f1c070e20 0x7f1f1c158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:38.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:38.117+0000 7f1f23160640 1 -- 172.21.15.148:0/2194207794 >> 172.21.15.148:0/2194207794 conn(0x7f1f1c0d2100 msgr2=0x7f1f1c06eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:38.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:38.118+0000 7f1f23160640 1 -- 172.21.15.148:0/2194207794 shutdown_connections 2024-03-29T04:10:38.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:38.118+0000 7f1f23160640 1 -- 172.21.15.148:0/2194207794 wait complete. 2024-03-29T04:10:38.108 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T04:10:38.108 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:10:38.108 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-03-29T04:10:38.109 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-03-29T04:10:38.109 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-29T04:10:38.109 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T04:10:38.109 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T04:10:38.109 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:10:38.109 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-03-29T04:10:38.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:38.109 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-03-29T04:10:38.124 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:10:38.125 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:10:38.125 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:10:38.125 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:38.125 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-03-29T04:10:38.125 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:38.125 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:10:38.125 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:38.125 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-03-29T04:10:38.125 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:10:38.125 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:38.125 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-03-29T04:10:38.125 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:10:38.125 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:38.125 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:10:38.125 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:38.126 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T04:10:38.126 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42704sec preferred_lft 42704sec 2024-03-29T04:10:38.126 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link 2024-03-29T04:10:38.126 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:38.126 INFO:teuthology.orchestra.run.smithi148.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:10:38.126 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 62:cb:c4:12:20:16 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:38.126 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T04:10:38.126 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:38.126 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::38f9:90ff:fe15:4429/64 scope link 2024-03-29T04:10:38.126 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:38.126 INFO:teuthology.orchestra.run.smithi148.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T04:10:38.127 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 62:cb:c4:12:20:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:10:38.127 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::60cb:c4ff:fe12:2016/64 scope link tentative 2024-03-29T04:10:38.127 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:38.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:38.127 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:10:38.190 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-03-29T04:10:38.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:38.191 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-03-29T04:10:38.247 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-03-29T04:10:38.247 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:10:38.247 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T04:10:38.247 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T04:10:38.247 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:10:38.248 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:10:38.248 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-29T04:10:38.248 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T04:10:38.248 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T04:10:38.248 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T04:10:38.248 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-29T04:10:38.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:10:38.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:38.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:38.387 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-03-29T04:10:38.402 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T04:10:38.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:38.403 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list-id 2024-03-29T04:10:38.457 INFO:teuthology.orchestra.run.smithi148.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-03-29T04:10:38.457 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:10:38.458 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-29T04:10:38.458 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:10:38.458 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-03-29T04:10:38.458 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-29T04:10:38.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:10:38.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:38.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:38.580 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-29T04:10:38.608 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:10:38.608 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:10:38.608 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:10:38.608 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:38.608 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-03-29T04:10:38.608 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:38.608 INFO:teuthology.orchestra.run.smithi148.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:10:38.609 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether d2:cc:a1:cb:0c:75 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-29T04:10:38.609 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-29T04:10:38.609 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:38.609 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::d0cc:a1ff:fecb:c75/64 scope link tentative 2024-03-29T04:10:38.609 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:38.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:38.610 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-29T04:10:38.677 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:10:38.677 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:10:38.677 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:10:38.677 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:38.677 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-03-29T04:10:38.677 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:38.677 INFO:teuthology.orchestra.run.smithi148.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:10:38.677 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether d2:cc:a1:cb:0c:75 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-29T04:10:38.677 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-29T04:10:38.677 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:38.677 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::d0cc:a1ff:fecb:c75/64 scope link 2024-03-29T04:10:38.677 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:38.679 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-03-29T04:10:38.679 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:10:38.679 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-29T04:10:38.679 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-03-29T04:10:38.679 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-29T04:10:38.679 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-03-29T04:10:38.679 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-03-29T04:10:38.679 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-03-29T04:10:38.679 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-29T04:10:38.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:10:38.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:38.846 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:10:38.846 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-29T04:10:38.847 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.1 up 2024-03-29T04:10:38.847 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set dev brx.1 master ceph-brx 2024-03-29T04:10:38.847 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-29T04:10:38.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:10:38.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:38.931 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-03-29T04:10:38.931 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:10:38.931 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-03-29T04:10:38.956 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-03-29T04:10:38.957 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:10:38.957 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-03-29T04:10:39.011 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2024-03-29T04:10:39.080 DEBUG:teuthology.orchestra.run.smithi148:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:10:39.135 INFO:teuthology.orchestra.run.smithi148.stdout:/proc 2024-03-29T04:10:39.135 INFO:teuthology.orchestra.run.smithi148.stdout:/sys 2024-03-29T04:10:39.135 INFO:teuthology.orchestra.run.smithi148.stdout:/dev 2024-03-29T04:10:39.136 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/security 2024-03-29T04:10:39.136 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/shm 2024-03-29T04:10:39.136 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/pts 2024-03-29T04:10:39.136 INFO:teuthology.orchestra.run.smithi148.stdout:/run 2024-03-29T04:10:39.136 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/cgroup 2024-03-29T04:10:39.136 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/pstore 2024-03-29T04:10:39.136 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/bpf 2024-03-29T04:10:39.136 INFO:teuthology.orchestra.run.smithi148.stdout:/ 2024-03-29T04:10:39.136 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/selinux 2024-03-29T04:10:39.136 INFO:teuthology.orchestra.run.smithi148.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:10:39.136 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/hugepages 2024-03-29T04:10:39.136 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/mqueue 2024-03-29T04:10:39.136 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/debug 2024-03-29T04:10:39.136 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/tracing 2024-03-29T04:10:39.137 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-29T04:10:39.137 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/fuse/connections 2024-03-29T04:10:39.137 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-sysctl.service 2024-03-29T04:10:39.137 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/config 2024-03-29T04:10:39.137 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-29T04:10:39.137 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/nfs/rpc_pipefs 2024-03-29T04:10:39.137 INFO:teuthology.orchestra.run.smithi148.stdout:/run/user/1000 2024-03-29T04:10:39.137 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph 2024-03-29T04:10:39.137 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-0 2024-03-29T04:10:39.137 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-1 2024-03-29T04:10:39.137 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-2 2024-03-29T04:10:39.137 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-3 2024-03-29T04:10:39.137 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns 2024-03-29T04:10:39.137 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:10:39.137 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:10:39.138 INFO:teuthology.orchestra.run.smithi148.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-29T04:10:39.138 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:10:39.138 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:10:39.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:39.139 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2024-03-29T04:10:39.194 INFO:teuthology.orchestra.run.smithi148.stdout:42 2024-03-29T04:10:39.194 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-03-29T04:10:39.195 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-03-29T04:10:39.237 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2024-03-29T04:10:39.264 DEBUG:teuthology.orchestra.run.smithi148:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:10:39.321 INFO:teuthology.orchestra.run.smithi148.stdout:/proc 2024-03-29T04:10:39.322 INFO:teuthology.orchestra.run.smithi148.stdout:/sys 2024-03-29T04:10:39.322 INFO:teuthology.orchestra.run.smithi148.stdout:/dev 2024-03-29T04:10:39.322 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/security 2024-03-29T04:10:39.322 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/shm 2024-03-29T04:10:39.322 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/pts 2024-03-29T04:10:39.322 INFO:teuthology.orchestra.run.smithi148.stdout:/run 2024-03-29T04:10:39.322 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/cgroup 2024-03-29T04:10:39.322 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/pstore 2024-03-29T04:10:39.322 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/bpf 2024-03-29T04:10:39.322 INFO:teuthology.orchestra.run.smithi148.stdout:/ 2024-03-29T04:10:39.322 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/selinux 2024-03-29T04:10:39.322 INFO:teuthology.orchestra.run.smithi148.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:10:39.322 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/hugepages 2024-03-29T04:10:39.323 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/mqueue 2024-03-29T04:10:39.323 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/debug 2024-03-29T04:10:39.323 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/tracing 2024-03-29T04:10:39.323 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-29T04:10:39.323 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/fuse/connections 2024-03-29T04:10:39.323 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-sysctl.service 2024-03-29T04:10:39.323 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/config 2024-03-29T04:10:39.323 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-29T04:10:39.323 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/nfs/rpc_pipefs 2024-03-29T04:10:39.323 INFO:teuthology.orchestra.run.smithi148.stdout:/run/user/1000 2024-03-29T04:10:39.324 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph 2024-03-29T04:10:39.324 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-0 2024-03-29T04:10:39.324 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-1 2024-03-29T04:10:39.324 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-2 2024-03-29T04:10:39.324 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-3 2024-03-29T04:10:39.324 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns 2024-03-29T04:10:39.324 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:10:39.324 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:10:39.324 INFO:teuthology.orchestra.run.smithi148.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-29T04:10:39.324 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:10:39.324 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:10:39.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:39.325 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2024-03-29T04:10:39.325 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi148.stderr:2024-03-29T04:10:39.344+0000 7f416ae3f480 -1 init, newargv = 0x564209932700 newargc=15 2024-03-29T04:10:39.325 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi148.stderr:ceph-fuse[48678]: starting ceph client 2024-03-29T04:10:39.328 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi148.stderr:ceph-fuse[48678]: starting fuse 2024-03-29T04:10:39.341 INFO:teuthology.orchestra.run.smithi148.stdout:42 2024-03-29T04:10:39.341 INFO:teuthology.orchestra.run.smithi148.stdout:44 2024-03-29T04:10:39.341 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 44] 2024-03-29T04:10:39.341 DEBUG:teuthology.orchestra.run.smithi148:> sudo stdin-killer -- python3 -c ' 2024-03-29T04:10:39.341 DEBUG:teuthology.orchestra.run.smithi148:> import glob 2024-03-29T04:10:39.341 DEBUG:teuthology.orchestra.run.smithi148:> import re 2024-03-29T04:10:39.341 DEBUG:teuthology.orchestra.run.smithi148:> import os 2024-03-29T04:10:39.341 DEBUG:teuthology.orchestra.run.smithi148:> import subprocess 2024-03-29T04:10:39.341 DEBUG:teuthology.orchestra.run.smithi148:> 2024-03-29T04:10:39.342 DEBUG:teuthology.orchestra.run.smithi148:> def _find_admin_socket(client_name): 2024-03-29T04:10:39.342 DEBUG:teuthology.orchestra.run.smithi148:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-03-29T04:10:39.342 DEBUG:teuthology.orchestra.run.smithi148:> files = glob.glob(asok_path) 2024-03-29T04:10:39.342 DEBUG:teuthology.orchestra.run.smithi148:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-03-29T04:10:39.342 DEBUG:teuthology.orchestra.run.smithi148:> 2024-03-29T04:10:39.342 DEBUG:teuthology.orchestra.run.smithi148:> # Given a non-glob path, it better be there 2024-03-29T04:10:39.342 DEBUG:teuthology.orchestra.run.smithi148:> if "*" not in asok_path: 2024-03-29T04:10:39.342 DEBUG:teuthology.orchestra.run.smithi148:> assert(len(files) == 1) 2024-03-29T04:10:39.342 DEBUG:teuthology.orchestra.run.smithi148:> return files[0] 2024-03-29T04:10:39.342 DEBUG:teuthology.orchestra.run.smithi148:> 2024-03-29T04:10:39.342 DEBUG:teuthology.orchestra.run.smithi148:> for f in files: 2024-03-29T04:10:39.342 DEBUG:teuthology.orchestra.run.smithi148:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T04:10:39.342 DEBUG:teuthology.orchestra.run.smithi148:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T04:10:39.342 DEBUG:teuthology.orchestra.run.smithi148:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T04:10:39.342 DEBUG:teuthology.orchestra.run.smithi148:> contents = proc_f.read() 2024-03-29T04:10:39.342 DEBUG:teuthology.orchestra.run.smithi148:> if mountpoint in contents: 2024-03-29T04:10:39.343 DEBUG:teuthology.orchestra.run.smithi148:> return f 2024-03-29T04:10:39.343 DEBUG:teuthology.orchestra.run.smithi148:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T04:10:39.343 DEBUG:teuthology.orchestra.run.smithi148:> 2024-03-29T04:10:39.343 DEBUG:teuthology.orchestra.run.smithi148:> print(_find_admin_socket("client.2")) 2024-03-29T04:10:39.343 DEBUG:teuthology.orchestra.run.smithi148:> ' 2024-03-29T04:10:39.462 INFO:teuthology.orchestra.run.smithi148.stdout:/var/run/ceph/ceph-client.2.48678.asok 2024-03-29T04:10:39.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:39.472 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.48678.asok 2024-03-29T04:10:39.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:39.472 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.48678.asok status 2024-03-29T04:10:39.609 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-03-29T04:10:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": { 2024-03-29T04:10:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_sha1": "69e2cd73c52284dfd5096c66b60c6e6b26a3c447", 2024-03-29T04:10:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_version": "ceph version 19.0.0-2520-g69e2cd73 (69e2cd73c52284dfd5096c66b60c6e6b26a3c447) squid (dev)", 2024-03-29T04:10:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: "entity_id": "2", 2024-03-29T04:10:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: "hostname": "smithi148", 2024-03-29T04:10:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-03-29T04:10:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: "pid": "48678", 2024-03-29T04:10:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: "root": "/" 2024-03-29T04:10:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:10:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: "dentry_count": 0, 2024-03-29T04:10:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: "dentry_pinned_count": 0, 2024-03-29T04:10:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 4923, 2024-03-29T04:10:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: "inst": { 2024-03-29T04:10:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: "name": { 2024-03-29T04:10:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client", 2024-03-29T04:10:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: "num": 4923 2024-03-29T04:10:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:10:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": { 2024-03-29T04:10:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2024-03-29T04:10:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "192.168.0.2:0", 2024-03-29T04:10:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 2153376680 2024-03-29T04:10:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:10:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:10:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": { 2024-03-29T04:10:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2024-03-29T04:10:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "192.168.0.2:0", 2024-03-29T04:10:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 2153376680 2024-03-29T04:10:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:10:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: "inst_str": "client.4923 192.168.0.2:0/2153376680", 2024-03-29T04:10:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: "addr_str": "192.168.0.2:0/2153376680", 2024-03-29T04:10:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: "inode_count": 1, 2024-03-29T04:10:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_epoch": 9, 2024-03-29T04:10:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_epoch": 24, 2024-03-29T04:10:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_epoch_barrier": 0, 2024-03-29T04:10:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklisted": false, 2024-03-29T04:10:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs" 2024-03-29T04:10:39.612 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-03-29T04:10:39.618 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T04:10:39.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.742+0000 7ff046f7c640 1 Processor -- start 2024-03-29T04:10:39.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.742+0000 7ff046f7c640 1 -- start start 2024-03-29T04:10:39.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.743+0000 7ff046f7c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff0400ba030 0x7ff0400d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:39.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.743+0000 7ff046f7c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0400d10d0 0x7ff0400d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:39.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.743+0000 7ff046f7c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff04015e7c0 con 0x7ff04015e3e0 2024-03-29T04:10:39.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.743+0000 7ff046f7c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7ff0400d3840 con 0x7ff0400ba030 2024-03-29T04:10:39.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.743+0000 7ff046f7c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff0400d2c30 con 0x7ff0400d10d0 2024-03-29T04:10:39.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.743+0000 7ff03ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff0400ba030 0x7ff0400d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:39.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.743+0000 7ff03ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff0400ba030 0x7ff0400d3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46194/0 (socket says 172.21.15.148:46194) 2024-03-29T04:10:39.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.743+0000 7ff03ffff640 1 -- 172.21.15.148:0/1345140646 learned_addr learned my addr 172.21.15.148:0/1345140646 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:39.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.743+0000 7ff045519640 1 --2- 172.21.15.148:0/1345140646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0400d10d0 0x7ff0400d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:39.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.744+0000 7ff03ffff640 1 -- 172.21.15.148:0/1345140646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0400d10d0 msgr2=0x7ff0400d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:39.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.744+0000 7ff03ffff640 1 --2- 172.21.15.148:0/1345140646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0400d10d0 0x7ff0400d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:39.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.744+0000 7ff03ffff640 1 -- 172.21.15.148:0/1345140646 >> v1:172.21.15.157:6789/0 conn(0x7ff04015e3e0 legacy=0x7ff0400d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:39.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.744+0000 7ff03ffff640 1 -- 172.21.15.148:0/1345140646 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0400d2f10 con 0x7ff0400ba030 2024-03-29T04:10:39.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.744+0000 7ff03ffff640 1 --2- 172.21.15.148:0/1345140646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0400ba030 0x7ff0400d3cd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff0400d05d0 tx=0x7ff02c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21996a8f3818043 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:39.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.744+0000 7ff03f7fe640 1 -- 172.21.15.148:0/1345140646 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c034070 con 0x7ff0400ba030 2024-03-29T04:10:39.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.744+0000 7ff03f7fe640 1 -- 172.21.15.148:0/1345140646 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff02c030610 con 0x7ff0400ba030 2024-03-29T04:10:39.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.745+0000 7ff03f7fe640 1 -- 172.21.15.148:0/1345140646 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c0309e0 con 0x7ff0400ba030 2024-03-29T04:10:39.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.745+0000 7ff046f7c640 1 -- 172.21.15.148:0/1345140646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0400ba030 msgr2=0x7ff0400d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:39.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.745+0000 7ff046f7c640 1 --2- 172.21.15.148:0/1345140646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0400ba030 0x7ff0400d3cd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff0400d05d0 tx=0x7ff02c02b960 comp rx=0 tx=0).stop 2024-03-29T04:10:39.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.745+0000 7ff046f7c640 1 -- 172.21.15.148:0/1345140646 shutdown_connections 2024-03-29T04:10:39.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.745+0000 7ff046f7c640 1 --2- 172.21.15.148:0/1345140646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0400d10d0 0x7ff0400d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:39.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.745+0000 7ff046f7c640 1 --2- 172.21.15.148:0/1345140646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0400ba030 0x7ff0400d3cd0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:39.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.745+0000 7ff046f7c640 1 -- 172.21.15.148:0/1345140646 >> 172.21.15.148:0/1345140646 conn(0x7ff0400b6210 msgr2=0x7ff0400b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:39.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.745+0000 7ff046f7c640 1 -- 172.21.15.148:0/1345140646 shutdown_connections 2024-03-29T04:10:39.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.745+0000 7ff046f7c640 1 -- 172.21.15.148:0/1345140646 wait complete. 2024-03-29T04:10:39.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.746+0000 7ff046f7c640 1 Processor -- start 2024-03-29T04:10:39.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.746+0000 7ff046f7c640 1 -- start start 2024-03-29T04:10:39.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.747+0000 7ff046f7c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0400ba030 0x7ff04015f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:39.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.747+0000 7ff046f7c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0400d10d0 0x7ff0401546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:39.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.747+0000 7ff046f7c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff04015e3e0 0x7ff040157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:39.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.747+0000 7ff046f7c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff040167f50 con 0x7ff0400d10d0 2024-03-29T04:10:39.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.747+0000 7ff046f7c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff0400d34d0 con 0x7ff0400ba030 2024-03-29T04:10:39.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.747+0000 7ff046f7c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff0401680d0 con 0x7ff04015e3e0 2024-03-29T04:10:39.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.747+0000 7ff03ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0400d10d0 0x7ff0401546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:39.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.747+0000 7ff03ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0400d10d0 0x7ff0401546e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46196/0 (socket says 172.21.15.148:46196) 2024-03-29T04:10:39.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.747+0000 7ff03ffff640 1 -- 172.21.15.148:0/1518817456 learned_addr learned my addr 172.21.15.148:0/1518817456 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:39.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.747+0000 7ff045519640 1 --2- 172.21.15.148:0/1518817456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff04015e3e0 0x7ff040157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:39.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.747+0000 7ff044d18640 1 --2- 172.21.15.148:0/1518817456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0400ba030 0x7ff04015f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:39.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.747+0000 7ff03ffff640 1 -- 172.21.15.148:0/1518817456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff04015e3e0 msgr2=0x7ff040157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:39.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.747+0000 7ff03ffff640 1 --2- 172.21.15.148:0/1518817456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff04015e3e0 0x7ff040157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:39.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.747+0000 7ff03ffff640 1 -- 172.21.15.148:0/1518817456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0400ba030 msgr2=0x7ff04015f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:39.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.747+0000 7ff03ffff640 1 --2- 172.21.15.148:0/1518817456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0400ba030 0x7ff04015f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:39.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.748+0000 7ff03ffff640 1 -- 172.21.15.148:0/1518817456 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04015fa30 con 0x7ff0400d10d0 2024-03-29T04:10:39.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.748+0000 7ff044d18640 1 --2- 172.21.15.148:0/1518817456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0400ba030 0x7ff04015f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:39.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.748+0000 7ff03ffff640 1 --2- 172.21.15.148:0/1518817456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0400d10d0 0x7ff0401546e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff02c000c00 tx=0x7ff02c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:39.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.748+0000 7ff03d7fa640 1 -- 172.21.15.148:0/1518817456 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c034040 con 0x7ff0400d10d0 2024-03-29T04:10:39.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.748+0000 7ff03d7fa640 1 -- 172.21.15.148:0/1518817456 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff02c039a90 con 0x7ff0400d10d0 2024-03-29T04:10:39.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.748+0000 7ff03d7fa640 1 -- 172.21.15.148:0/1518817456 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c039e60 con 0x7ff0400d10d0 2024-03-29T04:10:39.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.748+0000 7ff046f7c640 1 -- 172.21.15.148:0/1518817456 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff04015fd10 con 0x7ff0400d10d0 2024-03-29T04:10:39.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.748+0000 7ff046f7c640 1 -- 172.21.15.148:0/1518817456 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0401605b0 con 0x7ff0400d10d0 2024-03-29T04:10:39.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.749+0000 7ff046f7c640 1 -- 172.21.15.148:0/1518817456 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff04015fee0 con 0x7ff0400d10d0 2024-03-29T04:10:39.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.749+0000 7ff03d7fa640 1 -- 172.21.15.148:0/1518817456 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff02c04a060 con 0x7ff0400d10d0 2024-03-29T04:10:39.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.750+0000 7ff03d7fa640 1 --2- 172.21.15.148:0/1518817456 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff02406c530 0x7ff02406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:39.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.750+0000 7ff044d18640 1 --2- 172.21.15.148:0/1518817456 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff02406c530 0x7ff02406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:39.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.750+0000 7ff03d7fa640 1 -- 172.21.15.148:0/1518817456 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff02c0ad710 con 0x7ff0400d10d0 2024-03-29T04:10:39.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.750+0000 7ff044d18640 1 --2- 172.21.15.148:0/1518817456 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff02406c530 0x7ff02406e9f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff034003eb0 tx=0x7ff034009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:39.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:39.754+0000 7ff03d7fa640 1 -- 172.21.15.148:0/1518817456 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff02c03e070 con 0x7ff0400d10d0 2024-03-29T04:10:39.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:40.009+0000 7ff046f7c640 1 -- 172.21.15.148:0/1518817456 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff0400687c0 con 0x7ff0400d10d0 2024-03-29T04:10:39.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:40.009+0000 7ff03d7fa640 1 -- 172.21.15.148:0/1518817456 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7ff02c07ba80 con 0x7ff0400d10d0 2024-03-29T04:10:39.990 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-29T04:10:39.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:40.011+0000 7ff046f7c640 1 -- 172.21.15.148:0/1518817456 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff02406c530 msgr2=0x7ff02406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:39.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:40.011+0000 7ff046f7c640 1 --2- 172.21.15.148:0/1518817456 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff02406c530 0x7ff02406e9f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff034003eb0 tx=0x7ff034009040 comp rx=0 tx=0).stop 2024-03-29T04:10:39.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:40.011+0000 7ff046f7c640 1 -- 172.21.15.148:0/1518817456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0400d10d0 msgr2=0x7ff0401546e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:39.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:40.011+0000 7ff046f7c640 1 --2- 172.21.15.148:0/1518817456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0400d10d0 0x7ff0401546e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff02c000c00 tx=0x7ff02c0046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:39.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:40.012+0000 7ff046f7c640 1 -- 172.21.15.148:0/1518817456 shutdown_connections 2024-03-29T04:10:39.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:40.012+0000 7ff046f7c640 1 --2- 172.21.15.148:0/1518817456 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff02406c530 0x7ff02406e9f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:39.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:40.012+0000 7ff046f7c640 1 --2- 172.21.15.148:0/1518817456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff04015e3e0 0x7ff040157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:39.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:40.012+0000 7ff046f7c640 1 --2- 172.21.15.148:0/1518817456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0400d10d0 0x7ff0401546e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:39.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:40.012+0000 7ff046f7c640 1 --2- 172.21.15.148:0/1518817456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0400ba030 0x7ff04015f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:39.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:40.012+0000 7ff046f7c640 1 -- 172.21.15.148:0/1518817456 >> 172.21.15.148:0/1518817456 conn(0x7ff0400b6210 msgr2=0x7ff0400b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:39.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:40.012+0000 7ff046f7c640 1 -- 172.21.15.148:0/1518817456 shutdown_connections 2024-03-29T04:10:39.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:40.012+0000 7ff046f7c640 1 -- 172.21.15.148:0/1518817456 wait complete. 2024-03-29T04:10:40.003 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T04:10:40.003 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:10:40.003 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-03-29T04:10:40.003 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-29T04:10:40.003 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-29T04:10:40.003 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T04:10:40.003 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T04:10:40.003 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:10:40.003 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-29T04:10:40.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:40.004 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-03-29T04:10:40.022 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:10:40.022 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:10:40.022 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:10:40.023 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:40.023 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-03-29T04:10:40.023 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:40.023 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:10:40.023 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:40.023 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-03-29T04:10:40.023 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:10:40.023 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:40.023 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-03-29T04:10:40.023 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:10:40.023 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:40.023 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:10:40.023 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:40.023 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T04:10:40.024 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42721sec preferred_lft 42721sec 2024-03-29T04:10:40.024 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-03-29T04:10:40.024 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:40.024 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-29T04:10:40.024 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:10:40.024 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-03-29T04:10:40.024 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add name ceph-brx type bridge 2024-03-29T04:10:40.024 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip addr flush dev ceph-brx 2024-03-29T04:10:40.024 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set ceph-brx up 2024-03-29T04:10:40.024 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-29T04:10:40.024 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-03-29T04:10:40.116 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:10:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:10:40.326 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:10:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:40.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:40.333 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:10:40.361 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-03-29T04:10:40.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:40.362 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-03-29T04:10:40.434 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-03-29T04:10:40.435 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:10:40.435 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T04:10:40.435 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T04:10:40.435 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:10:40.435 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:10:40.435 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-03-29T04:10:40.435 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T04:10:40.435 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T04:10:40.435 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T04:10:40.435 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-03-29T04:10:40.523 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:10:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:10:40.755 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:10:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:40.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:40.763 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-03-29T04:10:40.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:40.778 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list-id 2024-03-29T04:10:40.833 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:10:40.833 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-03-29T04:10:40.833 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:10:40.833 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-29T04:10:40.833 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-03-29T04:10:40.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:10:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:10:40.952 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:10:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:40.960 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-29T04:10:40.961 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:10:40.961 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-03-29T04:10:40.961 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-29T04:10:40.961 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-29T04:10:40.961 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-29T04:10:40.961 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-29T04:10:40.961 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-29T04:10:40.961 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-03-29T04:10:41.010 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:10:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:10:41.098 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:10:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:41.104 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:10:41.104 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-03-29T04:10:41.104 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-03-29T04:10:41.104 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set dev brx.0 master ceph-brx 2024-03-29T04:10:41.105 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-03-29T04:10:41.153 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:10:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:10:41.181 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:10:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:41.187 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-03-29T04:10:41.187 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:10:41.187 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-29T04:10:41.202 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-29T04:10:41.203 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:10:41.203 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-29T04:10:41.257 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-03-29T04:10:41.326 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:10:41.382 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-03-29T04:10:41.382 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-03-29T04:10:41.382 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-03-29T04:10:41.382 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-03-29T04:10:41.382 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-03-29T04:10:41.382 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-03-29T04:10:41.382 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-03-29T04:10:41.383 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-03-29T04:10:41.383 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-03-29T04:10:41.383 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-03-29T04:10:41.383 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-03-29T04:10:41.383 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/selinux 2024-03-29T04:10:41.383 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:10:41.383 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-03-29T04:10:41.383 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-03-29T04:10:41.383 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-03-29T04:10:41.383 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-03-29T04:10:41.383 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-03-29T04:10:41.383 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-03-29T04:10:41.383 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysctl.service 2024-03-29T04:10:41.383 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-29T04:10:41.383 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-29T04:10:41.383 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/nfs/rpc_pipefs 2024-03-29T04:10:41.384 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-03-29T04:10:41.384 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-03-29T04:10:41.384 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-03-29T04:10:41.384 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-03-29T04:10:41.384 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-03-29T04:10:41.384 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-03-29T04:10:41.384 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-03-29T04:10:41.384 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:10:41.384 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:10:41.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:41.385 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-03-29T04:10:41.437 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T04:10:41.437 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-03-29T04:10:41.480 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-03-29T04:10:41.507 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:10:41.564 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-03-29T04:10:41.564 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-03-29T04:10:41.564 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-03-29T04:10:41.564 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-03-29T04:10:41.564 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-03-29T04:10:41.564 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-03-29T04:10:41.564 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-03-29T04:10:41.565 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-03-29T04:10:41.565 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-03-29T04:10:41.565 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-03-29T04:10:41.565 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-03-29T04:10:41.565 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/selinux 2024-03-29T04:10:41.565 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:10:41.565 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-03-29T04:10:41.565 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-03-29T04:10:41.565 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-03-29T04:10:41.565 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-03-29T04:10:41.565 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-03-29T04:10:41.565 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-03-29T04:10:41.565 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysctl.service 2024-03-29T04:10:41.566 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-29T04:10:41.566 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-29T04:10:41.566 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/nfs/rpc_pipefs 2024-03-29T04:10:41.566 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-03-29T04:10:41.566 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-03-29T04:10:41.566 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-03-29T04:10:41.566 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-03-29T04:10:41.566 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-03-29T04:10:41.566 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-03-29T04:10:41.566 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-03-29T04:10:41.566 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:10:41.566 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:10:41.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:41.567 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-03-29T04:10:41.567 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:2024-03-29T04:10:41.588+0000 7faf4dbfb480 -1 init, newargv = 0x55f91c44a450 newargc=15 2024-03-29T04:10:41.567 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[43302]: starting ceph client 2024-03-29T04:10:41.571 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[43302]: starting fuse 2024-03-29T04:10:41.582 INFO:teuthology.orchestra.run.smithi157.stdout:41 2024-03-29T04:10:41.582 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-03-29T04:10:41.582 DEBUG:teuthology.orchestra.run.smithi157:> sudo stdin-killer -- python3 -c ' 2024-03-29T04:10:41.582 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2024-03-29T04:10:41.583 DEBUG:teuthology.orchestra.run.smithi157:> import re 2024-03-29T04:10:41.583 DEBUG:teuthology.orchestra.run.smithi157:> import os 2024-03-29T04:10:41.583 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2024-03-29T04:10:41.583 DEBUG:teuthology.orchestra.run.smithi157:> 2024-03-29T04:10:41.583 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2024-03-29T04:10:41.583 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-03-29T04:10:41.583 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2024-03-29T04:10:41.583 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-03-29T04:10:41.583 DEBUG:teuthology.orchestra.run.smithi157:> 2024-03-29T04:10:41.583 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2024-03-29T04:10:41.583 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2024-03-29T04:10:41.583 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2024-03-29T04:10:41.583 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2024-03-29T04:10:41.583 DEBUG:teuthology.orchestra.run.smithi157:> 2024-03-29T04:10:41.583 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2024-03-29T04:10:41.584 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T04:10:41.584 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T04:10:41.584 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T04:10:41.584 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2024-03-29T04:10:41.584 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2024-03-29T04:10:41.584 DEBUG:teuthology.orchestra.run.smithi157:> return f 2024-03-29T04:10:41.584 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T04:10:41.584 DEBUG:teuthology.orchestra.run.smithi157:> 2024-03-29T04:10:41.584 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.1")) 2024-03-29T04:10:41.584 DEBUG:teuthology.orchestra.run.smithi157:> ' 2024-03-29T04:10:41.707 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.1.43302.asok 2024-03-29T04:10:41.710 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:10:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:41.717 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.43302.asok 2024-03-29T04:10:41.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:41.718 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.43302.asok status 2024-03-29T04:10:41.854 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "69e2cd73c52284dfd5096c66b60c6e6b26a3c447", 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 19.0.0-2520-g69e2cd73 (69e2cd73c52284dfd5096c66b60c6e6b26a3c447) squid (dev)", 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "1", 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "43302", 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 4771, 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 4771 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 653169293 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 653169293 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.4771 192.168.0.1:0/653169293", 2024-03-29T04:10:41.857 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/653169293", 2024-03-29T04:10:41.857 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2024-03-29T04:10:41.857 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 9, 2024-03-29T04:10:41.857 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 25, 2024-03-29T04:10:41.857 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2024-03-29T04:10:41.857 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2024-03-29T04:10:41.857 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2024-03-29T04:10:41.857 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-03-29T04:10:41.863 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T04:10:41.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.988+0000 7fe77d3e9640 1 Processor -- start 2024-03-29T04:10:41.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.989+0000 7fe77d3e9640 1 -- start start 2024-03-29T04:10:41.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.990+0000 7fe77d3e9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe7780d3430 0x7fe7780cf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:41.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.990+0000 7fe77d3e9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe7780cf8a0 0x7fe7780b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:41.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.990+0000 7fe77d3e9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7780b3ba0 con 0x7fe7780cef80 2024-03-29T04:10:41.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.990+0000 7fe77d3e9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fe77815ec20 con 0x7fe7780cf8a0 2024-03-29T04:10:41.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.990+0000 7fe77d3e9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe7780b47b0 con 0x7fe7780d3430 2024-03-29T04:10:41.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.990+0000 7fe7777fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe7780cf8a0 0x7fe7780b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:41.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.990+0000 7fe7777fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe7780cf8a0 0x7fe7780b4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46210/0 (socket says 172.21.15.148:46210) 2024-03-29T04:10:41.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.990+0000 7fe7777fe640 1 -- 172.21.15.148:0/1022161770 learned_addr learned my addr 172.21.15.148:0/1022161770 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:41.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.990+0000 7fe76e7fc640 1 --2- 172.21.15.148:0/1022161770 >> v2:172.21.15.157:3300/0 conn(0x7fe7780d3430 0x7fe7780cf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:41.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.990+0000 7fe7777fe640 1 -- 172.21.15.148:0/1022161770 >> v2:172.21.15.157:3300/0 conn(0x7fe7780d3430 msgr2=0x7fe7780cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:41.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.990+0000 7fe7777fe640 1 --2- 172.21.15.148:0/1022161770 >> v2:172.21.15.157:3300/0 conn(0x7fe7780d3430 0x7fe7780cf360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:41.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.990+0000 7fe7777fe640 1 -- 172.21.15.148:0/1022161770 >> v1:172.21.15.157:6789/0 conn(0x7fe7780cef80 legacy=0x7fe7780b7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:41.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.990+0000 7fe7777fe640 1 -- 172.21.15.148:0/1022161770 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7780b3e80 con 0x7fe7780cf8a0 2024-03-29T04:10:41.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.991+0000 7fe7777fe640 1 --2- 172.21.15.148:0/1022161770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7780cf8a0 0x7fe7780b4c40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe7780b5c90 tx=0x7fe77002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3b4dd5cdfa12f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:41.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.991+0000 7fe7767fc640 1 -- 172.21.15.148:0/1022161770 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe770034070 con 0x7fe7780cf8a0 2024-03-29T04:10:41.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.991+0000 7fe7767fc640 1 -- 172.21.15.148:0/1022161770 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe770030860 con 0x7fe7780cf8a0 2024-03-29T04:10:41.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.991+0000 7fe7767fc640 1 -- 172.21.15.148:0/1022161770 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe770030c30 con 0x7fe7780cf8a0 2024-03-29T04:10:41.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.992+0000 7fe77d3e9640 1 -- 172.21.15.148:0/1022161770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7780cf8a0 msgr2=0x7fe7780b4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:41.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.992+0000 7fe77d3e9640 1 --2- 172.21.15.148:0/1022161770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7780cf8a0 0x7fe7780b4c40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe7780b5c90 tx=0x7fe77002b9a0 comp rx=0 tx=0).stop 2024-03-29T04:10:41.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.992+0000 7fe77d3e9640 1 -- 172.21.15.148:0/1022161770 shutdown_connections 2024-03-29T04:10:41.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.992+0000 7fe77d3e9640 1 --2- 172.21.15.148:0/1022161770 >> v2:172.21.15.157:3300/0 conn(0x7fe7780d3430 0x7fe7780cf360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:41.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.992+0000 7fe77d3e9640 1 --2- 172.21.15.148:0/1022161770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7780cf8a0 0x7fe7780b4c40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:41.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.992+0000 7fe77d3e9640 1 -- 172.21.15.148:0/1022161770 >> 172.21.15.148:0/1022161770 conn(0x7fe7780d4300 msgr2=0x7fe7780d4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:41.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.992+0000 7fe77d3e9640 1 -- 172.21.15.148:0/1022161770 shutdown_connections 2024-03-29T04:10:41.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.992+0000 7fe77d3e9640 1 -- 172.21.15.148:0/1022161770 wait complete. 2024-03-29T04:10:41.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.993+0000 7fe77d3e9640 1 Processor -- start 2024-03-29T04:10:41.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.993+0000 7fe77d3e9640 1 -- start start 2024-03-29T04:10:41.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.994+0000 7fe77d3e9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7780cef80 0x7fe7780c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:41.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.994+0000 7fe77d3e9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7780cf8a0 0x7fe7781f48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.994+0000 7fe77d3e9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7780d3430 0x7fe77815f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.994+0000 7fe77d3e9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe7780d26b0 con 0x7fe7780cf8a0 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.994+0000 7fe77d3e9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe7780b4440 con 0x7fe7780cef80 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.994+0000 7fe77d3e9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe7780b3830 con 0x7fe7780d3430 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.994+0000 7fe76e7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7780cf8a0 0x7fe7781f48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.994+0000 7fe76e7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7780cf8a0 0x7fe7781f48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46220/0 (socket says 172.21.15.148:46220) 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.994+0000 7fe76e7fc640 1 -- 172.21.15.148:0/3818803256 learned_addr learned my addr 172.21.15.148:0/3818803256 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.994+0000 7fe776ffd640 1 --2- 172.21.15.148:0/3818803256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7780cef80 0x7fe7780c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.994+0000 7fe7777fe640 1 --2- 172.21.15.148:0/3818803256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7780d3430 0x7fe77815f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.994+0000 7fe76e7fc640 1 -- 172.21.15.148:0/3818803256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7780d3430 msgr2=0x7fe77815f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.994+0000 7fe76e7fc640 1 --2- 172.21.15.148:0/3818803256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7780d3430 0x7fe77815f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.994+0000 7fe76e7fc640 1 -- 172.21.15.148:0/3818803256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7780cef80 msgr2=0x7fe7780c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.994+0000 7fe76e7fc640 1 --2- 172.21.15.148:0/3818803256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7780cef80 0x7fe7780c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.994+0000 7fe76e7fc640 1 -- 172.21.15.148:0/3818803256 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7781f5100 con 0x7fe7780cf8a0 2024-03-29T04:10:41.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.995+0000 7fe76e7fc640 1 --2- 172.21.15.148:0/3818803256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7780cf8a0 0x7fe7781f48a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe76000d5d0 tx=0x7fe76000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:41.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.995+0000 7fe76ffff640 1 -- 172.21.15.148:0/3818803256 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7600125f0 con 0x7fe7780cf8a0 2024-03-29T04:10:41.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.995+0000 7fe76ffff640 1 -- 172.21.15.148:0/3818803256 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe760019070 con 0x7fe7780cf8a0 2024-03-29T04:10:41.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.995+0000 7fe76ffff640 1 -- 172.21.15.148:0/3818803256 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7600129f0 con 0x7fe7780cf8a0 2024-03-29T04:10:41.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.995+0000 7fe77d3e9640 1 -- 172.21.15.148:0/3818803256 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7781f53e0 con 0x7fe7780cf8a0 2024-03-29T04:10:41.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.995+0000 7fe77d3e9640 1 -- 172.21.15.148:0/3818803256 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe77815eff0 con 0x7fe7780cf8a0 2024-03-29T04:10:41.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.996+0000 7fe77d3e9640 1 -- 172.21.15.148:0/3818803256 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe778160d40 con 0x7fe7780cf8a0 2024-03-29T04:10:41.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.996+0000 7fe76ffff640 1 -- 172.21.15.148:0/3818803256 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe760012b90 con 0x7fe7780cf8a0 2024-03-29T04:10:41.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.997+0000 7fe76ffff640 1 --2- 172.21.15.148:0/3818803256 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe75806c530 0x7fe75806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:41.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.997+0000 7fe76ffff640 1 -- 172.21.15.148:0/3818803256 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe76008bd80 con 0x7fe7780cf8a0 2024-03-29T04:10:41.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.997+0000 7fe776ffd640 1 --2- 172.21.15.148:0/3818803256 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe75806c530 0x7fe75806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:41.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:41.997+0000 7fe776ffd640 1 --2- 172.21.15.148:0/3818803256 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe75806c530 0x7fe75806e9f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe768001f10 tx=0x7fe768009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:41.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:42.001+0000 7fe76ffff640 1 -- 172.21.15.148:0/3818803256 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe7600599e0 con 0x7fe7780cf8a0 2024-03-29T04:10:42.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:42.222+0000 7fe77d3e9640 1 -- 172.21.15.148:0/3818803256 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe77815ff80 con 0x7fe7780cf8a0 2024-03-29T04:10:42.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:42.222+0000 7fe76ffff640 1 -- 172.21.15.148:0/3818803256 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fe76005dce0 con 0x7fe7780cf8a0 2024-03-29T04:10:42.203 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-29T04:10:42.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:42.224+0000 7fe77d3e9640 1 -- 172.21.15.148:0/3818803256 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe75806c530 msgr2=0x7fe75806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:42.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:42.224+0000 7fe77d3e9640 1 --2- 172.21.15.148:0/3818803256 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe75806c530 0x7fe75806e9f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe768001f10 tx=0x7fe768009040 comp rx=0 tx=0).stop 2024-03-29T04:10:42.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:42.224+0000 7fe77d3e9640 1 -- 172.21.15.148:0/3818803256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7780cf8a0 msgr2=0x7fe7781f48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:42.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:42.224+0000 7fe77d3e9640 1 --2- 172.21.15.148:0/3818803256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7780cf8a0 0x7fe7781f48a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe76000d5d0 tx=0x7fe76000da90 comp rx=0 tx=0).stop 2024-03-29T04:10:42.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:42.224+0000 7fe77d3e9640 1 -- 172.21.15.148:0/3818803256 shutdown_connections 2024-03-29T04:10:42.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:42.224+0000 7fe77d3e9640 1 --2- 172.21.15.148:0/3818803256 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe75806c530 0x7fe75806e9f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:42.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:42.224+0000 7fe77d3e9640 1 --2- 172.21.15.148:0/3818803256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7780d3430 0x7fe77815f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:42.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:42.224+0000 7fe77d3e9640 1 --2- 172.21.15.148:0/3818803256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7780cf8a0 0x7fe7781f48a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:42.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:42.224+0000 7fe77d3e9640 1 --2- 172.21.15.148:0/3818803256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7780cef80 0x7fe7780c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:42.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:42.224+0000 7fe77d3e9640 1 -- 172.21.15.148:0/3818803256 >> 172.21.15.148:0/3818803256 conn(0x7fe7780d4300 msgr2=0x7fe7780c7170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:42.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:42.225+0000 7fe77d3e9640 1 -- 172.21.15.148:0/3818803256 shutdown_connections 2024-03-29T04:10:42.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:42.225+0000 7fe77d3e9640 1 -- 172.21.15.148:0/3818803256 wait complete. 2024-03-29T04:10:42.215 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T04:10:42.215 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:10:42.215 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-03-29T04:10:42.215 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-03-29T04:10:42.215 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-29T04:10:42.215 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T04:10:42.216 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T04:10:42.216 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:10:42.216 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-03-29T04:10:42.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:42.216 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-03-29T04:10:42.232 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:10:42.232 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:10:42.233 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:10:42.233 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:42.233 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-03-29T04:10:42.233 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:42.233 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:10:42.233 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:42.233 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-03-29T04:10:42.233 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:10:42.233 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:42.233 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-03-29T04:10:42.233 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:10:42.233 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:42.233 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:10:42.233 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:42.233 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T04:10:42.234 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42718sec preferred_lft 42718sec 2024-03-29T04:10:42.234 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-03-29T04:10:42.234 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:42.234 INFO:teuthology.orchestra.run.smithi157.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:10:42.234 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether d6:7c:73:cd:6f:d3 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:42.234 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T04:10:42.234 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:42.234 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4fd:14ff:fef9:9691/64 scope link tentative 2024-03-29T04:10:42.234 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:42.234 INFO:teuthology.orchestra.run.smithi157.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T04:10:42.234 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether d6:7c:73:cd:6f:d3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:10:42.234 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::d47c:73ff:fecd:6fd3/64 scope link tentative 2024-03-29T04:10:42.234 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:42.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:42.235 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:10:42.299 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-03-29T04:10:42.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:42.301 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-03-29T04:10:42.357 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-03-29T04:10:42.358 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:10:42.358 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T04:10:42.358 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T04:10:42.358 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:10:42.358 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:10:42.358 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-03-29T04:10:42.358 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T04:10:42.359 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T04:10:42.359 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T04:10:42.359 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-03-29T04:10:42.445 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:10:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:10:42.488 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:10:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:42.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:42.494 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-03-29T04:10:42.509 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-29T04:10:42.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:42.509 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list-id 2024-03-29T04:10:42.564 INFO:teuthology.orchestra.run.smithi157.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-03-29T04:10:42.564 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:10:42.564 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-03-29T04:10:42.564 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:10:42.564 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-03-29T04:10:42.564 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-03-29T04:10:42.655 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:10:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:10:42.683 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:10:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:42.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:42.689 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-29T04:10:42.715 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:10:42.715 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:10:42.715 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:10:42.716 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:42.716 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-03-29T04:10:42.716 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:42.716 INFO:teuthology.orchestra.run.smithi157.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:10:42.716 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether ba:b2:41:62:92:5e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-29T04:10:42.716 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-29T04:10:42.716 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:42.716 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::b8b2:41ff:fe62:925e/64 scope link tentative 2024-03-29T04:10:42.716 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:42.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:42.717 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-29T04:10:42.784 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:10:42.784 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:10:42.785 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:10:42.785 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:42.785 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-03-29T04:10:42.785 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:42.785 INFO:teuthology.orchestra.run.smithi157.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:10:42.785 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether ba:b2:41:62:92:5e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-29T04:10:42.785 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-29T04:10:42.785 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:42.785 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::b8b2:41ff:fe62:925e/64 scope link 2024-03-29T04:10:42.785 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:42.786 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-03-29T04:10:42.787 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:10:42.787 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-03-29T04:10:42.787 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-03-29T04:10:42.787 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-29T04:10:42.787 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-03-29T04:10:42.787 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-03-29T04:10:42.787 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-03-29T04:10:42.787 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-03-29T04:10:42.874 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:10:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:10:42.945 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:10:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:42.951 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:10:42.951 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-03-29T04:10:42.951 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.1 up 2024-03-29T04:10:42.952 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set dev brx.1 master ceph-brx 2024-03-29T04:10:42.952 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-03-29T04:10:42.999 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:10:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:10:43.027 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:10:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:43.034 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-03-29T04:10:43.034 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:10:43.034 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-03-29T04:10:43.048 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-03-29T04:10:43.048 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:10:43.048 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-03-29T04:10:43.103 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-03-29T04:10:43.171 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:10:43.227 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-03-29T04:10:43.227 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-03-29T04:10:43.227 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-03-29T04:10:43.227 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-03-29T04:10:43.227 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-03-29T04:10:43.227 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-03-29T04:10:43.228 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-03-29T04:10:43.228 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-03-29T04:10:43.228 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-03-29T04:10:43.228 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-03-29T04:10:43.228 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-03-29T04:10:43.228 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/selinux 2024-03-29T04:10:43.228 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:10:43.228 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-03-29T04:10:43.228 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-03-29T04:10:43.228 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-03-29T04:10:43.228 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-03-29T04:10:43.228 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-03-29T04:10:43.228 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-03-29T04:10:43.228 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysctl.service 2024-03-29T04:10:43.228 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-29T04:10:43.229 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-29T04:10:43.229 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/nfs/rpc_pipefs 2024-03-29T04:10:43.229 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-03-29T04:10:43.229 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-03-29T04:10:43.229 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-03-29T04:10:43.229 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-03-29T04:10:43.229 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-03-29T04:10:43.229 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-03-29T04:10:43.229 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-03-29T04:10:43.229 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:10:43.229 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:10:43.229 INFO:teuthology.orchestra.run.smithi157.stdout:/home/ubuntu/cephtest/mnt.1 2024-03-29T04:10:43.229 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:10:43.229 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:10:43.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:43.230 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-03-29T04:10:43.282 INFO:teuthology.orchestra.run.smithi157.stdout:41 2024-03-29T04:10:43.282 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2024-03-29T04:10:43.283 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-03-29T04:10:43.326 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-03-29T04:10:43.352 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:10:43.408 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-03-29T04:10:43.408 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-03-29T04:10:43.408 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-03-29T04:10:43.408 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-03-29T04:10:43.408 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-03-29T04:10:43.408 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-03-29T04:10:43.408 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-03-29T04:10:43.409 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-03-29T04:10:43.409 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-03-29T04:10:43.409 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-03-29T04:10:43.409 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-03-29T04:10:43.409 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/selinux 2024-03-29T04:10:43.409 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:10:43.409 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-03-29T04:10:43.409 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-03-29T04:10:43.409 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-03-29T04:10:43.409 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-03-29T04:10:43.409 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-03-29T04:10:43.409 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-03-29T04:10:43.409 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysctl.service 2024-03-29T04:10:43.409 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-29T04:10:43.409 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-29T04:10:43.410 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/nfs/rpc_pipefs 2024-03-29T04:10:43.410 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-03-29T04:10:43.410 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-03-29T04:10:43.410 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-03-29T04:10:43.410 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-03-29T04:10:43.410 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-03-29T04:10:43.410 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-03-29T04:10:43.410 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-03-29T04:10:43.410 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:10:43.410 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:10:43.410 INFO:teuthology.orchestra.run.smithi157.stdout:/home/ubuntu/cephtest/mnt.1 2024-03-29T04:10:43.410 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:10:43.410 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:10:43.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:43.411 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-03-29T04:10:43.411 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi157.stderr:2024-03-29T04:10:43.433+0000 7f1edd63c480 -1 init, newargv = 0x56041a5cf4e0 newargc=15 2024-03-29T04:10:43.412 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi157.stderr:ceph-fuse[43759]: starting ceph client 2024-03-29T04:10:43.416 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi157.stderr:ceph-fuse[43759]: starting fuse 2024-03-29T04:10:43.426 INFO:teuthology.orchestra.run.smithi157.stdout:41 2024-03-29T04:10:43.426 INFO:teuthology.orchestra.run.smithi157.stdout:44 2024-03-29T04:10:43.426 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 44] 2024-03-29T04:10:43.426 DEBUG:teuthology.orchestra.run.smithi157:> sudo stdin-killer -- python3 -c ' 2024-03-29T04:10:43.426 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2024-03-29T04:10:43.427 DEBUG:teuthology.orchestra.run.smithi157:> import re 2024-03-29T04:10:43.427 DEBUG:teuthology.orchestra.run.smithi157:> import os 2024-03-29T04:10:43.427 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2024-03-29T04:10:43.427 DEBUG:teuthology.orchestra.run.smithi157:> 2024-03-29T04:10:43.427 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2024-03-29T04:10:43.427 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-03-29T04:10:43.427 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2024-03-29T04:10:43.427 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-03-29T04:10:43.427 DEBUG:teuthology.orchestra.run.smithi157:> 2024-03-29T04:10:43.427 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2024-03-29T04:10:43.427 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2024-03-29T04:10:43.427 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2024-03-29T04:10:43.427 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2024-03-29T04:10:43.427 DEBUG:teuthology.orchestra.run.smithi157:> 2024-03-29T04:10:43.427 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2024-03-29T04:10:43.427 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T04:10:43.427 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T04:10:43.428 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T04:10:43.428 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2024-03-29T04:10:43.428 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2024-03-29T04:10:43.428 DEBUG:teuthology.orchestra.run.smithi157:> return f 2024-03-29T04:10:43.428 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T04:10:43.428 DEBUG:teuthology.orchestra.run.smithi157:> 2024-03-29T04:10:43.428 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.3")) 2024-03-29T04:10:43.428 DEBUG:teuthology.orchestra.run.smithi157:> ' 2024-03-29T04:10:43.546 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.3.43759.asok 2024-03-29T04:10:43.549 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:10:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:43.557 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.43759.asok 2024-03-29T04:10:43.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:43.557 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.43759.asok status 2024-03-29T04:10:43.696 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-03-29T04:10:43.696 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2024-03-29T04:10:43.696 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "69e2cd73c52284dfd5096c66b60c6e6b26a3c447", 2024-03-29T04:10:43.696 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 19.0.0-2520-g69e2cd73 (69e2cd73c52284dfd5096c66b60c6e6b26a3c447) squid (dev)", 2024-03-29T04:10:43.696 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "3", 2024-03-29T04:10:43.696 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2024-03-29T04:10:43.696 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-03-29T04:10:43.696 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "43759", 2024-03-29T04:10:43.696 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2024-03-29T04:10:43.696 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-29T04:10:43.696 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2024-03-29T04:10:43.696 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2024-03-29T04:10:43.697 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 4616, 2024-03-29T04:10:43.697 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2024-03-29T04:10:43.697 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2024-03-29T04:10:43.697 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2024-03-29T04:10:43.697 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 4616 2024-03-29T04:10:43.697 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-29T04:10:43.697 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-03-29T04:10:43.697 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-03-29T04:10:43.697 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.2:0", 2024-03-29T04:10:43.697 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 1450325018 2024-03-29T04:10:43.697 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-29T04:10:43.697 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-29T04:10:43.697 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-03-29T04:10:43.697 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-03-29T04:10:43.697 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.2:0", 2024-03-29T04:10:43.698 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 1450325018 2024-03-29T04:10:43.698 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-29T04:10:43.698 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.4616 192.168.0.2:0/1450325018", 2024-03-29T04:10:43.698 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.2:0/1450325018", 2024-03-29T04:10:43.698 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2024-03-29T04:10:43.698 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 9, 2024-03-29T04:10:43.698 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 26, 2024-03-29T04:10:43.698 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2024-03-29T04:10:43.698 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2024-03-29T04:10:43.698 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2024-03-29T04:10:43.698 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-03-29T04:10:43.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:43.705 DEBUG:teuthology.orchestra.run.smithi148:> stat --file-system '--printf=%T 2024-03-29T04:10:43.706 DEBUG:teuthology.orchestra.run.smithi148:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T04:10:43.722 INFO:teuthology.orchestra.run.smithi148.stdout:fuseblk 2024-03-29T04:10:43.722 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T04:10:43.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:43.722 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T04:10:43.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:43.793 DEBUG:teuthology.orchestra.run.smithi148:> stat --file-system '--printf=%T 2024-03-29T04:10:43.793 DEBUG:teuthology.orchestra.run.smithi148:> ' -- /home/ubuntu/cephtest/mnt.2 2024-03-29T04:10:43.849 INFO:teuthology.orchestra.run.smithi148.stdout:fuseblk 2024-03-29T04:10:43.849 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-03-29T04:10:43.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:43.849 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-03-29T04:10:43.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:43.917 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2024-03-29T04:10:43.917 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1 2024-03-29T04:10:43.933 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2024-03-29T04:10:43.934 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-03-29T04:10:43.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:43.934 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-29T04:10:44.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:44.001 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2024-03-29T04:10:44.001 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.3 2024-03-29T04:10:44.057 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2024-03-29T04:10:44.057 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-03-29T04:10:44.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:44.058 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-03-29T04:10:44.124 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-29T04:10:44.140 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:44.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.265+0000 7fb34d3f2640 1 Processor -- start 2024-03-29T04:10:44.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.265+0000 7fb34d3f2640 1 -- start start 2024-03-29T04:10:44.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.265+0000 7fb34d3f2640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb3480ba030 0x7fb3480d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:44.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.265+0000 7fb34d3f2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3480d10d0 0x7fb3480d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:44.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.265+0000 7fb34d3f2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb34815e7c0 con 0x7fb34815e3e0 2024-03-29T04:10:44.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.265+0000 7fb34d3f2640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fb3480d3840 con 0x7fb3480ba030 2024-03-29T04:10:44.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.265+0000 7fb34d3f2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb3480d2c30 con 0x7fb3480d10d0 2024-03-29T04:10:44.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.265+0000 7fb33ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb3480ba030 0x7fb3480d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:44.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.265+0000 7fb33ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb3480ba030 0x7fb3480d3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46236/0 (socket says 172.21.15.148:46236) 2024-03-29T04:10:44.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.265+0000 7fb33ffff640 1 -- 172.21.15.148:0/256329188 learned_addr learned my addr 172.21.15.148:0/256329188 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:44.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.265+0000 7fb3477fe640 1 --2- 172.21.15.148:0/256329188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3480d10d0 0x7fb3480d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:44.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.265+0000 7fb33ffff640 1 -- 172.21.15.148:0/256329188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3480d10d0 msgr2=0x7fb3480d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:44.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.265+0000 7fb33ffff640 1 --2- 172.21.15.148:0/256329188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3480d10d0 0x7fb3480d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.265+0000 7fb33ffff640 1 -- 172.21.15.148:0/256329188 >> v1:172.21.15.157:6789/0 conn(0x7fb34815e3e0 legacy=0x7fb3480d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:44.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.265+0000 7fb33ffff640 1 -- 172.21.15.148:0/256329188 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3480d2f10 con 0x7fb3480ba030 2024-03-29T04:10:44.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.265+0000 7fb3477fe640 1 --2- 172.21.15.148:0/256329188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3480d10d0 0x7fb3480d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:44.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.266+0000 7fb33ffff640 1 --2- 172.21.15.148:0/256329188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3480ba030 0x7fb3480d3cd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb330009870 tx=0x7fb33002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7874b7dc8d9fc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:44.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.267+0000 7fb3467fc640 1 -- 172.21.15.148:0/256329188 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33002f840 con 0x7fb3480ba030 2024-03-29T04:10:44.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.267+0000 7fb3467fc640 1 -- 172.21.15.148:0/256329188 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb33002f9e0 con 0x7fb3480ba030 2024-03-29T04:10:44.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.267+0000 7fb3467fc640 1 -- 172.21.15.148:0/256329188 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33002fdb0 con 0x7fb3480ba030 2024-03-29T04:10:44.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.267+0000 7fb34d3f2640 1 -- 172.21.15.148:0/256329188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3480ba030 msgr2=0x7fb3480d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:44.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.267+0000 7fb34d3f2640 1 --2- 172.21.15.148:0/256329188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3480ba030 0x7fb3480d3cd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb330009870 tx=0x7fb33002ba20 comp rx=0 tx=0).stop 2024-03-29T04:10:44.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.267+0000 7fb34d3f2640 1 -- 172.21.15.148:0/256329188 shutdown_connections 2024-03-29T04:10:44.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.267+0000 7fb34d3f2640 1 --2- 172.21.15.148:0/256329188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3480d10d0 0x7fb3480d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.267+0000 7fb34d3f2640 1 --2- 172.21.15.148:0/256329188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3480ba030 0x7fb3480d3cd0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.267+0000 7fb34d3f2640 1 -- 172.21.15.148:0/256329188 >> 172.21.15.148:0/256329188 conn(0x7fb3480b6210 msgr2=0x7fb3480b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:44.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.267+0000 7fb34d3f2640 1 -- 172.21.15.148:0/256329188 shutdown_connections 2024-03-29T04:10:44.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.267+0000 7fb34d3f2640 1 -- 172.21.15.148:0/256329188 wait complete. 2024-03-29T04:10:44.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.269+0000 7fb34d3f2640 1 Processor -- start 2024-03-29T04:10:44.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.269+0000 7fb34d3f2640 1 -- start start 2024-03-29T04:10:44.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.270+0000 7fb34d3f2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3480ba030 0x7fb34814eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:44.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.270+0000 7fb34d3f2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3480d10d0 0x7fb348154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:44.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.270+0000 7fb34d3f2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb34815e3e0 0x7fb348154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:44.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.270+0000 7fb34d3f2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb348167f50 con 0x7fb34815e3e0 2024-03-29T04:10:44.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.270+0000 7fb34d3f2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb3480d34d0 con 0x7fb3480ba030 2024-03-29T04:10:44.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.270+0000 7fb34d3f2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb3481680d0 con 0x7fb3480d10d0 2024-03-29T04:10:44.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.270+0000 7fb3477fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb34815e3e0 0x7fb348154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:44.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.270+0000 7fb3477fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb34815e3e0 0x7fb348154680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46250/0 (socket says 172.21.15.148:46250) 2024-03-29T04:10:44.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.270+0000 7fb3477fe640 1 -- 172.21.15.148:0/2379963221 learned_addr learned my addr 172.21.15.148:0/2379963221 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:44.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.270+0000 7fb33ffff640 1 --2- 172.21.15.148:0/2379963221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3480d10d0 0x7fb348154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:44.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.270+0000 7fb346ffd640 1 --2- 172.21.15.148:0/2379963221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3480ba030 0x7fb34814eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:44.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.270+0000 7fb3477fe640 1 -- 172.21.15.148:0/2379963221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3480d10d0 msgr2=0x7fb348154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:44.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.270+0000 7fb3477fe640 1 --2- 172.21.15.148:0/2379963221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3480d10d0 0x7fb348154140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.270+0000 7fb3477fe640 1 -- 172.21.15.148:0/2379963221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3480ba030 msgr2=0x7fb34814eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:44.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.270+0000 7fb3477fe640 1 --2- 172.21.15.148:0/2379963221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3480ba030 0x7fb34814eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.270+0000 7fb3477fe640 1 -- 172.21.15.148:0/2379963221 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb348156c60 con 0x7fb34815e3e0 2024-03-29T04:10:44.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.271+0000 7fb3477fe640 1 --2- 172.21.15.148:0/2379963221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb34815e3e0 0x7fb348154680 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb340007ef0 tx=0x7fb34000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:44.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.271+0000 7fb33f7fe640 1 -- 172.21.15.148:0/2379963221 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34001a070 con 0x7fb34815e3e0 2024-03-29T04:10:44.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.271+0000 7fb33f7fe640 1 -- 172.21.15.148:0/2379963221 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb340003b90 con 0x7fb34815e3e0 2024-03-29T04:10:44.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.271+0000 7fb34d3f2640 1 -- 172.21.15.148:0/2379963221 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb348156f20 con 0x7fb34815e3e0 2024-03-29T04:10:44.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.271+0000 7fb33f7fe640 1 -- 172.21.15.148:0/2379963221 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb340004030 con 0x7fb34815e3e0 2024-03-29T04:10:44.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.271+0000 7fb34d3f2640 1 -- 172.21.15.148:0/2379963221 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3481574e0 con 0x7fb34815e3e0 2024-03-29T04:10:44.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.272+0000 7fb34d3f2640 1 -- 172.21.15.148:0/2379963221 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3480687e0 con 0x7fb34815e3e0 2024-03-29T04:10:44.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.272+0000 7fb33f7fe640 1 -- 172.21.15.148:0/2379963221 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb34001d070 con 0x7fb34815e3e0 2024-03-29T04:10:44.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.273+0000 7fb33f7fe640 1 --2- 172.21.15.148:0/2379963221 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb32806c530 0x7fb32806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:44.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.273+0000 7fb33f7fe640 1 -- 172.21.15.148:0/2379963221 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb34008bca0 con 0x7fb34815e3e0 2024-03-29T04:10:44.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.273+0000 7fb346ffd640 1 --2- 172.21.15.148:0/2379963221 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb32806c530 0x7fb32806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:44.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.273+0000 7fb346ffd640 1 --2- 172.21.15.148:0/2379963221 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb32806c530 0x7fb32806e9f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb338000f60 tx=0x7fb338009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:44.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.278+0000 7fb33f7fe640 1 -- 172.21.15.148:0/2379963221 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb340010070 con 0x7fb34815e3e0 2024-03-29T04:10:44.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.460+0000 7fb34d3f2640 1 -- 172.21.15.148:0/2379963221 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb348155bd0 con 0x7fb34815e3e0 2024-03-29T04:10:44.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.461+0000 7fb33f7fe640 1 -- 172.21.15.148:0/2379963221 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18858 (secure 0 0 0) 0x7fb340059870 con 0x7fb34815e3e0 2024-03-29T04:10:44.441 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:44.441 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:41.662934+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:44.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.463+0000 7fb34d3f2640 1 -- 172.21.15.148:0/2379963221 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb32806c530 msgr2=0x7fb32806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:44.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.463+0000 7fb34d3f2640 1 --2- 172.21.15.148:0/2379963221 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb32806c530 0x7fb32806e9f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb338000f60 tx=0x7fb338009040 comp rx=0 tx=0).stop 2024-03-29T04:10:44.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.463+0000 7fb34d3f2640 1 -- 172.21.15.148:0/2379963221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb34815e3e0 msgr2=0x7fb348154680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:44.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.463+0000 7fb34d3f2640 1 --2- 172.21.15.148:0/2379963221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb34815e3e0 0x7fb348154680 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb340007ef0 tx=0x7fb34000d4a0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.463+0000 7fb34d3f2640 1 -- 172.21.15.148:0/2379963221 shutdown_connections 2024-03-29T04:10:44.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.463+0000 7fb34d3f2640 1 --2- 172.21.15.148:0/2379963221 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb32806c530 0x7fb32806e9f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.463+0000 7fb34d3f2640 1 --2- 172.21.15.148:0/2379963221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb34815e3e0 0x7fb348154680 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.463+0000 7fb34d3f2640 1 --2- 172.21.15.148:0/2379963221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3480d10d0 0x7fb348154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.463+0000 7fb34d3f2640 1 --2- 172.21.15.148:0/2379963221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3480ba030 0x7fb34814eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.463+0000 7fb34d3f2640 1 -- 172.21.15.148:0/2379963221 >> 172.21.15.148:0/2379963221 conn(0x7fb3480b6210 msgr2=0x7fb348154d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:44.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.463+0000 7fb34d3f2640 1 -- 172.21.15.148:0/2379963221 shutdown_connections 2024-03-29T04:10:44.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.463+0000 7fb34d3f2640 1 -- 172.21.15.148:0/2379963221 wait complete. 2024-03-29T04:10:44.454 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:47.898351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:57.922313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:58.829405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:10:02.551763+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:10:44.455 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:10:44.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.581+0000 7f52df8c8640 1 Processor -- start 2024-03-29T04:10:44.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.581+0000 7f52df8c8640 1 -- start start 2024-03-29T04:10:44.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.582+0000 7f52df8c8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f52d815e3c0 0x7f52d8161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:44.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.582+0000 7f52df8c8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52d80b2f00 0x7f52d81624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:44.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.582+0000 7f52df8c8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f52d80b3430 0x7f52d80d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:44.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.582+0000 7f52df8c8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f52d80cf960 con 0x7f52d815e3c0 2024-03-29T04:10:44.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.582+0000 7f52df8c8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f52d80cf330 con 0x7f52d80b3430 2024-03-29T04:10:44.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.582+0000 7f52df8c8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f52d8162cf0 con 0x7f52d80b2f00 2024-03-29T04:10:44.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.582+0000 7f52dd664640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f52d815e3c0 0x7f52d8161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:44.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.582+0000 7f52dd664640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f52d815e3c0 0x7f52d8161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46258/0 (socket says 172.21.15.148:46258) 2024-03-29T04:10:44.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.582+0000 7f52dd664640 1 -- 172.21.15.148:0/2807687778 learned_addr learned my addr 172.21.15.148:0/2807687778 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:44.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.582+0000 7f52dde65640 1 --2- 172.21.15.148:0/2807687778 >> v2:172.21.15.157:3300/0 conn(0x7f52d80b3430 0x7f52d80d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:44.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.582+0000 7f52dce63640 1 --2- 172.21.15.148:0/2807687778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52d80b2f00 0x7f52d81624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:44.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.583+0000 7f52dd664640 1 -- 172.21.15.148:0/2807687778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52d80b2f00 msgr2=0x7f52d81624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:44.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.583+0000 7f52dd664640 1 --2- 172.21.15.148:0/2807687778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52d80b2f00 0x7f52d81624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.583+0000 7f52dd664640 1 -- 172.21.15.148:0/2807687778 >> v2:172.21.15.157:3300/0 conn(0x7f52d80b3430 msgr2=0x7f52d80d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:44.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.583+0000 7f52dd664640 1 --2- 172.21.15.148:0/2807687778 >> v2:172.21.15.157:3300/0 conn(0x7f52d80b3430 0x7f52d80d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.583+0000 7f52dd664640 1 -- 172.21.15.148:0/2807687778 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d80cf5f0 con 0x7f52d815e3c0 2024-03-29T04:10:44.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.583+0000 7f52dde65640 1 --2- 172.21.15.148:0/2807687778 >> v2:172.21.15.157:3300/0 conn(0x7f52d80b3430 0x7f52d80d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:44.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.583+0000 7f52dd664640 1 --2- 172.21.15.148:0/2807687778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52d815e3c0 0x7f52d8161f70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f52d80cf0c0 tx=0x7f52d402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b76115f1b06aafe server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:44.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.583+0000 7f52cffff640 1 -- 172.21.15.148:0/2807687778 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d4035020 con 0x7f52d815e3c0 2024-03-29T04:10:44.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.583+0000 7f52cffff640 1 -- 172.21.15.148:0/2807687778 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d4031870 con 0x7f52d815e3c0 2024-03-29T04:10:44.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.583+0000 7f52cffff640 1 -- 172.21.15.148:0/2807687778 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d4031c40 con 0x7f52d815e3c0 2024-03-29T04:10:44.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.584+0000 7f52df8c8640 1 -- 172.21.15.148:0/2807687778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52d815e3c0 msgr2=0x7f52d8161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:44.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.584+0000 7f52df8c8640 1 --2- 172.21.15.148:0/2807687778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52d815e3c0 0x7f52d8161f70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f52d80cf0c0 tx=0x7f52d402b910 comp rx=0 tx=0).stop 2024-03-29T04:10:44.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.584+0000 7f52df8c8640 1 -- 172.21.15.148:0/2807687778 shutdown_connections 2024-03-29T04:10:44.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.584+0000 7f52df8c8640 1 --2- 172.21.15.148:0/2807687778 >> v2:172.21.15.157:3300/0 conn(0x7f52d80b3430 0x7f52d80d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.584+0000 7f52df8c8640 1 --2- 172.21.15.148:0/2807687778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52d80b2f00 0x7f52d81624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.584+0000 7f52df8c8640 1 --2- 172.21.15.148:0/2807687778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52d815e3c0 0x7f52d8161f70 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.584+0000 7f52df8c8640 1 -- 172.21.15.148:0/2807687778 >> 172.21.15.148:0/2807687778 conn(0x7f52d80b6430 msgr2=0x7f52d80d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:44.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.584+0000 7f52df8c8640 1 -- 172.21.15.148:0/2807687778 shutdown_connections 2024-03-29T04:10:44.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.584+0000 7f52df8c8640 1 -- 172.21.15.148:0/2807687778 wait complete. 2024-03-29T04:10:44.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.586+0000 7f52df8c8640 1 Processor -- start 2024-03-29T04:10:44.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.586+0000 7f52df8c8640 1 -- start start 2024-03-29T04:10:44.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.587+0000 7f52df8c8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52d80b3430 0x7f52d814dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:44.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.587+0000 7f52df8c8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52d815e3c0 0x7f52d8160a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:44.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.587+0000 7f52df8c8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52d8160f90 0x7f52d80c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:44.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.587+0000 7f52df8c8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f52d80b39a0 con 0x7f52d815e3c0 2024-03-29T04:10:44.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.587+0000 7f52df8c8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f52d815e7a0 con 0x7f52d80b3430 2024-03-29T04:10:44.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.587+0000 7f52df8c8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f52d80b3b20 con 0x7f52d8160f90 2024-03-29T04:10:44.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.587+0000 7f52dce63640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52d815e3c0 0x7f52d8160a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:44.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.587+0000 7f52dce63640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52d815e3c0 0x7f52d8160a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46268/0 (socket says 172.21.15.148:46268) 2024-03-29T04:10:44.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.587+0000 7f52dce63640 1 -- 172.21.15.148:0/3928842104 learned_addr learned my addr 172.21.15.148:0/3928842104 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:44.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.587+0000 7f52dd664640 1 --2- 172.21.15.148:0/3928842104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52d80b3430 0x7f52d814dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:44.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.587+0000 7f52dde65640 1 --2- 172.21.15.148:0/3928842104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52d8160f90 0x7f52d80c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:44.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.588+0000 7f52dce63640 1 -- 172.21.15.148:0/3928842104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52d8160f90 msgr2=0x7f52d80c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:44.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.588+0000 7f52dce63640 1 --2- 172.21.15.148:0/3928842104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52d8160f90 0x7f52d80c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.588+0000 7f52dce63640 1 -- 172.21.15.148:0/3928842104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52d80b3430 msgr2=0x7f52d814dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:44.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.588+0000 7f52dce63640 1 --2- 172.21.15.148:0/3928842104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52d80b3430 0x7f52d814dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.588+0000 7f52dce63640 1 -- 172.21.15.148:0/3928842104 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d814e100 con 0x7f52d815e3c0 2024-03-29T04:10:44.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.588+0000 7f52dce63640 1 --2- 172.21.15.148:0/3928842104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52d815e3c0 0x7f52d8160a50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f52c800d830 tx=0x7f52c800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:44.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.588+0000 7f52cdffb640 1 -- 172.21.15.148:0/3928842104 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c8018030 con 0x7f52d815e3c0 2024-03-29T04:10:44.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.588+0000 7f52cdffb640 1 -- 172.21.15.148:0/3928842104 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c800be10 con 0x7f52d815e3c0 2024-03-29T04:10:44.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.588+0000 7f52df8c8640 1 -- 172.21.15.148:0/3928842104 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52d80cbe50 con 0x7f52d815e3c0 2024-03-29T04:10:44.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.588+0000 7f52cdffb640 1 -- 172.21.15.148:0/3928842104 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c80098b0 con 0x7f52d815e3c0 2024-03-29T04:10:44.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.588+0000 7f52df8c8640 1 -- 172.21.15.148:0/3928842104 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52d80cc410 con 0x7f52d815e3c0 2024-03-29T04:10:44.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.590+0000 7f52df8c8640 1 -- 172.21.15.148:0/3928842104 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52d80d36b0 con 0x7f52d815e3c0 2024-03-29T04:10:44.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.590+0000 7f52cdffb640 1 -- 172.21.15.148:0/3928842104 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f52c801b070 con 0x7f52d815e3c0 2024-03-29T04:10:44.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.590+0000 7f52cdffb640 1 --2- 172.21.15.148:0/3928842104 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f52bc06c560 0x7f52bc06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:44.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.590+0000 7f52dd664640 1 --2- 172.21.15.148:0/3928842104 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f52bc06c560 0x7f52bc06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:44.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.590+0000 7f52cdffb640 1 -- 172.21.15.148:0/3928842104 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f52c808bb10 con 0x7f52d815e3c0 2024-03-29T04:10:44.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.591+0000 7f52dd664640 1 --2- 172.21.15.148:0/3928842104 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f52bc06c560 0x7f52bc06ea20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f52d4036000 tx=0x7f52d403c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:44.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.596+0000 7f52cdffb640 1 -- 172.21.15.148:0/3928842104 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f52d80d36b0 con 0x7f52d815e3c0 2024-03-29T04:10:44.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.779+0000 7f52df8c8640 1 -- 172.21.15.148:0/3928842104 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f52d80cf110 con 0x7f52d815e3c0 2024-03-29T04:10:44.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.779+0000 7f52cdffb640 1 -- 172.21.15.148:0/3928842104 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f52d814e100 con 0x7f52d815e3c0 2024-03-29T04:10:44.759 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-29T04:10:44.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.781+0000 7f52df8c8640 1 -- 172.21.15.148:0/3928842104 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f52bc06c560 msgr2=0x7f52bc06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:44.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.781+0000 7f52df8c8640 1 --2- 172.21.15.148:0/3928842104 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f52bc06c560 0x7f52bc06ea20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f52d4036000 tx=0x7f52d403c040 comp rx=0 tx=0).stop 2024-03-29T04:10:44.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.781+0000 7f52df8c8640 1 -- 172.21.15.148:0/3928842104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52d815e3c0 msgr2=0x7f52d8160a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:44.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.781+0000 7f52df8c8640 1 --2- 172.21.15.148:0/3928842104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52d815e3c0 0x7f52d8160a50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f52c800d830 tx=0x7f52c800dcf0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.781+0000 7f52df8c8640 1 -- 172.21.15.148:0/3928842104 shutdown_connections 2024-03-29T04:10:44.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.781+0000 7f52df8c8640 1 --2- 172.21.15.148:0/3928842104 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f52bc06c560 0x7f52bc06ea20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.781+0000 7f52df8c8640 1 --2- 172.21.15.148:0/3928842104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52d8160f90 0x7f52d80c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.781+0000 7f52df8c8640 1 --2- 172.21.15.148:0/3928842104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52d815e3c0 0x7f52d8160a50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.781+0000 7f52df8c8640 1 --2- 172.21.15.148:0/3928842104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52d80b3430 0x7f52d814dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.781+0000 7f52df8c8640 1 -- 172.21.15.148:0/3928842104 >> 172.21.15.148:0/3928842104 conn(0x7f52d80b6430 msgr2=0x7f52d80b54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:44.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.781+0000 7f52df8c8640 1 -- 172.21.15.148:0/3928842104 shutdown_connections 2024-03-29T04:10:44.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.781+0000 7f52df8c8640 1 -- 172.21.15.148:0/3928842104 wait complete. 2024-03-29T04:10:44.771 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:10:44.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.898+0000 7fbedbd7c640 1 Processor -- start 2024-03-29T04:10:44.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.899+0000 7fbedbd7c640 1 -- start start 2024-03-29T04:10:44.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.899+0000 7fbedbd7c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbed40b2f00 0x7fbed4162680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:44.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.899+0000 7fbedbd7c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbed40b3430 0x7fbed406b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:44.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.899+0000 7fbedbd7c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbed40cf330 con 0x7fbed415e3c0 2024-03-29T04:10:44.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.899+0000 7fbedbd7c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fbed40cf960 con 0x7fbed40b2f00 2024-03-29T04:10:44.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.899+0000 7fbedbd7c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbed4162cf0 con 0x7fbed40b3430 2024-03-29T04:10:44.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.900+0000 7fbed9b18640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbed415e3c0 0x7fbed4161f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49360/0 (socket says 172.21.15.148:49360) 2024-03-29T04:10:44.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.900+0000 7fbed9b18640 1 -- 172.21.15.148:0/430073001 learned_addr learned my addr 172.21.15.148:0/430073001 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:44.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.900+0000 7fbeda319640 1 --2- 172.21.15.148:0/430073001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbed40b3430 0x7fbed406b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:44.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.900+0000 7fbed9317640 1 --2- 172.21.15.148:0/430073001 >> v2:172.21.15.157:3300/0 conn(0x7fbed40b2f00 0x7fbed4162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:44.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.900+0000 7fbed8b16640 1 -- 172.21.15.148:0/430073001 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3153098155 0 0) 0x7fbed40cf330 con 0x7fbed415e3c0 2024-03-29T04:10:44.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.900+0000 7fbed8b16640 1 -- 172.21.15.148:0/430073001 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbec0003520 con 0x7fbed415e3c0 2024-03-29T04:10:44.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.900+0000 7fbed8b16640 1 -- 172.21.15.148:0/430073001 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3534984155 0 0) 0x7fbec0003520 con 0x7fbed415e3c0 2024-03-29T04:10:44.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.900+0000 7fbeda319640 1 -- 172.21.15.148:0/430073001 >> v2:172.21.15.157:3300/0 conn(0x7fbed40b2f00 msgr2=0x7fbed4162680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:44.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.900+0000 7fbeda319640 1 --2- 172.21.15.148:0/430073001 >> v2:172.21.15.157:3300/0 conn(0x7fbed40b2f00 0x7fbed4162680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.900+0000 7fbeda319640 1 -- 172.21.15.148:0/430073001 >> v1:172.21.15.148:6789/0 conn(0x7fbed415e3c0 legacy=0x7fbed4161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:44.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.900+0000 7fbeda319640 1 -- 172.21.15.148:0/430073001 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed40cf5f0 con 0x7fbed40b3430 2024-03-29T04:10:44.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.900+0000 7fbed9317640 1 --2- 172.21.15.148:0/430073001 >> v2:172.21.15.157:3300/0 conn(0x7fbed40b2f00 0x7fbed4162680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:44.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.901+0000 7fbeda319640 1 --2- 172.21.15.148:0/430073001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbed40b3430 0x7fbed406b7f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009d90 tx=0x7fbecc02bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c044b0eaba77243 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:44.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.901+0000 7fbed8b16640 1 -- 172.21.15.148:0/430073001 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc039070 con 0x7fbed40b3430 2024-03-29T04:10:44.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.901+0000 7fbed8b16640 1 -- 172.21.15.148:0/430073001 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbecc02c810 con 0x7fbed40b3430 2024-03-29T04:10:44.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.901+0000 7fbed8b16640 1 -- 172.21.15.148:0/430073001 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed0001670 con 0x7fbed40b3430 2024-03-29T04:10:44.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.902+0000 7fbedbd7c640 1 -- 172.21.15.148:0/430073001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbed40b3430 msgr2=0x7fbed406b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:44.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.902+0000 7fbedbd7c640 1 --2- 172.21.15.148:0/430073001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbed40b3430 0x7fbed406b7f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009d90 tx=0x7fbecc02bce0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.902+0000 7fbedbd7c640 1 -- 172.21.15.148:0/430073001 shutdown_connections 2024-03-29T04:10:44.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.902+0000 7fbedbd7c640 1 --2- 172.21.15.148:0/430073001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbed40b3430 0x7fbed406b7f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.902+0000 7fbedbd7c640 1 --2- 172.21.15.148:0/430073001 >> v2:172.21.15.157:3300/0 conn(0x7fbed40b2f00 0x7fbed4162680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.902+0000 7fbedbd7c640 1 -- 172.21.15.148:0/430073001 >> 172.21.15.148:0/430073001 conn(0x7fbed40b6430 msgr2=0x7fbed40d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:44.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.902+0000 7fbedbd7c640 1 -- 172.21.15.148:0/430073001 shutdown_connections 2024-03-29T04:10:44.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.902+0000 7fbedbd7c640 1 -- 172.21.15.148:0/430073001 wait complete. 2024-03-29T04:10:44.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.903+0000 7fbedbd7c640 1 Processor -- start 2024-03-29T04:10:44.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.904+0000 7fbedbd7c640 1 -- start start 2024-03-29T04:10:44.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.905+0000 7fbedbd7c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbed40b3430 0x7fbed415c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:44.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.905+0000 7fbedbd7c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbed415e3c0 0x7fbed40c9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:44.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.905+0000 7fbedbd7c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbed4160560 0x7fbed4160a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:44.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.905+0000 7fbedbd7c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbed40b3b20 con 0x7fbed4160560 2024-03-29T04:10:44.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.905+0000 7fbedbd7c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbed40b39a0 con 0x7fbed40b3430 2024-03-29T04:10:44.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.905+0000 7fbedbd7c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbed406bd30 con 0x7fbed415e3c0 2024-03-29T04:10:44.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.905+0000 7fbeda319640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbed4160560 0x7fbed4160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:44.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.905+0000 7fbeda319640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbed4160560 0x7fbed4160a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46284/0 (socket says 172.21.15.148:46284) 2024-03-29T04:10:44.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.905+0000 7fbeda319640 1 -- 172.21.15.148:0/3655783832 learned_addr learned my addr 172.21.15.148:0/3655783832 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:44.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.905+0000 7fbed9317640 1 --2- 172.21.15.148:0/3655783832 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbed415e3c0 0x7fbed40c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:44.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.905+0000 7fbed9b18640 1 --2- 172.21.15.148:0/3655783832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbed40b3430 0x7fbed415c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:44.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.905+0000 7fbeda319640 1 -- 172.21.15.148:0/3655783832 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbed415e3c0 msgr2=0x7fbed40c9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:44.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.905+0000 7fbeda319640 1 --2- 172.21.15.148:0/3655783832 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbed415e3c0 0x7fbed40c9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.905+0000 7fbeda319640 1 -- 172.21.15.148:0/3655783832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbed40b3430 msgr2=0x7fbed415c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:44.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.905+0000 7fbeda319640 1 --2- 172.21.15.148:0/3655783832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbed40b3430 0x7fbed415c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.905+0000 7fbeda319640 1 -- 172.21.15.148:0/3655783832 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed40c8bd0 con 0x7fbed4160560 2024-03-29T04:10:44.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.905+0000 7fbeda319640 1 --2- 172.21.15.148:0/3655783832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbed4160560 0x7fbed4160a10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009d90 tx=0x7fbecc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:44.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.906+0000 7fbeca7fc640 1 -- 172.21.15.148:0/3655783832 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc039050 con 0x7fbed4160560 2024-03-29T04:10:44.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.906+0000 7fbedbd7c640 1 -- 172.21.15.148:0/3655783832 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbed415d0d0 con 0x7fbed4160560 2024-03-29T04:10:44.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.906+0000 7fbeca7fc640 1 -- 172.21.15.148:0/3655783832 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbecc045070 con 0x7fbed4160560 2024-03-29T04:10:44.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.906+0000 7fbeca7fc640 1 -- 172.21.15.148:0/3655783832 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc009450 con 0x7fbed4160560 2024-03-29T04:10:44.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.906+0000 7fbedbd7c640 1 -- 172.21.15.148:0/3655783832 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbed41610d0 con 0x7fbed4160560 2024-03-29T04:10:44.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.907+0000 7fbedbd7c640 1 -- 172.21.15.148:0/3655783832 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbed4068850 con 0x7fbed4160560 2024-03-29T04:10:44.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.907+0000 7fbeca7fc640 1 -- 172.21.15.148:0/3655783832 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbecc040690 con 0x7fbed4160560 2024-03-29T04:10:44.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.907+0000 7fbeca7fc640 1 --2- 172.21.15.148:0/3655783832 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbeb806c530 0x7fbeb806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:44.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.908+0000 7fbeca7fc640 1 -- 172.21.15.148:0/3655783832 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fbecc0ae360 con 0x7fbed4160560 2024-03-29T04:10:44.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.908+0000 7fbed9b18640 1 --2- 172.21.15.148:0/3655783832 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbeb806c530 0x7fbeb806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:44.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.908+0000 7fbed9b18640 1 --2- 172.21.15.148:0/3655783832 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbeb806c530 0x7fbeb806e9f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbed41f75c0 tx=0x7fbed0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:44.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:44.912+0000 7fbeca7fc640 1 -- 172.21.15.148:0/3655783832 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbed40c8bd0 con 0x7fbed4160560 2024-03-29T04:10:45.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.093+0000 7fbedbd7c640 1 -- 172.21.15.148:0/3655783832 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbed40d37a0 con 0x7fbed4160560 2024-03-29T04:10:45.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.093+0000 7fbeca7fc640 1 -- 172.21.15.148:0/3655783832 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fbecc032340 con 0x7fbed4160560 2024-03-29T04:10:45.072 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:10:45.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.095+0000 7fbedbd7c640 1 -- 172.21.15.148:0/3655783832 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbeb806c530 msgr2=0x7fbeb806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:45.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.095+0000 7fbedbd7c640 1 --2- 172.21.15.148:0/3655783832 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbeb806c530 0x7fbeb806e9f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbed41f75c0 tx=0x7fbed0009040 comp rx=0 tx=0).stop 2024-03-29T04:10:45.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.095+0000 7fbedbd7c640 1 -- 172.21.15.148:0/3655783832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbed4160560 msgr2=0x7fbed4160a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:45.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.095+0000 7fbedbd7c640 1 --2- 172.21.15.148:0/3655783832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbed4160560 0x7fbed4160a10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009d90 tx=0x7fbecc004a40 comp rx=0 tx=0).stop 2024-03-29T04:10:45.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.095+0000 7fbedbd7c640 1 -- 172.21.15.148:0/3655783832 shutdown_connections 2024-03-29T04:10:45.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.095+0000 7fbedbd7c640 1 --2- 172.21.15.148:0/3655783832 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbeb806c530 0x7fbeb806e9f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.095+0000 7fbedbd7c640 1 --2- 172.21.15.148:0/3655783832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbed4160560 0x7fbed4160a10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.095+0000 7fbedbd7c640 1 --2- 172.21.15.148:0/3655783832 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbed415e3c0 0x7fbed40c9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.095+0000 7fbedbd7c640 1 --2- 172.21.15.148:0/3655783832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbed40b3430 0x7fbed415c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.095+0000 7fbedbd7c640 1 -- 172.21.15.148:0/3655783832 >> 172.21.15.148:0/3655783832 conn(0x7fbed40b6430 msgr2=0x7fbed40d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:45.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.095+0000 7fbedbd7c640 1 -- 172.21.15.148:0/3655783832 shutdown_connections 2024-03-29T04:10:45.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.095+0000 7fbedbd7c640 1 -- 172.21.15.148:0/3655783832 wait complete. 2024-03-29T04:10:45.085 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:10:45.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.210+0000 7fe1edf3d640 1 Processor -- start 2024-03-29T04:10:45.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.210+0000 7fe1edf3d640 1 -- start start 2024-03-29T04:10:45.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.212+0000 7fe1edf3d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe1e80b2f00 0x7fe1e8162680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:45.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.212+0000 7fe1edf3d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe1e80b3430 0x7fe1e806b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:45.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.212+0000 7fe1edf3d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1e80cf330 con 0x7fe1e815e3c0 2024-03-29T04:10:45.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.212+0000 7fe1edf3d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fe1e80cf960 con 0x7fe1e80b2f00 2024-03-29T04:10:45.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.212+0000 7fe1edf3d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe1e8162cf0 con 0x7fe1e80b3430 2024-03-29T04:10:45.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.212+0000 7fe1e6ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe1e80b2f00 0x7fe1e8162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:45.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.212+0000 7fe1e6ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe1e80b2f00 0x7fe1e8162680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46294/0 (socket says 172.21.15.148:46294) 2024-03-29T04:10:45.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.212+0000 7fe1e6ffd640 1 -- 172.21.15.148:0/624159730 learned_addr learned my addr 172.21.15.148:0/624159730 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:45.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.212+0000 7fe1e7fff640 1 --2- 172.21.15.148:0/624159730 >> v2:172.21.15.157:3300/0 conn(0x7fe1e80b3430 0x7fe1e806b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:45.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.212+0000 7fe1e6ffd640 1 -- 172.21.15.148:0/624159730 >> v2:172.21.15.157:3300/0 conn(0x7fe1e80b3430 msgr2=0x7fe1e806b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:45.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.212+0000 7fe1e6ffd640 1 --2- 172.21.15.148:0/624159730 >> v2:172.21.15.157:3300/0 conn(0x7fe1e80b3430 0x7fe1e806b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.212+0000 7fe1e6ffd640 1 -- 172.21.15.148:0/624159730 >> v1:172.21.15.157:6789/0 conn(0x7fe1e815e3c0 legacy=0x7fe1e8161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:45.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.212+0000 7fe1e6ffd640 1 -- 172.21.15.148:0/624159730 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e80cf5f0 con 0x7fe1e80b2f00 2024-03-29T04:10:45.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.213+0000 7fe1e6ffd640 1 --2- 172.21.15.148:0/624159730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1e80b2f00 0x7fe1e8162680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe1e80cf8d0 tx=0x7fe1dc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c058b0cefcec7f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:45.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.213+0000 7fe1e67fc640 1 -- 172.21.15.148:0/624159730 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc02e070 con 0x7fe1e80b2f00 2024-03-29T04:10:45.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.213+0000 7fe1e67fc640 1 -- 172.21.15.148:0/624159730 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1dc037070 con 0x7fe1e80b2f00 2024-03-29T04:10:45.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.213+0000 7fe1e67fc640 1 -- 172.21.15.148:0/624159730 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc031a00 con 0x7fe1e80b2f00 2024-03-29T04:10:45.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.214+0000 7fe1edf3d640 1 -- 172.21.15.148:0/624159730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1e80b2f00 msgr2=0x7fe1e8162680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:45.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.214+0000 7fe1edf3d640 1 --2- 172.21.15.148:0/624159730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1e80b2f00 0x7fe1e8162680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe1e80cf8d0 tx=0x7fe1dc02b880 comp rx=0 tx=0).stop 2024-03-29T04:10:45.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.214+0000 7fe1edf3d640 1 -- 172.21.15.148:0/624159730 shutdown_connections 2024-03-29T04:10:45.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.214+0000 7fe1edf3d640 1 --2- 172.21.15.148:0/624159730 >> v2:172.21.15.157:3300/0 conn(0x7fe1e80b3430 0x7fe1e806b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.214+0000 7fe1edf3d640 1 --2- 172.21.15.148:0/624159730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1e80b2f00 0x7fe1e8162680 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.214+0000 7fe1edf3d640 1 -- 172.21.15.148:0/624159730 >> 172.21.15.148:0/624159730 conn(0x7fe1e80b6430 msgr2=0x7fe1e80d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:45.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.214+0000 7fe1edf3d640 1 -- 172.21.15.148:0/624159730 shutdown_connections 2024-03-29T04:10:45.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.214+0000 7fe1edf3d640 1 -- 172.21.15.148:0/624159730 wait complete. 2024-03-29T04:10:45.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.215+0000 7fe1edf3d640 1 Processor -- start 2024-03-29T04:10:45.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.215+0000 7fe1edf3d640 1 -- start start 2024-03-29T04:10:45.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.216+0000 7fe1edf3d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1e80b3430 0x7fe1e81f3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:45.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.216+0000 7fe1edf3d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1e815e3c0 0x7fe1e815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:45.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.216+0000 7fe1edf3d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1e80ca850 0x7fe1e815ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:45.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.216+0000 7fe1edf3d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e80b3b20 con 0x7fe1e815e3c0 2024-03-29T04:10:45.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.216+0000 7fe1edf3d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e80b39a0 con 0x7fe1e80b3430 2024-03-29T04:10:45.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.216+0000 7fe1edf3d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe1e806bd30 con 0x7fe1e80ca850 2024-03-29T04:10:45.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.216+0000 7fe1e6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1e815e3c0 0x7fe1e815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:45.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.216+0000 7fe1e6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1e815e3c0 0x7fe1e815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46302/0 (socket says 172.21.15.148:46302) 2024-03-29T04:10:45.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.216+0000 7fe1e6ffd640 1 -- 172.21.15.148:0/760602232 learned_addr learned my addr 172.21.15.148:0/760602232 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:45.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.216+0000 7fe1e7fff640 1 --2- 172.21.15.148:0/760602232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1e80ca850 0x7fe1e815ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:45.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.216+0000 7fe1e77fe640 1 --2- 172.21.15.148:0/760602232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1e80b3430 0x7fe1e81f3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:45.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.216+0000 7fe1e6ffd640 1 -- 172.21.15.148:0/760602232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1e80ca850 msgr2=0x7fe1e815ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:45.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.216+0000 7fe1e6ffd640 1 --2- 172.21.15.148:0/760602232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1e80ca850 0x7fe1e815ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.216+0000 7fe1e6ffd640 1 -- 172.21.15.148:0/760602232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1e80b3430 msgr2=0x7fe1e81f3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:45.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.216+0000 7fe1e6ffd640 1 --2- 172.21.15.148:0/760602232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1e80b3430 0x7fe1e81f3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.216+0000 7fe1e6ffd640 1 -- 172.21.15.148:0/760602232 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e815ec20 con 0x7fe1e815e3c0 2024-03-29T04:10:45.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.216+0000 7fe1e7fff640 1 --2- 172.21.15.148:0/760602232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1e80ca850 0x7fe1e815ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:45.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.216+0000 7fe1e77fe640 1 --2- 172.21.15.148:0/760602232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1e80b3430 0x7fe1e81f3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:45.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.216+0000 7fe1e6ffd640 1 --2- 172.21.15.148:0/760602232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1e815e3c0 0x7fe1e815d5d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc02b8c0 tx=0x7fe1dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:45.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.216+0000 7fe1cffff640 1 -- 172.21.15.148:0/760602232 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc032880 con 0x7fe1e815e3c0 2024-03-29T04:10:45.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.216+0000 7fe1cffff640 1 -- 172.21.15.148:0/760602232 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1dc032a40 con 0x7fe1e815e3c0 2024-03-29T04:10:45.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.217+0000 7fe1edf3d640 1 -- 172.21.15.148:0/760602232 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1e81f4450 con 0x7fe1e815e3c0 2024-03-29T04:10:45.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.217+0000 7fe1cffff640 1 -- 172.21.15.148:0/760602232 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc032e30 con 0x7fe1e815e3c0 2024-03-29T04:10:45.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.217+0000 7fe1edf3d640 1 -- 172.21.15.148:0/760602232 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1e81f4a10 con 0x7fe1e815e3c0 2024-03-29T04:10:45.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.218+0000 7fe1edf3d640 1 -- 172.21.15.148:0/760602232 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1e8068850 con 0x7fe1e815e3c0 2024-03-29T04:10:45.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.219+0000 7fe1cffff640 1 -- 172.21.15.148:0/760602232 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe1dc04c060 con 0x7fe1e815e3c0 2024-03-29T04:10:45.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.219+0000 7fe1cffff640 1 --2- 172.21.15.148:0/760602232 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe1d406c530 0x7fe1d406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:45.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.219+0000 7fe1cffff640 1 -- 172.21.15.148:0/760602232 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe1dc0ad500 con 0x7fe1e815e3c0 2024-03-29T04:10:45.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.219+0000 7fe1e77fe640 1 --2- 172.21.15.148:0/760602232 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe1d406c530 0x7fe1d406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:45.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.219+0000 7fe1e77fe640 1 --2- 172.21.15.148:0/760602232 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe1d406c530 0x7fe1d406e9f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8006fd0 tx=0x7fe1d8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:45.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.223+0000 7fe1cffff640 1 -- 172.21.15.148:0/760602232 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe1dc040070 con 0x7fe1e815e3c0 2024-03-29T04:10:45.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.403+0000 7fe1edf3d640 1 -- 172.21.15.148:0/760602232 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe1e80cf110 con 0x7fe1e815e3c0 2024-03-29T04:10:45.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.404+0000 7fe1cffff640 1 -- 172.21.15.148:0/760602232 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fe1dc04c350 con 0x7fe1e815e3c0 2024-03-29T04:10:45.383 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-29T04:10:45.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.406+0000 7fe1edf3d640 1 -- 172.21.15.148:0/760602232 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe1d406c530 msgr2=0x7fe1d406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:45.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.406+0000 7fe1edf3d640 1 --2- 172.21.15.148:0/760602232 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe1d406c530 0x7fe1d406e9f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8006fd0 tx=0x7fe1d8008040 comp rx=0 tx=0).stop 2024-03-29T04:10:45.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.406+0000 7fe1edf3d640 1 -- 172.21.15.148:0/760602232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1e815e3c0 msgr2=0x7fe1e815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:45.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.406+0000 7fe1edf3d640 1 --2- 172.21.15.148:0/760602232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1e815e3c0 0x7fe1e815d5d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc02b8c0 tx=0x7fe1dc0046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.406+0000 7fe1edf3d640 1 -- 172.21.15.148:0/760602232 shutdown_connections 2024-03-29T04:10:45.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.406+0000 7fe1edf3d640 1 --2- 172.21.15.148:0/760602232 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe1d406c530 0x7fe1d406e9f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.406+0000 7fe1edf3d640 1 --2- 172.21.15.148:0/760602232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1e80ca850 0x7fe1e815ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.406+0000 7fe1edf3d640 1 --2- 172.21.15.148:0/760602232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1e815e3c0 0x7fe1e815d5d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.406+0000 7fe1edf3d640 1 --2- 172.21.15.148:0/760602232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1e80b3430 0x7fe1e81f3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.406+0000 7fe1edf3d640 1 -- 172.21.15.148:0/760602232 >> 172.21.15.148:0/760602232 conn(0x7fe1e80b6430 msgr2=0x7fe1e80b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:45.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.407+0000 7fe1edf3d640 1 -- 172.21.15.148:0/760602232 shutdown_connections 2024-03-29T04:10:45.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.407+0000 7fe1edf3d640 1 -- 172.21.15.148:0/760602232 wait complete. 2024-03-29T04:10:45.400 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:10:45.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.529+0000 7f51e7aa7640 1 Processor -- start 2024-03-29T04:10:45.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.530+0000 7f51e7aa7640 1 -- start start 2024-03-29T04:10:45.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.530+0000 7f51e7aa7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51e00ccec0 0x7f51e01693b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:45.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.530+0000 7f51e7aa7640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51e00b56c0 con 0x7f51e015f200 2024-03-29T04:10:45.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.530+0000 7f51e7aa7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51e00cfdd0 con 0x7f51e015f5e0 2024-03-29T04:10:45.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.530+0000 7f51e7aa7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f51e00d09e0 con 0x7f51e00ccec0 2024-03-29T04:10:45.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.530+0000 7f51e5843640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f51e015f200 0x7f51e00b4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50276/0 (socket says 172.21.15.148:50276) 2024-03-29T04:10:45.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.530+0000 7f51e5843640 1 -- 172.21.15.148:0/2819906945 learned_addr learned my addr 172.21.15.148:0/2819906945 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:45.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.531+0000 7f51e6044640 1 --2- 172.21.15.148:0/2819906945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51e00ccec0 0x7f51e01693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:45.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.531+0000 7f51e4841640 1 -- 172.21.15.148:0/2819906945 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3774521824 0 0) 0x7f51e00b56c0 con 0x7f51e015f200 2024-03-29T04:10:45.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.531+0000 7f51e4841640 1 -- 172.21.15.148:0/2819906945 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51cc003520 con 0x7f51e015f200 2024-03-29T04:10:45.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.531+0000 7f51e4841640 1 -- 172.21.15.148:0/2819906945 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3955880470 0 0) 0x7f51e00cfdd0 con 0x7f51e015f5e0 2024-03-29T04:10:45.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.531+0000 7f51e4841640 1 -- 172.21.15.148:0/2819906945 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51e00b56c0 con 0x7f51e015f5e0 2024-03-29T04:10:45.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.531+0000 7f51e6044640 1 -- 172.21.15.148:0/2819906945 >> v1:172.21.15.157:6789/0 conn(0x7f51e015f5e0 legacy=0x7f51e00cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:45.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.531+0000 7f51e6044640 1 -- 172.21.15.148:0/2819906945 >> v1:172.21.15.148:6789/0 conn(0x7f51e015f200 legacy=0x7f51e00b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:45.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.531+0000 7f51e6044640 1 -- 172.21.15.148:0/2819906945 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e00d00b0 con 0x7f51e00ccec0 2024-03-29T04:10:45.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.532+0000 7f51e6044640 1 --2- 172.21.15.148:0/2819906945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51e00ccec0 0x7f51e01693b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009dc0 tx=0x7f51d802bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c787b59c1f3c968 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:45.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.532+0000 7f51e4841640 1 -- 172.21.15.148:0/2819906945 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8039070 con 0x7f51e00ccec0 2024-03-29T04:10:45.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.532+0000 7f51e4841640 1 -- 172.21.15.148:0/2819906945 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51d802ccd0 con 0x7f51e00ccec0 2024-03-29T04:10:45.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.532+0000 7f51e4841640 1 -- 172.21.15.148:0/2819906945 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8034660 con 0x7f51e00ccec0 2024-03-29T04:10:45.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.533+0000 7f51e7aa7640 1 -- 172.21.15.148:0/2819906945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51e00ccec0 msgr2=0x7f51e01693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:45.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.533+0000 7f51e7aa7640 1 --2- 172.21.15.148:0/2819906945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51e00ccec0 0x7f51e01693b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009dc0 tx=0x7f51d802bf20 comp rx=0 tx=0).stop 2024-03-29T04:10:45.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.533+0000 7f51e7aa7640 1 -- 172.21.15.148:0/2819906945 shutdown_connections 2024-03-29T04:10:45.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.533+0000 7f51e7aa7640 1 --2- 172.21.15.148:0/2819906945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51e00ccec0 0x7f51e01693b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.533+0000 7f51e7aa7640 1 -- 172.21.15.148:0/2819906945 >> 172.21.15.148:0/2819906945 conn(0x7f51e00b3c70 msgr2=0x7f51e00b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:45.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.533+0000 7f51e7aa7640 1 -- 172.21.15.148:0/2819906945 shutdown_connections 2024-03-29T04:10:45.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.533+0000 7f51e7aa7640 1 -- 172.21.15.148:0/2819906945 wait complete. 2024-03-29T04:10:45.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.534+0000 7f51e7aa7640 1 Processor -- start 2024-03-29T04:10:45.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.534+0000 7f51e7aa7640 1 -- start start 2024-03-29T04:10:45.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.535+0000 7f51e7aa7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51e00ccec0 0x7f51e014c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:45.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.535+0000 7f51e7aa7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51e015f200 0x7f51e0154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:45.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.535+0000 7f51e7aa7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51e015f5e0 0x7f51e0151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:45.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.535+0000 7f51e7aa7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f51e00b7b40 con 0x7f51e015f200 2024-03-29T04:10:45.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.535+0000 7f51e7aa7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f51e00d0670 con 0x7f51e00ccec0 2024-03-29T04:10:45.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.535+0000 7f51e7aa7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f51e00b7cc0 con 0x7f51e015f5e0 2024-03-29T04:10:45.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.535+0000 7f51e5042640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51e015f200 0x7f51e0154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:45.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.535+0000 7f51e5042640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51e015f200 0x7f51e0154380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53128/0 (socket says 172.21.15.148:53128) 2024-03-29T04:10:45.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.535+0000 7f51e5042640 1 -- 172.21.15.148:0/2487349425 learned_addr learned my addr 172.21.15.148:0/2487349425 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:45.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.535+0000 7f51e6044640 1 --2- 172.21.15.148:0/2487349425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51e015f5e0 0x7f51e0151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:45.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.535+0000 7f51e5843640 1 --2- 172.21.15.148:0/2487349425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51e00ccec0 0x7f51e014c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:45.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.535+0000 7f51e5042640 1 -- 172.21.15.148:0/2487349425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51e015f5e0 msgr2=0x7f51e0151d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:45.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.535+0000 7f51e5042640 1 --2- 172.21.15.148:0/2487349425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51e015f5e0 0x7f51e0151d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.535+0000 7f51e5042640 1 -- 172.21.15.148:0/2487349425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51e00ccec0 msgr2=0x7f51e014c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:45.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.535+0000 7f51e5042640 1 --2- 172.21.15.148:0/2487349425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51e00ccec0 0x7f51e014c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.535+0000 7f51e5042640 1 -- 172.21.15.148:0/2487349425 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e01619b0 con 0x7f51e015f200 2024-03-29T04:10:45.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.535+0000 7f51e6044640 1 --2- 172.21.15.148:0/2487349425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51e015f5e0 0x7f51e0151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:45.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.535+0000 7f51e5843640 1 --2- 172.21.15.148:0/2487349425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51e00ccec0 0x7f51e014c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:45.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.535+0000 7f51e5042640 1 --2- 172.21.15.148:0/2487349425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51e015f200 0x7f51e0154380 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00c460 tx=0x7f51dc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:45.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.535+0000 7f51d67fc640 1 -- 172.21.15.148:0/2487349425 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc017070 con 0x7f51e015f200 2024-03-29T04:10:45.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.535+0000 7f51d67fc640 1 -- 172.21.15.148:0/2487349425 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51dc0052f0 con 0x7f51e015f200 2024-03-29T04:10:45.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.535+0000 7f51e7aa7640 1 -- 172.21.15.148:0/2487349425 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51e0161c90 con 0x7f51e015f200 2024-03-29T04:10:45.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.535+0000 7f51d67fc640 1 -- 172.21.15.148:0/2487349425 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc003830 con 0x7f51e015f200 2024-03-29T04:10:45.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.535+0000 7f51e7aa7640 1 -- 172.21.15.148:0/2487349425 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51e0162250 con 0x7f51e015f200 2024-03-29T04:10:45.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.536+0000 7f51e7aa7640 1 -- 172.21.15.148:0/2487349425 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51e00758d0 con 0x7f51e015f200 2024-03-29T04:10:45.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.537+0000 7f51d67fc640 1 -- 172.21.15.148:0/2487349425 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f51dc01a070 con 0x7f51e015f200 2024-03-29T04:10:45.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.537+0000 7f51d67fc640 1 --2- 172.21.15.148:0/2487349425 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f51c406c530 0x7f51c406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:45.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.537+0000 7f51d67fc640 1 -- 172.21.15.148:0/2487349425 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f51dc08aef0 con 0x7f51e015f200 2024-03-29T04:10:45.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.537+0000 7f51e5843640 1 --2- 172.21.15.148:0/2487349425 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f51c406c530 0x7f51c406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:45.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.538+0000 7f51e5843640 1 --2- 172.21.15.148:0/2487349425 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f51c406c530 0x7f51c406e9f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f51d00011e0 tx=0x7f51d0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:45.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.546+0000 7f51d67fc640 1 -- 172.21.15.148:0/2487349425 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f51dc059110 con 0x7f51e015f200 2024-03-29T04:10:45.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.724+0000 7f51e7aa7640 1 -- 172.21.15.148:0/2487349425 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f51e00ceda0 con 0x7f51e015f200 2024-03-29T04:10:45.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.725+0000 7f51d67fc640 1 -- 172.21.15.148:0/2487349425 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f51e00ceda0 con 0x7f51e015f200 2024-03-29T04:10:45.704 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:10:45.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.727+0000 7f51e7aa7640 1 -- 172.21.15.148:0/2487349425 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f51c406c530 msgr2=0x7f51c406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:45.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.727+0000 7f51e7aa7640 1 --2- 172.21.15.148:0/2487349425 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f51c406c530 0x7f51c406e9f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f51d00011e0 tx=0x7f51d0009040 comp rx=0 tx=0).stop 2024-03-29T04:10:45.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.727+0000 7f51e7aa7640 1 -- 172.21.15.148:0/2487349425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51e015f200 msgr2=0x7f51e0154380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:45.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.727+0000 7f51e7aa7640 1 --2- 172.21.15.148:0/2487349425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51e015f200 0x7f51e0154380 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00c460 tx=0x7f51dc00c920 comp rx=0 tx=0).stop 2024-03-29T04:10:45.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.727+0000 7f51e7aa7640 1 -- 172.21.15.148:0/2487349425 shutdown_connections 2024-03-29T04:10:45.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.727+0000 7f51e7aa7640 1 --2- 172.21.15.148:0/2487349425 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f51c406c530 0x7f51c406e9f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.727+0000 7f51e7aa7640 1 --2- 172.21.15.148:0/2487349425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51e015f5e0 0x7f51e0151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.727+0000 7f51e7aa7640 1 --2- 172.21.15.148:0/2487349425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51e015f200 0x7f51e0154380 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.727+0000 7f51e7aa7640 1 --2- 172.21.15.148:0/2487349425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51e00ccec0 0x7f51e014c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.727+0000 7f51e7aa7640 1 -- 172.21.15.148:0/2487349425 >> 172.21.15.148:0/2487349425 conn(0x7f51e00b3c70 msgr2=0x7f51e00b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:45.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.727+0000 7f51e7aa7640 1 -- 172.21.15.148:0/2487349425 shutdown_connections 2024-03-29T04:10:45.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.727+0000 7f51e7aa7640 1 -- 172.21.15.148:0/2487349425 wait complete. 2024-03-29T04:10:45.719 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:45.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.844+0000 7fda00028640 1 Processor -- start 2024-03-29T04:10:45.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.844+0000 7fda00028640 1 -- start start 2024-03-29T04:10:45.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.845+0000 7fda00028640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd9f815e380 0x7fd9f80afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:45.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.845+0000 7fda00028640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9f8162d50 con 0x7fd9f80b5e90 2024-03-29T04:10:45.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.845+0000 7fda00028640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9f806b930 con 0x7fd9f80b4fa0 2024-03-29T04:10:45.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.845+0000 7fda00028640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fd9f806bc10 con 0x7fd9f815e380 2024-03-29T04:10:45.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.845+0000 7fd9fddc4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd9f815e380 0x7fd9f80afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:45.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.845+0000 7fd9fd5c3640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd9f80b5e90 0x7fd9f8161f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50284/0 (socket says 172.21.15.148:50284) 2024-03-29T04:10:45.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.845+0000 7fd9fd5c3640 1 -- 172.21.15.148:0/1966937248 learned_addr learned my addr 172.21.15.148:0/1966937248 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:45.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.846+0000 7fd9fddc4640 1 -- 172.21.15.148:0/1966937248 >> v1:172.21.15.157:6789/0 conn(0x7fd9f80b4fa0 legacy=0x7fd9f8162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:45.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.846+0000 7fd9fddc4640 1 -- 172.21.15.148:0/1966937248 >> v1:172.21.15.148:6789/0 conn(0x7fd9f80b5e90 legacy=0x7fd9f8161f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:45.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.846+0000 7fd9fddc4640 1 -- 172.21.15.148:0/1966937248 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f806bef0 con 0x7fd9f815e380 2024-03-29T04:10:45.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.846+0000 7fd9fcdc2640 1 -- 172.21.15.148:0/1966937248 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1355709528 0 0) 0x7fd9f8162d50 con 0x7fd9f80b5e90 2024-03-29T04:10:45.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.846+0000 7fd9fddc4640 1 --2- 172.21.15.148:0/1966937248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f815e380 0x7fd9f80afeb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd9f40097a0 tx=0x7fd9f402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7130368a6408055 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:45.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.847+0000 7fd9fcdc2640 1 -- 172.21.15.148:0/1966937248 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4035020 con 0x7fd9f815e380 2024-03-29T04:10:45.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.847+0000 7fd9fcdc2640 1 -- 172.21.15.148:0/1966937248 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9f4031830 con 0x7fd9f815e380 2024-03-29T04:10:45.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.847+0000 7fd9fcdc2640 1 -- 172.21.15.148:0/1966937248 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4031c00 con 0x7fd9f815e380 2024-03-29T04:10:45.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.847+0000 7fda00028640 1 -- 172.21.15.148:0/1966937248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f815e380 msgr2=0x7fd9f80afeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:45.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.847+0000 7fda00028640 1 --2- 172.21.15.148:0/1966937248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f815e380 0x7fd9f80afeb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd9f40097a0 tx=0x7fd9f402b8d0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.847+0000 7fda00028640 1 -- 172.21.15.148:0/1966937248 shutdown_connections 2024-03-29T04:10:45.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.847+0000 7fda00028640 1 --2- 172.21.15.148:0/1966937248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f815e380 0x7fd9f80afeb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.847+0000 7fda00028640 1 -- 172.21.15.148:0/1966937248 >> 172.21.15.148:0/1966937248 conn(0x7fd9f80b2c60 msgr2=0x7fd9f80b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:45.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.847+0000 7fda00028640 1 -- 172.21.15.148:0/1966937248 shutdown_connections 2024-03-29T04:10:45.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.847+0000 7fda00028640 1 -- 172.21.15.148:0/1966937248 wait complete. 2024-03-29T04:10:45.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.848+0000 7fda00028640 1 Processor -- start 2024-03-29T04:10:45.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.848+0000 7fda00028640 1 -- start start 2024-03-29T04:10:45.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.849+0000 7fda00028640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9f80b4fa0 0x7fd9f8158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:45.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.849+0000 7fda00028640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f80b5e90 0x7fd9f815e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:45.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.849+0000 7fda00028640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9f815e380 0x7fd9f81489e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:45.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.849+0000 7fda00028640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f806b5b0 con 0x7fd9f80b5e90 2024-03-29T04:10:45.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.849+0000 7fda00028640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f815e760 con 0x7fd9f815e380 2024-03-29T04:10:45.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.849+0000 7fda00028640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd9f806c9d0 con 0x7fd9f80b4fa0 2024-03-29T04:10:45.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.849+0000 7fd9fd5c3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f80b5e90 0x7fd9f815e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:45.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.849+0000 7fd9fd5c3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f80b5e90 0x7fd9f815e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53146/0 (socket says 172.21.15.148:53146) 2024-03-29T04:10:45.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.849+0000 7fd9fd5c3640 1 -- 172.21.15.148:0/3604276737 learned_addr learned my addr 172.21.15.148:0/3604276737 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:45.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.849+0000 7fd9fe5c5640 1 --2- 172.21.15.148:0/3604276737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9f815e380 0x7fd9f81489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:45.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.850+0000 7fd9fddc4640 1 --2- 172.21.15.148:0/3604276737 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9f80b4fa0 0x7fd9f8158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:45.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.850+0000 7fd9fd5c3640 1 -- 172.21.15.148:0/3604276737 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9f80b4fa0 msgr2=0x7fd9f8158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:45.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.850+0000 7fd9fd5c3640 1 --2- 172.21.15.148:0/3604276737 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9f80b4fa0 0x7fd9f8158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.850+0000 7fd9fd5c3640 1 -- 172.21.15.148:0/3604276737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9f815e380 msgr2=0x7fd9f81489e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:45.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.850+0000 7fd9fd5c3640 1 --2- 172.21.15.148:0/3604276737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9f815e380 0x7fd9f81489e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.850+0000 7fd9fd5c3640 1 -- 172.21.15.148:0/3604276737 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f815ee90 con 0x7fd9f80b5e90 2024-03-29T04:10:45.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.850+0000 7fd9fe5c5640 1 --2- 172.21.15.148:0/3604276737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9f815e380 0x7fd9f81489e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:45.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.850+0000 7fd9fddc4640 1 --2- 172.21.15.148:0/3604276737 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9f80b4fa0 0x7fd9f8158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:45.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.850+0000 7fd9fd5c3640 1 --2- 172.21.15.148:0/3604276737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f80b5e90 0x7fd9f815e950 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800c460 tx=0x7fd9e800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:45.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.850+0000 7fd9ee7fc640 1 -- 172.21.15.148:0/3604276737 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e8018070 con 0x7fd9f80b5e90 2024-03-29T04:10:45.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.850+0000 7fd9ee7fc640 1 -- 172.21.15.148:0/3604276737 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9e8007520 con 0x7fd9f80b5e90 2024-03-29T04:10:45.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.850+0000 7fda00028640 1 -- 172.21.15.148:0/3604276737 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f81592e0 con 0x7fd9f80b5e90 2024-03-29T04:10:45.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.850+0000 7fd9ee7fc640 1 -- 172.21.15.148:0/3604276737 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e8004a20 con 0x7fd9f80b5e90 2024-03-29T04:10:45.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.850+0000 7fda00028640 1 -- 172.21.15.148:0/3604276737 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f81624b0 con 0x7fd9f80b5e90 2024-03-29T04:10:45.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.851+0000 7fda00028640 1 -- 172.21.15.148:0/3604276737 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f80687e0 con 0x7fd9f80b5e90 2024-03-29T04:10:45.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.851+0000 7fd9ee7fc640 1 -- 172.21.15.148:0/3604276737 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd9e8004bc0 con 0x7fd9f80b5e90 2024-03-29T04:10:45.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.852+0000 7fd9ee7fc640 1 --2- 172.21.15.148:0/3604276737 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd9dc06c530 0x7fd9dc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:45.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.852+0000 7fd9ee7fc640 1 -- 172.21.15.148:0/3604276737 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd9e808ace0 con 0x7fd9f80b5e90 2024-03-29T04:10:45.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.852+0000 7fd9fddc4640 1 --2- 172.21.15.148:0/3604276737 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd9dc06c530 0x7fd9dc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:45.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.852+0000 7fd9fddc4640 1 --2- 172.21.15.148:0/3604276737 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd9dc06c530 0x7fd9dc06e9f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4036000 tx=0x7fd9f4008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:45.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:45.856+0000 7fd9ee7fc640 1 -- 172.21.15.148:0/3604276737 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd9e80588c0 con 0x7fd9f80b5e90 2024-03-29T04:10:46.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.039+0000 7fda00028640 1 -- 172.21.15.148:0/3604276737 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9f815d100 con 0x7fd9f80b5e90 2024-03-29T04:10:46.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.040+0000 7fd9ee7fc640 1 -- 172.21.15.148:0/3604276737 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18858 (secure 0 0 0) 0x7fd9e805cbc0 con 0x7fd9f80b5e90 2024-03-29T04:10:46.020 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:46.020 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:41.662934+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:46.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.042+0000 7fda00028640 1 -- 172.21.15.148:0/3604276737 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd9dc06c530 msgr2=0x7fd9dc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:46.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.042+0000 7fda00028640 1 --2- 172.21.15.148:0/3604276737 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd9dc06c530 0x7fd9dc06e9f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4036000 tx=0x7fd9f4008030 comp rx=0 tx=0).stop 2024-03-29T04:10:46.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.042+0000 7fda00028640 1 -- 172.21.15.148:0/3604276737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f80b5e90 msgr2=0x7fd9f815e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:46.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.042+0000 7fda00028640 1 --2- 172.21.15.148:0/3604276737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f80b5e90 0x7fd9f815e950 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800c460 tx=0x7fd9e800c920 comp rx=0 tx=0).stop 2024-03-29T04:10:46.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.042+0000 7fda00028640 1 -- 172.21.15.148:0/3604276737 shutdown_connections 2024-03-29T04:10:46.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.042+0000 7fda00028640 1 --2- 172.21.15.148:0/3604276737 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd9dc06c530 0x7fd9dc06e9f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.042+0000 7fda00028640 1 --2- 172.21.15.148:0/3604276737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9f815e380 0x7fd9f81489e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.042+0000 7fda00028640 1 --2- 172.21.15.148:0/3604276737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f80b5e90 0x7fd9f815e950 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.042+0000 7fda00028640 1 --2- 172.21.15.148:0/3604276737 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9f80b4fa0 0x7fd9f8158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.042+0000 7fda00028640 1 -- 172.21.15.148:0/3604276737 >> 172.21.15.148:0/3604276737 conn(0x7fd9f80b2c60 msgr2=0x7fd9f815d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:46.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.042+0000 7fda00028640 1 -- 172.21.15.148:0/3604276737 shutdown_connections 2024-03-29T04:10:46.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.042+0000 7fda00028640 1 -- 172.21.15.148:0/3604276737 wait complete. 2024-03-29T04:10:46.032 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:47.898351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:57.922313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:58.829405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:10:02.551763+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:10:46.033 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:10:46.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.162+0000 7efd1f874640 1 Processor -- start 2024-03-29T04:10:46.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.162+0000 7efd1f874640 1 -- start start 2024-03-29T04:10:46.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.163+0000 7efd1f874640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd1806ccd0 0x7efd180b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:46.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.163+0000 7efd1f874640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd180b3d60 con 0x7efd18073120 2024-03-29T04:10:46.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.163+0000 7efd1f874640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd180b34c0 con 0x7efd18072d40 2024-03-29T04:10:46.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.163+0000 7efd1f874640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efd1815ee60 con 0x7efd1806ccd0 2024-03-29T04:10:46.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.163+0000 7efd1ce0f640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7efd18073120 0x7efd1806c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50296/0 (socket says 172.21.15.148:50296) 2024-03-29T04:10:46.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.163+0000 7efd1ce0f640 1 -- 172.21.15.148:0/856048714 learned_addr learned my addr 172.21.15.148:0/856048714 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:46.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.163+0000 7efd0ffff640 1 -- 172.21.15.148:0/856048714 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1984098564 0 0) 0x7efd180b3d60 con 0x7efd18073120 2024-03-29T04:10:46.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.163+0000 7efd0ffff640 1 -- 172.21.15.148:0/856048714 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd04003520 con 0x7efd18073120 2024-03-29T04:10:46.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.163+0000 7efd1de11640 1 --2- 172.21.15.148:0/856048714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd1806ccd0 0x7efd180b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:46.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.163+0000 7efd0ffff640 1 -- 172.21.15.148:0/856048714 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 992096971 0 0) 0x7efd04003520 con 0x7efd18073120 2024-03-29T04:10:46.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.163+0000 7efd0ffff640 1 -- 172.21.15.148:0/856048714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd1806ccd0 msgr2=0x7efd180b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:46.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.163+0000 7efd0ffff640 1 --2- 172.21.15.148:0/856048714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd1806ccd0 0x7efd180b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.163+0000 7efd0ffff640 1 -- 172.21.15.148:0/856048714 >> v1:172.21.15.157:6789/0 conn(0x7efd18072d40 legacy=0x7efd1815b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:46.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.163+0000 7efd0ffff640 1 -- 172.21.15.148:0/856048714 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd180b37a0 con 0x7efd18073120 2024-03-29T04:10:46.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.163+0000 7efd0ffff640 1 -- 172.21.15.148:0/856048714 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7efd08001670 con 0x7efd18073120 2024-03-29T04:10:46.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.163+0000 7efd0ffff640 1 -- 172.21.15.148:0/856048714 >> v1:172.21.15.148:6789/0 conn(0x7efd18073120 legacy=0x7efd1806c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:46.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.163+0000 7efd0ffff640 1 --2- 172.21.15.148:0/856048714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd040039e0 0x7efd04003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:46.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.163+0000 7efd0ffff640 1 --2- 172.21.15.148:0/856048714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd1806ccd0 0x7efd04004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:46.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.163+0000 7efd0ffff640 1 --2- 172.21.15.148:0/856048714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd04004d40 0x7efd040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:46.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.164+0000 7efd0ffff640 1 -- 172.21.15.148:0/856048714 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efd1815ee60 con 0x7efd04004d40 2024-03-29T04:10:46.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.164+0000 7efd0ffff640 1 -- 172.21.15.148:0/856048714 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efd0400baa0 con 0x7efd040039e0 2024-03-29T04:10:46.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.164+0000 7efd0ffff640 1 -- 172.21.15.148:0/856048714 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efd0400bca0 con 0x7efd1806ccd0 2024-03-29T04:10:46.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.164+0000 7efd1de11640 1 --2- 172.21.15.148:0/856048714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd04004d40 0x7efd040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:46.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.164+0000 7efd1d610640 1 --2- 172.21.15.148:0/856048714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd040039e0 0x7efd04003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:46.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.164+0000 7efd1ce0f640 1 --2- 172.21.15.148:0/856048714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd1806ccd0 0x7efd04004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:46.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.164+0000 7efd1de11640 1 -- 172.21.15.148:0/856048714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd1806ccd0 msgr2=0x7efd04004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:46.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.164+0000 7efd1de11640 1 --2- 172.21.15.148:0/856048714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd1806ccd0 0x7efd04004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.164+0000 7efd1de11640 1 -- 172.21.15.148:0/856048714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd040039e0 msgr2=0x7efd04003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:46.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.164+0000 7efd1de11640 1 --2- 172.21.15.148:0/856048714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd040039e0 0x7efd04003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.164+0000 7efd1de11640 1 -- 172.21.15.148:0/856048714 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd04003520 con 0x7efd04004d40 2024-03-29T04:10:46.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.165+0000 7efd1de11640 1 --2- 172.21.15.148:0/856048714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd04004d40 0x7efd040051c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efd1815c8b0 tx=0x7efd04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1dcd94bed0b779a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:46.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.165+0000 7efd0ffff640 1 -- 172.21.15.148:0/856048714 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10039070 con 0x7efd04004d40 2024-03-29T04:10:46.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.165+0000 7efd0ffff640 1 -- 172.21.15.148:0/856048714 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd1002cb90 con 0x7efd04004d40 2024-03-29T04:10:46.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.165+0000 7efd0ffff640 1 -- 172.21.15.148:0/856048714 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10009c10 con 0x7efd04004d40 2024-03-29T04:10:46.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.165+0000 7efd1f874640 1 -- 172.21.15.148:0/856048714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd04004d40 msgr2=0x7efd040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:46.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.165+0000 7efd1f874640 1 --2- 172.21.15.148:0/856048714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd04004d40 0x7efd040051c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efd1815c8b0 tx=0x7efd04005830 comp rx=0 tx=0).stop 2024-03-29T04:10:46.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.165+0000 7efd1de11640 1 -- 172.21.15.148:0/856048714 reap_dead start 2024-03-29T04:10:46.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.165+0000 7efd1f874640 1 -- 172.21.15.148:0/856048714 shutdown_connections 2024-03-29T04:10:46.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.165+0000 7efd1f874640 1 -- 172.21.15.148:0/856048714 >> 172.21.15.148:0/856048714 conn(0x7efd180d00a0 msgr2=0x7efd180d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:46.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.166+0000 7efd1f874640 1 -- 172.21.15.148:0/856048714 shutdown_connections 2024-03-29T04:10:46.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.166+0000 7efd1f874640 1 -- 172.21.15.148:0/856048714 wait complete. 2024-03-29T04:10:46.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.167+0000 7efd1f874640 1 Processor -- start 2024-03-29T04:10:46.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.167+0000 7efd1f874640 1 -- start start 2024-03-29T04:10:46.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.167+0000 7efd1f874640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd18151b00 0x7efd1813cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:46.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.167+0000 7efd1f874640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd18157760 0x7efd18157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:46.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.167+0000 7efd1f874640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd18159340 0x7efd181597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:46.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.167+0000 7efd1f874640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efd1815bf70 con 0x7efd18151b00 2024-03-29T04:10:46.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.167+0000 7efd1f874640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efd1815bdf0 con 0x7efd18157760 2024-03-29T04:10:46.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.167+0000 7efd1f874640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efd180b2e70 con 0x7efd18159340 2024-03-29T04:10:46.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.167+0000 7efd1d610640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd18151b00 0x7efd1813cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:46.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.168+0000 7efd1d610640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd18151b00 0x7efd1813cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53162/0 (socket says 172.21.15.148:53162) 2024-03-29T04:10:46.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.168+0000 7efd1d610640 1 -- 172.21.15.148:0/1903055148 learned_addr learned my addr 172.21.15.148:0/1903055148 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:46.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.168+0000 7efd1ce0f640 1 --2- 172.21.15.148:0/1903055148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd18157760 0x7efd18157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:46.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.168+0000 7efd1de11640 1 --2- 172.21.15.148:0/1903055148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd18159340 0x7efd181597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:46.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.168+0000 7efd1d610640 1 -- 172.21.15.148:0/1903055148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd18159340 msgr2=0x7efd181597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:46.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.168+0000 7efd1d610640 1 --2- 172.21.15.148:0/1903055148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd18159340 0x7efd181597d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.168+0000 7efd1d610640 1 -- 172.21.15.148:0/1903055148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd18157760 msgr2=0x7efd18157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:46.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.168+0000 7efd1d610640 1 --2- 172.21.15.148:0/1903055148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd18157760 0x7efd18157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.168+0000 7efd1d610640 1 -- 172.21.15.148:0/1903055148 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1815ae60 con 0x7efd18151b00 2024-03-29T04:10:46.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.168+0000 7efd1d610640 1 --2- 172.21.15.148:0/1903055148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd18151b00 0x7efd1813cd80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efd1400f820 tx=0x7efd1400fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:46.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.168+0000 7efd0dffb640 1 -- 172.21.15.148:0/1903055148 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd140050d0 con 0x7efd18151b00 2024-03-29T04:10:46.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.168+0000 7efd0dffb640 1 -- 172.21.15.148:0/1903055148 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd14018070 con 0x7efd18151b00 2024-03-29T04:10:46.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.168+0000 7efd0dffb640 1 -- 172.21.15.148:0/1903055148 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14005490 con 0x7efd18151b00 2024-03-29T04:10:46.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.168+0000 7efd1f874640 1 -- 172.21.15.148:0/1903055148 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd1815a8a0 con 0x7efd18151b00 2024-03-29T04:10:46.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.168+0000 7efd1f874640 1 -- 172.21.15.148:0/1903055148 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd1813e6d0 con 0x7efd18151b00 2024-03-29T04:10:46.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.169+0000 7efd1f874640 1 -- 172.21.15.148:0/1903055148 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd18068850 con 0x7efd18151b00 2024-03-29T04:10:46.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.170+0000 7efd0dffb640 1 -- 172.21.15.148:0/1903055148 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efd1400d730 con 0x7efd18151b00 2024-03-29T04:10:46.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.170+0000 7efd0dffb640 1 --2- 172.21.15.148:0/1903055148 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efcfc06c530 0x7efcfc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:46.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.170+0000 7efd0dffb640 1 -- 172.21.15.148:0/1903055148 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7efd1408b9a0 con 0x7efd18151b00 2024-03-29T04:10:46.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.171+0000 7efd1ce0f640 1 --2- 172.21.15.148:0/1903055148 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efcfc06c530 0x7efcfc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:46.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.171+0000 7efd1ce0f640 1 --2- 172.21.15.148:0/1903055148 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efcfc06c530 0x7efcfc06e9f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efd08001eb0 tx=0x7efd08009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:46.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.179+0000 7efd0dffb640 1 -- 172.21.15.148:0/1903055148 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efd14059bc0 con 0x7efd18151b00 2024-03-29T04:10:46.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.355+0000 7efd1f874640 1 -- 172.21.15.148:0/1903055148 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efd1813eab0 con 0x7efd18151b00 2024-03-29T04:10:46.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.355+0000 7efd0dffb640 1 -- 172.21.15.148:0/1903055148 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7efd140594c0 con 0x7efd18151b00 2024-03-29T04:10:46.334 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-29T04:10:46.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.357+0000 7efd1f874640 1 -- 172.21.15.148:0/1903055148 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efcfc06c530 msgr2=0x7efcfc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:46.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.357+0000 7efd1f874640 1 --2- 172.21.15.148:0/1903055148 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efcfc06c530 0x7efcfc06e9f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efd08001eb0 tx=0x7efd08009040 comp rx=0 tx=0).stop 2024-03-29T04:10:46.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.357+0000 7efd1f874640 1 -- 172.21.15.148:0/1903055148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd18151b00 msgr2=0x7efd1813cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:46.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.357+0000 7efd1f874640 1 --2- 172.21.15.148:0/1903055148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd18151b00 0x7efd1813cd80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efd1400f820 tx=0x7efd1400fce0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.357+0000 7efd1f874640 1 -- 172.21.15.148:0/1903055148 shutdown_connections 2024-03-29T04:10:46.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.357+0000 7efd1f874640 1 --2- 172.21.15.148:0/1903055148 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efcfc06c530 0x7efcfc06e9f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.357+0000 7efd1f874640 1 --2- 172.21.15.148:0/1903055148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd18159340 0x7efd181597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.357+0000 7efd1f874640 1 --2- 172.21.15.148:0/1903055148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd18157760 0x7efd18157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.357+0000 7efd1f874640 1 --2- 172.21.15.148:0/1903055148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd18151b00 0x7efd1813cd80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.357+0000 7efd1f874640 1 -- 172.21.15.148:0/1903055148 >> 172.21.15.148:0/1903055148 conn(0x7efd180d00a0 msgr2=0x7efd180ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:46.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.357+0000 7efd1f874640 1 -- 172.21.15.148:0/1903055148 shutdown_connections 2024-03-29T04:10:46.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.357+0000 7efd1f874640 1 -- 172.21.15.148:0/1903055148 wait complete. 2024-03-29T04:10:46.351 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:10:46.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.485+0000 7fcf1be63640 1 Processor -- start 2024-03-29T04:10:46.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.485+0000 7fcf1be63640 1 -- start start 2024-03-29T04:10:46.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.486+0000 7fcf1be63640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcf140b2f00 0x7fcf14162680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:46.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.486+0000 7fcf1be63640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcf140b3430 0x7fcf1406b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:46.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.486+0000 7fcf1be63640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf140cf330 con 0x7fcf1415e3c0 2024-03-29T04:10:46.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.486+0000 7fcf1be63640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fcf140cf960 con 0x7fcf140b2f00 2024-03-29T04:10:46.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.486+0000 7fcf1be63640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fcf14162cf0 con 0x7fcf140b3430 2024-03-29T04:10:46.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.486+0000 7fcf193fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcf140b2f00 0x7fcf14162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:46.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.486+0000 7fcf193fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcf140b2f00 0x7fcf14162680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53166/0 (socket says 172.21.15.148:53166) 2024-03-29T04:10:46.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.486+0000 7fcf193fe640 1 -- 172.21.15.148:0/1656018585 learned_addr learned my addr 172.21.15.148:0/1656018585 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:46.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.486+0000 7fcf1a400640 1 --2- 172.21.15.148:0/1656018585 >> v2:172.21.15.157:3300/0 conn(0x7fcf140b3430 0x7fcf1406b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:46.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.486+0000 7fcf193fe640 1 -- 172.21.15.148:0/1656018585 >> v2:172.21.15.157:3300/0 conn(0x7fcf140b3430 msgr2=0x7fcf1406b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:46.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.486+0000 7fcf193fe640 1 --2- 172.21.15.148:0/1656018585 >> v2:172.21.15.157:3300/0 conn(0x7fcf140b3430 0x7fcf1406b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.486+0000 7fcf193fe640 1 -- 172.21.15.148:0/1656018585 >> v1:172.21.15.157:6789/0 conn(0x7fcf1415e3c0 legacy=0x7fcf14161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:46.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.486+0000 7fcf193fe640 1 -- 172.21.15.148:0/1656018585 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf140cf5f0 con 0x7fcf140b2f00 2024-03-29T04:10:46.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.487+0000 7fcf193fe640 1 --2- 172.21.15.148:0/1656018585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf140b2f00 0x7fcf14162680 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcf140cf8d0 tx=0x7fcf0802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f2c12dc508d91d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:46.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.487+0000 7fcf18bfd640 1 -- 172.21.15.148:0/1656018585 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf08035020 con 0x7fcf140b2f00 2024-03-29T04:10:46.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.487+0000 7fcf18bfd640 1 -- 172.21.15.148:0/1656018585 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf08031870 con 0x7fcf140b2f00 2024-03-29T04:10:46.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.488+0000 7fcf18bfd640 1 -- 172.21.15.148:0/1656018585 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf08031c40 con 0x7fcf140b2f00 2024-03-29T04:10:46.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.488+0000 7fcf1be63640 1 -- 172.21.15.148:0/1656018585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf140b2f00 msgr2=0x7fcf14162680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:46.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.488+0000 7fcf1be63640 1 --2- 172.21.15.148:0/1656018585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf140b2f00 0x7fcf14162680 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcf140cf8d0 tx=0x7fcf0802b910 comp rx=0 tx=0).stop 2024-03-29T04:10:46.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.488+0000 7fcf1be63640 1 -- 172.21.15.148:0/1656018585 shutdown_connections 2024-03-29T04:10:46.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.488+0000 7fcf1be63640 1 --2- 172.21.15.148:0/1656018585 >> v2:172.21.15.157:3300/0 conn(0x7fcf140b3430 0x7fcf1406b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.488+0000 7fcf1be63640 1 --2- 172.21.15.148:0/1656018585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf140b2f00 0x7fcf14162680 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.488+0000 7fcf1be63640 1 -- 172.21.15.148:0/1656018585 >> 172.21.15.148:0/1656018585 conn(0x7fcf140b6430 msgr2=0x7fcf140d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:46.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.488+0000 7fcf1be63640 1 -- 172.21.15.148:0/1656018585 shutdown_connections 2024-03-29T04:10:46.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.488+0000 7fcf1be63640 1 -- 172.21.15.148:0/1656018585 wait complete. 2024-03-29T04:10:46.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.489+0000 7fcf1be63640 1 Processor -- start 2024-03-29T04:10:46.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.490+0000 7fcf1be63640 1 -- start start 2024-03-29T04:10:46.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.490+0000 7fcf1be63640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf140b3430 0x7fcf14159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:46.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.490+0000 7fcf1be63640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf1415e3c0 0x7fcf140d2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:46.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.490+0000 7fcf1be63640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf140d32a0 0x7fcf141608b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:46.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.490+0000 7fcf1be63640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcf140b3b20 con 0x7fcf140b3430 2024-03-29T04:10:46.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.490+0000 7fcf1be63640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcf140b39a0 con 0x7fcf1415e3c0 2024-03-29T04:10:46.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.490+0000 7fcf1be63640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcf1406bd30 con 0x7fcf140d32a0 2024-03-29T04:10:46.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.490+0000 7fcf19bff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf140b3430 0x7fcf14159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:46.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.490+0000 7fcf19bff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf140b3430 0x7fcf14159120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53170/0 (socket says 172.21.15.148:53170) 2024-03-29T04:10:46.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.490+0000 7fcf19bff640 1 -- 172.21.15.148:0/2556903058 learned_addr learned my addr 172.21.15.148:0/2556903058 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:46.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.490+0000 7fcf193fe640 1 --2- 172.21.15.148:0/2556903058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf1415e3c0 0x7fcf140d2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:46.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.490+0000 7fcf1a400640 1 --2- 172.21.15.148:0/2556903058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf140d32a0 0x7fcf141608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:46.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.491+0000 7fcf19bff640 1 -- 172.21.15.148:0/2556903058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf140d32a0 msgr2=0x7fcf141608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:46.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.491+0000 7fcf19bff640 1 --2- 172.21.15.148:0/2556903058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf140d32a0 0x7fcf141608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.491+0000 7fcf19bff640 1 -- 172.21.15.148:0/2556903058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf1415e3c0 msgr2=0x7fcf140d2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:46.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.491+0000 7fcf19bff640 1 --2- 172.21.15.148:0/2556903058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf1415e3c0 0x7fcf140d2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.491+0000 7fcf19bff640 1 -- 172.21.15.148:0/2556903058 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1415ea50 con 0x7fcf140b3430 2024-03-29T04:10:46.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.491+0000 7fcf193fe640 1 --2- 172.21.15.148:0/2556903058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf1415e3c0 0x7fcf140d2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:46.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.491+0000 7fcf19bff640 1 --2- 172.21.15.148:0/2556903058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf140b3430 0x7fcf14159120 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400c730 tx=0x7fcf0400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:46.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.491+0000 7fcf127fc640 1 -- 172.21.15.148:0/2556903058 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf04016020 con 0x7fcf140b3430 2024-03-29T04:10:46.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.491+0000 7fcf127fc640 1 -- 172.21.15.148:0/2556903058 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf04006d50 con 0x7fcf140b3430 2024-03-29T04:10:46.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.491+0000 7fcf1be63640 1 -- 172.21.15.148:0/2556903058 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf1415ed30 con 0x7fcf140b3430 2024-03-29T04:10:46.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.491+0000 7fcf127fc640 1 -- 172.21.15.148:0/2556903058 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf04003bb0 con 0x7fcf140b3430 2024-03-29T04:10:46.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.491+0000 7fcf1be63640 1 -- 172.21.15.148:0/2556903058 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf1415f2f0 con 0x7fcf140b3430 2024-03-29T04:10:46.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.492+0000 7fcf1be63640 1 -- 172.21.15.148:0/2556903058 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf14068850 con 0x7fcf140b3430 2024-03-29T04:10:46.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.493+0000 7fcf127fc640 1 -- 172.21.15.148:0/2556903058 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcf0400a850 con 0x7fcf140b3430 2024-03-29T04:10:46.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.494+0000 7fcf127fc640 1 --2- 172.21.15.148:0/2556903058 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fcf0006c560 0x7fcf0006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:46.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.494+0000 7fcf193fe640 1 --2- 172.21.15.148:0/2556903058 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fcf0006c560 0x7fcf0006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:46.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.494+0000 7fcf127fc640 1 -- 172.21.15.148:0/2556903058 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fcf0408b570 con 0x7fcf140b3430 2024-03-29T04:10:46.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.494+0000 7fcf193fe640 1 --2- 172.21.15.148:0/2556903058 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fcf0006c560 0x7fcf0006ea20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcf08036000 tx=0x7fcf0803c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:46.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.498+0000 7fcf127fc640 1 -- 172.21.15.148:0/2556903058 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcf04020020 con 0x7fcf140b3430 2024-03-29T04:10:46.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.678+0000 7fcf1be63640 1 -- 172.21.15.148:0/2556903058 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcf140d1b40 con 0x7fcf140b3430 2024-03-29T04:10:46.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.679+0000 7fcf127fc640 1 -- 172.21.15.148:0/2556903058 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fcf040591d0 con 0x7fcf140b3430 2024-03-29T04:10:46.658 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:10:46.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.681+0000 7fcf1be63640 1 -- 172.21.15.148:0/2556903058 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fcf0006c560 msgr2=0x7fcf0006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:46.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.681+0000 7fcf1be63640 1 --2- 172.21.15.148:0/2556903058 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fcf0006c560 0x7fcf0006ea20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcf08036000 tx=0x7fcf0803c040 comp rx=0 tx=0).stop 2024-03-29T04:10:46.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.681+0000 7fcf1be63640 1 -- 172.21.15.148:0/2556903058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf140b3430 msgr2=0x7fcf14159120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:46.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.681+0000 7fcf1be63640 1 --2- 172.21.15.148:0/2556903058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf140b3430 0x7fcf14159120 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400c730 tx=0x7fcf0400cbf0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.681+0000 7fcf1be63640 1 -- 172.21.15.148:0/2556903058 shutdown_connections 2024-03-29T04:10:46.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.681+0000 7fcf1be63640 1 --2- 172.21.15.148:0/2556903058 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fcf0006c560 0x7fcf0006ea20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.681+0000 7fcf1be63640 1 --2- 172.21.15.148:0/2556903058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf140d32a0 0x7fcf141608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.681+0000 7fcf1be63640 1 --2- 172.21.15.148:0/2556903058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf1415e3c0 0x7fcf140d2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.681+0000 7fcf1be63640 1 --2- 172.21.15.148:0/2556903058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf140b3430 0x7fcf14159120 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.681+0000 7fcf1be63640 1 -- 172.21.15.148:0/2556903058 >> 172.21.15.148:0/2556903058 conn(0x7fcf140b6430 msgr2=0x7fcf140b5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:46.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.681+0000 7fcf1be63640 1 -- 172.21.15.148:0/2556903058 shutdown_connections 2024-03-29T04:10:46.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.681+0000 7fcf1be63640 1 -- 172.21.15.148:0/2556903058 wait complete. 2024-03-29T04:10:46.671 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:10:46.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.833+0000 7fed79d3d640 1 Processor -- start 2024-03-29T04:10:46.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.833+0000 7fed79d3d640 1 -- start start 2024-03-29T04:10:46.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.834+0000 7fed79d3d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed7415a120 0x7fed740700d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:46.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.834+0000 7fed79d3d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fed740ce310 0x7fed7406e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:46.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.834+0000 7fed79d3d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed7415d460 con 0x7fed7406ee10 2024-03-29T04:10:46.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.834+0000 7fed79d3d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fed740d1b20 con 0x7fed740ce310 2024-03-29T04:10:46.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.834+0000 7fed79d3d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fed740d0f60 con 0x7fed7415a120 2024-03-29T04:10:46.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.834+0000 7fed72ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fed740ce310 0x7fed7406e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:46.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.834+0000 7fed73fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fed7406ee10 0x7fed740b6f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50300/0 (socket says 172.21.15.148:50300) 2024-03-29T04:10:46.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.834+0000 7fed73fff640 1 -- 172.21.15.148:0/2497198870 learned_addr learned my addr 172.21.15.148:0/2497198870 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:46.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.834+0000 7fed737fe640 1 --2- 172.21.15.148:0/2497198870 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed7415a120 0x7fed740700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:46.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.834+0000 7fed72ffd640 1 -- 172.21.15.148:0/2497198870 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed7415a120 msgr2=0x7fed740700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:46.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.834+0000 7fed72ffd640 1 --2- 172.21.15.148:0/2497198870 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed7415a120 0x7fed740700d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.834+0000 7fed72ffd640 1 -- 172.21.15.148:0/2497198870 >> v1:172.21.15.148:6789/0 conn(0x7fed7406ee10 legacy=0x7fed740b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:46.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.834+0000 7fed72ffd640 1 -- 172.21.15.148:0/2497198870 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7415d700 con 0x7fed740ce310 2024-03-29T04:10:46.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.834+0000 7fed727fc640 1 -- 172.21.15.148:0/2497198870 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2349056045 0 0) 0x7fed7415d460 con 0x7fed7406ee10 2024-03-29T04:10:46.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.834+0000 7fed737fe640 1 --2- 172.21.15.148:0/2497198870 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed7415a120 0x7fed740700d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:46.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.835+0000 7fed72ffd640 1 --2- 172.21.15.148:0/2497198870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed740ce310 0x7fed7406e8d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fed640097a0 tx=0x7fed6402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78e8c7221164c23 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:46.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.835+0000 7fed727fc640 1 -- 172.21.15.148:0/2497198870 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed64035020 con 0x7fed740ce310 2024-03-29T04:10:46.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.835+0000 7fed727fc640 1 -- 172.21.15.148:0/2497198870 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed64031830 con 0x7fed740ce310 2024-03-29T04:10:46.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.835+0000 7fed727fc640 1 -- 172.21.15.148:0/2497198870 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed64031c00 con 0x7fed740ce310 2024-03-29T04:10:46.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.836+0000 7fed79d3d640 1 -- 172.21.15.148:0/2497198870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed740ce310 msgr2=0x7fed7406e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:46.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.836+0000 7fed79d3d640 1 --2- 172.21.15.148:0/2497198870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed740ce310 0x7fed7406e8d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fed640097a0 tx=0x7fed6402b8d0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.836+0000 7fed79d3d640 1 -- 172.21.15.148:0/2497198870 shutdown_connections 2024-03-29T04:10:46.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.836+0000 7fed79d3d640 1 --2- 172.21.15.148:0/2497198870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed740ce310 0x7fed7406e8d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.836+0000 7fed79d3d640 1 --2- 172.21.15.148:0/2497198870 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed7415a120 0x7fed740700d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.836+0000 7fed79d3d640 1 -- 172.21.15.148:0/2497198870 >> 172.21.15.148:0/2497198870 conn(0x7fed740cead0 msgr2=0x7fed740ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:46.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.836+0000 7fed79d3d640 1 -- 172.21.15.148:0/2497198870 shutdown_connections 2024-03-29T04:10:46.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.836+0000 7fed79d3d640 1 -- 172.21.15.148:0/2497198870 wait complete. 2024-03-29T04:10:46.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.837+0000 7fed79d3d640 1 Processor -- start 2024-03-29T04:10:46.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.837+0000 7fed79d3d640 1 -- start start 2024-03-29T04:10:46.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.838+0000 7fed79d3d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed7406ee10 0x7fed74153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:46.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.838+0000 7fed79d3d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed740ce310 0x7fed740b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:46.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.838+0000 7fed79d3d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed7415a120 0x7fed740b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:46.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.838+0000 7fed79d3d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fed7406f5c0 con 0x7fed7415a120 2024-03-29T04:10:46.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.838+0000 7fed79d3d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fed740ce7d0 con 0x7fed740ce310 2024-03-29T04:10:46.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.838+0000 7fed79d3d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fed7406f740 con 0x7fed7406ee10 2024-03-29T04:10:46.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.838+0000 7fed73fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed7415a120 0x7fed740b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:46.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.838+0000 7fed73fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed7415a120 0x7fed740b62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53194/0 (socket says 172.21.15.148:53194) 2024-03-29T04:10:46.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.838+0000 7fed73fff640 1 -- 172.21.15.148:0/3043721058 learned_addr learned my addr 172.21.15.148:0/3043721058 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:46.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.838+0000 7fed737fe640 1 --2- 172.21.15.148:0/3043721058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed7406ee10 0x7fed74153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:46.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.838+0000 7fed72ffd640 1 --2- 172.21.15.148:0/3043721058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed740ce310 0x7fed740b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:46.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.838+0000 7fed73fff640 1 -- 172.21.15.148:0/3043721058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed7406ee10 msgr2=0x7fed74153120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:46.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.838+0000 7fed73fff640 1 --2- 172.21.15.148:0/3043721058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed7406ee10 0x7fed74153120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.838+0000 7fed73fff640 1 -- 172.21.15.148:0/3043721058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed740ce310 msgr2=0x7fed740b5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:46.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.838+0000 7fed73fff640 1 --2- 172.21.15.148:0/3043721058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed740ce310 0x7fed740b5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:46.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.838+0000 7fed73fff640 1 -- 172.21.15.148:0/3043721058 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed740c9490 con 0x7fed7415a120 2024-03-29T04:10:46.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.838+0000 7fed72ffd640 1 --2- 172.21.15.148:0/3043721058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed740ce310 0x7fed740b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:46.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.838+0000 7fed737fe640 1 --2- 172.21.15.148:0/3043721058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed7406ee10 0x7fed74153120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:46.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.838+0000 7fed73fff640 1 --2- 172.21.15.148:0/3043721058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed7415a120 0x7fed740b62b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00c410 tx=0x7fed6c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:46.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.839+0000 7fed6bfff640 1 -- 172.21.15.148:0/3043721058 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c0074e0 con 0x7fed7415a120 2024-03-29T04:10:46.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.839+0000 7fed6bfff640 1 -- 172.21.15.148:0/3043721058 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed6c0047e0 con 0x7fed7415a120 2024-03-29T04:10:46.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.839+0000 7fed6bfff640 1 -- 172.21.15.148:0/3043721058 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c004c10 con 0x7fed7415a120 2024-03-29T04:10:46.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.839+0000 7fed79d3d640 1 -- 172.21.15.148:0/3043721058 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed740c9770 con 0x7fed7415a120 2024-03-29T04:10:46.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.839+0000 7fed79d3d640 1 -- 172.21.15.148:0/3043721058 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed740ca010 con 0x7fed7415a120 2024-03-29T04:10:46.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.840+0000 7fed79d3d640 1 -- 172.21.15.148:0/3043721058 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed740c9b60 con 0x7fed7415a120 2024-03-29T04:10:46.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.840+0000 7fed6bfff640 1 -- 172.21.15.148:0/3043721058 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fed6c011070 con 0x7fed7415a120 2024-03-29T04:10:46.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.840+0000 7fed6bfff640 1 --2- 172.21.15.148:0/3043721058 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fed5406c530 0x7fed5406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:46.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.841+0000 7fed6bfff640 1 -- 172.21.15.148:0/3043721058 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fed6c08b570 con 0x7fed7415a120 2024-03-29T04:10:46.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.841+0000 7fed737fe640 1 --2- 172.21.15.148:0/3043721058 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fed5406c530 0x7fed5406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:46.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.841+0000 7fed737fe640 1 --2- 172.21.15.148:0/3043721058 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fed5406c530 0x7fed5406e9f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00d430 tx=0x7fed5c007530 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:46.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:46.845+0000 7fed6bfff640 1 -- 172.21.15.148:0/3043721058 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fed6c01a2b0 con 0x7fed7415a120 2024-03-29T04:10:47.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.022+0000 7fed79d3d640 1 -- 172.21.15.148:0/3043721058 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fed7415d2c0 con 0x7fed7415a120 2024-03-29T04:10:47.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.023+0000 7fed6bfff640 1 -- 172.21.15.148:0/3043721058 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fed7415d2c0 con 0x7fed7415a120 2024-03-29T04:10:47.002 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-29T04:10:47.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.025+0000 7fed79d3d640 1 -- 172.21.15.148:0/3043721058 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fed5406c530 msgr2=0x7fed5406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:47.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.025+0000 7fed79d3d640 1 --2- 172.21.15.148:0/3043721058 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fed5406c530 0x7fed5406e9f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00d430 tx=0x7fed5c007530 comp rx=0 tx=0).stop 2024-03-29T04:10:47.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.025+0000 7fed79d3d640 1 -- 172.21.15.148:0/3043721058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed7415a120 msgr2=0x7fed740b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:47.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.025+0000 7fed79d3d640 1 --2- 172.21.15.148:0/3043721058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed7415a120 0x7fed740b62b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00c410 tx=0x7fed6c00c8d0 comp rx=0 tx=0).stop 2024-03-29T04:10:47.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.025+0000 7fed79d3d640 1 -- 172.21.15.148:0/3043721058 shutdown_connections 2024-03-29T04:10:47.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.025+0000 7fed79d3d640 1 --2- 172.21.15.148:0/3043721058 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fed5406c530 0x7fed5406e9f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:47.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.025+0000 7fed79d3d640 1 --2- 172.21.15.148:0/3043721058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed7415a120 0x7fed740b62b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:47.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.025+0000 7fed79d3d640 1 --2- 172.21.15.148:0/3043721058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed740ce310 0x7fed740b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:47.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.025+0000 7fed79d3d640 1 --2- 172.21.15.148:0/3043721058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed7406ee10 0x7fed74153120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:47.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.025+0000 7fed79d3d640 1 -- 172.21.15.148:0/3043721058 >> 172.21.15.148:0/3043721058 conn(0x7fed740cead0 msgr2=0x7fed7415fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:47.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.025+0000 7fed79d3d640 1 -- 172.21.15.148:0/3043721058 shutdown_connections 2024-03-29T04:10:47.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.025+0000 7fed79d3d640 1 -- 172.21.15.148:0/3043721058 wait complete. 2024-03-29T04:10:47.015 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:10:47.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.142+0000 7f069cbd5640 1 Processor -- start 2024-03-29T04:10:47.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.142+0000 7f069cbd5640 1 -- start start 2024-03-29T04:10:47.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.143+0000 7f069cbd5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f069815a120 0x7f06980700d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:47.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.143+0000 7f069cbd5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f069806efe0 0x7f06980b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:47.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.143+0000 7f069cbd5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f069815d460 con 0x7f06980ce310 2024-03-29T04:10:47.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.143+0000 7f069cbd5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f06980d1b20 con 0x7f069806efe0 2024-03-29T04:10:47.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.143+0000 7f069cbd5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f06980d0f60 con 0x7f069815a120 2024-03-29T04:10:47.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.143+0000 7f0696d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f069806efe0 0x7f06980b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:47.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.143+0000 7f0695d9b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f06980ce310 0x7f069806e8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50310/0 (socket says 172.21.15.148:50310) 2024-03-29T04:10:47.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.143+0000 7f0695d9b640 1 -- 172.21.15.148:0/2403504810 learned_addr learned my addr 172.21.15.148:0/2403504810 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:47.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.143+0000 7f069659c640 1 --2- 172.21.15.148:0/2403504810 >> v2:172.21.15.157:3300/0 conn(0x7f069815a120 0x7f06980700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:47.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.144+0000 7f0696d9d640 1 -- 172.21.15.148:0/2403504810 >> v2:172.21.15.157:3300/0 conn(0x7f069815a120 msgr2=0x7f06980700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:47.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.144+0000 7f0696d9d640 1 --2- 172.21.15.148:0/2403504810 >> v2:172.21.15.157:3300/0 conn(0x7f069815a120 0x7f06980700d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:47.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.144+0000 7f0696d9d640 1 -- 172.21.15.148:0/2403504810 >> v1:172.21.15.148:6789/0 conn(0x7f06980ce310 legacy=0x7f069806e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:47.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.144+0000 7f0696d9d640 1 -- 172.21.15.148:0/2403504810 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069815d700 con 0x7f069806efe0 2024-03-29T04:10:47.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.144+0000 7f069559a640 1 -- 172.21.15.148:0/2403504810 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1876609442 0 0) 0x7f069815d460 con 0x7f06980ce310 2024-03-29T04:10:47.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.145+0000 7f0696d9d640 1 --2- 172.21.15.148:0/2403504810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f069806efe0 0x7f06980b6f20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0698073010 tx=0x7f068802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ca1d3e196582b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:47.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.145+0000 7f069559a640 1 -- 172.21.15.148:0/2403504810 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0688035020 con 0x7f069806efe0 2024-03-29T04:10:47.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.145+0000 7f069559a640 1 -- 172.21.15.148:0/2403504810 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0688031830 con 0x7f069806efe0 2024-03-29T04:10:47.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.145+0000 7f069559a640 1 -- 172.21.15.148:0/2403504810 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0688031c00 con 0x7f069806efe0 2024-03-29T04:10:47.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.145+0000 7f069cbd5640 1 -- 172.21.15.148:0/2403504810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f069806efe0 msgr2=0x7f06980b6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:47.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.145+0000 7f069cbd5640 1 --2- 172.21.15.148:0/2403504810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f069806efe0 0x7f06980b6f20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0698073010 tx=0x7f068802b8d0 comp rx=0 tx=0).stop 2024-03-29T04:10:47.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.145+0000 7f069cbd5640 1 -- 172.21.15.148:0/2403504810 shutdown_connections 2024-03-29T04:10:47.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.145+0000 7f069cbd5640 1 --2- 172.21.15.148:0/2403504810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f069806efe0 0x7f06980b6f20 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:47.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.145+0000 7f069cbd5640 1 --2- 172.21.15.148:0/2403504810 >> v2:172.21.15.157:3300/0 conn(0x7f069815a120 0x7f06980700d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:47.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.145+0000 7f069cbd5640 1 -- 172.21.15.148:0/2403504810 >> 172.21.15.148:0/2403504810 conn(0x7f06980cead0 msgr2=0x7f06980ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:47.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.146+0000 7f069cbd5640 1 -- 172.21.15.148:0/2403504810 shutdown_connections 2024-03-29T04:10:47.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.146+0000 7f069cbd5640 1 -- 172.21.15.148:0/2403504810 wait complete. 2024-03-29T04:10:47.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.147+0000 7f069cbd5640 1 Processor -- start 2024-03-29T04:10:47.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.147+0000 7f069cbd5640 1 -- start start 2024-03-29T04:10:47.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.148+0000 7f069cbd5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f069806efe0 0x7f0698159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:47.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.148+0000 7f069cbd5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06980ce310 0x7f06980cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:47.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.148+0000 7f069cbd5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f069815a120 0x7f06980cb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:47.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.148+0000 7f069cbd5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f069806f5c0 con 0x7f06980ce310 2024-03-29T04:10:47.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.148+0000 7f069cbd5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f06980ce7d0 con 0x7f069815a120 2024-03-29T04:10:47.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.148+0000 7f069cbd5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f069806f740 con 0x7f069806efe0 2024-03-29T04:10:47.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.148+0000 7f0695d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06980ce310 0x7f06980cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:47.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.148+0000 7f0695d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06980ce310 0x7f06980cb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53216/0 (socket says 172.21.15.148:53216) 2024-03-29T04:10:47.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.148+0000 7f0695d9b640 1 -- 172.21.15.148:0/1146664469 learned_addr learned my addr 172.21.15.148:0/1146664469 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:47.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.148+0000 7f0696d9d640 1 --2- 172.21.15.148:0/1146664469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f069815a120 0x7f06980cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:47.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.148+0000 7f069659c640 1 --2- 172.21.15.148:0/1146664469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f069806efe0 0x7f0698159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:47.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.148+0000 7f0695d9b640 1 -- 172.21.15.148:0/1146664469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f069806efe0 msgr2=0x7f0698159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:47.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.148+0000 7f0695d9b640 1 --2- 172.21.15.148:0/1146664469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f069806efe0 0x7f0698159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:47.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.148+0000 7f0695d9b640 1 -- 172.21.15.148:0/1146664469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f069815a120 msgr2=0x7f06980cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:47.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.148+0000 7f0695d9b640 1 --2- 172.21.15.148:0/1146664469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f069815a120 0x7f06980cb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:47.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.148+0000 7f0695d9b640 1 -- 172.21.15.148:0/1146664469 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06980c68c0 con 0x7f06980ce310 2024-03-29T04:10:47.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.148+0000 7f069659c640 1 --2- 172.21.15.148:0/1146664469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f069806efe0 0x7f0698159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:47.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.148+0000 7f0695d9b640 1 --2- 172.21.15.148:0/1146664469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06980ce310 0x7f06980cb1d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f068000c4d0 tx=0x7f068000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:47.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.148+0000 7f068effd640 1 -- 172.21.15.148:0/1146664469 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680017070 con 0x7f06980ce310 2024-03-29T04:10:47.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.148+0000 7f068effd640 1 -- 172.21.15.148:0/1146664469 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0680005d00 con 0x7f06980ce310 2024-03-29T04:10:47.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.148+0000 7f068effd640 1 -- 172.21.15.148:0/1146664469 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06800072b0 con 0x7f06980ce310 2024-03-29T04:10:47.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.148+0000 7f069cbd5640 1 -- 172.21.15.148:0/1146664469 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06980c6ba0 con 0x7f06980ce310 2024-03-29T04:10:47.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.148+0000 7f069cbd5640 1 -- 172.21.15.148:0/1146664469 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06980c7160 con 0x7f06980ce310 2024-03-29T04:10:47.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.150+0000 7f069cbd5640 1 -- 172.21.15.148:0/1146664469 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0698068850 con 0x7f06980ce310 2024-03-29T04:10:47.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.150+0000 7f068effd640 1 -- 172.21.15.148:0/1146664469 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f068001a070 con 0x7f06980ce310 2024-03-29T04:10:47.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.151+0000 7f068effd640 1 --2- 172.21.15.148:0/1146664469 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f067806c530 0x7f067806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:47.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.151+0000 7f068effd640 1 -- 172.21.15.148:0/1146664469 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f068008adb0 con 0x7f06980ce310 2024-03-29T04:10:47.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.151+0000 7f069659c640 1 --2- 172.21.15.148:0/1146664469 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f067806c530 0x7f067806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:47.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.151+0000 7f069659c640 1 --2- 172.21.15.148:0/1146664469 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f067806c530 0x7f067806e9f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0690009fd0 tx=0x7f069000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:47.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.155+0000 7f068effd640 1 -- 172.21.15.148:0/1146664469 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0680058980 con 0x7f06980ce310 2024-03-29T04:10:47.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.493+0000 7f069cbd5640 1 -- 172.21.15.148:0/1146664469 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f06980d1cb0 con 0x7f06980ce310 2024-03-29T04:10:47.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.494+0000 7f068effd640 1 -- 172.21.15.148:0/1146664469 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f068005cc80 con 0x7f06980ce310 2024-03-29T04:10:47.473 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:10:47.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.496+0000 7f069cbd5640 1 -- 172.21.15.148:0/1146664469 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f067806c530 msgr2=0x7f067806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:47.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.496+0000 7f069cbd5640 1 --2- 172.21.15.148:0/1146664469 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f067806c530 0x7f067806e9f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0690009fd0 tx=0x7f069000b040 comp rx=0 tx=0).stop 2024-03-29T04:10:47.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.496+0000 7f069cbd5640 1 -- 172.21.15.148:0/1146664469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06980ce310 msgr2=0x7f06980cb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:47.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.496+0000 7f069cbd5640 1 --2- 172.21.15.148:0/1146664469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06980ce310 0x7f06980cb1d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f068000c4d0 tx=0x7f068000c990 comp rx=0 tx=0).stop 2024-03-29T04:10:47.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.496+0000 7f069cbd5640 1 -- 172.21.15.148:0/1146664469 shutdown_connections 2024-03-29T04:10:47.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.496+0000 7f069cbd5640 1 --2- 172.21.15.148:0/1146664469 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f067806c530 0x7f067806e9f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:47.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.496+0000 7f069cbd5640 1 --2- 172.21.15.148:0/1146664469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f069815a120 0x7f06980cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:47.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.496+0000 7f069cbd5640 1 --2- 172.21.15.148:0/1146664469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06980ce310 0x7f06980cb1d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:47.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.496+0000 7f069cbd5640 1 --2- 172.21.15.148:0/1146664469 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f069806efe0 0x7f0698159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:47.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.496+0000 7f069cbd5640 1 -- 172.21.15.148:0/1146664469 >> 172.21.15.148:0/1146664469 conn(0x7f06980cead0 msgr2=0x7f0698070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:47.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.496+0000 7f069cbd5640 1 -- 172.21.15.148:0/1146664469 shutdown_connections 2024-03-29T04:10:47.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.496+0000 7f069cbd5640 1 -- 172.21.15.148:0/1146664469 wait complete. 2024-03-29T04:10:47.488 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:47.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.612+0000 7fb8e0a93640 1 Processor -- start 2024-03-29T04:10:47.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.612+0000 7fb8e0a93640 1 -- start start 2024-03-29T04:10:47.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.612+0000 7fb8e0a93640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb8dc0d17e0 0x7fb8dc0d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:47.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.612+0000 7fb8e0a93640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8dc15e7c0 con 0x7fb8dc0ba030 2024-03-29T04:10:47.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.613+0000 7fb8e0a93640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8dc0d2c30 con 0x7fb8dc15e3e0 2024-03-29T04:10:47.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.613+0000 7fb8e0a93640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb8dc0d3840 con 0x7fb8dc0d17e0 2024-03-29T04:10:47.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.613+0000 7fb8d9d9b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb8dc0ba030 0x7fb8dc0d10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50326/0 (socket says 172.21.15.148:50326) 2024-03-29T04:10:47.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.613+0000 7fb8d9d9b640 1 -- 172.21.15.148:0/545454009 learned_addr learned my addr 172.21.15.148:0/545454009 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:47.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.613+0000 7fb8dad9d640 1 --2- 172.21.15.148:0/545454009 >> v2:172.21.15.157:3300/0 conn(0x7fb8dc0d17e0 0x7fb8dc0d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:47.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.613+0000 7fb8d959a640 1 -- 172.21.15.148:0/545454009 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3327237191 0 0) 0x7fb8dc15e7c0 con 0x7fb8dc0ba030 2024-03-29T04:10:47.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.613+0000 7fb8d959a640 1 -- 172.21.15.148:0/545454009 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8c4003520 con 0x7fb8dc0ba030 2024-03-29T04:10:47.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.613+0000 7fb8d959a640 1 -- 172.21.15.148:0/545454009 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2241970679 0 0) 0x7fb8dc0d2c30 con 0x7fb8dc15e3e0 2024-03-29T04:10:47.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.613+0000 7fb8d959a640 1 -- 172.21.15.148:0/545454009 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8dc15e7c0 con 0x7fb8dc15e3e0 2024-03-29T04:10:47.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.613+0000 7fb8d959a640 1 -- 172.21.15.148:0/545454009 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3602499083 0 0) 0x7fb8c4003520 con 0x7fb8dc0ba030 2024-03-29T04:10:47.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.613+0000 7fb8dad9d640 1 -- 172.21.15.148:0/545454009 >> v1:172.21.15.157:6789/0 conn(0x7fb8dc15e3e0 legacy=0x7fb8dc0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:47.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.613+0000 7fb8dad9d640 1 -- 172.21.15.148:0/545454009 >> v1:172.21.15.148:6789/0 conn(0x7fb8dc0ba030 legacy=0x7fb8dc0d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:47.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.613+0000 7fb8dad9d640 1 -- 172.21.15.148:0/545454009 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8dc0d2f10 con 0x7fb8dc0d17e0 2024-03-29T04:10:47.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.614+0000 7fb8dad9d640 1 --2- 172.21.15.148:0/545454009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8dc0d17e0 0x7fb8dc0d3cd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4009d90 tx=0x7fb8d402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e2660765a74e655 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:47.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.614+0000 7fb8d959a640 1 -- 172.21.15.148:0/545454009 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4039070 con 0x7fb8dc0d17e0 2024-03-29T04:10:47.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.614+0000 7fb8d959a640 1 -- 172.21.15.148:0/545454009 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8d402c9d0 con 0x7fb8dc0d17e0 2024-03-29T04:10:47.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.614+0000 7fb8d959a640 1 -- 172.21.15.148:0/545454009 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0001670 con 0x7fb8dc0d17e0 2024-03-29T04:10:47.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.615+0000 7fb8e0a93640 1 -- 172.21.15.148:0/545454009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8dc0d17e0 msgr2=0x7fb8dc0d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:47.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.615+0000 7fb8e0a93640 1 --2- 172.21.15.148:0/545454009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8dc0d17e0 0x7fb8dc0d3cd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4009d90 tx=0x7fb8d402bea0 comp rx=0 tx=0).stop 2024-03-29T04:10:47.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.615+0000 7fb8e0a93640 1 -- 172.21.15.148:0/545454009 shutdown_connections 2024-03-29T04:10:47.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.615+0000 7fb8e0a93640 1 --2- 172.21.15.148:0/545454009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8dc0d17e0 0x7fb8dc0d3cd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:47.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.615+0000 7fb8e0a93640 1 -- 172.21.15.148:0/545454009 >> 172.21.15.148:0/545454009 conn(0x7fb8dc0b6210 msgr2=0x7fb8dc0b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:47.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.615+0000 7fb8e0a93640 1 -- 172.21.15.148:0/545454009 shutdown_connections 2024-03-29T04:10:47.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.615+0000 7fb8e0a93640 1 -- 172.21.15.148:0/545454009 wait complete. 2024-03-29T04:10:47.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.616+0000 7fb8e0a93640 1 Processor -- start 2024-03-29T04:10:47.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.616+0000 7fb8e0a93640 1 -- start start 2024-03-29T04:10:47.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.617+0000 7fb8e0a93640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8dc0ba030 0x7fb8dc1465b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:47.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.617+0000 7fb8e0a93640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8dc0d17e0 0x7fb8dc0ce280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:47.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.617+0000 7fb8e0a93640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8dc15e3e0 0x7fb8dc160d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:47.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.617+0000 7fb8e0a93640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc168f60 con 0x7fb8dc0d17e0 2024-03-29T04:10:47.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.617+0000 7fb8e0a93640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc0d34d0 con 0x7fb8dc0ba030 2024-03-29T04:10:47.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.617+0000 7fb8e0a93640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb8dc1690e0 con 0x7fb8dc15e3e0 2024-03-29T04:10:47.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.617+0000 7fb8d9d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8dc0d17e0 0x7fb8dc0ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:47.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.617+0000 7fb8d9d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8dc0d17e0 0x7fb8dc0ce280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53224/0 (socket says 172.21.15.148:53224) 2024-03-29T04:10:47.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.617+0000 7fb8d9d9b640 1 -- 172.21.15.148:0/1458969512 learned_addr learned my addr 172.21.15.148:0/1458969512 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:47.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.617+0000 7fb8da59c640 1 --2- 172.21.15.148:0/1458969512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8dc0ba030 0x7fb8dc1465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:47.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.617+0000 7fb8dad9d640 1 --2- 172.21.15.148:0/1458969512 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8dc15e3e0 0x7fb8dc160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:47.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.617+0000 7fb8d9d9b640 1 -- 172.21.15.148:0/1458969512 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8dc15e3e0 msgr2=0x7fb8dc160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:47.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.617+0000 7fb8d9d9b640 1 --2- 172.21.15.148:0/1458969512 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8dc15e3e0 0x7fb8dc160d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:47.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.617+0000 7fb8d9d9b640 1 -- 172.21.15.148:0/1458969512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8dc0ba030 msgr2=0x7fb8dc1465b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:47.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.617+0000 7fb8d9d9b640 1 --2- 172.21.15.148:0/1458969512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8dc0ba030 0x7fb8dc1465b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:47.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.617+0000 7fb8d9d9b640 1 -- 172.21.15.148:0/1458969512 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8dc146af0 con 0x7fb8dc0d17e0 2024-03-29T04:10:47.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.617+0000 7fb8da59c640 1 --2- 172.21.15.148:0/1458969512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8dc0ba030 0x7fb8dc1465b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:47.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.617+0000 7fb8d9d9b640 1 --2- 172.21.15.148:0/1458969512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8dc0d17e0 0x7fb8dc0ce280 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000c780 tx=0x7fb8d000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:47.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.618+0000 7fb8caffd640 1 -- 172.21.15.148:0/1458969512 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0017070 con 0x7fb8dc0d17e0 2024-03-29T04:10:47.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.618+0000 7fb8caffd640 1 -- 172.21.15.148:0/1458969512 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8d0005ad0 con 0x7fb8dc0d17e0 2024-03-29T04:10:47.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.618+0000 7fb8e0a93640 1 -- 172.21.15.148:0/1458969512 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8dc15fcc0 con 0x7fb8dc0d17e0 2024-03-29T04:10:47.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.618+0000 7fb8caffd640 1 -- 172.21.15.148:0/1458969512 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0007070 con 0x7fb8dc0d17e0 2024-03-29T04:10:47.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.618+0000 7fb8e0a93640 1 -- 172.21.15.148:0/1458969512 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8dc160280 con 0x7fb8dc0d17e0 2024-03-29T04:10:47.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.619+0000 7fb8e0a93640 1 -- 172.21.15.148:0/1458969512 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8dc0d0720 con 0x7fb8dc0d17e0 2024-03-29T04:10:47.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.619+0000 7fb8caffd640 1 -- 172.21.15.148:0/1458969512 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb8d001a070 con 0x7fb8dc0d17e0 2024-03-29T04:10:47.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.619+0000 7fb8caffd640 1 --2- 172.21.15.148:0/1458969512 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb8bc06c530 0x7fb8bc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:47.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.620+0000 7fb8caffd640 1 -- 172.21.15.148:0/1458969512 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb8d008b130 con 0x7fb8dc0d17e0 2024-03-29T04:10:47.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.620+0000 7fb8da59c640 1 --2- 172.21.15.148:0/1458969512 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb8bc06c530 0x7fb8bc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:47.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.620+0000 7fb8da59c640 1 --2- 172.21.15.148:0/1458969512 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb8bc06c530 0x7fb8bc06e9f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc0011e0 tx=0x7fb8cc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:47.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.624+0000 7fb8caffd640 1 -- 172.21.15.148:0/1458969512 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb8dc0d0720 con 0x7fb8dc0d17e0 2024-03-29T04:10:47.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.803+0000 7fb8e0a93640 1 -- 172.21.15.148:0/1458969512 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8dc0d2340 con 0x7fb8dc0d17e0 2024-03-29T04:10:47.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.805+0000 7fb8caffd640 1 -- 172.21.15.148:0/1458969512 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18858 (secure 0 0 0) 0x7fb8d00592c0 con 0x7fb8dc0d17e0 2024-03-29T04:10:47.784 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:47.784 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:41.662934+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:47.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.807+0000 7fb8e0a93640 1 -- 172.21.15.148:0/1458969512 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb8bc06c530 msgr2=0x7fb8bc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:47.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.807+0000 7fb8e0a93640 1 --2- 172.21.15.148:0/1458969512 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb8bc06c530 0x7fb8bc06e9f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc0011e0 tx=0x7fb8cc009040 comp rx=0 tx=0).stop 2024-03-29T04:10:47.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.807+0000 7fb8e0a93640 1 -- 172.21.15.148:0/1458969512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8dc0d17e0 msgr2=0x7fb8dc0ce280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:47.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.807+0000 7fb8e0a93640 1 --2- 172.21.15.148:0/1458969512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8dc0d17e0 0x7fb8dc0ce280 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000c780 tx=0x7fb8d000cc40 comp rx=0 tx=0).stop 2024-03-29T04:10:47.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.807+0000 7fb8e0a93640 1 -- 172.21.15.148:0/1458969512 shutdown_connections 2024-03-29T04:10:47.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.807+0000 7fb8e0a93640 1 --2- 172.21.15.148:0/1458969512 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb8bc06c530 0x7fb8bc06e9f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:47.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.807+0000 7fb8e0a93640 1 --2- 172.21.15.148:0/1458969512 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8dc15e3e0 0x7fb8dc160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:47.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.807+0000 7fb8e0a93640 1 --2- 172.21.15.148:0/1458969512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8dc0d17e0 0x7fb8dc0ce280 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:47.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.807+0000 7fb8e0a93640 1 --2- 172.21.15.148:0/1458969512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8dc0ba030 0x7fb8dc1465b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:47.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.807+0000 7fb8e0a93640 1 -- 172.21.15.148:0/1458969512 >> 172.21.15.148:0/1458969512 conn(0x7fb8dc0b6210 msgr2=0x7fb8dc0b7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:47.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.807+0000 7fb8e0a93640 1 -- 172.21.15.148:0/1458969512 shutdown_connections 2024-03-29T04:10:47.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.807+0000 7fb8e0a93640 1 -- 172.21.15.148:0/1458969512 wait complete. 2024-03-29T04:10:47.798 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:47.898351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:57.922313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:58.829405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:10:02.551763+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:10:47.798 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:10:47.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.925+0000 7f4668515640 1 Processor -- start 2024-03-29T04:10:47.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.925+0000 7f4668515640 1 -- start start 2024-03-29T04:10:47.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.926+0000 7f4668515640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f466015e3c0 0x7f4660161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:47.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.926+0000 7f4668515640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46600cf330 con 0x7f46600b3430 2024-03-29T04:10:47.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.926+0000 7f4668515640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4660162cf0 con 0x7f46600b2f00 2024-03-29T04:10:47.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.926+0000 7f4668515640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f46600cf960 con 0x7f466015e3c0 2024-03-29T04:10:47.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.926+0000 7f4666ab2640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f46600b3430 0x7f466006b7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50336/0 (socket says 172.21.15.148:50336) 2024-03-29T04:10:47.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.926+0000 7f4666ab2640 1 -- 172.21.15.148:0/781791001 learned_addr learned my addr 172.21.15.148:0/781791001 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:47.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.926+0000 7f46652af640 1 -- 172.21.15.148:0/781791001 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 508761976 0 0) 0x7f46600cf330 con 0x7f46600b3430 2024-03-29T04:10:47.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.926+0000 7f46662b1640 1 --2- 172.21.15.148:0/781791001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f466015e3c0 0x7f4660161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:47.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.926+0000 7f46652af640 1 -- 172.21.15.148:0/781791001 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f464c003520 con 0x7f46600b3430 2024-03-29T04:10:47.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.926+0000 7f46652af640 1 -- 172.21.15.148:0/781791001 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2130681760 0 0) 0x7f464c003520 con 0x7f46600b3430 2024-03-29T04:10:47.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.926+0000 7f46652af640 1 -- 172.21.15.148:0/781791001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f466015e3c0 msgr2=0x7f4660161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:47.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.926+0000 7f46652af640 1 --2- 172.21.15.148:0/781791001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f466015e3c0 0x7f4660161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:47.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.926+0000 7f46652af640 1 -- 172.21.15.148:0/781791001 >> v1:172.21.15.157:6789/0 conn(0x7f46600b2f00 legacy=0x7f46601624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:47.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.926+0000 7f46652af640 1 -- 172.21.15.148:0/781791001 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46600cf5f0 con 0x7f46600b3430 2024-03-29T04:10:47.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.926+0000 7f46652af640 1 -- 172.21.15.148:0/781791001 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f46580019f0 con 0x7f46600b3430 2024-03-29T04:10:47.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.926+0000 7f46652af640 1 -- 172.21.15.148:0/781791001 >> v1:172.21.15.148:6789/0 conn(0x7f46600b3430 legacy=0x7f466006b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:47.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.926+0000 7f46652af640 1 --2- 172.21.15.148:0/781791001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f464c0039e0 0x7f464c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:47.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.926+0000 7f46652af640 1 --2- 172.21.15.148:0/781791001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f466015e3c0 0x7f464c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:47.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.926+0000 7f46652af640 1 --2- 172.21.15.148:0/781791001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f464c004d40 0x7f464c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:47.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.926+0000 7f46652af640 1 -- 172.21.15.148:0/781791001 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f46600cf960 con 0x7f464c004d40 2024-03-29T04:10:47.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.926+0000 7f46652af640 1 -- 172.21.15.148:0/781791001 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f464c00bb50 con 0x7f464c0039e0 2024-03-29T04:10:47.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.926+0000 7f46652af640 1 -- 172.21.15.148:0/781791001 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f464c00bd20 con 0x7f466015e3c0 2024-03-29T04:10:47.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.927+0000 7f4666ab2640 1 --2- 172.21.15.148:0/781791001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f464c004d40 0x7f464c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:47.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.927+0000 7f46662b1640 1 --2- 172.21.15.148:0/781791001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f464c0039e0 0x7f464c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:47.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.927+0000 7f4665ab0640 1 --2- 172.21.15.148:0/781791001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f466015e3c0 0x7f464c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:47.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.927+0000 7f4666ab2640 1 -- 172.21.15.148:0/781791001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f466015e3c0 msgr2=0x7f464c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:47.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.927+0000 7f4666ab2640 1 --2- 172.21.15.148:0/781791001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f466015e3c0 0x7f464c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:47.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.927+0000 7f4666ab2640 1 -- 172.21.15.148:0/781791001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f464c0039e0 msgr2=0x7f464c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:47.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.927+0000 7f4666ab2640 1 --2- 172.21.15.148:0/781791001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f464c0039e0 0x7f464c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:47.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.927+0000 7f4666ab2640 1 -- 172.21.15.148:0/781791001 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f464c003520 con 0x7f464c004d40 2024-03-29T04:10:47.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.928+0000 7f4666ab2640 1 --2- 172.21.15.148:0/781791001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f464c004d40 0x7f464c0051c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f465800a840 tx=0x7f464c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3046bb622689d23 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:47.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.928+0000 7f46652af640 1 -- 172.21.15.148:0/781791001 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465800a460 con 0x7f464c004d40 2024-03-29T04:10:47.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.928+0000 7f46652af640 1 -- 172.21.15.148:0/781791001 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4658034650 con 0x7f464c004d40 2024-03-29T04:10:47.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.928+0000 7f46652af640 1 -- 172.21.15.148:0/781791001 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4658034a40 con 0x7f464c004d40 2024-03-29T04:10:47.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.928+0000 7f4668515640 1 -- 172.21.15.148:0/781791001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f464c004d40 msgr2=0x7f464c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:47.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.928+0000 7f4668515640 1 --2- 172.21.15.148:0/781791001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f464c004d40 0x7f464c0051c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f465800a840 tx=0x7f464c005830 comp rx=0 tx=0).stop 2024-03-29T04:10:47.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.928+0000 7f4666ab2640 1 -- 172.21.15.148:0/781791001 reap_dead start 2024-03-29T04:10:47.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.928+0000 7f4668515640 1 -- 172.21.15.148:0/781791001 shutdown_connections 2024-03-29T04:10:47.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.928+0000 7f4668515640 1 -- 172.21.15.148:0/781791001 >> 172.21.15.148:0/781791001 conn(0x7f46600b6430 msgr2=0x7f46600d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:47.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.928+0000 7f4668515640 1 -- 172.21.15.148:0/781791001 shutdown_connections 2024-03-29T04:10:47.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.928+0000 7f4668515640 1 -- 172.21.15.148:0/781791001 wait complete. 2024-03-29T04:10:47.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.930+0000 7f4668515640 1 Processor -- start 2024-03-29T04:10:47.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.930+0000 7f4668515640 1 -- start start 2024-03-29T04:10:47.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.930+0000 7f4668515640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f464c004d40 0x7f46601f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:47.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.930+0000 7f4668515640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4660159830 0x7f46600cd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:47.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.930+0000 7f4668515640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46600cd5d0 0x7f466015ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:47.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.930+0000 7f4668515640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f46600b3b20 con 0x7f46600cd5d0 2024-03-29T04:10:47.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.930+0000 7f4668515640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f46600b39a0 con 0x7f464c004d40 2024-03-29T04:10:47.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.930+0000 7f4668515640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4660161f70 con 0x7f4660159830 2024-03-29T04:10:47.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.930+0000 7f4666ab2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46600cd5d0 0x7f466015ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:47.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.930+0000 7f4666ab2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46600cd5d0 0x7f466015ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53248/0 (socket says 172.21.15.148:53248) 2024-03-29T04:10:47.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.930+0000 7f4666ab2640 1 -- 172.21.15.148:0/1382398203 learned_addr learned my addr 172.21.15.148:0/1382398203 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:47.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.931+0000 7f4665ab0640 1 --2- 172.21.15.148:0/1382398203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4660159830 0x7f46600cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:47.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.931+0000 7f46662b1640 1 --2- 172.21.15.148:0/1382398203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f464c004d40 0x7f46601f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:47.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.931+0000 7f4666ab2640 1 -- 172.21.15.148:0/1382398203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4660159830 msgr2=0x7f46600cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:47.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.931+0000 7f4666ab2640 1 --2- 172.21.15.148:0/1382398203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4660159830 0x7f46600cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:47.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.931+0000 7f4666ab2640 1 -- 172.21.15.148:0/1382398203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f464c004d40 msgr2=0x7f46601f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:47.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.931+0000 7f4666ab2640 1 --2- 172.21.15.148:0/1382398203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f464c004d40 0x7f46601f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:47.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.931+0000 7f4666ab2640 1 -- 172.21.15.148:0/1382398203 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46601f4260 con 0x7f46600cd5d0 2024-03-29T04:10:47.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.931+0000 7f4665ab0640 1 --2- 172.21.15.148:0/1382398203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4660159830 0x7f46600cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:47.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.931+0000 7f46662b1640 1 --2- 172.21.15.148:0/1382398203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f464c004d40 0x7f46601f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:47.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.931+0000 7f4666ab2640 1 --2- 172.21.15.148:0/1382398203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46600cd5d0 0x7f466015ca70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4658000f80 tx=0x7f46600cf7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:47.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.931+0000 7f4656ffd640 1 -- 172.21.15.148:0/1382398203 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4658001fb0 con 0x7f46600cd5d0 2024-03-29T04:10:47.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.931+0000 7f4656ffd640 1 -- 172.21.15.148:0/1382398203 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4658044070 con 0x7f46600cd5d0 2024-03-29T04:10:47.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.931+0000 7f4668515640 1 -- 172.21.15.148:0/1382398203 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46601f4540 con 0x7f46600cd5d0 2024-03-29T04:10:47.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.931+0000 7f4656ffd640 1 -- 172.21.15.148:0/1382398203 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4658034a00 con 0x7f46600cd5d0 2024-03-29T04:10:47.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.931+0000 7f4668515640 1 -- 172.21.15.148:0/1382398203 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46601f4b00 con 0x7f46600cd5d0 2024-03-29T04:10:47.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.932+0000 7f4668515640 1 -- 172.21.15.148:0/1382398203 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4660068850 con 0x7f46600cd5d0 2024-03-29T04:10:47.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.933+0000 7f4656ffd640 1 -- 172.21.15.148:0/1382398203 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4658034ba0 con 0x7f46600cd5d0 2024-03-29T04:10:47.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.933+0000 7f4656ffd640 1 --2- 172.21.15.148:0/1382398203 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f464406c530 0x7f464406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:47.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.933+0000 7f4656ffd640 1 -- 172.21.15.148:0/1382398203 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f46580adf40 con 0x7f46600cd5d0 2024-03-29T04:10:47.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.933+0000 7f46662b1640 1 --2- 172.21.15.148:0/1382398203 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f464406c530 0x7f464406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:47.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.933+0000 7f46662b1640 1 --2- 172.21.15.148:0/1382398203 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f464406c530 0x7f464406e9f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4650009bc0 tx=0x7f4650008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:47.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:47.937+0000 7f4656ffd640 1 -- 172.21.15.148:0/1382398203 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f465807c1f0 con 0x7f46600cd5d0 2024-03-29T04:10:48.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.226+0000 7f4668515640 1 -- 172.21.15.148:0/1382398203 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f46601f4e10 con 0x7f46600cd5d0 2024-03-29T04:10:48.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.227+0000 7f4656ffd640 1 -- 172.21.15.148:0/1382398203 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f465807baf0 con 0x7f46600cd5d0 2024-03-29T04:10:48.206 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-29T04:10:48.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.229+0000 7f4668515640 1 -- 172.21.15.148:0/1382398203 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f464406c530 msgr2=0x7f464406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:48.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.229+0000 7f4668515640 1 --2- 172.21.15.148:0/1382398203 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f464406c530 0x7f464406e9f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4650009bc0 tx=0x7f4650008040 comp rx=0 tx=0).stop 2024-03-29T04:10:48.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.229+0000 7f4668515640 1 -- 172.21.15.148:0/1382398203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46600cd5d0 msgr2=0x7f466015ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:48.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.229+0000 7f4668515640 1 --2- 172.21.15.148:0/1382398203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46600cd5d0 0x7f466015ca70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4658000f80 tx=0x7f46600cf7c0 comp rx=0 tx=0).stop 2024-03-29T04:10:48.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.229+0000 7f4668515640 1 -- 172.21.15.148:0/1382398203 shutdown_connections 2024-03-29T04:10:48.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.229+0000 7f4668515640 1 --2- 172.21.15.148:0/1382398203 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f464406c530 0x7f464406e9f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:48.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.229+0000 7f4668515640 1 --2- 172.21.15.148:0/1382398203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46600cd5d0 0x7f466015ca70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:48.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.229+0000 7f4668515640 1 --2- 172.21.15.148:0/1382398203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4660159830 0x7f46600cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:48.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.229+0000 7f4668515640 1 --2- 172.21.15.148:0/1382398203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f464c004d40 0x7f46601f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:48.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.229+0000 7f4668515640 1 -- 172.21.15.148:0/1382398203 >> 172.21.15.148:0/1382398203 conn(0x7f46600b6430 msgr2=0x7f46600b5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:48.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.229+0000 7f4668515640 1 -- 172.21.15.148:0/1382398203 shutdown_connections 2024-03-29T04:10:48.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.229+0000 7f4668515640 1 -- 172.21.15.148:0/1382398203 wait complete. 2024-03-29T04:10:48.219 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:10:48.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.345+0000 7fa44578d640 1 Processor -- start 2024-03-29T04:10:48.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.345+0000 7fa44578d640 1 -- start start 2024-03-29T04:10:48.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.346+0000 7fa44578d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa44015e3c0 0x7fa440161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:48.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.346+0000 7fa44578d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa4400b2f00 0x7fa4401624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:48.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.346+0000 7fa44578d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4400cf330 con 0x7fa4400b3430 2024-03-29T04:10:48.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.346+0000 7fa44578d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fa4400cf960 con 0x7fa44015e3c0 2024-03-29T04:10:48.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.346+0000 7fa44578d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa440162cf0 con 0x7fa4400b2f00 2024-03-29T04:10:48.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.346+0000 7fa43effd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa44015e3c0 0x7fa440161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:48.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.346+0000 7fa43f7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa4400b3430 0x7fa4400d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50340/0 (socket says 172.21.15.148:50340) 2024-03-29T04:10:48.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.346+0000 7fa43f7fe640 1 -- 172.21.15.148:0/3235895678 learned_addr learned my addr 172.21.15.148:0/3235895678 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:48.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.346+0000 7fa43e7fc640 1 --2- 172.21.15.148:0/3235895678 >> v2:172.21.15.157:3300/0 conn(0x7fa4400b2f00 0x7fa4401624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:48.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.346+0000 7fa43effd640 1 -- 172.21.15.148:0/3235895678 >> v2:172.21.15.157:3300/0 conn(0x7fa4400b2f00 msgr2=0x7fa4401624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:48.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.346+0000 7fa43effd640 1 --2- 172.21.15.148:0/3235895678 >> v2:172.21.15.157:3300/0 conn(0x7fa4400b2f00 0x7fa4401624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:48.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.346+0000 7fa43effd640 1 -- 172.21.15.148:0/3235895678 >> v1:172.21.15.148:6789/0 conn(0x7fa4400b3430 legacy=0x7fa4400d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:48.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.346+0000 7fa43effd640 1 -- 172.21.15.148:0/3235895678 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4400cf5f0 con 0x7fa44015e3c0 2024-03-29T04:10:48.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.346+0000 7fa43dffb640 1 -- 172.21.15.148:0/3235895678 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2920506123 0 0) 0x7fa4400cf330 con 0x7fa4400b3430 2024-03-29T04:10:48.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.347+0000 7fa43effd640 1 --2- 172.21.15.148:0/3235895678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa44015e3c0 0x7fa440161f70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa4400c7be0 tx=0x7fa43802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7dbbbfdf6ddf1a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:48.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.347+0000 7fa43dffb640 1 -- 172.21.15.148:0/3235895678 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa438035020 con 0x7fa44015e3c0 2024-03-29T04:10:48.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.347+0000 7fa43dffb640 1 -- 172.21.15.148:0/3235895678 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa438031830 con 0x7fa44015e3c0 2024-03-29T04:10:48.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.347+0000 7fa43dffb640 1 -- 172.21.15.148:0/3235895678 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa438031c00 con 0x7fa44015e3c0 2024-03-29T04:10:48.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.348+0000 7fa44578d640 1 -- 172.21.15.148:0/3235895678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa44015e3c0 msgr2=0x7fa440161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:48.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.348+0000 7fa44578d640 1 --2- 172.21.15.148:0/3235895678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa44015e3c0 0x7fa440161f70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa4400c7be0 tx=0x7fa43802b8d0 comp rx=0 tx=0).stop 2024-03-29T04:10:48.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.348+0000 7fa44578d640 1 -- 172.21.15.148:0/3235895678 shutdown_connections 2024-03-29T04:10:48.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.348+0000 7fa44578d640 1 --2- 172.21.15.148:0/3235895678 >> v2:172.21.15.157:3300/0 conn(0x7fa4400b2f00 0x7fa4401624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:48.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.348+0000 7fa44578d640 1 --2- 172.21.15.148:0/3235895678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa44015e3c0 0x7fa440161f70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:48.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.348+0000 7fa44578d640 1 -- 172.21.15.148:0/3235895678 >> 172.21.15.148:0/3235895678 conn(0x7fa4400b6430 msgr2=0x7fa4400d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:48.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.348+0000 7fa44578d640 1 -- 172.21.15.148:0/3235895678 shutdown_connections 2024-03-29T04:10:48.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.348+0000 7fa44578d640 1 -- 172.21.15.148:0/3235895678 wait complete. 2024-03-29T04:10:48.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.349+0000 7fa44578d640 1 Processor -- start 2024-03-29T04:10:48.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.349+0000 7fa44578d640 1 -- start start 2024-03-29T04:10:48.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.350+0000 7fa44578d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4400b3430 0x7fa4400b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:48.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.350+0000 7fa44578d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa44015e3c0 0x7fa44015b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:48.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.350+0000 7fa44578d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4400c87f0 0x7fa4400cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:48.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.350+0000 7fa44578d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa4400b39a0 con 0x7fa4400c87f0 2024-03-29T04:10:48.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.350+0000 7fa44578d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa44015e7a0 con 0x7fa4400b3430 2024-03-29T04:10:48.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.350+0000 7fa44578d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa4400b3b20 con 0x7fa44015e3c0 2024-03-29T04:10:48.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.350+0000 7fa43f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4400c87f0 0x7fa4400cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:48.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.350+0000 7fa43f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4400c87f0 0x7fa4400cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53266/0 (socket says 172.21.15.148:53266) 2024-03-29T04:10:48.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.350+0000 7fa43f7fe640 1 -- 172.21.15.148:0/1839327928 learned_addr learned my addr 172.21.15.148:0/1839327928 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:48.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.350+0000 7fa43effd640 1 --2- 172.21.15.148:0/1839327928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4400b3430 0x7fa4400b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:48.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.350+0000 7fa43e7fc640 1 --2- 172.21.15.148:0/1839327928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa44015e3c0 0x7fa44015b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:48.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.350+0000 7fa43f7fe640 1 -- 172.21.15.148:0/1839327928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa44015e3c0 msgr2=0x7fa44015b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:48.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.350+0000 7fa43f7fe640 1 --2- 172.21.15.148:0/1839327928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa44015e3c0 0x7fa44015b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:48.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.350+0000 7fa43f7fe640 1 -- 172.21.15.148:0/1839327928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4400b3430 msgr2=0x7fa4400b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:48.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.350+0000 7fa43f7fe640 1 --2- 172.21.15.148:0/1839327928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4400b3430 0x7fa4400b2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:48.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.350+0000 7fa43f7fe640 1 -- 172.21.15.148:0/1839327928 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4401f3ec0 con 0x7fa4400c87f0 2024-03-29T04:10:48.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.351+0000 7fa43f7fe640 1 --2- 172.21.15.148:0/1839327928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4400c87f0 0x7fa4400cd570 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa43000c7a0 tx=0x7fa43000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:48.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.351+0000 7fa4377fe640 1 -- 172.21.15.148:0/1839327928 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa430018070 con 0x7fa4400c87f0 2024-03-29T04:10:48.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.351+0000 7fa4377fe640 1 -- 172.21.15.148:0/1839327928 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa43000a640 con 0x7fa4400c87f0 2024-03-29T04:10:48.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.351+0000 7fa4377fe640 1 -- 172.21.15.148:0/1839327928 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43000aab0 con 0x7fa4400c87f0 2024-03-29T04:10:48.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.351+0000 7fa44578d640 1 -- 172.21.15.148:0/1839327928 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4401f41a0 con 0x7fa4400c87f0 2024-03-29T04:10:48.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.351+0000 7fa44578d640 1 -- 172.21.15.148:0/1839327928 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4401f4760 con 0x7fa4400c87f0 2024-03-29T04:10:48.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.352+0000 7fa44578d640 1 -- 172.21.15.148:0/1839327928 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa440068850 con 0x7fa4400c87f0 2024-03-29T04:10:48.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.352+0000 7fa4377fe640 1 -- 172.21.15.148:0/1839327928 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa43001b070 con 0x7fa4400c87f0 2024-03-29T04:10:48.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.353+0000 7fa4377fe640 1 --2- 172.21.15.148:0/1839327928 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa42006c530 0x7fa42006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:48.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.353+0000 7fa4377fe640 1 -- 172.21.15.148:0/1839327928 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa43008b090 con 0x7fa4400c87f0 2024-03-29T04:10:48.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.353+0000 7fa43effd640 1 --2- 172.21.15.148:0/1839327928 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa42006c530 0x7fa42006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:48.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.353+0000 7fa43effd640 1 --2- 172.21.15.148:0/1839327928 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa42006c530 0x7fa42006e9f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa438036000 tx=0x7fa438008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:48.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.357+0000 7fa4377fe640 1 -- 172.21.15.148:0/1839327928 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa430059340 con 0x7fa4400c87f0 2024-03-29T04:10:48.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.537+0000 7fa44578d640 1 -- 172.21.15.148:0/1839327928 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa44006c680 con 0x7fa4400c87f0 2024-03-29T04:10:48.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.537+0000 7fa4377fe640 1 -- 172.21.15.148:0/1839327928 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fa430058c40 con 0x7fa4400c87f0 2024-03-29T04:10:48.516 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:10:48.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.539+0000 7fa44578d640 1 -- 172.21.15.148:0/1839327928 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa42006c530 msgr2=0x7fa42006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:48.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.539+0000 7fa44578d640 1 --2- 172.21.15.148:0/1839327928 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa42006c530 0x7fa42006e9f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa438036000 tx=0x7fa438008030 comp rx=0 tx=0).stop 2024-03-29T04:10:48.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.539+0000 7fa44578d640 1 -- 172.21.15.148:0/1839327928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4400c87f0 msgr2=0x7fa4400cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:48.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.539+0000 7fa44578d640 1 --2- 172.21.15.148:0/1839327928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4400c87f0 0x7fa4400cd570 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa43000c7a0 tx=0x7fa43000cc60 comp rx=0 tx=0).stop 2024-03-29T04:10:48.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.539+0000 7fa44578d640 1 -- 172.21.15.148:0/1839327928 shutdown_connections 2024-03-29T04:10:48.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.539+0000 7fa44578d640 1 --2- 172.21.15.148:0/1839327928 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa42006c530 0x7fa42006e9f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:48.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.539+0000 7fa44578d640 1 --2- 172.21.15.148:0/1839327928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4400c87f0 0x7fa4400cd570 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:48.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.539+0000 7fa44578d640 1 --2- 172.21.15.148:0/1839327928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa44015e3c0 0x7fa44015b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:48.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.539+0000 7fa44578d640 1 --2- 172.21.15.148:0/1839327928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4400b3430 0x7fa4400b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:48.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.539+0000 7fa44578d640 1 -- 172.21.15.148:0/1839327928 >> 172.21.15.148:0/1839327928 conn(0x7fa4400b6430 msgr2=0x7fa4400b5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:48.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.539+0000 7fa44578d640 1 -- 172.21.15.148:0/1839327928 shutdown_connections 2024-03-29T04:10:48.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.539+0000 7fa44578d640 1 -- 172.21.15.148:0/1839327928 wait complete. 2024-03-29T04:10:48.534 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:10:48.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.672+0000 7fe812c0d640 1 Processor -- start 2024-03-29T04:10:48.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.672+0000 7fe812c0d640 1 -- start start 2024-03-29T04:10:48.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.674+0000 7fe812c0d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe80c15e3c0 0x7fe80c161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:48.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.674+0000 7fe812c0d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe80c0b3430 0x7fe80c06b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:48.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.674+0000 7fe812c0d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe80c0cf330 con 0x7fe80c0b2f00 2024-03-29T04:10:48.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.674+0000 7fe812c0d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fe80c0cf960 con 0x7fe80c15e3c0 2024-03-29T04:10:48.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.674+0000 7fe812c0d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe80c162cf0 con 0x7fe80c0b3430 2024-03-29T04:10:48.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.674+0000 7fe8109a9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe80c15e3c0 0x7fe80c161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:48.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.674+0000 7fe8109a9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe80c15e3c0 0x7fe80c161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53268/0 (socket says 172.21.15.148:53268) 2024-03-29T04:10:48.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.674+0000 7fe8109a9640 1 -- 172.21.15.148:0/1829984698 learned_addr learned my addr 172.21.15.148:0/1829984698 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:48.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.674+0000 7fe8109a9640 1 -- 172.21.15.148:0/1829984698 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe80c0b3430 msgr2=0x7fe80c06b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:48.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.674+0000 7fe8109a9640 1 --2- 172.21.15.148:0/1829984698 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe80c0b3430 0x7fe80c06b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:48.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.674+0000 7fe8109a9640 1 -- 172.21.15.148:0/1829984698 >> v1:172.21.15.157:6789/0 conn(0x7fe80c0b2f00 legacy=0x7fe80c1624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:48.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.674+0000 7fe8109a9640 1 -- 172.21.15.148:0/1829984698 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80c0cf5f0 con 0x7fe80c15e3c0 2024-03-29T04:10:48.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.675+0000 7fe8109a9640 1 --2- 172.21.15.148:0/1829984698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe80c15e3c0 0x7fe80c161f70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe800009870 tx=0x7fe80002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3318eb4a88777a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:48.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.675+0000 7fe80b7fe640 1 -- 172.21.15.148:0/1829984698 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8000317c0 con 0x7fe80c15e3c0 2024-03-29T04:10:48.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.675+0000 7fe80b7fe640 1 -- 172.21.15.148:0/1829984698 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe800037070 con 0x7fe80c15e3c0 2024-03-29T04:10:48.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.675+0000 7fe80b7fe640 1 -- 172.21.15.148:0/1829984698 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe800031b80 con 0x7fe80c15e3c0 2024-03-29T04:10:48.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.676+0000 7fe812c0d640 1 -- 172.21.15.148:0/1829984698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe80c15e3c0 msgr2=0x7fe80c161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:48.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.676+0000 7fe812c0d640 1 --2- 172.21.15.148:0/1829984698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe80c15e3c0 0x7fe80c161f70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe800009870 tx=0x7fe80002b860 comp rx=0 tx=0).stop 2024-03-29T04:10:48.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.676+0000 7fe812c0d640 1 -- 172.21.15.148:0/1829984698 shutdown_connections 2024-03-29T04:10:48.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.676+0000 7fe812c0d640 1 --2- 172.21.15.148:0/1829984698 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe80c0b3430 0x7fe80c06b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:48.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.676+0000 7fe812c0d640 1 --2- 172.21.15.148:0/1829984698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe80c15e3c0 0x7fe80c161f70 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:48.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.676+0000 7fe812c0d640 1 -- 172.21.15.148:0/1829984698 >> 172.21.15.148:0/1829984698 conn(0x7fe80c0b6430 msgr2=0x7fe80c0d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:48.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.676+0000 7fe812c0d640 1 -- 172.21.15.148:0/1829984698 shutdown_connections 2024-03-29T04:10:48.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.676+0000 7fe812c0d640 1 -- 172.21.15.148:0/1829984698 wait complete. 2024-03-29T04:10:48.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.677+0000 7fe812c0d640 1 Processor -- start 2024-03-29T04:10:48.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.677+0000 7fe812c0d640 1 -- start start 2024-03-29T04:10:48.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.678+0000 7fe812c0d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe80c0b3430 0x7fe80c1e8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:48.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.678+0000 7fe812c0d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe80c15e3c0 0x7fe80c14c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:48.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.678+0000 7fe812c0d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe80c0ccac0 0x7fe80c0cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:48.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.678+0000 7fe812c0d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c0b3b20 con 0x7fe80c0ccac0 2024-03-29T04:10:48.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.678+0000 7fe812c0d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c0b39a0 con 0x7fe80c15e3c0 2024-03-29T04:10:48.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.678+0000 7fe812c0d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe80c06bd30 con 0x7fe80c0b3430 2024-03-29T04:10:48.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.678+0000 7fe8111aa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe80c0ccac0 0x7fe80c0cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:48.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.678+0000 7fe8111aa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe80c0ccac0 0x7fe80c0cd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53272/0 (socket says 172.21.15.148:53272) 2024-03-29T04:10:48.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.678+0000 7fe8111aa640 1 -- 172.21.15.148:0/1644534536 learned_addr learned my addr 172.21.15.148:0/1644534536 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:48.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.678+0000 7fe8109a9640 1 --2- 172.21.15.148:0/1644534536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe80c0b3430 0x7fe80c1e8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:48.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.678+0000 7fe80bfff640 1 --2- 172.21.15.148:0/1644534536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe80c15e3c0 0x7fe80c14c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:48.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.678+0000 7fe8111aa640 1 -- 172.21.15.148:0/1644534536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe80c0b3430 msgr2=0x7fe80c1e8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:48.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.678+0000 7fe8111aa640 1 --2- 172.21.15.148:0/1644534536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe80c0b3430 0x7fe80c1e8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:48.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.678+0000 7fe8111aa640 1 -- 172.21.15.148:0/1644534536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe80c15e3c0 msgr2=0x7fe80c14c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:48.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.678+0000 7fe8111aa640 1 --2- 172.21.15.148:0/1644534536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe80c15e3c0 0x7fe80c14c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:48.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.678+0000 7fe8111aa640 1 -- 172.21.15.148:0/1644534536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80c1e8fd0 con 0x7fe80c0ccac0 2024-03-29T04:10:48.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.679+0000 7fe8111aa640 1 --2- 172.21.15.148:0/1644534536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe80c0ccac0 0x7fe80c0cd110 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe80400b9a0 tx=0x7fe80400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:48.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.679+0000 7fe8097fa640 1 -- 172.21.15.148:0/1644534536 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe804019070 con 0x7fe80c0ccac0 2024-03-29T04:10:48.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.679+0000 7fe8097fa640 1 -- 172.21.15.148:0/1644534536 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe80400cb60 con 0x7fe80c0ccac0 2024-03-29T04:10:48.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.679+0000 7fe812c0d640 1 -- 172.21.15.148:0/1644534536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe80c1e92b0 con 0x7fe80c0ccac0 2024-03-29T04:10:48.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.679+0000 7fe8097fa640 1 -- 172.21.15.148:0/1644534536 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe804014440 con 0x7fe80c0ccac0 2024-03-29T04:10:48.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.679+0000 7fe812c0d640 1 -- 172.21.15.148:0/1644534536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe80c1e9870 con 0x7fe80c0ccac0 2024-03-29T04:10:48.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.680+0000 7fe812c0d640 1 -- 172.21.15.148:0/1644534536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe80c068850 con 0x7fe80c0ccac0 2024-03-29T04:10:48.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.680+0000 7fe8097fa640 1 -- 172.21.15.148:0/1644534536 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe80401c070 con 0x7fe80c0ccac0 2024-03-29T04:10:48.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.681+0000 7fe8097fa640 1 --2- 172.21.15.148:0/1644534536 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe7f006c530 0x7fe7f006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:48.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.681+0000 7fe8097fa640 1 -- 172.21.15.148:0/1644534536 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe80408b750 con 0x7fe80c0ccac0 2024-03-29T04:10:48.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.681+0000 7fe8109a9640 1 --2- 172.21.15.148:0/1644534536 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe7f006c530 0x7fe7f006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:48.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.681+0000 7fe8109a9640 1 --2- 172.21.15.148:0/1644534536 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe7f006c530 0x7fe7f006e9f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe800036000 tx=0x7fe80003d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:48.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.685+0000 7fe8097fa640 1 -- 172.21.15.148:0/1644534536 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe804059320 con 0x7fe80c0ccac0 2024-03-29T04:10:48.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.860+0000 7fe812c0d640 1 -- 172.21.15.148:0/1644534536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe80c0d00f0 con 0x7fe80c0ccac0 2024-03-29T04:10:48.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.860+0000 7fe8097fa640 1 -- 172.21.15.148:0/1644534536 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fe80405d620 con 0x7fe80c0ccac0 2024-03-29T04:10:48.840 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-29T04:10:48.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.862+0000 7fe812c0d640 1 -- 172.21.15.148:0/1644534536 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe7f006c530 msgr2=0x7fe7f006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:48.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.862+0000 7fe812c0d640 1 --2- 172.21.15.148:0/1644534536 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe7f006c530 0x7fe7f006e9f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe800036000 tx=0x7fe80003d040 comp rx=0 tx=0).stop 2024-03-29T04:10:48.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.862+0000 7fe812c0d640 1 -- 172.21.15.148:0/1644534536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe80c0ccac0 msgr2=0x7fe80c0cd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:48.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.862+0000 7fe812c0d640 1 --2- 172.21.15.148:0/1644534536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe80c0ccac0 0x7fe80c0cd110 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe80400b9a0 tx=0x7fe80400be60 comp rx=0 tx=0).stop 2024-03-29T04:10:48.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.862+0000 7fe812c0d640 1 -- 172.21.15.148:0/1644534536 shutdown_connections 2024-03-29T04:10:48.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.862+0000 7fe812c0d640 1 --2- 172.21.15.148:0/1644534536 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe7f006c530 0x7fe7f006e9f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:48.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.862+0000 7fe812c0d640 1 --2- 172.21.15.148:0/1644534536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe80c0ccac0 0x7fe80c0cd110 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:48.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.862+0000 7fe812c0d640 1 --2- 172.21.15.148:0/1644534536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe80c15e3c0 0x7fe80c14c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:48.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.862+0000 7fe812c0d640 1 --2- 172.21.15.148:0/1644534536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe80c0b3430 0x7fe80c1e8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:48.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.862+0000 7fe812c0d640 1 -- 172.21.15.148:0/1644534536 >> 172.21.15.148:0/1644534536 conn(0x7fe80c0b6430 msgr2=0x7fe80c15c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:48.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.862+0000 7fe812c0d640 1 -- 172.21.15.148:0/1644534536 shutdown_connections 2024-03-29T04:10:48.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:48.863+0000 7fe812c0d640 1 -- 172.21.15.148:0/1644534536 wait complete. 2024-03-29T04:10:48.852 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:10:48.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.013+0000 7f50fb6d9640 1 Processor -- start 2024-03-29T04:10:48.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.013+0000 7f50fb6d9640 1 -- start start 2024-03-29T04:10:48.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.014+0000 7f50fb6d9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f415e3c0 0x7f50f4161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:48.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.014+0000 7f50fb6d9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f50f40b3430 0x7f50f406b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:48.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.014+0000 7f50fb6d9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50f40cf330 con 0x7f50f40b2f00 2024-03-29T04:10:48.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.014+0000 7f50fb6d9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f50f40cf960 con 0x7f50f40b3430 2024-03-29T04:10:48.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.014+0000 7f50fb6d9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f50f4162cf0 con 0x7f50f415e3c0 2024-03-29T04:10:48.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.014+0000 7f50f8c74640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f50f40b2f00 0x7f50f41624b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50348/0 (socket says 172.21.15.148:50348) 2024-03-29T04:10:48.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.014+0000 7f50f8c74640 1 -- 172.21.15.148:0/3715249464 learned_addr learned my addr 172.21.15.148:0/3715249464 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:48.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.014+0000 7f50ebfff640 1 -- 172.21.15.148:0/3715249464 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2084375322 0 0) 0x7f50f40cf330 con 0x7f50f40b2f00 2024-03-29T04:10:48.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.014+0000 7f50f9c76640 1 --2- 172.21.15.148:0/3715249464 >> v2:172.21.15.157:3300/0 conn(0x7f50f40b3430 0x7f50f406b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:48.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.014+0000 7f50ebfff640 1 -- 172.21.15.148:0/3715249464 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50e0003520 con 0x7f50f40b2f00 2024-03-29T04:10:48.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.014+0000 7f50f9475640 1 --2- 172.21.15.148:0/3715249464 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f415e3c0 0x7f50f4161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:48.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.015+0000 7f50ebfff640 1 -- 172.21.15.148:0/3715249464 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3664175117 0 0) 0x7f50e0003520 con 0x7f50f40b2f00 2024-03-29T04:10:48.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.015+0000 7f50ebfff640 1 -- 172.21.15.148:0/3715249464 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f415e3c0 msgr2=0x7f50f4161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:48.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.015+0000 7f50ebfff640 1 --2- 172.21.15.148:0/3715249464 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f415e3c0 0x7f50f4161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:48.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.015+0000 7f50ebfff640 1 -- 172.21.15.148:0/3715249464 >> v2:172.21.15.157:3300/0 conn(0x7f50f40b3430 msgr2=0x7f50f406b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:48.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.015+0000 7f50ebfff640 1 --2- 172.21.15.148:0/3715249464 >> v2:172.21.15.157:3300/0 conn(0x7f50f40b3430 0x7f50f406b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:48.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.015+0000 7f50ebfff640 1 -- 172.21.15.148:0/3715249464 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f40cf5f0 con 0x7f50f40b2f00 2024-03-29T04:10:48.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.015+0000 7f50ebfff640 1 -- 172.21.15.148:0/3715249464 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f50e4001670 con 0x7f50f40b2f00 2024-03-29T04:10:48.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.015+0000 7f50ebfff640 1 -- 172.21.15.148:0/3715249464 >> v1:172.21.15.148:6789/0 conn(0x7f50f40b2f00 legacy=0x7f50f41624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:48.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.015+0000 7f50ebfff640 1 --2- 172.21.15.148:0/3715249464 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f415e3c0 0x7f50e00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:48.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.015+0000 7f50ebfff640 1 --2- 172.21.15.148:0/3715249464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50e0003f30 0x7f50e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:48.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.015+0000 7f50ebfff640 1 --2- 172.21.15.148:0/3715249464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50e0004d40 0x7f50e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:48.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.015+0000 7f50ebfff640 1 -- 172.21.15.148:0/3715249464 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f50f40cf960 con 0x7f50e0004d40 2024-03-29T04:10:48.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.015+0000 7f50ebfff640 1 -- 172.21.15.148:0/3715249464 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4162cf0 con 0x7f50e0003f30 2024-03-29T04:10:48.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.015+0000 7f50ebfff640 1 -- 172.21.15.148:0/3715249464 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f50e000ba10 con 0x7f50f415e3c0 2024-03-29T04:10:48.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.015+0000 7f50f9c76640 1 --2- 172.21.15.148:0/3715249464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50e0004d40 0x7f50e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:48.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.015+0000 7f50f9475640 1 --2- 172.21.15.148:0/3715249464 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f415e3c0 0x7f50e00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:48.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.015+0000 7f50f8c74640 1 --2- 172.21.15.148:0/3715249464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50e0003f30 0x7f50e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:48.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.015+0000 7f50f9c76640 1 -- 172.21.15.148:0/3715249464 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f415e3c0 msgr2=0x7f50e00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:48.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.015+0000 7f50f9c76640 1 --2- 172.21.15.148:0/3715249464 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f415e3c0 0x7f50e00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:48.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.015+0000 7f50f9c76640 1 -- 172.21.15.148:0/3715249464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50e0003f30 msgr2=0x7f50e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:48.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.015+0000 7f50f9c76640 1 --2- 172.21.15.148:0/3715249464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50e0003f30 0x7f50e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:48.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.015+0000 7f50f9c76640 1 -- 172.21.15.148:0/3715249464 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e0003520 con 0x7f50e0004d40 2024-03-29T04:10:48.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.016+0000 7f50f9c76640 1 --2- 172.21.15.148:0/3715249464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50e0004d40 0x7f50e00051c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00e170 tx=0x7f50e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd95b587292e4373 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:48.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.016+0000 7f50ebfff640 1 -- 172.21.15.148:0/3715249464 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec039070 con 0x7f50e0004d40 2024-03-29T04:10:48.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.016+0000 7f50ebfff640 1 -- 172.21.15.148:0/3715249464 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50ec02cc20 con 0x7f50e0004d40 2024-03-29T04:10:48.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.016+0000 7f50ebfff640 1 -- 172.21.15.148:0/3715249464 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec009c10 con 0x7f50e0004d40 2024-03-29T04:10:49.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.017+0000 7f50fb6d9640 1 -- 172.21.15.148:0/3715249464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50e0004d40 msgr2=0x7f50e00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:49.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.017+0000 7f50fb6d9640 1 --2- 172.21.15.148:0/3715249464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50e0004d40 0x7f50e00051c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00e170 tx=0x7f50e0005830 comp rx=0 tx=0).stop 2024-03-29T04:10:49.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.017+0000 7f50f9c76640 1 -- 172.21.15.148:0/3715249464 reap_dead start 2024-03-29T04:10:49.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.017+0000 7f50fb6d9640 1 -- 172.21.15.148:0/3715249464 shutdown_connections 2024-03-29T04:10:49.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.017+0000 7f50fb6d9640 1 -- 172.21.15.148:0/3715249464 >> 172.21.15.148:0/3715249464 conn(0x7f50f40b6430 msgr2=0x7f50f40d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:49.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.017+0000 7f50fb6d9640 1 -- 172.21.15.148:0/3715249464 shutdown_connections 2024-03-29T04:10:49.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.017+0000 7f50fb6d9640 1 -- 172.21.15.148:0/3715249464 wait complete. 2024-03-29T04:10:49.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.018+0000 7f50fb6d9640 1 Processor -- start 2024-03-29T04:10:49.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.018+0000 7f50fb6d9640 1 -- start start 2024-03-29T04:10:49.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.018+0000 7f50fb6d9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50e0004d40 0x7f50f40cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:49.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.019+0000 7f50fb6d9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50f41526d0 0x7f50f4155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:49.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.019+0000 7f50fb6d9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f4156100 0x7f50f40cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:49.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.019+0000 7f50fb6d9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f50f40b3b20 con 0x7f50f41526d0 2024-03-29T04:10:49.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.019+0000 7f50fb6d9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f50f40b39a0 con 0x7f50e0004d40 2024-03-29T04:10:49.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.019+0000 7f50fb6d9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f50f4161f70 con 0x7f50f4156100 2024-03-29T04:10:49.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.019+0000 7f50f8c74640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50f41526d0 0x7f50f4155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:49.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.019+0000 7f50f8c74640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50f41526d0 0x7f50f4155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53288/0 (socket says 172.21.15.148:53288) 2024-03-29T04:10:49.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.019+0000 7f50f8c74640 1 -- 172.21.15.148:0/1507898581 learned_addr learned my addr 172.21.15.148:0/1507898581 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:49.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.019+0000 7f50f9c76640 1 --2- 172.21.15.148:0/1507898581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f4156100 0x7f50f40cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:49.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.019+0000 7f50f9475640 1 --2- 172.21.15.148:0/1507898581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50e0004d40 0x7f50f40cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:49.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.019+0000 7f50f8c74640 1 -- 172.21.15.148:0/1507898581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f4156100 msgr2=0x7f50f40cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:49.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.019+0000 7f50f8c74640 1 --2- 172.21.15.148:0/1507898581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f4156100 0x7f50f40cca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.019+0000 7f50f8c74640 1 -- 172.21.15.148:0/1507898581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50e0004d40 msgr2=0x7f50f40cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:49.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.019+0000 7f50f8c74640 1 --2- 172.21.15.148:0/1507898581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50e0004d40 0x7f50f40cc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.019+0000 7f50f8c74640 1 -- 172.21.15.148:0/1507898581 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f40ca300 con 0x7f50f41526d0 2024-03-29T04:10:49.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.019+0000 7f50f8c74640 1 --2- 172.21.15.148:0/1507898581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50f41526d0 0x7f50f4155bc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f50e4007340 tx=0x7f50e4007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:49.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.019+0000 7f50e9ffb640 1 -- 172.21.15.148:0/1507898581 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e400f940 con 0x7f50f41526d0 2024-03-29T04:10:49.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.019+0000 7f50e9ffb640 1 -- 172.21.15.148:0/1507898581 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50e4018070 con 0x7f50f41526d0 2024-03-29T04:10:49.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.019+0000 7f50fb6d9640 1 -- 172.21.15.148:0/1507898581 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f40ca5a0 con 0x7f50f41526d0 2024-03-29T04:10:49.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.019+0000 7f50e9ffb640 1 -- 172.21.15.148:0/1507898581 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e400fd40 con 0x7f50f41526d0 2024-03-29T04:10:49.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.020+0000 7f50fb6d9640 1 -- 172.21.15.148:0/1507898581 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f40cab60 con 0x7f50f41526d0 2024-03-29T04:10:49.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.021+0000 7f50fb6d9640 1 -- 172.21.15.148:0/1507898581 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50f4068850 con 0x7f50f41526d0 2024-03-29T04:10:49.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.021+0000 7f50e9ffb640 1 -- 172.21.15.148:0/1507898581 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f50e401d460 con 0x7f50f41526d0 2024-03-29T04:10:49.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.021+0000 7f50e9ffb640 1 --2- 172.21.15.148:0/1507898581 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f50d806c530 0x7f50d806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:49.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.021+0000 7f50e9ffb640 1 -- 172.21.15.148:0/1507898581 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f50e408ba30 con 0x7f50f41526d0 2024-03-29T04:10:49.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.021+0000 7f50f9475640 1 --2- 172.21.15.148:0/1507898581 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f50d806c530 0x7f50d806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:49.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.022+0000 7f50f9475640 1 --2- 172.21.15.148:0/1507898581 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f50d806c530 0x7f50d806e9f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f50f41f7940 tx=0x7f50f0009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:49.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.026+0000 7f50e9ffb640 1 -- 172.21.15.148:0/1507898581 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f50e4059690 con 0x7f50f41526d0 2024-03-29T04:10:49.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.206+0000 7f50fb6d9640 1 -- 172.21.15.148:0/1507898581 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f50f41564e0 con 0x7f50f41526d0 2024-03-29T04:10:49.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.206+0000 7f50e9ffb640 1 -- 172.21.15.148:0/1507898581 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f50e405d990 con 0x7f50f41526d0 2024-03-29T04:10:49.185 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:10:49.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.208+0000 7f50fb6d9640 1 -- 172.21.15.148:0/1507898581 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f50d806c530 msgr2=0x7f50d806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:49.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.208+0000 7f50fb6d9640 1 --2- 172.21.15.148:0/1507898581 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f50d806c530 0x7f50d806e9f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f50f41f7940 tx=0x7f50f0009190 comp rx=0 tx=0).stop 2024-03-29T04:10:49.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.208+0000 7f50fb6d9640 1 -- 172.21.15.148:0/1507898581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50f41526d0 msgr2=0x7f50f4155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:49.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.208+0000 7f50fb6d9640 1 --2- 172.21.15.148:0/1507898581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50f41526d0 0x7f50f4155bc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f50e4007340 tx=0x7f50e4007800 comp rx=0 tx=0).stop 2024-03-29T04:10:49.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.208+0000 7f50fb6d9640 1 -- 172.21.15.148:0/1507898581 shutdown_connections 2024-03-29T04:10:49.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.208+0000 7f50fb6d9640 1 --2- 172.21.15.148:0/1507898581 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f50d806c530 0x7f50d806e9f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.208+0000 7f50fb6d9640 1 --2- 172.21.15.148:0/1507898581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50f4156100 0x7f50f40cca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.208+0000 7f50fb6d9640 1 --2- 172.21.15.148:0/1507898581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50f41526d0 0x7f50f4155bc0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.208+0000 7f50fb6d9640 1 --2- 172.21.15.148:0/1507898581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50e0004d40 0x7f50f40cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.208+0000 7f50fb6d9640 1 -- 172.21.15.148:0/1507898581 >> 172.21.15.148:0/1507898581 conn(0x7f50f40b6430 msgr2=0x7f50f40b4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:49.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.208+0000 7f50fb6d9640 1 -- 172.21.15.148:0/1507898581 shutdown_connections 2024-03-29T04:10:49.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.208+0000 7f50fb6d9640 1 -- 172.21.15.148:0/1507898581 wait complete. 2024-03-29T04:10:49.198 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:49.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.320+0000 7faeeaf9d640 1 Processor -- start 2024-03-29T04:10:49.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.320+0000 7faeeaf9d640 1 -- start start 2024-03-29T04:10:49.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.321+0000 7faeeaf9d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee415e3e0 0x7faee40d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:49.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.321+0000 7faeeaf9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faee40ba030 0x7faee40d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:49.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.321+0000 7faeeaf9d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faee415e7c0 con 0x7faee40d10d0 2024-03-29T04:10:49.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.321+0000 7faeeaf9d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7faee40d3840 con 0x7faee40ba030 2024-03-29T04:10:49.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.321+0000 7faeeaf9d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faee40d2c30 con 0x7faee415e3e0 2024-03-29T04:10:49.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.322+0000 7faee953a640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7faee40d10d0 0x7faee40b72d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50358/0 (socket says 172.21.15.148:50358) 2024-03-29T04:10:49.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.322+0000 7faee953a640 1 -- 172.21.15.148:0/4282263401 learned_addr learned my addr 172.21.15.148:0/4282263401 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:49.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.322+0000 7faee8d39640 1 --2- 172.21.15.148:0/4282263401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee415e3e0 0x7faee40d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:49.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.322+0000 7faee3fff640 1 --2- 172.21.15.148:0/4282263401 >> v2:172.21.15.157:3300/0 conn(0x7faee40ba030 0x7faee40d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:49.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.322+0000 7faee37fe640 1 -- 172.21.15.148:0/4282263401 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3248289333 0 0) 0x7faee415e7c0 con 0x7faee40d10d0 2024-03-29T04:10:49.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.322+0000 7faee37fe640 1 -- 172.21.15.148:0/4282263401 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faed4003520 con 0x7faee40d10d0 2024-03-29T04:10:49.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.322+0000 7faee37fe640 1 -- 172.21.15.148:0/4282263401 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3814498691 0 0) 0x7faed4003520 con 0x7faee40d10d0 2024-03-29T04:10:49.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.322+0000 7faee37fe640 1 -- 172.21.15.148:0/4282263401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee415e3e0 msgr2=0x7faee40d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:49.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.322+0000 7faee37fe640 1 --2- 172.21.15.148:0/4282263401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee415e3e0 0x7faee40d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.322+0000 7faee37fe640 1 -- 172.21.15.148:0/4282263401 >> v2:172.21.15.157:3300/0 conn(0x7faee40ba030 msgr2=0x7faee40d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:49.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.322+0000 7faee37fe640 1 --2- 172.21.15.148:0/4282263401 >> v2:172.21.15.157:3300/0 conn(0x7faee40ba030 0x7faee40d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.322+0000 7faee37fe640 1 -- 172.21.15.148:0/4282263401 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee40d2f10 con 0x7faee40d10d0 2024-03-29T04:10:49.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.322+0000 7faee37fe640 1 -- 172.21.15.148:0/4282263401 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7faedc0019f0 con 0x7faee40d10d0 2024-03-29T04:10:49.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.322+0000 7faee37fe640 1 -- 172.21.15.148:0/4282263401 >> v1:172.21.15.148:6789/0 conn(0x7faee40d10d0 legacy=0x7faee40b72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:49.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.322+0000 7faee37fe640 1 --2- 172.21.15.148:0/4282263401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faed40039b0 0x7faed4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:49.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.322+0000 7faee37fe640 1 --2- 172.21.15.148:0/4282263401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faed4004800 0x7faed4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:49.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.322+0000 7faee37fe640 1 --2- 172.21.15.148:0/4282263401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee415e3e0 0x7faed40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:49.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.322+0000 7faee37fe640 1 -- 172.21.15.148:0/4282263401 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faee40d3840 con 0x7faed40039b0 2024-03-29T04:10:49.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.322+0000 7faee37fe640 1 -- 172.21.15.148:0/4282263401 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faee40d2c30 con 0x7faed4004800 2024-03-29T04:10:49.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.322+0000 7faee37fe640 1 -- 172.21.15.148:0/4282263401 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faed400b9f0 con 0x7faee415e3e0 2024-03-29T04:10:49.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.322+0000 7faee8d39640 1 --2- 172.21.15.148:0/4282263401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faed40039b0 0x7faed4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:49.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.322+0000 7faee3fff640 1 --2- 172.21.15.148:0/4282263401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faed4004800 0x7faed4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:49.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.322+0000 7faee953a640 1 --2- 172.21.15.148:0/4282263401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee415e3e0 0x7faed40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:49.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.323+0000 7faee8d39640 1 -- 172.21.15.148:0/4282263401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee415e3e0 msgr2=0x7faed40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:49.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.323+0000 7faee8d39640 1 --2- 172.21.15.148:0/4282263401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee415e3e0 0x7faed40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.323+0000 7faee8d39640 1 -- 172.21.15.148:0/4282263401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faed4004800 msgr2=0x7faed4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:49.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.323+0000 7faee8d39640 1 --2- 172.21.15.148:0/4282263401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faed4004800 0x7faed4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.323+0000 7faee8d39640 1 -- 172.21.15.148:0/4282263401 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed4003520 con 0x7faed40039b0 2024-03-29T04:10:49.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.323+0000 7faee953a640 1 --2- 172.21.15.148:0/4282263401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee415e3e0 0x7faed40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:49.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.323+0000 7faee8d39640 1 --2- 172.21.15.148:0/4282263401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faed40039b0 0x7faed4003dd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faed8009ac0 tx=0x7faed4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14c83bda78b7474 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:49.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.323+0000 7faee37fe640 1 -- 172.21.15.148:0/4282263401 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed8039070 con 0x7faed40039b0 2024-03-29T04:10:49.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.323+0000 7faee37fe640 1 -- 172.21.15.148:0/4282263401 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed802c6b0 con 0x7faed40039b0 2024-03-29T04:10:49.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.323+0000 7faee37fe640 1 -- 172.21.15.148:0/4282263401 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed8009690 con 0x7faed40039b0 2024-03-29T04:10:49.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.324+0000 7faeeaf9d640 1 -- 172.21.15.148:0/4282263401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faed40039b0 msgr2=0x7faed4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:49.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.324+0000 7faeeaf9d640 1 --2- 172.21.15.148:0/4282263401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faed40039b0 0x7faed4003dd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faed8009ac0 tx=0x7faed4005830 comp rx=0 tx=0).stop 2024-03-29T04:10:49.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.324+0000 7faee953a640 1 -- 172.21.15.148:0/4282263401 reap_dead start 2024-03-29T04:10:49.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.324+0000 7faeeaf9d640 1 -- 172.21.15.148:0/4282263401 shutdown_connections 2024-03-29T04:10:49.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.324+0000 7faeeaf9d640 1 -- 172.21.15.148:0/4282263401 >> 172.21.15.148:0/4282263401 conn(0x7faee40b6210 msgr2=0x7faee40b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:49.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.324+0000 7faeeaf9d640 1 -- 172.21.15.148:0/4282263401 shutdown_connections 2024-03-29T04:10:49.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.324+0000 7faeeaf9d640 1 -- 172.21.15.148:0/4282263401 wait complete. 2024-03-29T04:10:49.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.325+0000 7faeeaf9d640 1 Processor -- start 2024-03-29T04:10:49.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.325+0000 7faeeaf9d640 1 -- start start 2024-03-29T04:10:49.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.326+0000 7faeeaf9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faed40039b0 0x7faee4154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:49.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.326+0000 7faeeaf9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faee4158120 0x7faee4154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:49.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.326+0000 7faeeaf9d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee4155510 0x7faee41f8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:49.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.326+0000 7faeeaf9d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faee40d2870 con 0x7faee4158120 2024-03-29T04:10:49.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.326+0000 7faeeaf9d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faee40d34d0 con 0x7faed40039b0 2024-03-29T04:10:49.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.326+0000 7faeeaf9d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faee40d29f0 con 0x7faee4155510 2024-03-29T04:10:49.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.326+0000 7faee3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faee4158120 0x7faee4154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:49.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.326+0000 7faee3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faee4158120 0x7faee4154fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53314/0 (socket says 172.21.15.148:53314) 2024-03-29T04:10:49.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.326+0000 7faee3fff640 1 -- 172.21.15.148:0/1175460652 learned_addr learned my addr 172.21.15.148:0/1175460652 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:49.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.326+0000 7faee8d39640 1 --2- 172.21.15.148:0/1175460652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faed40039b0 0x7faee4154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:49.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.326+0000 7faee953a640 1 --2- 172.21.15.148:0/1175460652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee4155510 0x7faee41f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:49.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.326+0000 7faee3fff640 1 -- 172.21.15.148:0/1175460652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee4155510 msgr2=0x7faee41f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:49.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.326+0000 7faee3fff640 1 --2- 172.21.15.148:0/1175460652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee4155510 0x7faee41f8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.326+0000 7faee3fff640 1 -- 172.21.15.148:0/1175460652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faed40039b0 msgr2=0x7faee4154840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:49.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.326+0000 7faee3fff640 1 --2- 172.21.15.148:0/1175460652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faed40039b0 0x7faee4154840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.326+0000 7faee3fff640 1 -- 172.21.15.148:0/1175460652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee41573d0 con 0x7faee4158120 2024-03-29T04:10:49.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.327+0000 7faee3fff640 1 --2- 172.21.15.148:0/1175460652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faee4158120 0x7faee4154fd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7faed0011aa0 tx=0x7faed0011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:49.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.327+0000 7faee17fa640 1 -- 172.21.15.148:0/1175460652 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed000fb80 con 0x7faee4158120 2024-03-29T04:10:49.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.327+0000 7faee17fa640 1 -- 172.21.15.148:0/1175460652 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed000fd20 con 0x7faee4158120 2024-03-29T04:10:49.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.327+0000 7faeeaf9d640 1 -- 172.21.15.148:0/1175460652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee4154d80 con 0x7faee4158120 2024-03-29T04:10:49.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.327+0000 7faee17fa640 1 -- 172.21.15.148:0/1175460652 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed0014660 con 0x7faee4158120 2024-03-29T04:10:49.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.327+0000 7faeeaf9d640 1 -- 172.21.15.148:0/1175460652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee4157920 con 0x7faee4158120 2024-03-29T04:10:49.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.328+0000 7faeeaf9d640 1 -- 172.21.15.148:0/1175460652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faee4155920 con 0x7faee4158120 2024-03-29T04:10:49.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.328+0000 7faee17fa640 1 -- 172.21.15.148:0/1175460652 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7faed0012040 con 0x7faee4158120 2024-03-29T04:10:49.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.329+0000 7faee17fa640 1 --2- 172.21.15.148:0/1175460652 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7faec806c530 0x7faec806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:49.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.329+0000 7faee17fa640 1 -- 172.21.15.148:0/1175460652 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7faed008c040 con 0x7faee4158120 2024-03-29T04:10:49.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.329+0000 7faee8d39640 1 --2- 172.21.15.148:0/1175460652 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7faec806c530 0x7faec806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:49.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.329+0000 7faee8d39640 1 --2- 172.21.15.148:0/1175460652 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7faec806c530 0x7faec806e9f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7faed8036000 tx=0x7faed8030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:49.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.337+0000 7faee17fa640 1 -- 172.21.15.148:0/1175460652 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7faee4155920 con 0x7faee4158120 2024-03-29T04:10:49.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.513+0000 7faeeaf9d640 1 -- 172.21.15.148:0/1175460652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faee40687e0 con 0x7faee4158120 2024-03-29T04:10:49.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.514+0000 7faee17fa640 1 -- 172.21.15.148:0/1175460652 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18858 (secure 0 0 0) 0x7faed0019030 con 0x7faee4158120 2024-03-29T04:10:49.494 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:49.494 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:41.662934+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:49.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.516+0000 7faeeaf9d640 1 -- 172.21.15.148:0/1175460652 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7faec806c530 msgr2=0x7faec806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:49.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.516+0000 7faeeaf9d640 1 --2- 172.21.15.148:0/1175460652 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7faec806c530 0x7faec806e9f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7faed8036000 tx=0x7faed8030000 comp rx=0 tx=0).stop 2024-03-29T04:10:49.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.516+0000 7faeeaf9d640 1 -- 172.21.15.148:0/1175460652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faee4158120 msgr2=0x7faee4154fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:49.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.516+0000 7faeeaf9d640 1 --2- 172.21.15.148:0/1175460652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faee4158120 0x7faee4154fd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7faed0011aa0 tx=0x7faed0011f60 comp rx=0 tx=0).stop 2024-03-29T04:10:49.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.516+0000 7faeeaf9d640 1 -- 172.21.15.148:0/1175460652 shutdown_connections 2024-03-29T04:10:49.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.516+0000 7faeeaf9d640 1 --2- 172.21.15.148:0/1175460652 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7faec806c530 0x7faec806e9f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.516+0000 7faeeaf9d640 1 --2- 172.21.15.148:0/1175460652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee4155510 0x7faee41f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.516+0000 7faeeaf9d640 1 --2- 172.21.15.148:0/1175460652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faee4158120 0x7faee4154fd0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.516+0000 7faeeaf9d640 1 --2- 172.21.15.148:0/1175460652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faed40039b0 0x7faee4154840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.516+0000 7faeeaf9d640 1 -- 172.21.15.148:0/1175460652 >> 172.21.15.148:0/1175460652 conn(0x7faee40b6210 msgr2=0x7faee40b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:49.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.516+0000 7faeeaf9d640 1 -- 172.21.15.148:0/1175460652 shutdown_connections 2024-03-29T04:10:49.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.516+0000 7faeeaf9d640 1 -- 172.21.15.148:0/1175460652 wait complete. 2024-03-29T04:10:49.507 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:47.898351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:57.922313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:58.829405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:10:02.551763+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:10:49.507 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:10:49.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.633+0000 7f1469d22640 1 Processor -- start 2024-03-29T04:10:49.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.634+0000 7f1469d22640 1 -- start start 2024-03-29T04:10:49.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.635+0000 7f1469d22640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f146415a120 0x7f14640700d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:49.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.635+0000 7f1469d22640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f14640ce310 0x7f146406e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:49.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.635+0000 7f1469d22640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f146406ee10 0x7f14640b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:49.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.635+0000 7f1469d22640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f14640d1b20 con 0x7f14640ce310 2024-03-29T04:10:49.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.635+0000 7f1469d22640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f146406f260 con 0x7f146415a120 2024-03-29T04:10:49.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.635+0000 7f1469d22640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f146415d460 con 0x7f146406ee10 2024-03-29T04:10:49.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.635+0000 7f1462ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f14640ce310 0x7f146406e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:49.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.635+0000 7f1462ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f14640ce310 0x7f146406e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53322/0 (socket says 172.21.15.148:53322) 2024-03-29T04:10:49.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.635+0000 7f1462ffd640 1 -- 172.21.15.148:0/831934921 learned_addr learned my addr 172.21.15.148:0/831934921 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:49.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.635+0000 7f14637fe640 1 --2- 172.21.15.148:0/831934921 >> v2:172.21.15.157:3300/0 conn(0x7f146415a120 0x7f14640700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:49.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.635+0000 7f1462ffd640 1 -- 172.21.15.148:0/831934921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f146406ee10 msgr2=0x7f14640b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:49.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.635+0000 7f1462ffd640 1 --2- 172.21.15.148:0/831934921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f146406ee10 0x7f14640b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.635+0000 7f1462ffd640 1 -- 172.21.15.148:0/831934921 >> v2:172.21.15.157:3300/0 conn(0x7f146415a120 msgr2=0x7f14640700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:49.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.635+0000 7f1462ffd640 1 --2- 172.21.15.148:0/831934921 >> v2:172.21.15.157:3300/0 conn(0x7f146415a120 0x7f14640700d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.635+0000 7f1462ffd640 1 -- 172.21.15.148:0/831934921 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f146415d700 con 0x7f14640ce310 2024-03-29T04:10:49.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.637+0000 7f1462ffd640 1 --2- 172.21.15.148:0/831934921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14640ce310 0x7f146406e8d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1464073010 tx=0x7f144c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc961e25d4ff47a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:49.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.637+0000 7f14627fc640 1 -- 172.21.15.148:0/831934921 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c035020 con 0x7f14640ce310 2024-03-29T04:10:49.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.637+0000 7f14627fc640 1 -- 172.21.15.148:0/831934921 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f144c031870 con 0x7f14640ce310 2024-03-29T04:10:49.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.637+0000 7f14627fc640 1 -- 172.21.15.148:0/831934921 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c031c40 con 0x7f14640ce310 2024-03-29T04:10:49.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.637+0000 7f1469d22640 1 -- 172.21.15.148:0/831934921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14640ce310 msgr2=0x7f146406e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:49.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.637+0000 7f1469d22640 1 --2- 172.21.15.148:0/831934921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14640ce310 0x7f146406e8d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1464073010 tx=0x7f144c02b910 comp rx=0 tx=0).stop 2024-03-29T04:10:49.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.637+0000 7f1469d22640 1 -- 172.21.15.148:0/831934921 shutdown_connections 2024-03-29T04:10:49.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.637+0000 7f1469d22640 1 --2- 172.21.15.148:0/831934921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14640ce310 0x7f146406e8d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.637+0000 7f1469d22640 1 --2- 172.21.15.148:0/831934921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f146406ee10 0x7f14640b6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.637+0000 7f1469d22640 1 --2- 172.21.15.148:0/831934921 >> v2:172.21.15.157:3300/0 conn(0x7f146415a120 0x7f14640700d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.637+0000 7f1469d22640 1 -- 172.21.15.148:0/831934921 >> 172.21.15.148:0/831934921 conn(0x7f14640cead0 msgr2=0x7f14640ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:49.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.638+0000 7f1469d22640 1 -- 172.21.15.148:0/831934921 shutdown_connections 2024-03-29T04:10:49.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.638+0000 7f1469d22640 1 -- 172.21.15.148:0/831934921 wait complete. 2024-03-29T04:10:49.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.639+0000 7f1469d22640 1 Processor -- start 2024-03-29T04:10:49.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.639+0000 7f1469d22640 1 -- start start 2024-03-29T04:10:49.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.639+0000 7f1469d22640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f146406ee10 0x7f14640b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:49.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.640+0000 7f1469d22640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14640ce310 0x7f1464158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:49.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.640+0000 7f1469d22640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f146415a120 0x7f1464158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:49.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.640+0000 7f1469d22640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f146406f5c0 con 0x7f14640ce310 2024-03-29T04:10:49.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.640+0000 7f1469d22640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f14640ce7d0 con 0x7f146415a120 2024-03-29T04:10:49.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.640+0000 7f1469d22640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f146406f740 con 0x7f146406ee10 2024-03-29T04:10:49.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.640+0000 7f1462ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14640ce310 0x7f1464158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:49.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.640+0000 7f1462ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14640ce310 0x7f1464158560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53338/0 (socket says 172.21.15.148:53338) 2024-03-29T04:10:49.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.640+0000 7f1462ffd640 1 -- 172.21.15.148:0/1060583302 learned_addr learned my addr 172.21.15.148:0/1060583302 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:49.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.640+0000 7f1463fff640 1 --2- 172.21.15.148:0/1060583302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f146415a120 0x7f1464158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:49.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.640+0000 7f14637fe640 1 --2- 172.21.15.148:0/1060583302 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f146406ee10 0x7f14640b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:49.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.640+0000 7f1462ffd640 1 -- 172.21.15.148:0/1060583302 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f146406ee10 msgr2=0x7f14640b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:49.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.640+0000 7f1462ffd640 1 --2- 172.21.15.148:0/1060583302 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f146406ee10 0x7f14640b5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.640+0000 7f1462ffd640 1 -- 172.21.15.148:0/1060583302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f146415a120 msgr2=0x7f1464158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:49.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.640+0000 7f1462ffd640 1 --2- 172.21.15.148:0/1060583302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f146415a120 0x7f1464158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.640+0000 7f1462ffd640 1 -- 172.21.15.148:0/1060583302 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14640b62f0 con 0x7f14640ce310 2024-03-29T04:10:49.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.640+0000 7f1462ffd640 1 --2- 172.21.15.148:0/1060583302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14640ce310 0x7f1464158560 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f144c02b950 tx=0x7f144c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:49.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.640+0000 7f145bfff640 1 -- 172.21.15.148:0/1060583302 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c040070 con 0x7f14640ce310 2024-03-29T04:10:49.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.640+0000 7f145bfff640 1 -- 172.21.15.148:0/1060583302 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f144c030d00 con 0x7f14640ce310 2024-03-29T04:10:49.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.640+0000 7f1469d22640 1 -- 172.21.15.148:0/1060583302 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14640b65b0 con 0x7f14640ce310 2024-03-29T04:10:49.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.640+0000 7f145bfff640 1 -- 172.21.15.148:0/1060583302 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c035040 con 0x7f14640ce310 2024-03-29T04:10:49.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.640+0000 7f1469d22640 1 -- 172.21.15.148:0/1060583302 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14640b6b70 con 0x7f14640ce310 2024-03-29T04:10:49.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.641+0000 7f1469d22640 1 -- 172.21.15.148:0/1060583302 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1464068850 con 0x7f14640ce310 2024-03-29T04:10:49.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.642+0000 7f145bfff640 1 -- 172.21.15.148:0/1060583302 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f144c04b060 con 0x7f14640ce310 2024-03-29T04:10:49.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.642+0000 7f145bfff640 1 --2- 172.21.15.148:0/1060583302 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f144406c530 0x7f144406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:49.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.642+0000 7f14637fe640 1 --2- 172.21.15.148:0/1060583302 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f144406c530 0x7f144406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:49.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.642+0000 7f145bfff640 1 -- 172.21.15.148:0/1060583302 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f144c0ad8f0 con 0x7f14640ce310 2024-03-29T04:10:49.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.643+0000 7f14637fe640 1 --2- 172.21.15.148:0/1060583302 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f144406c530 0x7f144406e9f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1454006fd0 tx=0x7f1454008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:49.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.647+0000 7f145bfff640 1 -- 172.21.15.148:0/1060583302 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f144c07bb10 con 0x7f14640ce310 2024-03-29T04:10:49.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.864+0000 7f1469d22640 1 -- 172.21.15.148:0/1060583302 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f14640b7eb0 con 0x7f14640ce310 2024-03-29T04:10:49.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.864+0000 7f145bfff640 1 -- 172.21.15.148:0/1060583302 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f144c07b410 con 0x7f14640ce310 2024-03-29T04:10:49.844 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-29T04:10:49.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.867+0000 7f1469d22640 1 -- 172.21.15.148:0/1060583302 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f144406c530 msgr2=0x7f144406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:49.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.867+0000 7f1469d22640 1 --2- 172.21.15.148:0/1060583302 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f144406c530 0x7f144406e9f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1454006fd0 tx=0x7f1454008040 comp rx=0 tx=0).stop 2024-03-29T04:10:49.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.867+0000 7f1469d22640 1 -- 172.21.15.148:0/1060583302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14640ce310 msgr2=0x7f1464158560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:49.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.867+0000 7f1469d22640 1 --2- 172.21.15.148:0/1060583302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14640ce310 0x7f1464158560 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f144c02b950 tx=0x7f144c0046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.867+0000 7f1469d22640 1 -- 172.21.15.148:0/1060583302 shutdown_connections 2024-03-29T04:10:49.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.867+0000 7f1469d22640 1 --2- 172.21.15.148:0/1060583302 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f144406c530 0x7f144406e9f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.867+0000 7f1469d22640 1 --2- 172.21.15.148:0/1060583302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f146415a120 0x7f1464158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.867+0000 7f1469d22640 1 --2- 172.21.15.148:0/1060583302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14640ce310 0x7f1464158560 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.867+0000 7f1469d22640 1 --2- 172.21.15.148:0/1060583302 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f146406ee10 0x7f14640b5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.867+0000 7f1469d22640 1 -- 172.21.15.148:0/1060583302 >> 172.21.15.148:0/1060583302 conn(0x7f14640cead0 msgr2=0x7f14640cff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:49.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.867+0000 7f1469d22640 1 -- 172.21.15.148:0/1060583302 shutdown_connections 2024-03-29T04:10:49.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.867+0000 7f1469d22640 1 -- 172.21.15.148:0/1060583302 wait complete. 2024-03-29T04:10:49.857 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:10:49.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.981+0000 7f4b9bf11640 1 Processor -- start 2024-03-29T04:10:49.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.982+0000 7f4b9bf11640 1 -- start start 2024-03-29T04:10:49.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.982+0000 7f4b9bf11640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4b9415e3c0 0x7f4b94161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:49.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.982+0000 7f4b9bf11640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4b940b2f00 0x7f4b941624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:49.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.982+0000 7f4b9bf11640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b940cf330 con 0x7f4b940b3430 2024-03-29T04:10:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.982+0000 7f4b9bf11640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f4b940cf960 con 0x7f4b940b2f00 2024-03-29T04:10:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.982+0000 7f4b9bf11640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f4b94162cf0 con 0x7f4b9415e3c0 2024-03-29T04:10:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.983+0000 7f4b994ac640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4b940b2f00 0x7f4b941624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.983+0000 7f4b994ac640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4b940b2f00 0x7f4b941624b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53352/0 (socket says 172.21.15.148:53352) 2024-03-29T04:10:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.983+0000 7f4b994ac640 1 -- 172.21.15.148:0/2137776792 learned_addr learned my addr 172.21.15.148:0/2137776792 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.983+0000 7f4b994ac640 1 -- 172.21.15.148:0/2137776792 >> v2:172.21.15.157:3300/0 conn(0x7f4b9415e3c0 msgr2=0x7f4b94161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.983+0000 7f4b99cad640 1 --2- 172.21.15.148:0/2137776792 >> v2:172.21.15.157:3300/0 conn(0x7f4b9415e3c0 0x7f4b94161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.983+0000 7f4b994ac640 1 --2- 172.21.15.148:0/2137776792 >> v2:172.21.15.157:3300/0 conn(0x7f4b9415e3c0 0x7f4b94161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.983+0000 7f4b994ac640 1 -- 172.21.15.148:0/2137776792 >> v1:172.21.15.157:6789/0 conn(0x7f4b940b3430 legacy=0x7f4b940d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.983+0000 7f4b994ac640 1 -- 172.21.15.148:0/2137776792 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b940cf5f0 con 0x7f4b940b2f00 2024-03-29T04:10:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.984+0000 7f4b994ac640 1 --2- 172.21.15.148:0/2137776792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b940b2f00 0x7f4b941624b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4b940d2670 tx=0x7f4b8402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6275a54aa9c118c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.984+0000 7f4b98cab640 1 -- 172.21.15.148:0/2137776792 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b84035020 con 0x7f4b940b2f00 2024-03-29T04:10:49.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.984+0000 7f4b98cab640 1 -- 172.21.15.148:0/2137776792 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b84031870 con 0x7f4b940b2f00 2024-03-29T04:10:49.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.984+0000 7f4b98cab640 1 -- 172.21.15.148:0/2137776792 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b84031c40 con 0x7f4b940b2f00 2024-03-29T04:10:49.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.984+0000 7f4b9bf11640 1 -- 172.21.15.148:0/2137776792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b940b2f00 msgr2=0x7f4b941624b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:49.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.984+0000 7f4b9bf11640 1 --2- 172.21.15.148:0/2137776792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b940b2f00 0x7f4b941624b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4b940d2670 tx=0x7f4b8402b910 comp rx=0 tx=0).stop 2024-03-29T04:10:49.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.984+0000 7f4b9bf11640 1 -- 172.21.15.148:0/2137776792 shutdown_connections 2024-03-29T04:10:49.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.984+0000 7f4b9bf11640 1 --2- 172.21.15.148:0/2137776792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b940b2f00 0x7f4b941624b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.984+0000 7f4b9bf11640 1 --2- 172.21.15.148:0/2137776792 >> v2:172.21.15.157:3300/0 conn(0x7f4b9415e3c0 0x7f4b94161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.984+0000 7f4b9bf11640 1 -- 172.21.15.148:0/2137776792 >> 172.21.15.148:0/2137776792 conn(0x7f4b940b6430 msgr2=0x7f4b940d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:49.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.984+0000 7f4b9bf11640 1 -- 172.21.15.148:0/2137776792 shutdown_connections 2024-03-29T04:10:49.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.984+0000 7f4b9bf11640 1 -- 172.21.15.148:0/2137776792 wait complete. 2024-03-29T04:10:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.986+0000 7f4b9bf11640 1 Processor -- start 2024-03-29T04:10:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.986+0000 7f4b9bf11640 1 -- start start 2024-03-29T04:10:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.986+0000 7f4b9bf11640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b940b3430 0x7f4b94149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.986+0000 7f4b9bf11640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b9415e3c0 0x7f4b9415eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.986+0000 7f4b9bf11640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b9415f020 0x7f4b94160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.986+0000 7f4b9bf11640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4b940b39a0 con 0x7f4b940b3430 2024-03-29T04:10:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.986+0000 7f4b9bf11640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9415e7a0 con 0x7f4b9415e3c0 2024-03-29T04:10:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.986+0000 7f4b9bf11640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4b940b3b20 con 0x7f4b9415f020 2024-03-29T04:10:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.986+0000 7f4b99cad640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b940b3430 0x7f4b94149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.986+0000 7f4b99cad640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b940b3430 0x7f4b94149750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53368/0 (socket says 172.21.15.148:53368) 2024-03-29T04:10:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.986+0000 7f4b99cad640 1 -- 172.21.15.148:0/309605541 learned_addr learned my addr 172.21.15.148:0/309605541 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.987+0000 7f4b994ac640 1 --2- 172.21.15.148:0/309605541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b9415e3c0 0x7f4b9415eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.987+0000 7f4b9a4ae640 1 --2- 172.21.15.148:0/309605541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b9415f020 0x7f4b94160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.987+0000 7f4b99cad640 1 -- 172.21.15.148:0/309605541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b9415f020 msgr2=0x7f4b94160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.987+0000 7f4b99cad640 1 --2- 172.21.15.148:0/309605541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b9415f020 0x7f4b94160ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.987+0000 7f4b99cad640 1 -- 172.21.15.148:0/309605541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b9415e3c0 msgr2=0x7f4b9415eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.987+0000 7f4b99cad640 1 --2- 172.21.15.148:0/309605541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b9415e3c0 0x7f4b9415eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.987+0000 7f4b99cad640 1 -- 172.21.15.148:0/309605541 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b940cb440 con 0x7f4b940b3430 2024-03-29T04:10:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.987+0000 7f4b9a4ae640 1 --2- 172.21.15.148:0/309605541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b9415f020 0x7f4b94160ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.987+0000 7f4b994ac640 1 --2- 172.21.15.148:0/309605541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b9415e3c0 0x7f4b9415eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.987+0000 7f4b99cad640 1 --2- 172.21.15.148:0/309605541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b940b3430 0x7f4b94149750 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c00cd00 tx=0x7f4b8c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.987+0000 7f4b8a7fc640 1 -- 172.21.15.148:0/309605541 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c003990 con 0x7f4b940b3430 2024-03-29T04:10:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.987+0000 7f4b8a7fc640 1 -- 172.21.15.148:0/309605541 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b8c003b30 con 0x7f4b940b3430 2024-03-29T04:10:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.987+0000 7f4b9bf11640 1 -- 172.21.15.148:0/309605541 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b940cb6e0 con 0x7f4b940b3430 2024-03-29T04:10:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.987+0000 7f4b8a7fc640 1 -- 172.21.15.148:0/309605541 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c004020 con 0x7f4b940b3430 2024-03-29T04:10:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.987+0000 7f4b9bf11640 1 -- 172.21.15.148:0/309605541 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b941617e0 con 0x7f4b940b3430 2024-03-29T04:10:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.988+0000 7f4b9bf11640 1 -- 172.21.15.148:0/309605541 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b9415f430 con 0x7f4b940b3430 2024-03-29T04:10:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.988+0000 7f4b8a7fc640 1 -- 172.21.15.148:0/309605541 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4b8c0041c0 con 0x7f4b940b3430 2024-03-29T04:10:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.989+0000 7f4b8a7fc640 1 --2- 172.21.15.148:0/309605541 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4b7806c530 0x7f4b7806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.989+0000 7f4b8a7fc640 1 -- 172.21.15.148:0/309605541 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f4b8c08bf70 con 0x7f4b940b3430 2024-03-29T04:10:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.989+0000 7f4b994ac640 1 --2- 172.21.15.148:0/309605541 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4b7806c530 0x7f4b7806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.989+0000 7f4b994ac640 1 --2- 172.21.15.148:0/309605541 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4b7806c530 0x7f4b7806e9f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4b84036000 tx=0x7f4b8403c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:49.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:49.993+0000 7f4b8a7fc640 1 -- 172.21.15.148:0/309605541 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4b9415f430 con 0x7f4b940b3430 2024-03-29T04:10:50.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.174+0000 7f4b9bf11640 1 -- 172.21.15.148:0/309605541 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4b94068850 con 0x7f4b940b3430 2024-03-29T04:10:50.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.174+0000 7f4b8a7fc640 1 -- 172.21.15.148:0/309605541 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f4b8c014030 con 0x7f4b940b3430 2024-03-29T04:10:50.154 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:10:50.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.176+0000 7f4b9bf11640 1 -- 172.21.15.148:0/309605541 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4b7806c530 msgr2=0x7f4b7806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.176+0000 7f4b9bf11640 1 --2- 172.21.15.148:0/309605541 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4b7806c530 0x7f4b7806e9f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4b84036000 tx=0x7f4b8403c040 comp rx=0 tx=0).stop 2024-03-29T04:10:50.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.176+0000 7f4b9bf11640 1 -- 172.21.15.148:0/309605541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b940b3430 msgr2=0x7f4b94149750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.176+0000 7f4b9bf11640 1 --2- 172.21.15.148:0/309605541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b940b3430 0x7f4b94149750 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c00cd00 tx=0x7f4b8c00a5c0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.176+0000 7f4b9bf11640 1 -- 172.21.15.148:0/309605541 shutdown_connections 2024-03-29T04:10:50.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.176+0000 7f4b9bf11640 1 --2- 172.21.15.148:0/309605541 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4b7806c530 0x7f4b7806e9f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.176+0000 7f4b9bf11640 1 --2- 172.21.15.148:0/309605541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b9415f020 0x7f4b94160ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.176+0000 7f4b9bf11640 1 --2- 172.21.15.148:0/309605541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b9415e3c0 0x7f4b9415eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.176+0000 7f4b9bf11640 1 --2- 172.21.15.148:0/309605541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b940b3430 0x7f4b94149750 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.176+0000 7f4b9bf11640 1 -- 172.21.15.148:0/309605541 >> 172.21.15.148:0/309605541 conn(0x7f4b940b6430 msgr2=0x7f4b9406c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:50.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.176+0000 7f4b9bf11640 1 -- 172.21.15.148:0/309605541 shutdown_connections 2024-03-29T04:10:50.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.176+0000 7f4b9bf11640 1 -- 172.21.15.148:0/309605541 wait complete. 2024-03-29T04:10:50.166 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:10:50.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.295+0000 7f4222f59640 1 Processor -- start 2024-03-29T04:10:50.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.296+0000 7f4222f59640 1 -- start start 2024-03-29T04:10:50.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.296+0000 7f4222f59640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f421c15e3c0 0x7f421c161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.296+0000 7f4222f59640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f421c0b2f00 0x7f421c1624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.296+0000 7f4222f59640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f421c0b3430 0x7f421c0d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.296+0000 7f4222f59640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f421c0cf960 con 0x7f421c0b3430 2024-03-29T04:10:50.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.296+0000 7f4222f59640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f421c0cf330 con 0x7f421c15e3c0 2024-03-29T04:10:50.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.296+0000 7f4222f59640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f421c162cf0 con 0x7f421c0b2f00 2024-03-29T04:10:50.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.296+0000 7f42214f6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f421c0b3430 0x7f421c0d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.297+0000 7f42214f6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f421c0b3430 0x7f421c0d26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53380/0 (socket says 172.21.15.148:53380) 2024-03-29T04:10:50.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.297+0000 7f42214f6640 1 -- 172.21.15.148:0/1148342163 learned_addr learned my addr 172.21.15.148:0/1148342163 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:50.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.297+0000 7f421bfff640 1 --2- 172.21.15.148:0/1148342163 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f421c0b2f00 0x7f421c1624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.297+0000 7f4220cf5640 1 --2- 172.21.15.148:0/1148342163 >> v2:172.21.15.157:3300/0 conn(0x7f421c15e3c0 0x7f421c161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.297+0000 7f42214f6640 1 -- 172.21.15.148:0/1148342163 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f421c0b2f00 msgr2=0x7f421c1624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:50.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.297+0000 7f42214f6640 1 --2- 172.21.15.148:0/1148342163 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f421c0b2f00 0x7f421c1624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.297+0000 7f42214f6640 1 -- 172.21.15.148:0/1148342163 >> v2:172.21.15.157:3300/0 conn(0x7f421c15e3c0 msgr2=0x7f421c161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:50.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.297+0000 7f42214f6640 1 --2- 172.21.15.148:0/1148342163 >> v2:172.21.15.157:3300/0 conn(0x7f421c15e3c0 0x7f421c161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.297+0000 7f42214f6640 1 -- 172.21.15.148:0/1148342163 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421c0cf5f0 con 0x7f421c0b3430 2024-03-29T04:10:50.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.298+0000 7f42214f6640 1 --2- 172.21.15.148:0/1148342163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f421c0b3430 0x7f421c0d26a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f421c0d2670 tx=0x7f421402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db53e705d57c4b33 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:50.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.298+0000 7f421b7fe640 1 -- 172.21.15.148:0/1148342163 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421402ca70 con 0x7f421c0b3430 2024-03-29T04:10:50.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.298+0000 7f421b7fe640 1 -- 172.21.15.148:0/1148342163 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f421402cc10 con 0x7f421c0b3430 2024-03-29T04:10:50.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.298+0000 7f421b7fe640 1 -- 172.21.15.148:0/1148342163 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42140343f0 con 0x7f421c0b3430 2024-03-29T04:10:50.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.298+0000 7f4222f59640 1 -- 172.21.15.148:0/1148342163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f421c0b3430 msgr2=0x7f421c0d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.298+0000 7f4222f59640 1 --2- 172.21.15.148:0/1148342163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f421c0b3430 0x7f421c0d26a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f421c0d2670 tx=0x7f421402bed0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.298+0000 7f4222f59640 1 -- 172.21.15.148:0/1148342163 shutdown_connections 2024-03-29T04:10:50.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.298+0000 7f4222f59640 1 --2- 172.21.15.148:0/1148342163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f421c0b3430 0x7f421c0d26a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.298+0000 7f4222f59640 1 --2- 172.21.15.148:0/1148342163 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f421c0b2f00 0x7f421c1624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.298+0000 7f4222f59640 1 --2- 172.21.15.148:0/1148342163 >> v2:172.21.15.157:3300/0 conn(0x7f421c15e3c0 0x7f421c161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.298+0000 7f4222f59640 1 -- 172.21.15.148:0/1148342163 >> 172.21.15.148:0/1148342163 conn(0x7f421c0b6430 msgr2=0x7f421c0d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:50.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.298+0000 7f4222f59640 1 -- 172.21.15.148:0/1148342163 shutdown_connections 2024-03-29T04:10:50.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.298+0000 7f4222f59640 1 -- 172.21.15.148:0/1148342163 wait complete. 2024-03-29T04:10:50.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.299+0000 7f4222f59640 1 Processor -- start 2024-03-29T04:10:50.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.299+0000 7f4222f59640 1 -- start start 2024-03-29T04:10:50.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.300+0000 7f4222f59640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f421c0b3430 0x7f421c15cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.300+0000 7f4222f59640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f421c15e3c0 0x7f421c0ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.300+0000 7f4222f59640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f421c160020 0x7f421c0ca560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.300+0000 7f4222f59640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f421c0b39a0 con 0x7f421c0b3430 2024-03-29T04:10:50.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.300+0000 7f4222f59640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f421c15e7a0 con 0x7f421c15e3c0 2024-03-29T04:10:50.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.300+0000 7f4222f59640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f421c0b3b20 con 0x7f421c160020 2024-03-29T04:10:50.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.300+0000 7f4220cf5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f421c0b3430 0x7f421c15cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.301+0000 7f4220cf5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f421c0b3430 0x7f421c15cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53394/0 (socket says 172.21.15.148:53394) 2024-03-29T04:10:50.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.301+0000 7f4220cf5640 1 -- 172.21.15.148:0/3693436619 learned_addr learned my addr 172.21.15.148:0/3693436619 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:50.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.301+0000 7f421bfff640 1 --2- 172.21.15.148:0/3693436619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f421c15e3c0 0x7f421c0ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.301+0000 7f42214f6640 1 --2- 172.21.15.148:0/3693436619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f421c160020 0x7f421c0ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.301+0000 7f4220cf5640 1 -- 172.21.15.148:0/3693436619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f421c160020 msgr2=0x7f421c0ca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.301+0000 7f4220cf5640 1 --2- 172.21.15.148:0/3693436619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f421c160020 0x7f421c0ca560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.301+0000 7f4220cf5640 1 -- 172.21.15.148:0/3693436619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f421c15e3c0 msgr2=0x7f421c0ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.301+0000 7f4220cf5640 1 --2- 172.21.15.148:0/3693436619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f421c15e3c0 0x7f421c0ca020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.301+0000 7f4220cf5640 1 -- 172.21.15.148:0/3693436619 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421c1616b0 con 0x7f421c0b3430 2024-03-29T04:10:50.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.301+0000 7f4220cf5640 1 --2- 172.21.15.148:0/3693436619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f421c0b3430 0x7f421c15cb80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f421000d800 tx=0x7f421000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:50.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.301+0000 7f42197fa640 1 -- 172.21.15.148:0/3693436619 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421000b6c0 con 0x7f421c0b3430 2024-03-29T04:10:50.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.301+0000 7f4222f59640 1 -- 172.21.15.148:0/3693436619 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f421c15f540 con 0x7f421c0b3430 2024-03-29T04:10:50.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.301+0000 7f42197fa640 1 -- 172.21.15.148:0/3693436619 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4210019070 con 0x7f421c0b3430 2024-03-29T04:10:50.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.301+0000 7f42197fa640 1 -- 172.21.15.148:0/3693436619 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421000ba80 con 0x7f421c0b3430 2024-03-29T04:10:50.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.301+0000 7f4222f59640 1 -- 172.21.15.148:0/3693436619 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f421c15fb00 con 0x7f421c0b3430 2024-03-29T04:10:50.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.302+0000 7f4222f59640 1 -- 172.21.15.148:0/3693436619 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f421c068850 con 0x7f421c0b3430 2024-03-29T04:10:50.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.303+0000 7f42197fa640 1 -- 172.21.15.148:0/3693436619 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f421000bc20 con 0x7f421c0b3430 2024-03-29T04:10:50.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.303+0000 7f42197fa640 1 --2- 172.21.15.148:0/3693436619 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f420006c530 0x7f420006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.303+0000 7f421bfff640 1 --2- 172.21.15.148:0/3693436619 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f420006c530 0x7f420006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.303+0000 7f42197fa640 1 -- 172.21.15.148:0/3693436619 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f421008beb0 con 0x7f421c0b3430 2024-03-29T04:10:50.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.304+0000 7f421bfff640 1 --2- 172.21.15.148:0/3693436619 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f420006c530 0x7f420006e9f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f421c160520 tx=0x7f4208008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:50.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.308+0000 7f42197fa640 1 -- 172.21.15.148:0/3693436619 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4210059a00 con 0x7f421c0b3430 2024-03-29T04:10:50.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.486+0000 7f4222f59640 1 -- 172.21.15.148:0/3693436619 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f421c0cf110 con 0x7f421c0b3430 2024-03-29T04:10:50.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.487+0000 7f42197fa640 1 -- 172.21.15.148:0/3693436619 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f421005dd00 con 0x7f421c0b3430 2024-03-29T04:10:50.466 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-29T04:10:50.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.488+0000 7f4222f59640 1 -- 172.21.15.148:0/3693436619 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f420006c530 msgr2=0x7f420006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.488+0000 7f4222f59640 1 --2- 172.21.15.148:0/3693436619 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f420006c530 0x7f420006e9f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f421c160520 tx=0x7f4208008040 comp rx=0 tx=0).stop 2024-03-29T04:10:50.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.488+0000 7f4222f59640 1 -- 172.21.15.148:0/3693436619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f421c0b3430 msgr2=0x7f421c15cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.488+0000 7f4222f59640 1 --2- 172.21.15.148:0/3693436619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f421c0b3430 0x7f421c15cb80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f421000d800 tx=0x7f421000dcc0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.488+0000 7f4222f59640 1 -- 172.21.15.148:0/3693436619 shutdown_connections 2024-03-29T04:10:50.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.488+0000 7f4222f59640 1 --2- 172.21.15.148:0/3693436619 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f420006c530 0x7f420006e9f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.488+0000 7f4222f59640 1 --2- 172.21.15.148:0/3693436619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f421c160020 0x7f421c0ca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.488+0000 7f4222f59640 1 --2- 172.21.15.148:0/3693436619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f421c15e3c0 0x7f421c0ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.488+0000 7f4222f59640 1 --2- 172.21.15.148:0/3693436619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f421c0b3430 0x7f421c15cb80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.488+0000 7f4222f59640 1 -- 172.21.15.148:0/3693436619 >> 172.21.15.148:0/3693436619 conn(0x7f421c0b6430 msgr2=0x7f421c0d3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:50.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.489+0000 7f4222f59640 1 -- 172.21.15.148:0/3693436619 shutdown_connections 2024-03-29T04:10:50.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.489+0000 7f4222f59640 1 -- 172.21.15.148:0/3693436619 wait complete. 2024-03-29T04:10:50.479 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:10:50.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.605+0000 7fef9d1e1640 1 Processor -- start 2024-03-29T04:10:50.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.605+0000 7fef9d1e1640 1 -- start start 2024-03-29T04:10:50.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.606+0000 7fef9d1e1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fef980b3430 0x7fef98162680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.606+0000 7fef9d1e1640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef980cf330 con 0x7fef980b2f00 2024-03-29T04:10:50.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.606+0000 7fef9d1e1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef98162cf0 con 0x7fef9815e3c0 2024-03-29T04:10:50.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.606+0000 7fef9d1e1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fef980cf960 con 0x7fef980b3430 2024-03-29T04:10:50.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.606+0000 7fef9759e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fef980b3430 0x7fef98162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.606+0000 7fef9759e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fef980b3430 0x7fef98162680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53396/0 (socket says 172.21.15.148:53396) 2024-03-29T04:10:50.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.606+0000 7fef9759e640 1 -- 172.21.15.148:0/1435507109 learned_addr learned my addr 172.21.15.148:0/1435507109 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:50.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.607+0000 7fef9759e640 1 -- 172.21.15.148:0/1435507109 >> v1:172.21.15.157:6789/0 conn(0x7fef9815e3c0 legacy=0x7fef98161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:50.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.607+0000 7fef9759e640 1 -- 172.21.15.148:0/1435507109 >> v1:172.21.15.148:6789/0 conn(0x7fef980b2f00 legacy=0x7fef9806b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.607+0000 7fef9759e640 1 -- 172.21.15.148:0/1435507109 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef980cf5f0 con 0x7fef980b3430 2024-03-29T04:10:50.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.607+0000 7fef9659c640 1 -- 172.21.15.148:0/1435507109 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 197494947 0 0) 0x7fef980cf330 con 0x7fef980b2f00 2024-03-29T04:10:50.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.607+0000 7fef9759e640 1 --2- 172.21.15.148:0/1435507109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef980b3430 0x7fef98162680 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fef90009e50 tx=0x7fef9002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19321edd6e48a006 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:50.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.608+0000 7fef9659c640 1 -- 172.21.15.148:0/1435507109 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9002ca80 con 0x7fef980b3430 2024-03-29T04:10:50.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.608+0000 7fef9659c640 1 -- 172.21.15.148:0/1435507109 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef9002cc20 con 0x7fef980b3430 2024-03-29T04:10:50.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.608+0000 7fef9659c640 1 -- 172.21.15.148:0/1435507109 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef900343f0 con 0x7fef980b3430 2024-03-29T04:10:50.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.608+0000 7fef9d1e1640 1 -- 172.21.15.148:0/1435507109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef980b3430 msgr2=0x7fef98162680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.608+0000 7fef9d1e1640 1 --2- 172.21.15.148:0/1435507109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef980b3430 0x7fef98162680 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fef90009e50 tx=0x7fef9002bee0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.608+0000 7fef9d1e1640 1 -- 172.21.15.148:0/1435507109 shutdown_connections 2024-03-29T04:10:50.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.608+0000 7fef9d1e1640 1 --2- 172.21.15.148:0/1435507109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef980b3430 0x7fef98162680 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.608+0000 7fef9d1e1640 1 -- 172.21.15.148:0/1435507109 >> 172.21.15.148:0/1435507109 conn(0x7fef980b6430 msgr2=0x7fef980d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:50.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.608+0000 7fef9d1e1640 1 -- 172.21.15.148:0/1435507109 shutdown_connections 2024-03-29T04:10:50.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.608+0000 7fef9d1e1640 1 -- 172.21.15.148:0/1435507109 wait complete. 2024-03-29T04:10:50.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.609+0000 7fef9d1e1640 1 Processor -- start 2024-03-29T04:10:50.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.610+0000 7fef9d1e1640 1 -- start start 2024-03-29T04:10:50.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.610+0000 7fef9d1e1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef980b3430 0x7fef980cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.610+0000 7fef9d1e1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef9815e3c0 0x7fef980cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.610+0000 7fef9d1e1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef98152ca0 0x7fef980b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.610+0000 7fef9d1e1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fef980b3b20 con 0x7fef980b3430 2024-03-29T04:10:50.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.610+0000 7fef9d1e1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fef980b39a0 con 0x7fef98152ca0 2024-03-29T04:10:50.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.610+0000 7fef9d1e1640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fef9806bf00 con 0x7fef9815e3c0 2024-03-29T04:10:50.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.610+0000 7fef96d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef980b3430 0x7fef980cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.610+0000 7fef96d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef980b3430 0x7fef980cbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53404/0 (socket says 172.21.15.148:53404) 2024-03-29T04:10:50.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.610+0000 7fef96d9d640 1 -- 172.21.15.148:0/2213478345 learned_addr learned my addr 172.21.15.148:0/2213478345 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:50.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.611+0000 7fef9759e640 1 --2- 172.21.15.148:0/2213478345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef98152ca0 0x7fef980b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.611+0000 7fef8ffff640 1 --2- 172.21.15.148:0/2213478345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef9815e3c0 0x7fef980cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.611+0000 7fef96d9d640 1 -- 172.21.15.148:0/2213478345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef9815e3c0 msgr2=0x7fef980cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.611+0000 7fef96d9d640 1 --2- 172.21.15.148:0/2213478345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef9815e3c0 0x7fef980cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.611+0000 7fef96d9d640 1 -- 172.21.15.148:0/2213478345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef98152ca0 msgr2=0x7fef980b8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.611+0000 7fef96d9d640 1 --2- 172.21.15.148:0/2213478345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef98152ca0 0x7fef980b8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.611+0000 7fef96d9d640 1 -- 172.21.15.148:0/2213478345 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef98153150 con 0x7fef980b3430 2024-03-29T04:10:50.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.611+0000 7fef9759e640 1 --2- 172.21.15.148:0/2213478345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef98152ca0 0x7fef980b8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:50.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.611+0000 7fef96d9d640 1 --2- 172.21.15.148:0/2213478345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef980b3430 0x7fef980cbe40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fef8800b3f0 tx=0x7fef8800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:50.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.611+0000 7fef8f7fe640 1 -- 172.21.15.148:0/2213478345 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88015020 con 0x7fef980b3430 2024-03-29T04:10:50.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.611+0000 7fef8f7fe640 1 -- 172.21.15.148:0/2213478345 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef88011830 con 0x7fef980b3430 2024-03-29T04:10:50.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.611+0000 7fef9d1e1640 1 -- 172.21.15.148:0/2213478345 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef980ccdf0 con 0x7fef980b3430 2024-03-29T04:10:50.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.611+0000 7fef8f7fe640 1 -- 172.21.15.148:0/2213478345 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88011c20 con 0x7fef980b3430 2024-03-29T04:10:50.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.611+0000 7fef9d1e1640 1 -- 172.21.15.148:0/2213478345 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef980cd3b0 con 0x7fef980b3430 2024-03-29T04:10:50.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.612+0000 7fef9d1e1640 1 -- 172.21.15.148:0/2213478345 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef98068850 con 0x7fef980b3430 2024-03-29T04:10:50.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.612+0000 7fef8f7fe640 1 -- 172.21.15.148:0/2213478345 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fef88018020 con 0x7fef980b3430 2024-03-29T04:10:50.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.613+0000 7fef8f7fe640 1 --2- 172.21.15.148:0/2213478345 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fef7806c560 0x7fef7806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.613+0000 7fef8f7fe640 1 -- 172.21.15.148:0/2213478345 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fef8808b950 con 0x7fef980b3430 2024-03-29T04:10:50.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.613+0000 7fef8ffff640 1 --2- 172.21.15.148:0/2213478345 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fef7806c560 0x7fef7806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.613+0000 7fef8ffff640 1 --2- 172.21.15.148:0/2213478345 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fef7806c560 0x7fef7806ea20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fef981600f0 tx=0x7fef80009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:50.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.618+0000 7fef8f7fe640 1 -- 172.21.15.148:0/2213478345 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fef8801f020 con 0x7fef980b3430 2024-03-29T04:10:50.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.799+0000 7fef9d1e1640 1 -- 172.21.15.148:0/2213478345 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fef980b9f80 con 0x7fef980b3430 2024-03-29T04:10:50.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.799+0000 7fef8f7fe640 1 -- 172.21.15.148:0/2213478345 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fef98153150 con 0x7fef980b3430 2024-03-29T04:10:50.778 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:10:50.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.801+0000 7fef9d1e1640 1 -- 172.21.15.148:0/2213478345 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fef7806c560 msgr2=0x7fef7806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.801+0000 7fef9d1e1640 1 --2- 172.21.15.148:0/2213478345 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fef7806c560 0x7fef7806ea20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fef981600f0 tx=0x7fef80009040 comp rx=0 tx=0).stop 2024-03-29T04:10:50.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.801+0000 7fef9d1e1640 1 -- 172.21.15.148:0/2213478345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef980b3430 msgr2=0x7fef980cbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.801+0000 7fef9d1e1640 1 --2- 172.21.15.148:0/2213478345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef980b3430 0x7fef980cbe40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fef8800b3f0 tx=0x7fef8800b8b0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.801+0000 7fef9d1e1640 1 -- 172.21.15.148:0/2213478345 shutdown_connections 2024-03-29T04:10:50.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.801+0000 7fef9d1e1640 1 --2- 172.21.15.148:0/2213478345 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fef7806c560 0x7fef7806ea20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.801+0000 7fef9d1e1640 1 --2- 172.21.15.148:0/2213478345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef98152ca0 0x7fef980b8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.801+0000 7fef9d1e1640 1 --2- 172.21.15.148:0/2213478345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef9815e3c0 0x7fef980cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.801+0000 7fef9d1e1640 1 --2- 172.21.15.148:0/2213478345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef980b3430 0x7fef980cbe40 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.801+0000 7fef9d1e1640 1 -- 172.21.15.148:0/2213478345 >> 172.21.15.148:0/2213478345 conn(0x7fef980b6430 msgr2=0x7fef980b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:50.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.801+0000 7fef9d1e1640 1 -- 172.21.15.148:0/2213478345 shutdown_connections 2024-03-29T04:10:50.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.801+0000 7fef9d1e1640 1 -- 172.21.15.148:0/2213478345 wait complete. 2024-03-29T04:10:50.797 INFO:tasks.cephfs_test_runner:Starting test: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-29T04:10:50.798 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:50.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.935+0000 7fd9e6aeb640 1 Processor -- start 2024-03-29T04:10:50.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.935+0000 7fd9e6aeb640 1 -- start start 2024-03-29T04:10:50.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.936+0000 7fd9e6aeb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9e015e770 0x7fd9e00ce710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.936+0000 7fd9e6aeb640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd9e006b5b0 0x7fd9e0072520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.936+0000 7fd9e6aeb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9e006dc80 con 0x7fd9e0072a60 2024-03-29T04:10:50.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.936+0000 7fd9e6aeb640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fd9e00b4090 con 0x7fd9e006b5b0 2024-03-29T04:10:50.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.936+0000 7fd9e6aeb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd9e00b5ec0 con 0x7fd9e015e770 2024-03-29T04:10:50.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.936+0000 7fd9dffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd9e006b5b0 0x7fd9e0072520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.936+0000 7fd9dffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd9e006b5b0 0x7fd9e0072520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53412/0 (socket says 172.21.15.148:53412) 2024-03-29T04:10:50.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.936+0000 7fd9dffff640 1 -- 172.21.15.148:0/2632545966 learned_addr learned my addr 172.21.15.148:0/2632545966 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:50.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.936+0000 7fd9e4887640 1 --2- 172.21.15.148:0/2632545966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9e015e770 0x7fd9e00ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.936+0000 7fd9dffff640 1 -- 172.21.15.148:0/2632545966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9e015e770 msgr2=0x7fd9e00ce710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:50.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.936+0000 7fd9dffff640 1 --2- 172.21.15.148:0/2632545966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9e015e770 0x7fd9e00ce710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.936+0000 7fd9dffff640 1 -- 172.21.15.148:0/2632545966 >> v1:172.21.15.157:6789/0 conn(0x7fd9e0072a60 legacy=0x7fd9e00cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.936+0000 7fd9dffff640 1 -- 172.21.15.148:0/2632545966 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e00b61a0 con 0x7fd9e006b5b0 2024-03-29T04:10:50.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.937+0000 7fd9dffff640 1 --2- 172.21.15.148:0/2632545966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9e006b5b0 0x7fd9e0072520 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc009870 tx=0x7fd9cc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d618d2d71714633f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:50.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.937+0000 7fd9df7fe640 1 -- 172.21.15.148:0/2632545966 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc034070 con 0x7fd9e006b5b0 2024-03-29T04:10:50.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.937+0000 7fd9df7fe640 1 -- 172.21.15.148:0/2632545966 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9cc030840 con 0x7fd9e006b5b0 2024-03-29T04:10:50.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.937+0000 7fd9df7fe640 1 -- 172.21.15.148:0/2632545966 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc030c10 con 0x7fd9e006b5b0 2024-03-29T04:10:50.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.938+0000 7fd9e6aeb640 1 -- 172.21.15.148:0/2632545966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9e006b5b0 msgr2=0x7fd9e0072520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.938+0000 7fd9e6aeb640 1 --2- 172.21.15.148:0/2632545966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9e006b5b0 0x7fd9e0072520 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc009870 tx=0x7fd9cc02b980 comp rx=0 tx=0).stop 2024-03-29T04:10:50.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.938+0000 7fd9e6aeb640 1 -- 172.21.15.148:0/2632545966 shutdown_connections 2024-03-29T04:10:50.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.938+0000 7fd9e6aeb640 1 --2- 172.21.15.148:0/2632545966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9e006b5b0 0x7fd9e0072520 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.938+0000 7fd9e6aeb640 1 --2- 172.21.15.148:0/2632545966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9e015e770 0x7fd9e00ce710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.938+0000 7fd9e6aeb640 1 -- 172.21.15.148:0/2632545966 >> 172.21.15.148:0/2632545966 conn(0x7fd9e00d0f50 msgr2=0x7fd9e00d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:50.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.938+0000 7fd9e6aeb640 1 -- 172.21.15.148:0/2632545966 shutdown_connections 2024-03-29T04:10:50.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.938+0000 7fd9e6aeb640 1 -- 172.21.15.148:0/2632545966 wait complete. 2024-03-29T04:10:50.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.939+0000 7fd9e6aeb640 1 Processor -- start 2024-03-29T04:10:50.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.939+0000 7fd9e6aeb640 1 -- start start 2024-03-29T04:10:50.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.940+0000 7fd9e6aeb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9e006b5b0 0x7fd9e0142660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.940+0000 7fd9e6aeb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9e0072a60 0x7fd9e015aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.940+0000 7fd9e6aeb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9e015e770 0x7fd9e006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.940+0000 7fd9e6aeb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e006da20 con 0x7fd9e015e770 2024-03-29T04:10:50.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.940+0000 7fd9e6aeb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e006d8a0 con 0x7fd9e006b5b0 2024-03-29T04:10:50.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.940+0000 7fd9e6aeb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd9e00b5ae0 con 0x7fd9e0072a60 2024-03-29T04:10:50.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.941+0000 7fd9e5088640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9e015e770 0x7fd9e006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.941+0000 7fd9e5088640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9e015e770 0x7fd9e006ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53428/0 (socket says 172.21.15.148:53428) 2024-03-29T04:10:50.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.941+0000 7fd9e5088640 1 -- 172.21.15.148:0/445879964 learned_addr learned my addr 172.21.15.148:0/445879964 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:50.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.941+0000 7fd9e4887640 1 --2- 172.21.15.148:0/445879964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9e006b5b0 0x7fd9e0142660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.941+0000 7fd9dffff640 1 --2- 172.21.15.148:0/445879964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9e0072a60 0x7fd9e015aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.941+0000 7fd9e5088640 1 -- 172.21.15.148:0/445879964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9e0072a60 msgr2=0x7fd9e015aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.941+0000 7fd9e5088640 1 --2- 172.21.15.148:0/445879964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9e0072a60 0x7fd9e015aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.941+0000 7fd9e5088640 1 -- 172.21.15.148:0/445879964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9e006b5b0 msgr2=0x7fd9e0142660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.941+0000 7fd9e5088640 1 --2- 172.21.15.148:0/445879964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9e006b5b0 0x7fd9e0142660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.941+0000 7fd9e5088640 1 -- 172.21.15.148:0/445879964 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e0142ba0 con 0x7fd9e015e770 2024-03-29T04:10:50.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.941+0000 7fd9e5088640 1 --2- 172.21.15.148:0/445879964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9e015e770 0x7fd9e006ca10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800c7d0 tx=0x7fd9d800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:50.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.941+0000 7fd9dd7fa640 1 -- 172.21.15.148:0/445879964 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8003b70 con 0x7fd9e015e770 2024-03-29T04:10:50.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.941+0000 7fd9dd7fa640 1 -- 172.21.15.148:0/445879964 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9d8003d30 con 0x7fd9e015e770 2024-03-29T04:10:50.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.941+0000 7fd9e6aeb640 1 -- 172.21.15.148:0/445879964 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9e015cbf0 con 0x7fd9e015e770 2024-03-29T04:10:50.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.941+0000 7fd9dd7fa640 1 -- 172.21.15.148:0/445879964 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d800aa90 con 0x7fd9e015e770 2024-03-29T04:10:50.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.941+0000 7fd9e6aeb640 1 -- 172.21.15.148:0/445879964 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9e015d490 con 0x7fd9e015e770 2024-03-29T04:10:50.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.942+0000 7fd9e6aeb640 1 -- 172.21.15.148:0/445879964 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9e015cfe0 con 0x7fd9e015e770 2024-03-29T04:10:50.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.943+0000 7fd9dd7fa640 1 -- 172.21.15.148:0/445879964 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd9d800ac30 con 0x7fd9e015e770 2024-03-29T04:10:50.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.943+0000 7fd9dd7fa640 1 --2- 172.21.15.148:0/445879964 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd9c406c530 0x7fd9c406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.943+0000 7fd9dd7fa640 1 -- 172.21.15.148:0/445879964 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd9d808b980 con 0x7fd9e015e770 2024-03-29T04:10:50.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.943+0000 7fd9e4887640 1 --2- 172.21.15.148:0/445879964 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd9c406c530 0x7fd9c406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.944+0000 7fd9e4887640 1 --2- 172.21.15.148:0/445879964 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd9c406c530 0x7fd9c406e9f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4006fd0 tx=0x7fd9d4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:50.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:50.948+0000 7fd9dd7fa640 1 -- 172.21.15.148:0/445879964 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd9d80594c0 con 0x7fd9e015e770 2024-03-29T04:10:51.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.129+0000 7fd9e6aeb640 1 -- 172.21.15.148:0/445879964 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9e00d0bb0 con 0x7fd9e015e770 2024-03-29T04:10:51.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.130+0000 7fd9dd7fa640 1 -- 172.21.15.148:0/445879964 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18858 (secure 0 0 0) 0x7fd9e00d0bb0 con 0x7fd9e015e770 2024-03-29T04:10:51.110 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:51.110 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:41.662934+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:51.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.132+0000 7fd9e6aeb640 1 -- 172.21.15.148:0/445879964 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd9c406c530 msgr2=0x7fd9c406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:51.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.132+0000 7fd9e6aeb640 1 --2- 172.21.15.148:0/445879964 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd9c406c530 0x7fd9c406e9f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4006fd0 tx=0x7fd9d4008040 comp rx=0 tx=0).stop 2024-03-29T04:10:51.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.132+0000 7fd9e6aeb640 1 -- 172.21.15.148:0/445879964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9e015e770 msgr2=0x7fd9e006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:51.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.132+0000 7fd9e6aeb640 1 --2- 172.21.15.148:0/445879964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9e015e770 0x7fd9e006ca10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800c7d0 tx=0x7fd9d800cc90 comp rx=0 tx=0).stop 2024-03-29T04:10:51.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.133+0000 7fd9e6aeb640 1 -- 172.21.15.148:0/445879964 shutdown_connections 2024-03-29T04:10:51.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.133+0000 7fd9e6aeb640 1 --2- 172.21.15.148:0/445879964 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd9c406c530 0x7fd9c406e9f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:51.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.133+0000 7fd9e6aeb640 1 --2- 172.21.15.148:0/445879964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9e015e770 0x7fd9e006ca10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:51.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.133+0000 7fd9e6aeb640 1 --2- 172.21.15.148:0/445879964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9e0072a60 0x7fd9e015aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:51.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.133+0000 7fd9e6aeb640 1 --2- 172.21.15.148:0/445879964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9e006b5b0 0x7fd9e0142660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:51.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.133+0000 7fd9e6aeb640 1 -- 172.21.15.148:0/445879964 >> 172.21.15.148:0/445879964 conn(0x7fd9e00d0f50 msgr2=0x7fd9e00adc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:51.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.133+0000 7fd9e6aeb640 1 -- 172.21.15.148:0/445879964 shutdown_connections 2024-03-29T04:10:51.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.133+0000 7fd9e6aeb640 1 -- 172.21.15.148:0/445879964 wait complete. 2024-03-29T04:10:51.125 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:47.898351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:57.922313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:58.829405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:10:02.551763+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:10:51.125 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:10:51.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.251+0000 7f65bf132640 1 Processor -- start 2024-03-29T04:10:51.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.251+0000 7f65bf132640 1 -- start start 2024-03-29T04:10:51.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.252+0000 7f65bf132640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f65b815e3c0 0x7f65b8161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:51.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.252+0000 7f65bf132640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65b80b3430 0x7f65b806b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:51.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.252+0000 7f65bf132640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65b80cf330 con 0x7f65b80b2f00 2024-03-29T04:10:51.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.252+0000 7f65bf132640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f65b80cf960 con 0x7f65b815e3c0 2024-03-29T04:10:51.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.252+0000 7f65bf132640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f65b8162cf0 con 0x7f65b80b3430 2024-03-29T04:10:51.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.253+0000 7f65bd6cf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65b80b3430 0x7f65b806b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:51.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.253+0000 7f65bcece640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f65b815e3c0 0x7f65b8161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:51.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.253+0000 7f65bd6cf640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65b80b3430 0x7f65b806b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.148:59780/0 (socket says 172.21.15.148:59780) 2024-03-29T04:10:51.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.253+0000 7f65bd6cf640 1 -- 172.21.15.148:0/3812550128 learned_addr learned my addr 172.21.15.148:0/3812550128 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:51.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.253+0000 7f65b7fff640 1 --1- 172.21.15.148:0/3812550128 >> v1:172.21.15.157:6789/0 conn(0x7f65b80b2f00 0x7f65b81624b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.148:51312/0 (socket says 172.21.15.148:51312) 2024-03-29T04:10:51.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.253+0000 7f65bd6cf640 1 -- 172.21.15.148:0/3812550128 >> v2:172.21.15.157:3300/0 conn(0x7f65b815e3c0 msgr2=0x7f65b8161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:51.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.253+0000 7f65bd6cf640 1 --2- 172.21.15.148:0/3812550128 >> v2:172.21.15.157:3300/0 conn(0x7f65b815e3c0 0x7f65b8161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:51.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.253+0000 7f65bd6cf640 1 -- 172.21.15.148:0/3812550128 >> v1:172.21.15.157:6789/0 conn(0x7f65b80b2f00 legacy=0x7f65b81624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:51.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.253+0000 7f65bd6cf640 1 -- 172.21.15.148:0/3812550128 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b80cf5f0 con 0x7f65b80b3430 2024-03-29T04:10:51.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.253+0000 7f65bcece640 1 --2- 172.21.15.148:0/3812550128 >> v2:172.21.15.157:3300/0 conn(0x7f65b815e3c0 0x7f65b8161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:51.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.254+0000 7f65bd6cf640 1 --2- 172.21.15.148:0/3812550128 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65b80b3430 0x7f65b806b7f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009e50 tx=0x7f65b002bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc0ff9eda98628f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:51.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.254+0000 7f65b77fe640 1 -- 172.21.15.148:0/3812550128 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b002ca70 con 0x7f65b80b3430 2024-03-29T04:10:51.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.254+0000 7f65b77fe640 1 -- 172.21.15.148:0/3812550128 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65b002cc10 con 0x7f65b80b3430 2024-03-29T04:10:51.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.254+0000 7f65b77fe640 1 -- 172.21.15.148:0/3812550128 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b00343f0 con 0x7f65b80b3430 2024-03-29T04:10:51.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.255+0000 7f65bf132640 1 -- 172.21.15.148:0/3812550128 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65b80b3430 msgr2=0x7f65b806b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:51.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.255+0000 7f65bf132640 1 --2- 172.21.15.148:0/3812550128 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65b80b3430 0x7f65b806b7f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009e50 tx=0x7f65b002bed0 comp rx=0 tx=0).stop 2024-03-29T04:10:51.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.255+0000 7f65bf132640 1 -- 172.21.15.148:0/3812550128 shutdown_connections 2024-03-29T04:10:51.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.255+0000 7f65bf132640 1 --2- 172.21.15.148:0/3812550128 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65b80b3430 0x7f65b806b7f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:51.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.255+0000 7f65bf132640 1 --2- 172.21.15.148:0/3812550128 >> v2:172.21.15.157:3300/0 conn(0x7f65b815e3c0 0x7f65b8161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:51.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.255+0000 7f65bf132640 1 -- 172.21.15.148:0/3812550128 >> 172.21.15.148:0/3812550128 conn(0x7f65b80b6430 msgr2=0x7f65b80d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:51.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.255+0000 7f65bf132640 1 -- 172.21.15.148:0/3812550128 shutdown_connections 2024-03-29T04:10:51.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.255+0000 7f65bf132640 1 -- 172.21.15.148:0/3812550128 wait complete. 2024-03-29T04:10:51.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.256+0000 7f65bf132640 1 Processor -- start 2024-03-29T04:10:51.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.256+0000 7f65bf132640 1 -- start start 2024-03-29T04:10:51.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.257+0000 7f65bf132640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65b80b3430 0x7f65b815ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:51.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.257+0000 7f65bf132640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65b815e3c0 0x7f65b815e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:51.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.257+0000 7f65bf132640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65b815eef0 0x7f65b8155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:51.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.257+0000 7f65bf132640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f65b80b3b20 con 0x7f65b815eef0 2024-03-29T04:10:51.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.257+0000 7f65bf132640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f65b80b39a0 con 0x7f65b815e3c0 2024-03-29T04:10:51.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.257+0000 7f65bf132640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f65b806bd30 con 0x7f65b80b3430 2024-03-29T04:10:51.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.257+0000 7f65bd6cf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65b815eef0 0x7f65b8155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:51.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.257+0000 7f65bd6cf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65b815eef0 0x7f65b8155390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53434/0 (socket says 172.21.15.148:53434) 2024-03-29T04:10:51.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.257+0000 7f65bd6cf640 1 -- 172.21.15.148:0/1711040462 learned_addr learned my addr 172.21.15.148:0/1711040462 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:51.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.257+0000 7f65bcece640 1 --2- 172.21.15.148:0/1711040462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65b80b3430 0x7f65b815ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:51.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.257+0000 7f65b7fff640 1 --2- 172.21.15.148:0/1711040462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65b815e3c0 0x7f65b815e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:51.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.257+0000 7f65bd6cf640 1 -- 172.21.15.148:0/1711040462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65b80b3430 msgr2=0x7f65b815ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:51.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.257+0000 7f65bd6cf640 1 --2- 172.21.15.148:0/1711040462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65b80b3430 0x7f65b815ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:51.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.257+0000 7f65bd6cf640 1 -- 172.21.15.148:0/1711040462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65b815e3c0 msgr2=0x7f65b815e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:51.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.257+0000 7f65bd6cf640 1 --2- 172.21.15.148:0/1711040462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65b815e3c0 0x7f65b815e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:51.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.257+0000 7f65bd6cf640 1 -- 172.21.15.148:0/1711040462 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b80cb3a0 con 0x7f65b815eef0 2024-03-29T04:10:51.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.257+0000 7f65bd6cf640 1 --2- 172.21.15.148:0/1711040462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65b815eef0 0x7f65b8155390 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009e50 tx=0x7f65b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:51.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.258+0000 7f65b57fa640 1 -- 172.21.15.148:0/1711040462 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0043020 con 0x7f65b815eef0 2024-03-29T04:10:51.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.258+0000 7f65b57fa640 1 -- 172.21.15.148:0/1711040462 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65b003e450 con 0x7f65b815eef0 2024-03-29T04:10:51.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.258+0000 7f65b57fa640 1 -- 172.21.15.148:0/1711040462 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0009510 con 0x7f65b815eef0 2024-03-29T04:10:51.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.258+0000 7f65bf132640 1 -- 172.21.15.148:0/1711040462 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b815f370 con 0x7f65b815eef0 2024-03-29T04:10:51.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.258+0000 7f65bf132640 1 -- 172.21.15.148:0/1711040462 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b80cb900 con 0x7f65b815eef0 2024-03-29T04:10:51.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.259+0000 7f65bf132640 1 -- 172.21.15.148:0/1711040462 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65b8068850 con 0x7f65b815eef0 2024-03-29T04:10:51.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.259+0000 7f65b57fa640 1 -- 172.21.15.148:0/1711040462 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f65b0046020 con 0x7f65b815eef0 2024-03-29T04:10:51.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.259+0000 7f65b57fa640 1 --2- 172.21.15.148:0/1711040462 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f659c06c530 0x7f659c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:51.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.259+0000 7f65b57fa640 1 -- 172.21.15.148:0/1711040462 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f65b00ae0e0 con 0x7f65b815eef0 2024-03-29T04:10:51.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.259+0000 7f65bcece640 1 --2- 172.21.15.148:0/1711040462 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f659c06c530 0x7f659c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:51.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.260+0000 7f65bcece640 1 --2- 172.21.15.148:0/1711040462 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f659c06c530 0x7f659c06e9f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009560 tx=0x7f65ac009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:51.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.264+0000 7f65b57fa640 1 -- 172.21.15.148:0/1711040462 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f65b004d030 con 0x7f65b815eef0 2024-03-29T04:10:51.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.543+0000 7f65bf132640 1 -- 172.21.15.148:0/1711040462 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f65b80cbad0 con 0x7f65b815eef0 2024-03-29T04:10:51.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.543+0000 7f65b57fa640 1 -- 172.21.15.148:0/1711040462 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f65b80cb3a0 con 0x7f65b815eef0 2024-03-29T04:10:51.522 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-29T04:10:51.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.545+0000 7f65bf132640 1 -- 172.21.15.148:0/1711040462 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f659c06c530 msgr2=0x7f659c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:51.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.545+0000 7f65bf132640 1 --2- 172.21.15.148:0/1711040462 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f659c06c530 0x7f659c06e9f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009560 tx=0x7f65ac009190 comp rx=0 tx=0).stop 2024-03-29T04:10:51.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.545+0000 7f65bf132640 1 -- 172.21.15.148:0/1711040462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65b815eef0 msgr2=0x7f65b8155390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:51.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.545+0000 7f65bf132640 1 --2- 172.21.15.148:0/1711040462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65b815eef0 0x7f65b8155390 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009e50 tx=0x7f65b0004a40 comp rx=0 tx=0).stop 2024-03-29T04:10:51.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.545+0000 7f65bf132640 1 -- 172.21.15.148:0/1711040462 shutdown_connections 2024-03-29T04:10:51.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.545+0000 7f65bf132640 1 --2- 172.21.15.148:0/1711040462 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f659c06c530 0x7f659c06e9f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:51.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.545+0000 7f65bf132640 1 --2- 172.21.15.148:0/1711040462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65b815eef0 0x7f65b8155390 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:51.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.545+0000 7f65bf132640 1 --2- 172.21.15.148:0/1711040462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65b815e3c0 0x7f65b815e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:51.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.545+0000 7f65bf132640 1 --2- 172.21.15.148:0/1711040462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65b80b3430 0x7f65b815ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:51.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.545+0000 7f65bf132640 1 -- 172.21.15.148:0/1711040462 >> 172.21.15.148:0/1711040462 conn(0x7f65b80b6430 msgr2=0x7f65b80d0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:51.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.545+0000 7f65bf132640 1 -- 172.21.15.148:0/1711040462 shutdown_connections 2024-03-29T04:10:51.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.545+0000 7f65bf132640 1 -- 172.21.15.148:0/1711040462 wait complete. 2024-03-29T04:10:51.536 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:10:51.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.661+0000 7f6d4bab7640 1 Processor -- start 2024-03-29T04:10:51.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.662+0000 7f6d4bab7640 1 -- start start 2024-03-29T04:10:51.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.662+0000 7f6d4bab7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6d440b3430 0x7f6d44162680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:51.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.662+0000 7f6d4bab7640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d440cf330 con 0x7f6d440b2f00 2024-03-29T04:10:51.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.662+0000 7f6d4bab7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d44162cf0 con 0x7f6d4415e3c0 2024-03-29T04:10:51.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.662+0000 7f6d4bab7640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f6d440cf960 con 0x7f6d440b3430 2024-03-29T04:10:51.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.663+0000 7f6d4a054640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6d440b3430 0x7f6d44162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:51.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.663+0000 7f6d4a054640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6d440b3430 0x7f6d44162680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53436/0 (socket says 172.21.15.148:53436) 2024-03-29T04:10:51.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.663+0000 7f6d49052640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6d440b2f00 0x7f6d4406b7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50370/0 (socket says 172.21.15.148:50370) 2024-03-29T04:10:51.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.663+0000 7f6d4a054640 1 -- 172.21.15.148:0/2814286231 learned_addr learned my addr 172.21.15.148:0/2814286231 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:51.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.663+0000 7f6d4a054640 1 -- 172.21.15.148:0/2814286231 >> v1:172.21.15.157:6789/0 conn(0x7f6d4415e3c0 legacy=0x7f6d44161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:51.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.663+0000 7f6d4a054640 1 -- 172.21.15.148:0/2814286231 >> v1:172.21.15.148:6789/0 conn(0x7f6d440b2f00 legacy=0x7f6d4406b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:51.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.663+0000 7f6d48851640 1 -- 172.21.15.148:0/2814286231 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 90386786 0 0) 0x7f6d440cf330 con 0x7f6d440b2f00 2024-03-29T04:10:51.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.663+0000 7f6d4a054640 1 -- 172.21.15.148:0/2814286231 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d440cf5f0 con 0x7f6d440b3430 2024-03-29T04:10:51.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.664+0000 7f6d4a054640 1 --2- 172.21.15.148:0/2814286231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d440b3430 0x7f6d44162680 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009e50 tx=0x7f6d4002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a374eece1bb7e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:51.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.664+0000 7f6d48851640 1 -- 172.21.15.148:0/2814286231 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4002cb20 con 0x7f6d440b3430 2024-03-29T04:10:51.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.664+0000 7f6d48851640 1 -- 172.21.15.148:0/2814286231 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d4002ccc0 con 0x7f6d440b3430 2024-03-29T04:10:51.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.664+0000 7f6d48851640 1 -- 172.21.15.148:0/2814286231 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d40034610 con 0x7f6d440b3430 2024-03-29T04:10:51.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.664+0000 7f6d4bab7640 1 -- 172.21.15.148:0/2814286231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d440b3430 msgr2=0x7f6d44162680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:51.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.664+0000 7f6d4bab7640 1 --2- 172.21.15.148:0/2814286231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d440b3430 0x7f6d44162680 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009e50 tx=0x7f6d4002bf80 comp rx=0 tx=0).stop 2024-03-29T04:10:51.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.664+0000 7f6d4bab7640 1 -- 172.21.15.148:0/2814286231 shutdown_connections 2024-03-29T04:10:51.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.664+0000 7f6d4bab7640 1 --2- 172.21.15.148:0/2814286231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d440b3430 0x7f6d44162680 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:51.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.664+0000 7f6d4bab7640 1 -- 172.21.15.148:0/2814286231 >> 172.21.15.148:0/2814286231 conn(0x7f6d440b6430 msgr2=0x7f6d440d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:51.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.664+0000 7f6d4bab7640 1 -- 172.21.15.148:0/2814286231 shutdown_connections 2024-03-29T04:10:51.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.664+0000 7f6d4bab7640 1 -- 172.21.15.148:0/2814286231 wait complete. 2024-03-29T04:10:51.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.666+0000 7f6d4bab7640 1 Processor -- start 2024-03-29T04:10:51.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.666+0000 7f6d4bab7640 1 -- start start 2024-03-29T04:10:51.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.666+0000 7f6d4bab7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d440b3430 0x7f6d441f1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:51.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.666+0000 7f6d4bab7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d4415e3c0 0x7f6d441f4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:51.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.666+0000 7f6d4bab7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d441f5060 0x7f6d44161c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:51.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.666+0000 7f6d4bab7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6d440b3b20 con 0x7f6d441f5060 2024-03-29T04:10:51.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.666+0000 7f6d4bab7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6d440b39a0 con 0x7f6d4415e3c0 2024-03-29T04:10:51.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.666+0000 7f6d4bab7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6d4406bf00 con 0x7f6d440b3430 2024-03-29T04:10:51.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.666+0000 7f6d4a054640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d441f5060 0x7f6d44161c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:51.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.666+0000 7f6d4a054640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d441f5060 0x7f6d44161c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53444/0 (socket says 172.21.15.148:53444) 2024-03-29T04:10:51.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.666+0000 7f6d4a054640 1 -- 172.21.15.148:0/26075103 learned_addr learned my addr 172.21.15.148:0/26075103 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:51.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.667+0000 7f6d49052640 1 --2- 172.21.15.148:0/26075103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d4415e3c0 0x7f6d441f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:51.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.667+0000 7f6d49853640 1 --2- 172.21.15.148:0/26075103 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d440b3430 0x7f6d441f1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:51.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.667+0000 7f6d4a054640 1 -- 172.21.15.148:0/26075103 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d440b3430 msgr2=0x7f6d441f1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:51.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.667+0000 7f6d4a054640 1 --2- 172.21.15.148:0/26075103 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d440b3430 0x7f6d441f1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:51.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.667+0000 7f6d4a054640 1 -- 172.21.15.148:0/26075103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d4415e3c0 msgr2=0x7f6d441f4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:51.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.667+0000 7f6d4a054640 1 --2- 172.21.15.148:0/26075103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d4415e3c0 0x7f6d441f4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:51.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.667+0000 7f6d4a054640 1 -- 172.21.15.148:0/26075103 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d441f4e60 con 0x7f6d441f5060 2024-03-29T04:10:51.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.667+0000 7f6d4a054640 1 --2- 172.21.15.148:0/26075103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d441f5060 0x7f6d44161c70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009f70 tx=0x7f6d40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:51.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.667+0000 7f6d3a7fc640 1 -- 172.21.15.148:0/26075103 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d40043020 con 0x7f6d441f5060 2024-03-29T04:10:51.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.667+0000 7f6d3a7fc640 1 -- 172.21.15.148:0/26075103 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d4003e450 con 0x7f6d441f5060 2024-03-29T04:10:51.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.667+0000 7f6d3a7fc640 1 -- 172.21.15.148:0/26075103 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d40009510 con 0x7f6d441f5060 2024-03-29T04:10:51.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.667+0000 7f6d4bab7640 1 -- 172.21.15.148:0/26075103 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d440cb6d0 con 0x7f6d441f5060 2024-03-29T04:10:51.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.667+0000 7f6d4bab7640 1 -- 172.21.15.148:0/26075103 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d440cbc90 con 0x7f6d441f5060 2024-03-29T04:10:51.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.668+0000 7f6d4bab7640 1 -- 172.21.15.148:0/26075103 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d44068850 con 0x7f6d441f5060 2024-03-29T04:10:51.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.669+0000 7f6d3a7fc640 1 -- 172.21.15.148:0/26075103 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6d40046020 con 0x7f6d441f5060 2024-03-29T04:10:51.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.669+0000 7f6d3a7fc640 1 --2- 172.21.15.148:0/26075103 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6d2806c530 0x7f6d2806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:51.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.669+0000 7f6d3a7fc640 1 -- 172.21.15.148:0/26075103 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f6d400ae250 con 0x7f6d441f5060 2024-03-29T04:10:51.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.669+0000 7f6d49853640 1 --2- 172.21.15.148:0/26075103 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6d2806c530 0x7f6d2806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:51.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.670+0000 7f6d49853640 1 --2- 172.21.15.148:0/26075103 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6d2806c530 0x7f6d2806e9f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6d440b85e0 tx=0x7f6d3c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:51.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.674+0000 7f6d3a7fc640 1 -- 172.21.15.148:0/26075103 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6d4004d030 con 0x7f6d441f5060 2024-03-29T04:10:51.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.854+0000 7f6d4bab7640 1 -- 172.21.15.148:0/26075103 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6d440008d0 con 0x7f6d441f5060 2024-03-29T04:10:51.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.855+0000 7f6d3a7fc640 1 -- 172.21.15.148:0/26075103 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f6d441f4e60 con 0x7f6d441f5060 2024-03-29T04:10:51.834 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:10:51.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.856+0000 7f6d4bab7640 1 -- 172.21.15.148:0/26075103 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6d2806c530 msgr2=0x7f6d2806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:51.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.857+0000 7f6d4bab7640 1 --2- 172.21.15.148:0/26075103 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6d2806c530 0x7f6d2806e9f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6d440b85e0 tx=0x7f6d3c009040 comp rx=0 tx=0).stop 2024-03-29T04:10:51.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.857+0000 7f6d4bab7640 1 -- 172.21.15.148:0/26075103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d441f5060 msgr2=0x7f6d44161c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:51.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.857+0000 7f6d4bab7640 1 --2- 172.21.15.148:0/26075103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d441f5060 0x7f6d44161c70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009f70 tx=0x7f6d40004a40 comp rx=0 tx=0).stop 2024-03-29T04:10:51.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.857+0000 7f6d4bab7640 1 -- 172.21.15.148:0/26075103 shutdown_connections 2024-03-29T04:10:51.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.857+0000 7f6d4bab7640 1 --2- 172.21.15.148:0/26075103 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6d2806c530 0x7f6d2806e9f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:51.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.857+0000 7f6d4bab7640 1 --2- 172.21.15.148:0/26075103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d441f5060 0x7f6d44161c70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:51.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.857+0000 7f6d4bab7640 1 --2- 172.21.15.148:0/26075103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d4415e3c0 0x7f6d441f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:51.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.857+0000 7f6d4bab7640 1 --2- 172.21.15.148:0/26075103 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d440b3430 0x7f6d441f1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:51.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.857+0000 7f6d4bab7640 1 -- 172.21.15.148:0/26075103 >> 172.21.15.148:0/26075103 conn(0x7f6d440b6430 msgr2=0x7f6d440b4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:51.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.857+0000 7f6d4bab7640 1 -- 172.21.15.148:0/26075103 shutdown_connections 2024-03-29T04:10:51.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.857+0000 7f6d4bab7640 1 -- 172.21.15.148:0/26075103 wait complete. 2024-03-29T04:10:51.847 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:10:51.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.974+0000 7f3928147640 1 Processor -- start 2024-03-29T04:10:51.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.974+0000 7f3928147640 1 -- start start 2024-03-29T04:10:51.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.975+0000 7f3928147640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f392015a120 0x7f39200700d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:51.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.975+0000 7f3928147640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f39200ce310 0x7f392006e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:51.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.975+0000 7f3928147640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f392015d460 con 0x7f392006ee10 2024-03-29T04:10:51.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.975+0000 7f3928147640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f39200d1b20 con 0x7f39200ce310 2024-03-29T04:10:51.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.975+0000 7f3928147640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f39200d0f60 con 0x7f392015a120 2024-03-29T04:10:51.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.975+0000 7f39256e2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f39200ce310 0x7f392006e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:51.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.975+0000 7f3925ee3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f392015a120 0x7f39200700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:51.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.975+0000 7f39266e4640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f392006ee10 0x7f39200b6f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.148:51322/0 (socket says 172.21.15.148:51322) 2024-03-29T04:10:51.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.975+0000 7f39256e2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f39200ce310 0x7f392006e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:43496/0 (socket says 172.21.15.148:43496) 2024-03-29T04:10:51.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.975+0000 7f3925ee3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f392015a120 0x7f39200700d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.148:59818/0 (socket says 172.21.15.148:59818) 2024-03-29T04:10:51.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.976+0000 7f39266e4640 1 -- 172.21.15.148:0/3694548103 learned_addr learned my addr 172.21.15.148:0/3694548103 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:51.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.976+0000 7f39256e2640 1 -- 172.21.15.148:0/3694548103 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f392015a120 msgr2=0x7f39200700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:51.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.976+0000 7f39256e2640 1 --2- 172.21.15.148:0/3694548103 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f392015a120 0x7f39200700d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:51.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.976+0000 7f39256e2640 1 -- 172.21.15.148:0/3694548103 >> v1:172.21.15.157:6789/0 conn(0x7f392006ee10 legacy=0x7f39200b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:51.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.976+0000 7f39256e2640 1 -- 172.21.15.148:0/3694548103 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f392015d700 con 0x7f39200ce310 2024-03-29T04:10:51.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.977+0000 7f39256e2640 1 --2- 172.21.15.148:0/3694548103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39200ce310 0x7f392006e8d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f39100097a0 tx=0x7f391002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5961b9ba584b81b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:51.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.977+0000 7f3924ee1640 1 -- 172.21.15.148:0/3694548103 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910035020 con 0x7f39200ce310 2024-03-29T04:10:51.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.977+0000 7f3924ee1640 1 -- 172.21.15.148:0/3694548103 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3910031830 con 0x7f39200ce310 2024-03-29T04:10:51.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.977+0000 7f3924ee1640 1 -- 172.21.15.148:0/3694548103 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910031c00 con 0x7f39200ce310 2024-03-29T04:10:51.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.978+0000 7f3928147640 1 -- 172.21.15.148:0/3694548103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39200ce310 msgr2=0x7f392006e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:51.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.978+0000 7f3928147640 1 --2- 172.21.15.148:0/3694548103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39200ce310 0x7f392006e8d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f39100097a0 tx=0x7f391002b8d0 comp rx=0 tx=0).stop 2024-03-29T04:10:51.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.978+0000 7f3928147640 1 -- 172.21.15.148:0/3694548103 shutdown_connections 2024-03-29T04:10:51.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.978+0000 7f3928147640 1 --2- 172.21.15.148:0/3694548103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39200ce310 0x7f392006e8d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:51.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.978+0000 7f3928147640 1 --2- 172.21.15.148:0/3694548103 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f392015a120 0x7f39200700d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:51.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.978+0000 7f3928147640 1 -- 172.21.15.148:0/3694548103 >> 172.21.15.148:0/3694548103 conn(0x7f39200cead0 msgr2=0x7f39200ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:51.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.978+0000 7f3928147640 1 -- 172.21.15.148:0/3694548103 shutdown_connections 2024-03-29T04:10:51.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.978+0000 7f3928147640 1 -- 172.21.15.148:0/3694548103 wait complete. 2024-03-29T04:10:51.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.979+0000 7f3928147640 1 Processor -- start 2024-03-29T04:10:51.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.979+0000 7f3928147640 1 -- start start 2024-03-29T04:10:51.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.980+0000 7f3928147640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f392006ee10 0x7f39200c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:51.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.980+0000 7f3928147640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39200ce310 0x7f39200c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:51.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.980+0000 7f3928147640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f392015a120 0x7f39200c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:51.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.980+0000 7f3928147640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f392006f5c0 con 0x7f392015a120 2024-03-29T04:10:51.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.980+0000 7f3928147640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f39200ce7d0 con 0x7f39200ce310 2024-03-29T04:10:51.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.980+0000 7f3928147640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f392006f740 con 0x7f392006ee10 2024-03-29T04:10:51.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.980+0000 7f39266e4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f392015a120 0x7f39200c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:51.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.980+0000 7f39266e4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f392015a120 0x7f39200c69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53446/0 (socket says 172.21.15.148:53446) 2024-03-29T04:10:51.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.980+0000 7f39266e4640 1 -- 172.21.15.148:0/2526107752 learned_addr learned my addr 172.21.15.148:0/2526107752 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:51.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.980+0000 7f3925ee3640 1 --2- 172.21.15.148:0/2526107752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f392006ee10 0x7f39200c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:51.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.980+0000 7f39256e2640 1 --2- 172.21.15.148:0/2526107752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39200ce310 0x7f39200c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:51.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.980+0000 7f39266e4640 1 -- 172.21.15.148:0/2526107752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f392006ee10 msgr2=0x7f39200c5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:51.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.980+0000 7f39266e4640 1 --2- 172.21.15.148:0/2526107752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f392006ee10 0x7f39200c5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:51.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.980+0000 7f39266e4640 1 -- 172.21.15.148:0/2526107752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39200ce310 msgr2=0x7f39200c64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:51.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.980+0000 7f39266e4640 1 --2- 172.21.15.148:0/2526107752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39200ce310 0x7f39200c64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:51.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.980+0000 7f39266e4640 1 -- 172.21.15.148:0/2526107752 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39200b6af0 con 0x7f392015a120 2024-03-29T04:10:51.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.980+0000 7f39266e4640 1 --2- 172.21.15.148:0/2526107752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f392015a120 0x7f39200c69e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f391800ca80 tx=0x7f391800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:51.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.980+0000 7f39167fc640 1 -- 172.21.15.148:0/2526107752 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3918017070 con 0x7f392015a120 2024-03-29T04:10:51.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.980+0000 7f39167fc640 1 -- 172.21.15.148:0/2526107752 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f391800ac30 con 0x7f392015a120 2024-03-29T04:10:51.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.980+0000 7f3928147640 1 -- 172.21.15.148:0/2526107752 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39200b6dd0 con 0x7f392015a120 2024-03-29T04:10:51.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.980+0000 7f39167fc640 1 -- 172.21.15.148:0/2526107752 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3918012600 con 0x7f392015a120 2024-03-29T04:10:51.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.981+0000 7f3928147640 1 -- 172.21.15.148:0/2526107752 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39200b7390 con 0x7f392015a120 2024-03-29T04:10:51.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.982+0000 7f3928147640 1 -- 172.21.15.148:0/2526107752 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39200b75d0 con 0x7f392015a120 2024-03-29T04:10:51.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.983+0000 7f39167fc640 1 -- 172.21.15.148:0/2526107752 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f391801a070 con 0x7f392015a120 2024-03-29T04:10:51.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.983+0000 7f39167fc640 1 --2- 172.21.15.148:0/2526107752 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f390406c530 0x7f390406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:51.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.983+0000 7f39167fc640 1 -- 172.21.15.148:0/2526107752 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f391808b6c0 con 0x7f392015a120 2024-03-29T04:10:51.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.983+0000 7f3925ee3640 1 --2- 172.21.15.148:0/2526107752 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f390406c530 0x7f390406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:51.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.983+0000 7f3925ee3640 1 --2- 172.21.15.148:0/2526107752 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f390406c530 0x7f390406e9f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f391c009b40 tx=0x7f391c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:51.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:51.987+0000 7f39167fc640 1 -- 172.21.15.148:0/2526107752 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f39200b75d0 con 0x7f392015a120 2024-03-29T04:10:52.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.169+0000 7f3928147640 1 -- 172.21.15.148:0/2526107752 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3920068850 con 0x7f392015a120 2024-03-29T04:10:52.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.169+0000 7f39167fc640 1 -- 172.21.15.148:0/2526107752 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f391800ae20 con 0x7f392015a120 2024-03-29T04:10:52.148 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-29T04:10:52.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.172+0000 7f3928147640 1 -- 172.21.15.148:0/2526107752 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f390406c530 msgr2=0x7f390406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:52.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.172+0000 7f3928147640 1 --2- 172.21.15.148:0/2526107752 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f390406c530 0x7f390406e9f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f391c009b40 tx=0x7f391c009190 comp rx=0 tx=0).stop 2024-03-29T04:10:52.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.172+0000 7f3928147640 1 -- 172.21.15.148:0/2526107752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f392015a120 msgr2=0x7f39200c69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:52.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.172+0000 7f3928147640 1 --2- 172.21.15.148:0/2526107752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f392015a120 0x7f39200c69e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f391800ca80 tx=0x7f391800cf40 comp rx=0 tx=0).stop 2024-03-29T04:10:52.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.172+0000 7f3928147640 1 -- 172.21.15.148:0/2526107752 shutdown_connections 2024-03-29T04:10:52.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.172+0000 7f3928147640 1 --2- 172.21.15.148:0/2526107752 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f390406c530 0x7f390406e9f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:52.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.172+0000 7f3928147640 1 --2- 172.21.15.148:0/2526107752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f392015a120 0x7f39200c69e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:52.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.172+0000 7f3928147640 1 --2- 172.21.15.148:0/2526107752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39200ce310 0x7f39200c64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:52.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.172+0000 7f3928147640 1 --2- 172.21.15.148:0/2526107752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f392006ee10 0x7f39200c5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:52.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.172+0000 7f3928147640 1 -- 172.21.15.148:0/2526107752 >> 172.21.15.148:0/2526107752 conn(0x7f39200cead0 msgr2=0x7f39200b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:52.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.172+0000 7f3928147640 1 -- 172.21.15.148:0/2526107752 shutdown_connections 2024-03-29T04:10:52.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.172+0000 7f3928147640 1 -- 172.21.15.148:0/2526107752 wait complete. 2024-03-29T04:10:52.165 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:10:52.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.291+0000 7f54ca033640 1 Processor -- start 2024-03-29T04:10:52.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.291+0000 7f54ca033640 1 -- start start 2024-03-29T04:10:52.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.292+0000 7f54ca033640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f54c4144980 0x7f54c406fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:52.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.292+0000 7f54ca033640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c40cfda0 0x7f54c406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:52.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.292+0000 7f54ca033640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54c40cc660 con 0x7f54c40cc0f0 2024-03-29T04:10:52.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.292+0000 7f54ca033640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f54c40cd270 con 0x7f54c4144980 2024-03-29T04:10:52.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.292+0000 7f54ca033640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f54c40cc940 con 0x7f54c40cfda0 2024-03-29T04:10:52.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.292+0000 7f54baffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f54c40cc0f0 0x7f54c40cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50382/0 (socket says 172.21.15.148:50382) 2024-03-29T04:10:52.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.292+0000 7f54baffd640 1 -- 172.21.15.148:0/1912595553 learned_addr learned my addr 172.21.15.148:0/1912595553 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:52.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.292+0000 7f54c37fe640 1 --2- 172.21.15.148:0/1912595553 >> v2:172.21.15.157:3300/0 conn(0x7f54c4144980 0x7f54c406fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:52.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.292+0000 7f54c3fff640 1 --2- 172.21.15.148:0/1912595553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c40cfda0 0x7f54c406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:52.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.293+0000 7f54c2ffd640 1 -- 172.21.15.148:0/1912595553 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3140591373 0 0) 0x7f54c40cc660 con 0x7f54c40cc0f0 2024-03-29T04:10:52.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.293+0000 7f54c2ffd640 1 -- 172.21.15.148:0/1912595553 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54b0003520 con 0x7f54c40cc0f0 2024-03-29T04:10:52.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.293+0000 7f54c37fe640 1 -- 172.21.15.148:0/1912595553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c40cfda0 msgr2=0x7f54c406c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:52.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.293+0000 7f54c37fe640 1 --2- 172.21.15.148:0/1912595553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c40cfda0 0x7f54c406c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:52.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.293+0000 7f54c37fe640 1 -- 172.21.15.148:0/1912595553 >> v1:172.21.15.148:6789/0 conn(0x7f54c40cc0f0 legacy=0x7f54c40cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:52.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.293+0000 7f54c37fe640 1 -- 172.21.15.148:0/1912595553 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c40ccc20 con 0x7f54c4144980 2024-03-29T04:10:52.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.293+0000 7f54c3fff640 1 --2- 172.21.15.148:0/1912595553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c40cfda0 0x7f54c406c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:52.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.294+0000 7f54c37fe640 1 --2- 172.21.15.148:0/1912595553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54c4144980 0x7f54c406fab0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f54b40097a0 tx=0x7f54b402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ceb730c77ef587d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:52.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.294+0000 7f54c2ffd640 1 -- 172.21.15.148:0/1912595553 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4035020 con 0x7f54c4144980 2024-03-29T04:10:52.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.294+0000 7f54c2ffd640 1 -- 172.21.15.148:0/1912595553 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54b4031830 con 0x7f54c4144980 2024-03-29T04:10:52.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.294+0000 7f54c2ffd640 1 -- 172.21.15.148:0/1912595553 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4031c00 con 0x7f54c4144980 2024-03-29T04:10:52.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.294+0000 7f54ca033640 1 -- 172.21.15.148:0/1912595553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54c4144980 msgr2=0x7f54c406fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:52.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.294+0000 7f54ca033640 1 --2- 172.21.15.148:0/1912595553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54c4144980 0x7f54c406fab0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f54b40097a0 tx=0x7f54b402b8d0 comp rx=0 tx=0).stop 2024-03-29T04:10:52.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.294+0000 7f54ca033640 1 -- 172.21.15.148:0/1912595553 shutdown_connections 2024-03-29T04:10:52.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.294+0000 7f54ca033640 1 --2- 172.21.15.148:0/1912595553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c40cfda0 0x7f54c406c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:52.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.294+0000 7f54ca033640 1 --2- 172.21.15.148:0/1912595553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54c4144980 0x7f54c406fab0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:52.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.294+0000 7f54ca033640 1 -- 172.21.15.148:0/1912595553 >> 172.21.15.148:0/1912595553 conn(0x7f54c40b2c60 msgr2=0x7f54c40b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:52.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.294+0000 7f54ca033640 1 -- 172.21.15.148:0/1912595553 shutdown_connections 2024-03-29T04:10:52.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.294+0000 7f54ca033640 1 -- 172.21.15.148:0/1912595553 wait complete. 2024-03-29T04:10:52.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.295+0000 7f54ca033640 1 Processor -- start 2024-03-29T04:10:52.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.296+0000 7f54ca033640 1 -- start start 2024-03-29T04:10:52.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.296+0000 7f54ca033640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54c40cc0f0 0x7f54c41598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:52.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.296+0000 7f54ca033640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54c40cfda0 0x7f54c415d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:52.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.296+0000 7f54ca033640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c4144980 0x7f54c4072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:52.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.296+0000 7f54ca033640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f54c40cd700 con 0x7f54c40cc0f0 2024-03-29T04:10:52.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.296+0000 7f54ca033640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f54c4144d60 con 0x7f54c40cfda0 2024-03-29T04:10:52.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.296+0000 7f54ca033640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f54c40cd880 con 0x7f54c4144980 2024-03-29T04:10:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.297+0000 7f54c37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54c40cc0f0 0x7f54c41598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.297+0000 7f54c37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54c40cc0f0 0x7f54c41598a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53460/0 (socket says 172.21.15.148:53460) 2024-03-29T04:10:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.297+0000 7f54c37fe640 1 -- 172.21.15.148:0/1689074893 learned_addr learned my addr 172.21.15.148:0/1689074893 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.297+0000 7f54baffd640 1 --2- 172.21.15.148:0/1689074893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54c40cfda0 0x7f54c415d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.297+0000 7f54c3fff640 1 --2- 172.21.15.148:0/1689074893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c4144980 0x7f54c4072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.297+0000 7f54c37fe640 1 -- 172.21.15.148:0/1689074893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c4144980 msgr2=0x7f54c4072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.297+0000 7f54c37fe640 1 --2- 172.21.15.148:0/1689074893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c4144980 0x7f54c4072cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.297+0000 7f54c37fe640 1 -- 172.21.15.148:0/1689074893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54c40cfda0 msgr2=0x7f54c415d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.297+0000 7f54c37fe640 1 --2- 172.21.15.148:0/1689074893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54c40cfda0 0x7f54c415d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.297+0000 7f54c37fe640 1 -- 172.21.15.148:0/1689074893 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c415e790 con 0x7f54c40cc0f0 2024-03-29T04:10:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.297+0000 7f54c3fff640 1 --2- 172.21.15.148:0/1689074893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c4144980 0x7f54c4072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.297+0000 7f54baffd640 1 --2- 172.21.15.148:0/1689074893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54c40cfda0 0x7f54c415d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.297+0000 7f54c37fe640 1 --2- 172.21.15.148:0/1689074893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54c40cc0f0 0x7f54c41598a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f54b4034040 tx=0x7f54b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.297+0000 7f54c0ff9640 1 -- 172.21.15.148:0/1689074893 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4040070 con 0x7f54c40cc0f0 2024-03-29T04:10:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.297+0000 7f54c0ff9640 1 -- 172.21.15.148:0/1689074893 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54b403a910 con 0x7f54c40cc0f0 2024-03-29T04:10:52.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.297+0000 7f54c0ff9640 1 -- 172.21.15.148:0/1689074893 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4035040 con 0x7f54c40cc0f0 2024-03-29T04:10:52.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.297+0000 7f54ca033640 1 -- 172.21.15.148:0/1689074893 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c415def0 con 0x7f54c40cc0f0 2024-03-29T04:10:52.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.297+0000 7f54ca033640 1 -- 172.21.15.148:0/1689074893 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c415e4b0 con 0x7f54c40cc0f0 2024-03-29T04:10:52.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.298+0000 7f54ca033640 1 -- 172.21.15.148:0/1689074893 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54c4068850 con 0x7f54c40cc0f0 2024-03-29T04:10:52.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.299+0000 7f54c0ff9640 1 -- 172.21.15.148:0/1689074893 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f54b4038020 con 0x7f54c40cc0f0 2024-03-29T04:10:52.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.300+0000 7f54c0ff9640 1 --2- 172.21.15.148:0/1689074893 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f54a406c530 0x7f54a406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:52.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.300+0000 7f54c0ff9640 1 -- 172.21.15.148:0/1689074893 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f54b40ad300 con 0x7f54c40cc0f0 2024-03-29T04:10:52.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.300+0000 7f54baffd640 1 --2- 172.21.15.148:0/1689074893 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f54a406c530 0x7f54a406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:52.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.300+0000 7f54baffd640 1 --2- 172.21.15.148:0/1689074893 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f54a406c530 0x7f54a406e9f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f54ac007fd0 tx=0x7f54ac009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:52.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.304+0000 7f54c0ff9640 1 -- 172.21.15.148:0/1689074893 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f54b407b520 con 0x7f54c40cc0f0 2024-03-29T04:10:52.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.485+0000 7f54ca033640 1 -- 172.21.15.148:0/1689074893 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f54c406b5b0 con 0x7f54c40cc0f0 2024-03-29T04:10:52.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.485+0000 7f54c0ff9640 1 -- 172.21.15.148:0/1689074893 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f54b407ae20 con 0x7f54c40cc0f0 2024-03-29T04:10:52.465 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:10:52.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.487+0000 7f54ca033640 1 -- 172.21.15.148:0/1689074893 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f54a406c530 msgr2=0x7f54a406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:52.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.487+0000 7f54ca033640 1 --2- 172.21.15.148:0/1689074893 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f54a406c530 0x7f54a406e9f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f54ac007fd0 tx=0x7f54ac009040 comp rx=0 tx=0).stop 2024-03-29T04:10:52.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.487+0000 7f54ca033640 1 -- 172.21.15.148:0/1689074893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54c40cc0f0 msgr2=0x7f54c41598a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:52.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.487+0000 7f54ca033640 1 --2- 172.21.15.148:0/1689074893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54c40cc0f0 0x7f54c41598a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f54b4034040 tx=0x7f54b40046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:52.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.487+0000 7f54ca033640 1 -- 172.21.15.148:0/1689074893 shutdown_connections 2024-03-29T04:10:52.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.487+0000 7f54ca033640 1 --2- 172.21.15.148:0/1689074893 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f54a406c530 0x7f54a406e9f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:52.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.487+0000 7f54ca033640 1 --2- 172.21.15.148:0/1689074893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c4144980 0x7f54c4072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:52.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.487+0000 7f54ca033640 1 --2- 172.21.15.148:0/1689074893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54c40cfda0 0x7f54c415d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:52.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.487+0000 7f54ca033640 1 --2- 172.21.15.148:0/1689074893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54c40cc0f0 0x7f54c41598a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:52.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.487+0000 7f54ca033640 1 -- 172.21.15.148:0/1689074893 >> 172.21.15.148:0/1689074893 conn(0x7f54c40b2c60 msgr2=0x7f54c40cf660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:52.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.487+0000 7f54ca033640 1 -- 172.21.15.148:0/1689074893 shutdown_connections 2024-03-29T04:10:52.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.487+0000 7f54ca033640 1 -- 172.21.15.148:0/1689074893 wait complete. 2024-03-29T04:10:52.478 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining' 2024-03-29T04:10:52.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.603+0000 7f7f67724640 1 Processor -- start 2024-03-29T04:10:52.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.604+0000 7f7f67724640 1 -- start start 2024-03-29T04:10:52.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.604+0000 7f7f67724640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7f600ceb50 0x7f7f600cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:52.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.604+0000 7f7f67724640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f601608b0 0x7f7f6015cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:52.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.604+0000 7f7f67724640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f600d0250 con 0x7f7f6015c5f0 2024-03-29T04:10:52.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.604+0000 7f7f67724640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f7f600d19d0 con 0x7f7f600ceb50 2024-03-29T04:10:52.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.604+0000 7f7f67724640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7f600ccde0 con 0x7f7f601608b0 2024-03-29T04:10:52.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.605+0000 7f7f654c0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7f600ceb50 0x7f7f600cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:52.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.605+0000 7f7f65cc1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f601608b0 0x7f7f6015cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:52.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.605+0000 7f7f65cc1640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f601608b0 0x7f7f6015cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.148:59852/0 (socket says 172.21.15.148:59852) 2024-03-29T04:10:52.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.605+0000 7f7f65cc1640 1 -- 172.21.15.148:0/1315896677 learned_addr learned my addr 172.21.15.148:0/1315896677 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:52.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.605+0000 7f7f64cbf640 1 --1- 172.21.15.148:0/1315896677 >> v1:172.21.15.157:6789/0 conn(0x7f7f6015c5f0 0x7f7f600cef30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.148:51332/0 (socket says 172.21.15.148:51332) 2024-03-29T04:10:52.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.605+0000 7f7f654c0640 1 -- 172.21.15.148:0/1315896677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f601608b0 msgr2=0x7f7f6015cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:52.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.605+0000 7f7f654c0640 1 --2- 172.21.15.148:0/1315896677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f601608b0 0x7f7f6015cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:52.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.605+0000 7f7f654c0640 1 -- 172.21.15.148:0/1315896677 >> v1:172.21.15.157:6789/0 conn(0x7f7f6015c5f0 legacy=0x7f7f600cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:52.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.605+0000 7f7f654c0640 1 -- 172.21.15.148:0/1315896677 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f600ccb00 con 0x7f7f600ceb50 2024-03-29T04:10:52.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.605+0000 7f7f65cc1640 1 --2- 172.21.15.148:0/1315896677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f601608b0 0x7f7f6015cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:52.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.606+0000 7f7f654c0640 1 --2- 172.21.15.148:0/1315896677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f600ceb50 0x7f7f600cfd10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c0097a0 tx=0x7f7f5c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69996d58094f5800 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:52.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.606+0000 7f7f57fff640 1 -- 172.21.15.148:0/1315896677 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c035020 con 0x7f7f600ceb50 2024-03-29T04:10:52.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.606+0000 7f7f57fff640 1 -- 172.21.15.148:0/1315896677 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f5c031830 con 0x7f7f600ceb50 2024-03-29T04:10:52.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.606+0000 7f7f57fff640 1 -- 172.21.15.148:0/1315896677 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c031c00 con 0x7f7f600ceb50 2024-03-29T04:10:52.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.606+0000 7f7f67724640 1 -- 172.21.15.148:0/1315896677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f600ceb50 msgr2=0x7f7f600cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:52.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.606+0000 7f7f67724640 1 --2- 172.21.15.148:0/1315896677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f600ceb50 0x7f7f600cfd10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c0097a0 tx=0x7f7f5c02b8d0 comp rx=0 tx=0).stop 2024-03-29T04:10:52.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.607+0000 7f7f67724640 1 -- 172.21.15.148:0/1315896677 shutdown_connections 2024-03-29T04:10:52.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.607+0000 7f7f67724640 1 --2- 172.21.15.148:0/1315896677 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f601608b0 0x7f7f6015cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:52.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.607+0000 7f7f67724640 1 --2- 172.21.15.148:0/1315896677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f600ceb50 0x7f7f600cfd10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:52.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.607+0000 7f7f67724640 1 -- 172.21.15.148:0/1315896677 >> 172.21.15.148:0/1315896677 conn(0x7f7f600b3c70 msgr2=0x7f7f600b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:52.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.607+0000 7f7f67724640 1 -- 172.21.15.148:0/1315896677 shutdown_connections 2024-03-29T04:10:52.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.607+0000 7f7f67724640 1 -- 172.21.15.148:0/1315896677 wait complete. 2024-03-29T04:10:52.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.608+0000 7f7f67724640 1 Processor -- start 2024-03-29T04:10:52.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.608+0000 7f7f67724640 1 -- start start 2024-03-29T04:10:52.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.608+0000 7f7f67724640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f600ceb50 0x7f7f601f53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:52.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.608+0000 7f7f67724640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f601608b0 0x7f7f6015aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:52.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.608+0000 7f7f67724640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f601f7fa0 0x7f7f600cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:52.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.608+0000 7f7f67724640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7f600ce540 con 0x7f7f601608b0 2024-03-29T04:10:52.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.608+0000 7f7f67724640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7f600cc790 con 0x7f7f600ceb50 2024-03-29T04:10:52.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.608+0000 7f7f67724640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7f600ce6c0 con 0x7f7f601f7fa0 2024-03-29T04:10:52.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.609+0000 7f7f64cbf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f601608b0 0x7f7f6015aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:52.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.609+0000 7f7f64cbf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f601608b0 0x7f7f6015aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53472/0 (socket says 172.21.15.148:53472) 2024-03-29T04:10:52.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.609+0000 7f7f64cbf640 1 -- 172.21.15.148:0/2766603241 learned_addr learned my addr 172.21.15.148:0/2766603241 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:52.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.609+0000 7f7f654c0640 1 --2- 172.21.15.148:0/2766603241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f600ceb50 0x7f7f601f53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:52.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.609+0000 7f7f65cc1640 1 --2- 172.21.15.148:0/2766603241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f601f7fa0 0x7f7f600cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:52.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.609+0000 7f7f64cbf640 1 -- 172.21.15.148:0/2766603241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f601f7fa0 msgr2=0x7f7f600cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:52.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.609+0000 7f7f64cbf640 1 --2- 172.21.15.148:0/2766603241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f601f7fa0 0x7f7f600cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:52.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.609+0000 7f7f64cbf640 1 -- 172.21.15.148:0/2766603241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f600ceb50 msgr2=0x7f7f601f53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:52.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.609+0000 7f7f64cbf640 1 --2- 172.21.15.148:0/2766603241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f600ceb50 0x7f7f601f53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:52.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.609+0000 7f7f64cbf640 1 -- 172.21.15.148:0/2766603241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f601f5900 con 0x7f7f601608b0 2024-03-29T04:10:52.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.609+0000 7f7f64cbf640 1 --2- 172.21.15.148:0/2766603241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f601608b0 0x7f7f6015aff0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7f5000c3d0 tx=0x7f7f5000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:52.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.609+0000 7f7f55ffb640 1 -- 172.21.15.148:0/2766603241 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f500074e0 con 0x7f7f601608b0 2024-03-29T04:10:52.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.609+0000 7f7f55ffb640 1 -- 172.21.15.148:0/2766603241 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f50007680 con 0x7f7f601608b0 2024-03-29T04:10:52.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.609+0000 7f7f67724640 1 -- 172.21.15.148:0/2766603241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f601f8980 con 0x7f7f601608b0 2024-03-29T04:10:52.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.609+0000 7f7f55ffb640 1 -- 172.21.15.148:0/2766603241 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50004a00 con 0x7f7f601608b0 2024-03-29T04:10:52.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.610+0000 7f7f67724640 1 -- 172.21.15.148:0/2766603241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f601f4bc0 con 0x7f7f601608b0 2024-03-29T04:10:52.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.611+0000 7f7f55ffb640 1 -- 172.21.15.148:0/2766603241 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7f5001a020 con 0x7f7f601608b0 2024-03-29T04:10:52.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.611+0000 7f7f67724640 1 -- 172.21.15.148:0/2766603241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f600687e0 con 0x7f7f601608b0 2024-03-29T04:10:52.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.611+0000 7f7f55ffb640 1 --2- 172.21.15.148:0/2766603241 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7f4406c530 0x7f7f4406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:52.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.611+0000 7f7f654c0640 1 --2- 172.21.15.148:0/2766603241 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7f4406c530 0x7f7f4406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:52.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.611+0000 7f7f55ffb640 1 -- 172.21.15.148:0/2766603241 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f7f5008b0b0 con 0x7f7f601608b0 2024-03-29T04:10:52.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.612+0000 7f7f654c0640 1 --2- 172.21.15.148:0/2766603241 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7f4406c530 0x7f7f4406e9f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c036000 tx=0x7f7f5c008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:52.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.615+0000 7f7f55ffb640 1 -- 172.21.15.148:0/2766603241 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7f50058d10 con 0x7f7f601608b0 2024-03-29T04:10:52.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:52.910+0000 7f7f67724640 1 -- 172.21.15.148:0/2766603241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining"]} v 0) -- 0x7f7f600c8f10 con 0x7f7f601608b0 2024-03-29T04:10:53.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.713+0000 7f7f55ffb640 1 -- 172.21.15.148:0/2766603241 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining"]}]=0 v72) ==== 150+0+0 (secure 0 0 0) 0x7f7f5005d010 con 0x7f7f601608b0 2024-03-29T04:10:53.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.715+0000 7f7f67724640 1 -- 172.21.15.148:0/2766603241 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7f4406c530 msgr2=0x7f7f4406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:53.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.715+0000 7f7f67724640 1 --2- 172.21.15.148:0/2766603241 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7f4406c530 0x7f7f4406e9f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c036000 tx=0x7f7f5c008030 comp rx=0 tx=0).stop 2024-03-29T04:10:53.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.715+0000 7f7f67724640 1 -- 172.21.15.148:0/2766603241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f601608b0 msgr2=0x7f7f6015aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:53.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.715+0000 7f7f67724640 1 --2- 172.21.15.148:0/2766603241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f601608b0 0x7f7f6015aff0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7f5000c3d0 tx=0x7f7f5000c890 comp rx=0 tx=0).stop 2024-03-29T04:10:53.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.715+0000 7f7f67724640 1 -- 172.21.15.148:0/2766603241 shutdown_connections 2024-03-29T04:10:53.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.715+0000 7f7f67724640 1 --2- 172.21.15.148:0/2766603241 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7f4406c530 0x7f7f4406e9f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:53.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.715+0000 7f7f67724640 1 --2- 172.21.15.148:0/2766603241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7f601f7fa0 0x7f7f600cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:53.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.715+0000 7f7f67724640 1 --2- 172.21.15.148:0/2766603241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f601608b0 0x7f7f6015aff0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:53.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.715+0000 7f7f67724640 1 --2- 172.21.15.148:0/2766603241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f600ceb50 0x7f7f601f53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:53.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.715+0000 7f7f67724640 1 -- 172.21.15.148:0/2766603241 >> 172.21.15.148:0/2766603241 conn(0x7f7f600b3c70 msgr2=0x7f7f600b53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:53.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.715+0000 7f7f67724640 1 -- 172.21.15.148:0/2766603241 shutdown_connections 2024-03-29T04:10:53.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.715+0000 7f7f67724640 1 -- 172.21.15.148:0/2766603241 wait complete. 2024-03-29T04:10:53.706 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-29T04:10:53.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.832+0000 7f9aced15640 1 Processor -- start 2024-03-29T04:10:53.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.832+0000 7f9aced15640 1 -- start start 2024-03-29T04:10:53.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.833+0000 7f9aced15640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9ac815e3c0 0x7f9ac8161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:53.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.833+0000 7f9aced15640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ac80cf330 con 0x7f9ac80b3430 2024-03-29T04:10:53.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.833+0000 7f9aced15640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ac8162cf0 con 0x7f9ac80b2f00 2024-03-29T04:10:53.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.833+0000 7f9aced15640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f9ac80cf960 con 0x7f9ac815e3c0 2024-03-29T04:10:53.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.833+0000 7f9accab1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9ac815e3c0 0x7f9ac8161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:53.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.833+0000 7f9acd2b2640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9ac80b3430 0x7f9ac806b7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50392/0 (socket says 172.21.15.148:50392) 2024-03-29T04:10:53.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.833+0000 7f9acd2b2640 1 -- 172.21.15.148:0/3316113026 learned_addr learned my addr 172.21.15.148:0/3316113026 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:53.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.834+0000 7f9accab1640 1 -- 172.21.15.148:0/3316113026 >> v1:172.21.15.157:6789/0 conn(0x7f9ac80b2f00 legacy=0x7f9ac81624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:53.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.834+0000 7f9accab1640 1 -- 172.21.15.148:0/3316113026 >> v1:172.21.15.148:6789/0 conn(0x7f9ac80b3430 legacy=0x7f9ac806b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:53.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.834+0000 7f9ac77fe640 1 -- 172.21.15.148:0/3316113026 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1714885149 0 0) 0x7f9ac80cf330 con 0x7f9ac80b3430 2024-03-29T04:10:53.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.834+0000 7f9accab1640 1 -- 172.21.15.148:0/3316113026 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac80cf5f0 con 0x7f9ac815e3c0 2024-03-29T04:10:53.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.835+0000 7f9accab1640 1 --2- 172.21.15.148:0/3316113026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac815e3c0 0x7f9ac8161f70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0097a0 tx=0x7f9abc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8203dccc0f1c87e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:53.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.835+0000 7f9ac77fe640 1 -- 172.21.15.148:0/3316113026 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc035020 con 0x7f9ac815e3c0 2024-03-29T04:10:53.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.835+0000 7f9ac77fe640 1 -- 172.21.15.148:0/3316113026 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9abc031830 con 0x7f9ac815e3c0 2024-03-29T04:10:53.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.835+0000 7f9ac77fe640 1 -- 172.21.15.148:0/3316113026 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc031c00 con 0x7f9ac815e3c0 2024-03-29T04:10:53.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.835+0000 7f9aced15640 1 -- 172.21.15.148:0/3316113026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac815e3c0 msgr2=0x7f9ac8161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:53.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.835+0000 7f9aced15640 1 --2- 172.21.15.148:0/3316113026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac815e3c0 0x7f9ac8161f70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0097a0 tx=0x7f9abc02b8d0 comp rx=0 tx=0).stop 2024-03-29T04:10:53.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.836+0000 7f9aced15640 1 -- 172.21.15.148:0/3316113026 shutdown_connections 2024-03-29T04:10:53.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.836+0000 7f9aced15640 1 --2- 172.21.15.148:0/3316113026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac815e3c0 0x7f9ac8161f70 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:53.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.836+0000 7f9aced15640 1 -- 172.21.15.148:0/3316113026 >> 172.21.15.148:0/3316113026 conn(0x7f9ac80b6430 msgr2=0x7f9ac80d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:53.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.836+0000 7f9aced15640 1 -- 172.21.15.148:0/3316113026 shutdown_connections 2024-03-29T04:10:53.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.836+0000 7f9aced15640 1 -- 172.21.15.148:0/3316113026 wait complete. 2024-03-29T04:10:53.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.837+0000 7f9aced15640 1 Processor -- start 2024-03-29T04:10:53.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.837+0000 7f9aced15640 1 -- start start 2024-03-29T04:10:53.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.838+0000 7f9aced15640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ac80b3430 0x7f9ac80cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:53.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.838+0000 7f9aced15640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ac815e3c0 0x7f9ac8159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:53.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.838+0000 7f9aced15640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac814f5d0 0x7f9ac80c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:53.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.838+0000 7f9aced15640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac80b3b20 con 0x7f9ac814f5d0 2024-03-29T04:10:53.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.838+0000 7f9aced15640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac80b39a0 con 0x7f9ac815e3c0 2024-03-29T04:10:53.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.838+0000 7f9aced15640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9ac806bf00 con 0x7f9ac80b3430 2024-03-29T04:10:53.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.838+0000 7f9acd2b2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac814f5d0 0x7f9ac80c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:53.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.838+0000 7f9acd2b2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac814f5d0 0x7f9ac80c8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53490/0 (socket says 172.21.15.148:53490) 2024-03-29T04:10:53.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.838+0000 7f9acd2b2640 1 -- 172.21.15.148:0/952755805 learned_addr learned my addr 172.21.15.148:0/952755805 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:53.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.838+0000 7f9accab1640 1 --2- 172.21.15.148:0/952755805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ac80b3430 0x7f9ac80cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:53.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.838+0000 7f9ac7fff640 1 --2- 172.21.15.148:0/952755805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ac815e3c0 0x7f9ac8159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:53.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.838+0000 7f9acd2b2640 1 -- 172.21.15.148:0/952755805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ac80b3430 msgr2=0x7f9ac80cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:53.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.838+0000 7f9acd2b2640 1 --2- 172.21.15.148:0/952755805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ac80b3430 0x7f9ac80cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:53.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.838+0000 7f9acd2b2640 1 -- 172.21.15.148:0/952755805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ac815e3c0 msgr2=0x7f9ac8159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:53.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.838+0000 7f9acd2b2640 1 --2- 172.21.15.148:0/952755805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ac815e3c0 0x7f9ac8159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:53.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.838+0000 7f9acd2b2640 1 -- 172.21.15.148:0/952755805 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac81599c0 con 0x7f9ac814f5d0 2024-03-29T04:10:53.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.838+0000 7f9acd2b2640 1 --2- 172.21.15.148:0/952755805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac814f5d0 0x7f9ac80c8330 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000c7d0 tx=0x7f9ac000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:53.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.838+0000 7f9ac57fa640 1 -- 172.21.15.148:0/952755805 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0018070 con 0x7f9ac814f5d0 2024-03-29T04:10:53.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.838+0000 7f9ac57fa640 1 -- 172.21.15.148:0/952755805 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ac000a640 con 0x7f9ac814f5d0 2024-03-29T04:10:53.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.838+0000 7f9ac57fa640 1 -- 172.21.15.148:0/952755805 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac000aa70 con 0x7f9ac814f5d0 2024-03-29T04:10:53.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.838+0000 7f9aced15640 1 -- 172.21.15.148:0/952755805 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ac80cba30 con 0x7f9ac814f5d0 2024-03-29T04:10:53.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.838+0000 7f9aced15640 1 -- 172.21.15.148:0/952755805 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ac80cc2d0 con 0x7f9ac814f5d0 2024-03-29T04:10:53.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.840+0000 7f9aced15640 1 -- 172.21.15.148:0/952755805 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ac8068850 con 0x7f9ac814f5d0 2024-03-29T04:10:53.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.840+0000 7f9ac57fa640 1 -- 172.21.15.148:0/952755805 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9ac001b070 con 0x7f9ac814f5d0 2024-03-29T04:10:53.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.840+0000 7f9ac57fa640 1 --2- 172.21.15.148:0/952755805 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9aac06c530 0x7f9aac06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:53.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.840+0000 7f9ac57fa640 1 -- 172.21.15.148:0/952755805 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f9ac008b0d0 con 0x7f9ac814f5d0 2024-03-29T04:10:53.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.840+0000 7f9accab1640 1 --2- 172.21.15.148:0/952755805 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9aac06c530 0x7f9aac06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:53.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.841+0000 7f9accab1640 1 --2- 172.21.15.148:0/952755805 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9aac06c530 0x7f9aac06e9f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9abc036000 tx=0x7f9abc03c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:53.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:53.849+0000 7f9ac57fa640 1 -- 172.21.15.148:0/952755805 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9ac00592f0 con 0x7f9ac814f5d0 2024-03-29T04:10:54.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.032+0000 7f9aced15640 1 -- 172.21.15.148:0/952755805 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f9ac80b9cf0 con 0x7f9ac814f5d0 2024-03-29T04:10:54.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.037+0000 7f9ac57fa640 1 -- 172.21.15.148:0/952755805 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f9ac0058bf0 con 0x7f9ac814f5d0 2024-03-29T04:10:54.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.039+0000 7f9aced15640 1 -- 172.21.15.148:0/952755805 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9aac06c530 msgr2=0x7f9aac06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:54.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.039+0000 7f9aced15640 1 --2- 172.21.15.148:0/952755805 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9aac06c530 0x7f9aac06e9f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9abc036000 tx=0x7f9abc03c040 comp rx=0 tx=0).stop 2024-03-29T04:10:54.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.039+0000 7f9aced15640 1 -- 172.21.15.148:0/952755805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac814f5d0 msgr2=0x7f9ac80c8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:54.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.039+0000 7f9aced15640 1 --2- 172.21.15.148:0/952755805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac814f5d0 0x7f9ac80c8330 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000c7d0 tx=0x7f9ac000cc90 comp rx=0 tx=0).stop 2024-03-29T04:10:54.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.039+0000 7f9aced15640 1 -- 172.21.15.148:0/952755805 shutdown_connections 2024-03-29T04:10:54.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.039+0000 7f9aced15640 1 --2- 172.21.15.148:0/952755805 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9aac06c530 0x7f9aac06e9f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.039+0000 7f9aced15640 1 --2- 172.21.15.148:0/952755805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac814f5d0 0x7f9ac80c8330 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.039+0000 7f9aced15640 1 --2- 172.21.15.148:0/952755805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ac815e3c0 0x7f9ac8159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.039+0000 7f9aced15640 1 --2- 172.21.15.148:0/952755805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ac80b3430 0x7f9ac80cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.039+0000 7f9aced15640 1 -- 172.21.15.148:0/952755805 >> 172.21.15.148:0/952755805 conn(0x7f9ac80b6430 msgr2=0x7f9ac815cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:54.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.039+0000 7f9aced15640 1 -- 172.21.15.148:0/952755805 shutdown_connections 2024-03-29T04:10:54.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.039+0000 7f9aced15640 1 -- 172.21.15.148:0/952755805 wait complete. 2024-03-29T04:10:54.030 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-29T04:10:54.030 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T04:10:54.067 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T04:10:54.115 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-29T04:10:54.116 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:10:54.116 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:10:54.132 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:10:54.132 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:10:54.187 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:54.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.350+0000 7f0a82e3c640 1 Processor -- start 2024-03-29T04:10:54.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.350+0000 7f0a82e3c640 1 -- start start 2024-03-29T04:10:54.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.350+0000 7f0a82e3c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0a7c0ba030 0x7f0a7c0d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:54.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.350+0000 7f0a82e3c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a7c15e7c0 con 0x7f0a7c0d10d0 2024-03-29T04:10:54.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.350+0000 7f0a82e3c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a7c0d2c30 con 0x7f0a7c15e3e0 2024-03-29T04:10:54.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.350+0000 7f0a82e3c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f0a7c0d3840 con 0x7f0a7c0ba030 2024-03-29T04:10:54.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.351+0000 7f0a813d9640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0a7c0d10d0 0x7f0a7c167f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50394/0 (socket says 172.21.15.148:50394) 2024-03-29T04:10:54.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.351+0000 7f0a813d9640 1 -- 172.21.15.148:0/1946199207 learned_addr learned my addr 172.21.15.148:0/1946199207 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:54.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.351+0000 7f0a7b7fe640 1 -- 172.21.15.148:0/1946199207 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3899488281 0 0) 0x7f0a7c15e7c0 con 0x7f0a7c0d10d0 2024-03-29T04:10:54.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.351+0000 7f0a7bfff640 1 --2- 172.21.15.148:0/1946199207 >> v2:172.21.15.157:3300/0 conn(0x7f0a7c0ba030 0x7f0a7c0d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:54.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.351+0000 7f0a7b7fe640 1 -- 172.21.15.148:0/1946199207 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a6c003520 con 0x7f0a7c0d10d0 2024-03-29T04:10:54.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.351+0000 7f0a7b7fe640 1 -- 172.21.15.148:0/1946199207 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1110161403 0 0) 0x7f0a6c003520 con 0x7f0a7c0d10d0 2024-03-29T04:10:54.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.351+0000 7f0a7b7fe640 1 -- 172.21.15.148:0/1946199207 >> v2:172.21.15.157:3300/0 conn(0x7f0a7c0ba030 msgr2=0x7f0a7c0d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:54.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.351+0000 7f0a7b7fe640 1 --2- 172.21.15.148:0/1946199207 >> v2:172.21.15.157:3300/0 conn(0x7f0a7c0ba030 0x7f0a7c0d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.351+0000 7f0a7b7fe640 1 -- 172.21.15.148:0/1946199207 >> v1:172.21.15.157:6789/0 conn(0x7f0a7c15e3e0 legacy=0x7f0a7c0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:54.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.351+0000 7f0a7b7fe640 1 -- 172.21.15.148:0/1946199207 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7c0d2f10 con 0x7f0a7c0d10d0 2024-03-29T04:10:54.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.351+0000 7f0a7b7fe640 1 -- 172.21.15.148:0/1946199207 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f0a740019f0 con 0x7f0a7c0d10d0 2024-03-29T04:10:54.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.351+0000 7f0a7b7fe640 1 -- 172.21.15.148:0/1946199207 >> v1:172.21.15.148:6789/0 conn(0x7f0a7c0d10d0 legacy=0x7f0a7c167f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:54.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.351+0000 7f0a7b7fe640 1 --2- 172.21.15.148:0/1946199207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a6c0039b0 0x7f0a6c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:54.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.351+0000 7f0a7b7fe640 1 --2- 172.21.15.148:0/1946199207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a6c004800 0x7f0a6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:54.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.351+0000 7f0a7b7fe640 1 --2- 172.21.15.148:0/1946199207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a6c0051c0 0x7f0a6c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:54.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.351+0000 7f0a7b7fe640 1 -- 172.21.15.148:0/1946199207 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c0d3840 con 0x7f0a6c0051c0 2024-03-29T04:10:54.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.351+0000 7f0a7b7fe640 1 -- 172.21.15.148:0/1946199207 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6c00c070 con 0x7f0a6c004800 2024-03-29T04:10:54.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.351+0000 7f0a7b7fe640 1 -- 172.21.15.148:0/1946199207 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0a6c00c270 con 0x7f0a6c0039b0 2024-03-29T04:10:54.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.351+0000 7f0a813d9640 1 --2- 172.21.15.148:0/1946199207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a6c0051c0 0x7f0a6c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:54.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.351+0000 7f0a7bfff640 1 --2- 172.21.15.148:0/1946199207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a6c004800 0x7f0a6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:54.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.351+0000 7f0a80bd8640 1 --2- 172.21.15.148:0/1946199207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a6c0039b0 0x7f0a6c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:54.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.352+0000 7f0a813d9640 1 -- 172.21.15.148:0/1946199207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a6c0039b0 msgr2=0x7f0a6c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:54.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.352+0000 7f0a813d9640 1 --2- 172.21.15.148:0/1946199207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a6c0039b0 0x7f0a6c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.352+0000 7f0a813d9640 1 -- 172.21.15.148:0/1946199207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a6c004800 msgr2=0x7f0a6c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:54.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.352+0000 7f0a813d9640 1 --2- 172.21.15.148:0/1946199207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a6c004800 0x7f0a6c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.352+0000 7f0a813d9640 1 -- 172.21.15.148:0/1946199207 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a6c003520 con 0x7f0a6c0051c0 2024-03-29T04:10:54.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.352+0000 7f0a813d9640 1 -- 172.21.15.148:0/1946199207 reap_dead start 2024-03-29T04:10:54.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.352+0000 7f0a813d9640 1 --2- 172.21.15.148:0/1946199207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a6c0051c0 0x7f0a6c00b9c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400a840 tx=0x7f0a6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c7fde2b0bdf35 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:54.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.353+0000 7f0a7b7fe640 1 -- 172.21.15.148:0/1946199207 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7400a460 con 0x7f0a6c0051c0 2024-03-29T04:10:54.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.353+0000 7f0a7b7fe640 1 -- 172.21.15.148:0/1946199207 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a74034650 con 0x7f0a6c0051c0 2024-03-29T04:10:54.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.353+0000 7f0a7b7fe640 1 -- 172.21.15.148:0/1946199207 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a74034a40 con 0x7f0a6c0051c0 2024-03-29T04:10:54.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.353+0000 7f0a82e3c640 1 -- 172.21.15.148:0/1946199207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a6c0051c0 msgr2=0x7f0a6c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:54.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.353+0000 7f0a82e3c640 1 --2- 172.21.15.148:0/1946199207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a6c0051c0 0x7f0a6c00b9c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400a840 tx=0x7f0a6c005830 comp rx=0 tx=0).stop 2024-03-29T04:10:54.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.353+0000 7f0a82e3c640 1 -- 172.21.15.148:0/1946199207 shutdown_connections 2024-03-29T04:10:54.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.353+0000 7f0a82e3c640 1 --2- 172.21.15.148:0/1946199207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a6c0051c0 0x7f0a6c00b9c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.353+0000 7f0a82e3c640 1 -- 172.21.15.148:0/1946199207 >> 172.21.15.148:0/1946199207 conn(0x7f0a7c0b6210 msgr2=0x7f0a7c0b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:54.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.353+0000 7f0a82e3c640 1 -- 172.21.15.148:0/1946199207 shutdown_connections 2024-03-29T04:10:54.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.354+0000 7f0a82e3c640 1 -- 172.21.15.148:0/1946199207 wait complete. 2024-03-29T04:10:54.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.355+0000 7f0a82e3c640 1 Processor -- start 2024-03-29T04:10:54.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.355+0000 7f0a82e3c640 1 -- start start 2024-03-29T04:10:54.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.355+0000 7f0a82e3c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a6c0051c0 0x7f0a7c14e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:54.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.355+0000 7f0a82e3c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a7c15cb80 0x7f0a7c15d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:54.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.355+0000 7f0a82e3c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a7c15f9f0 0x7f0a7c15fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:54.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.355+0000 7f0a82e3c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c0d3cd0 con 0x7f0a6c0051c0 2024-03-29T04:10:54.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.355+0000 7f0a82e3c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c0d34d0 con 0x7f0a7c15cb80 2024-03-29T04:10:54.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.355+0000 7f0a82e3c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0a7c0d3e50 con 0x7f0a7c15f9f0 2024-03-29T04:10:54.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.356+0000 7f0a80bd8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a6c0051c0 0x7f0a7c14e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:54.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.356+0000 7f0a80bd8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a6c0051c0 0x7f0a7c14e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53520/0 (socket says 172.21.15.148:53520) 2024-03-29T04:10:54.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.356+0000 7f0a80bd8640 1 -- 172.21.15.148:0/3375234224 learned_addr learned my addr 172.21.15.148:0/3375234224 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:54.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.356+0000 7f0a813d9640 1 --2- 172.21.15.148:0/3375234224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a7c15f9f0 0x7f0a7c15fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:54.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.356+0000 7f0a7bfff640 1 --2- 172.21.15.148:0/3375234224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a7c15cb80 0x7f0a7c15d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:54.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.356+0000 7f0a80bd8640 1 -- 172.21.15.148:0/3375234224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a7c15f9f0 msgr2=0x7f0a7c15fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:54.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.356+0000 7f0a80bd8640 1 --2- 172.21.15.148:0/3375234224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a7c15f9f0 0x7f0a7c15fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.356+0000 7f0a80bd8640 1 -- 172.21.15.148:0/3375234224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a7c15cb80 msgr2=0x7f0a7c15d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:54.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.356+0000 7f0a80bd8640 1 --2- 172.21.15.148:0/3375234224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a7c15cb80 0x7f0a7c15d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.356+0000 7f0a80bd8640 1 -- 172.21.15.148:0/3375234224 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7c1609f0 con 0x7f0a6c0051c0 2024-03-29T04:10:54.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.356+0000 7f0a7bfff640 1 --2- 172.21.15.148:0/3375234224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a7c15cb80 0x7f0a7c15d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:54.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.356+0000 7f0a80bd8640 1 --2- 172.21.15.148:0/3375234224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a6c0051c0 0x7f0a7c14e0d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0a7000f620 tx=0x7f0a7000fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:54.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.356+0000 7f0a797fa640 1 -- 172.21.15.148:0/3375234224 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a700052e0 con 0x7f0a6c0051c0 2024-03-29T04:10:54.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.356+0000 7f0a797fa640 1 -- 172.21.15.148:0/3375234224 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a70018070 con 0x7f0a6c0051c0 2024-03-29T04:10:54.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.356+0000 7f0a797fa640 1 -- 172.21.15.148:0/3375234224 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a70003810 con 0x7f0a6c0051c0 2024-03-29T04:10:54.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.356+0000 7f0a82e3c640 1 -- 172.21.15.148:0/3375234224 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a7c160710 con 0x7f0a6c0051c0 2024-03-29T04:10:54.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.356+0000 7f0a82e3c640 1 -- 172.21.15.148:0/3375234224 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a7c15a850 con 0x7f0a6c0051c0 2024-03-29T04:10:54.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.357+0000 7f0a82e3c640 1 -- 172.21.15.148:0/3375234224 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a7c15d540 con 0x7f0a6c0051c0 2024-03-29T04:10:54.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.358+0000 7f0a797fa640 1 -- 172.21.15.148:0/3375234224 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0a700039b0 con 0x7f0a6c0051c0 2024-03-29T04:10:54.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.358+0000 7f0a797fa640 1 --2- 172.21.15.148:0/3375234224 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f0a6006c560 0x7f0a6006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:54.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.358+0000 7f0a797fa640 1 -- 172.21.15.148:0/3375234224 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f0a7008b7d0 con 0x7f0a6c0051c0 2024-03-29T04:10:54.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.358+0000 7f0a7bfff640 1 --2- 172.21.15.148:0/3375234224 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f0a6006c560 0x7f0a6006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:54.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.359+0000 7f0a7bfff640 1 --2- 172.21.15.148:0/3375234224 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f0a6006c560 0x7f0a6006ea20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0a680093d0 tx=0x7f0a68008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:54.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.363+0000 7f0a797fa640 1 -- 172.21.15.148:0/3375234224 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0a7c15d540 con 0x7f0a6c0051c0 2024-03-29T04:10:54.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.544+0000 7f0a82e3c640 1 -- 172.21.15.148:0/3375234224 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0a7c0687e0 con 0x7f0a6c0051c0 2024-03-29T04:10:54.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.545+0000 7f0a797fa640 1 -- 172.21.15.148:0/3375234224 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18858 (secure 0 0 0) 0x7f0a70059a80 con 0x7f0a6c0051c0 2024-03-29T04:10:54.525 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:54.525 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:41.662934+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:54.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.548+0000 7f0a82e3c640 1 -- 172.21.15.148:0/3375234224 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f0a6006c560 msgr2=0x7f0a6006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:54.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.548+0000 7f0a82e3c640 1 --2- 172.21.15.148:0/3375234224 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f0a6006c560 0x7f0a6006ea20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0a680093d0 tx=0x7f0a68008040 comp rx=0 tx=0).stop 2024-03-29T04:10:54.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.548+0000 7f0a82e3c640 1 -- 172.21.15.148:0/3375234224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a6c0051c0 msgr2=0x7f0a7c14e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:54.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.548+0000 7f0a82e3c640 1 --2- 172.21.15.148:0/3375234224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a6c0051c0 0x7f0a7c14e0d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0a7000f620 tx=0x7f0a7000fae0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.548+0000 7f0a82e3c640 1 -- 172.21.15.148:0/3375234224 shutdown_connections 2024-03-29T04:10:54.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.548+0000 7f0a82e3c640 1 --2- 172.21.15.148:0/3375234224 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f0a6006c560 0x7f0a6006ea20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.548+0000 7f0a82e3c640 1 --2- 172.21.15.148:0/3375234224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a7c15f9f0 0x7f0a7c15fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.548+0000 7f0a82e3c640 1 --2- 172.21.15.148:0/3375234224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a7c15cb80 0x7f0a7c15d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.548+0000 7f0a82e3c640 1 --2- 172.21.15.148:0/3375234224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a6c0051c0 0x7f0a7c14e0d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.548+0000 7f0a82e3c640 1 -- 172.21.15.148:0/3375234224 >> 172.21.15.148:0/3375234224 conn(0x7f0a7c0b6210 msgr2=0x7f0a7c0b7120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:54.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.548+0000 7f0a82e3c640 1 -- 172.21.15.148:0/3375234224 shutdown_connections 2024-03-29T04:10:54.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.548+0000 7f0a82e3c640 1 -- 172.21.15.148:0/3375234224 wait complete. 2024-03-29T04:10:54.540 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:47.898351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:57.922313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:58.829405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:10:02.551763+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:10:54.541 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:10:54.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.666+0000 7fdfcaae4640 1 Processor -- start 2024-03-29T04:10:54.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.666+0000 7fdfcaae4640 1 -- start start 2024-03-29T04:10:54.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.667+0000 7fdfcaae4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfc415e3c0 0x7fdfc4161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:54.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.667+0000 7fdfcaae4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdfc40b3430 0x7fdfc406b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:54.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.667+0000 7fdfcaae4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfc40cf330 con 0x7fdfc40b2f00 2024-03-29T04:10:54.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.667+0000 7fdfcaae4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fdfc40cf960 con 0x7fdfc40b3430 2024-03-29T04:10:54.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.667+0000 7fdfcaae4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdfc4162cf0 con 0x7fdfc415e3c0 2024-03-29T04:10:54.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.667+0000 7fdfc3fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fdfc40b2f00 0x7fdfc41624b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50404/0 (socket says 172.21.15.148:50404) 2024-03-29T04:10:54.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.667+0000 7fdfc3fff640 1 -- 172.21.15.148:0/1335375354 learned_addr learned my addr 172.21.15.148:0/1335375354 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:54.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.667+0000 7fdfc9081640 1 --2- 172.21.15.148:0/1335375354 >> v2:172.21.15.157:3300/0 conn(0x7fdfc40b3430 0x7fdfc406b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:54.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.667+0000 7fdfc8880640 1 --2- 172.21.15.148:0/1335375354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfc415e3c0 0x7fdfc4161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:54.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.667+0000 7fdfc37fe640 1 -- 172.21.15.148:0/1335375354 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 851426048 0 0) 0x7fdfc40cf330 con 0x7fdfc40b2f00 2024-03-29T04:10:54.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.668+0000 7fdfc37fe640 1 -- 172.21.15.148:0/1335375354 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdfb4003520 con 0x7fdfc40b2f00 2024-03-29T04:10:54.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.668+0000 7fdfc9081640 1 -- 172.21.15.148:0/1335375354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfc415e3c0 msgr2=0x7fdfc4161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:54.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.668+0000 7fdfc9081640 1 --2- 172.21.15.148:0/1335375354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfc415e3c0 0x7fdfc4161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.668+0000 7fdfc9081640 1 -- 172.21.15.148:0/1335375354 >> v1:172.21.15.148:6789/0 conn(0x7fdfc40b2f00 legacy=0x7fdfc41624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:54.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.668+0000 7fdfc9081640 1 -- 172.21.15.148:0/1335375354 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc40cf5f0 con 0x7fdfc40b3430 2024-03-29T04:10:54.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.669+0000 7fdfc9081640 1 --2- 172.21.15.148:0/1335375354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc40b3430 0x7fdfc406b7f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc009d90 tx=0x7fdfbc02be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=578d543ca0f6af64 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:54.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.669+0000 7fdfc37fe640 1 -- 172.21.15.148:0/1335375354 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc039070 con 0x7fdfc40b3430 2024-03-29T04:10:54.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.669+0000 7fdfc37fe640 1 -- 172.21.15.148:0/1335375354 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfbc02cbc0 con 0x7fdfc40b3430 2024-03-29T04:10:54.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.669+0000 7fdfc37fe640 1 -- 172.21.15.148:0/1335375354 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc034440 con 0x7fdfc40b3430 2024-03-29T04:10:54.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.670+0000 7fdfcaae4640 1 -- 172.21.15.148:0/1335375354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc40b3430 msgr2=0x7fdfc406b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:54.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.670+0000 7fdfcaae4640 1 --2- 172.21.15.148:0/1335375354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc40b3430 0x7fdfc406b7f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc009d90 tx=0x7fdfbc02be10 comp rx=0 tx=0).stop 2024-03-29T04:10:54.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.670+0000 7fdfcaae4640 1 -- 172.21.15.148:0/1335375354 shutdown_connections 2024-03-29T04:10:54.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.670+0000 7fdfcaae4640 1 --2- 172.21.15.148:0/1335375354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc40b3430 0x7fdfc406b7f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.670+0000 7fdfcaae4640 1 --2- 172.21.15.148:0/1335375354 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfc415e3c0 0x7fdfc4161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.670+0000 7fdfcaae4640 1 -- 172.21.15.148:0/1335375354 >> 172.21.15.148:0/1335375354 conn(0x7fdfc40b6430 msgr2=0x7fdfc40d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:54.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.670+0000 7fdfcaae4640 1 -- 172.21.15.148:0/1335375354 shutdown_connections 2024-03-29T04:10:54.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.670+0000 7fdfcaae4640 1 -- 172.21.15.148:0/1335375354 wait complete. 2024-03-29T04:10:54.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.671+0000 7fdfcaae4640 1 Processor -- start 2024-03-29T04:10:54.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.671+0000 7fdfcaae4640 1 -- start start 2024-03-29T04:10:54.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.672+0000 7fdfcaae4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc40b3430 0x7fdfc40c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:54.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.672+0000 7fdfcaae4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfc415e3c0 0x7fdfc415b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:54.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.672+0000 7fdfcaae4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfc415f3c0 0x7fdfc415b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:54.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.672+0000 7fdfcaae4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc40b3b20 con 0x7fdfc415e3c0 2024-03-29T04:10:54.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.672+0000 7fdfcaae4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc40b39a0 con 0x7fdfc40b3430 2024-03-29T04:10:54.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.672+0000 7fdfcaae4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdfc406bd30 con 0x7fdfc415f3c0 2024-03-29T04:10:54.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.672+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfc415e3c0 0x7fdfc415b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:54.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.672+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfc415e3c0 0x7fdfc415b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53530/0 (socket says 172.21.15.148:53530) 2024-03-29T04:10:54.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.672+0000 7fdfc3fff640 1 -- 172.21.15.148:0/2947612269 learned_addr learned my addr 172.21.15.148:0/2947612269 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:54.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.672+0000 7fdfc9081640 1 --2- 172.21.15.148:0/2947612269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfc415f3c0 0x7fdfc415b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:54.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.672+0000 7fdfc8880640 1 --2- 172.21.15.148:0/2947612269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc40b3430 0x7fdfc40c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:54.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.672+0000 7fdfc3fff640 1 -- 172.21.15.148:0/2947612269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfc415f3c0 msgr2=0x7fdfc415b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:54.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.672+0000 7fdfc3fff640 1 --2- 172.21.15.148:0/2947612269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfc415f3c0 0x7fdfc415b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.672+0000 7fdfc3fff640 1 -- 172.21.15.148:0/2947612269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc40b3430 msgr2=0x7fdfc40c9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:54.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.672+0000 7fdfc3fff640 1 --2- 172.21.15.148:0/2947612269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc40b3430 0x7fdfc40c9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.672+0000 7fdfc3fff640 1 -- 172.21.15.148:0/2947612269 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc415f8c0 con 0x7fdfc415e3c0 2024-03-29T04:10:54.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.672+0000 7fdfc3fff640 1 --2- 172.21.15.148:0/2947612269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfc415e3c0 0x7fdfc415b110 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdfb000c460 tx=0x7fdfb000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:54.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.672+0000 7fdfc17fa640 1 -- 172.21.15.148:0/2947612269 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0017070 con 0x7fdfc415e3c0 2024-03-29T04:10:54.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.672+0000 7fdfc17fa640 1 -- 172.21.15.148:0/2947612269 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfb0005ce0 con 0x7fdfc415e3c0 2024-03-29T04:10:54.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.672+0000 7fdfcaae4640 1 -- 172.21.15.148:0/2947612269 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc415fba0 con 0x7fdfc415e3c0 2024-03-29T04:10:54.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.673+0000 7fdfc17fa640 1 -- 172.21.15.148:0/2947612269 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0007290 con 0x7fdfc415e3c0 2024-03-29T04:10:54.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.673+0000 7fdfcaae4640 1 -- 172.21.15.148:0/2947612269 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc4160160 con 0x7fdfc415e3c0 2024-03-29T04:10:54.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.674+0000 7fdfcaae4640 1 -- 172.21.15.148:0/2947612269 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfc4068850 con 0x7fdfc415e3c0 2024-03-29T04:10:54.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.674+0000 7fdfc17fa640 1 -- 172.21.15.148:0/2947612269 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdfb001a070 con 0x7fdfc415e3c0 2024-03-29T04:10:54.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.674+0000 7fdfc17fa640 1 --2- 172.21.15.148:0/2947612269 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fdfa806c530 0x7fdfa806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:54.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.674+0000 7fdfc17fa640 1 -- 172.21.15.148:0/2947612269 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fdfb008ae60 con 0x7fdfc415e3c0 2024-03-29T04:10:54.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.674+0000 7fdfc8880640 1 --2- 172.21.15.148:0/2947612269 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fdfa806c530 0x7fdfa806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:54.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.675+0000 7fdfc8880640 1 --2- 172.21.15.148:0/2947612269 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fdfa806c530 0x7fdfa806e9f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdfc415bf70 tx=0x7fdfb8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:54.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.679+0000 7fdfc17fa640 1 -- 172.21.15.148:0/2947612269 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fdfb0059080 con 0x7fdfc415e3c0 2024-03-29T04:10:54.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.871+0000 7fdfcaae4640 1 -- 172.21.15.148:0/2947612269 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdfc40b7050 con 0x7fdfc415e3c0 2024-03-29T04:10:54.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.872+0000 7fdfc17fa640 1 -- 172.21.15.148:0/2947612269 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fdfb0058980 con 0x7fdfc415e3c0 2024-03-29T04:10:54.851 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-29T04:10:54.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.874+0000 7fdfcaae4640 1 -- 172.21.15.148:0/2947612269 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fdfa806c530 msgr2=0x7fdfa806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:54.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.874+0000 7fdfcaae4640 1 --2- 172.21.15.148:0/2947612269 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fdfa806c530 0x7fdfa806e9f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdfc415bf70 tx=0x7fdfb8008040 comp rx=0 tx=0).stop 2024-03-29T04:10:54.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.874+0000 7fdfcaae4640 1 -- 172.21.15.148:0/2947612269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfc415e3c0 msgr2=0x7fdfc415b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:54.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.874+0000 7fdfcaae4640 1 --2- 172.21.15.148:0/2947612269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfc415e3c0 0x7fdfc415b110 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdfb000c460 tx=0x7fdfb000c920 comp rx=0 tx=0).stop 2024-03-29T04:10:54.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.874+0000 7fdfcaae4640 1 -- 172.21.15.148:0/2947612269 shutdown_connections 2024-03-29T04:10:54.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.874+0000 7fdfcaae4640 1 --2- 172.21.15.148:0/2947612269 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fdfa806c530 0x7fdfa806e9f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.874+0000 7fdfcaae4640 1 --2- 172.21.15.148:0/2947612269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfc415f3c0 0x7fdfc415b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.874+0000 7fdfcaae4640 1 --2- 172.21.15.148:0/2947612269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfc415e3c0 0x7fdfc415b110 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.874+0000 7fdfcaae4640 1 --2- 172.21.15.148:0/2947612269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc40b3430 0x7fdfc40c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.874+0000 7fdfcaae4640 1 -- 172.21.15.148:0/2947612269 >> 172.21.15.148:0/2947612269 conn(0x7fdfc40b6430 msgr2=0x7fdfc40d2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:54.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.874+0000 7fdfcaae4640 1 -- 172.21.15.148:0/2947612269 shutdown_connections 2024-03-29T04:10:54.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.874+0000 7fdfcaae4640 1 -- 172.21.15.148:0/2947612269 wait complete. 2024-03-29T04:10:54.867 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:10:54.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.993+0000 7ff3df063640 1 Processor -- start 2024-03-29T04:10:54.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.993+0000 7ff3df063640 1 -- start start 2024-03-29T04:10:54.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.994+0000 7ff3df063640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3d815e3c0 0x7ff3d8161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:54.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.994+0000 7ff3df063640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff3d80b3430 0x7ff3d806b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:54.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.994+0000 7ff3df063640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3d80cf330 con 0x7ff3d80b2f00 2024-03-29T04:10:54.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.994+0000 7ff3df063640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7ff3d80cf960 con 0x7ff3d80b3430 2024-03-29T04:10:54.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.994+0000 7ff3df063640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff3d8162cf0 con 0x7ff3d815e3c0 2024-03-29T04:10:54.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.994+0000 7ff3dd600640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff3d80b3430 0x7ff3d806b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:54.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.994+0000 7ff3dd600640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff3d80b3430 0x7ff3d806b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53544/0 (socket says 172.21.15.148:53544) 2024-03-29T04:10:54.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.994+0000 7ff3dd600640 1 -- 172.21.15.148:0/268797110 learned_addr learned my addr 172.21.15.148:0/268797110 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:54.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.994+0000 7ff3d7fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff3d80b2f00 0x7ff3d81624b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50414/0 (socket says 172.21.15.148:50414) 2024-03-29T04:10:54.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.994+0000 7ff3dcdff640 1 --2- 172.21.15.148:0/268797110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3d815e3c0 0x7ff3d8161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:54.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.994+0000 7ff3dd600640 1 -- 172.21.15.148:0/268797110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3d815e3c0 msgr2=0x7ff3d8161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:54.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.994+0000 7ff3dd600640 1 --2- 172.21.15.148:0/268797110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3d815e3c0 0x7ff3d8161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.994+0000 7ff3dd600640 1 -- 172.21.15.148:0/268797110 >> v1:172.21.15.148:6789/0 conn(0x7ff3d80b2f00 legacy=0x7ff3d81624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:54.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.994+0000 7ff3dd600640 1 -- 172.21.15.148:0/268797110 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d80cf5f0 con 0x7ff3d80b3430 2024-03-29T04:10:54.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.994+0000 7ff3d77fe640 1 -- 172.21.15.148:0/268797110 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3777364881 0 0) 0x7ff3d80cf330 con 0x7ff3d80b2f00 2024-03-29T04:10:54.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.994+0000 7ff3dcdff640 1 --2- 172.21.15.148:0/268797110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3d815e3c0 0x7ff3d8161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:54.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.995+0000 7ff3dd600640 1 --2- 172.21.15.148:0/268797110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3d80b3430 0x7ff3d806b7f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0009e50 tx=0x7ff3d002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66faf1a22e657f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:54.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.995+0000 7ff3d77fe640 1 -- 172.21.15.148:0/268797110 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d002ca70 con 0x7ff3d80b3430 2024-03-29T04:10:54.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.995+0000 7ff3d77fe640 1 -- 172.21.15.148:0/268797110 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3d002cc10 con 0x7ff3d80b3430 2024-03-29T04:10:54.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.995+0000 7ff3d77fe640 1 -- 172.21.15.148:0/268797110 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d00343f0 con 0x7ff3d80b3430 2024-03-29T04:10:54.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.996+0000 7ff3df063640 1 -- 172.21.15.148:0/268797110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3d80b3430 msgr2=0x7ff3d806b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:54.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.996+0000 7ff3df063640 1 --2- 172.21.15.148:0/268797110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3d80b3430 0x7ff3d806b7f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0009e50 tx=0x7ff3d002bed0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.996+0000 7ff3df063640 1 -- 172.21.15.148:0/268797110 shutdown_connections 2024-03-29T04:10:54.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.996+0000 7ff3df063640 1 --2- 172.21.15.148:0/268797110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3d80b3430 0x7ff3d806b7f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.996+0000 7ff3df063640 1 --2- 172.21.15.148:0/268797110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3d815e3c0 0x7ff3d8161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.996+0000 7ff3df063640 1 -- 172.21.15.148:0/268797110 >> 172.21.15.148:0/268797110 conn(0x7ff3d80b6430 msgr2=0x7ff3d80d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:54.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.996+0000 7ff3df063640 1 -- 172.21.15.148:0/268797110 shutdown_connections 2024-03-29T04:10:54.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.996+0000 7ff3df063640 1 -- 172.21.15.148:0/268797110 wait complete. 2024-03-29T04:10:54.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.998+0000 7ff3df063640 1 Processor -- start 2024-03-29T04:10:54.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.998+0000 7ff3df063640 1 -- start start 2024-03-29T04:10:54.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.999+0000 7ff3df063640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3d80b3430 0x7ff3d815fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:54.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.999+0000 7ff3df063640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3d815e3c0 0x7ff3d81556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:54.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.999+0000 7ff3df063640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3d8155c20 0x7ff3d81f4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:54.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.999+0000 7ff3df063640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d80b3b20 con 0x7ff3d815e3c0 2024-03-29T04:10:54.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.999+0000 7ff3df063640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d80b39a0 con 0x7ff3d8155c20 2024-03-29T04:10:54.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.999+0000 7ff3df063640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff3d806bd30 con 0x7ff3d80b3430 2024-03-29T04:10:54.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.999+0000 7ff3d7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3d815e3c0 0x7ff3d81556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:54.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.999+0000 7ff3d7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3d815e3c0 0x7ff3d81556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53546/0 (socket says 172.21.15.148:53546) 2024-03-29T04:10:54.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.999+0000 7ff3d7fff640 1 -- 172.21.15.148:0/324352644 learned_addr learned my addr 172.21.15.148:0/324352644 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:54.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.999+0000 7ff3dd600640 1 --2- 172.21.15.148:0/324352644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3d8155c20 0x7ff3d81f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:54.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.999+0000 7ff3dcdff640 1 --2- 172.21.15.148:0/324352644 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3d80b3430 0x7ff3d815fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:54.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.999+0000 7ff3d7fff640 1 -- 172.21.15.148:0/324352644 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3d80b3430 msgr2=0x7ff3d815fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:54.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.999+0000 7ff3d7fff640 1 --2- 172.21.15.148:0/324352644 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3d80b3430 0x7ff3d815fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.999+0000 7ff3d7fff640 1 -- 172.21.15.148:0/324352644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3d8155c20 msgr2=0x7ff3d81f4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:54.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.999+0000 7ff3d7fff640 1 --2- 172.21.15.148:0/324352644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3d8155c20 0x7ff3d81f4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:54.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.999+0000 7ff3d7fff640 1 -- 172.21.15.148:0/324352644 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d8160f20 con 0x7ff3d815e3c0 2024-03-29T04:10:54.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:54.999+0000 7ff3dd600640 1 --2- 172.21.15.148:0/324352644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3d8155c20 0x7ff3d81f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:54.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.000+0000 7ff3d7fff640 1 --2- 172.21.15.148:0/324352644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3d815e3c0 0x7ff3d81556e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc00c4d0 tx=0x7ff3cc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:54.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.000+0000 7ff3d57fa640 1 -- 172.21.15.148:0/324352644 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc018070 con 0x7ff3d815e3c0 2024-03-29T04:10:54.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.000+0000 7ff3d57fa640 1 -- 172.21.15.148:0/324352644 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3cc007500 con 0x7ff3d815e3c0 2024-03-29T04:10:54.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.000+0000 7ff3d57fa640 1 -- 172.21.15.148:0/324352644 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc004a20 con 0x7ff3d815e3c0 2024-03-29T04:10:54.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.000+0000 7ff3df063640 1 -- 172.21.15.148:0/324352644 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d815ee00 con 0x7ff3d815e3c0 2024-03-29T04:10:54.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.000+0000 7ff3df063640 1 -- 172.21.15.148:0/324352644 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d815f3c0 con 0x7ff3d815e3c0 2024-03-29T04:10:54.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.001+0000 7ff3df063640 1 -- 172.21.15.148:0/324352644 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3d8156030 con 0x7ff3d815e3c0 2024-03-29T04:10:54.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.001+0000 7ff3d57fa640 1 -- 172.21.15.148:0/324352644 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff3cc01b070 con 0x7ff3d815e3c0 2024-03-29T04:10:54.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.002+0000 7ff3d57fa640 1 --2- 172.21.15.148:0/324352644 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff3c406c560 0x7ff3c406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:54.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.002+0000 7ff3d57fa640 1 -- 172.21.15.148:0/324352644 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff3cc08add0 con 0x7ff3d815e3c0 2024-03-29T04:10:54.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.002+0000 7ff3dcdff640 1 --2- 172.21.15.148:0/324352644 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff3c406c560 0x7ff3c406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:54.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.002+0000 7ff3dcdff640 1 --2- 172.21.15.148:0/324352644 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff3c406c560 0x7ff3c406ea20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800a950 tx=0x7ff3c8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:54.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.006+0000 7ff3d57fa640 1 -- 172.21.15.148:0/324352644 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff3d8156030 con 0x7ff3d815e3c0 2024-03-29T04:10:55.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.227+0000 7ff3df063640 1 -- 172.21.15.148:0/324352644 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff3d8068850 con 0x7ff3d815e3c0 2024-03-29T04:10:55.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.227+0000 7ff3d57fa640 1 -- 172.21.15.148:0/324352644 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7ff3cc058ff0 con 0x7ff3d815e3c0 2024-03-29T04:10:55.206 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:10:55.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.229+0000 7ff3df063640 1 -- 172.21.15.148:0/324352644 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff3c406c560 msgr2=0x7ff3c406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:55.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.229+0000 7ff3df063640 1 --2- 172.21.15.148:0/324352644 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff3c406c560 0x7ff3c406ea20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800a950 tx=0x7ff3c8008040 comp rx=0 tx=0).stop 2024-03-29T04:10:55.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.229+0000 7ff3df063640 1 -- 172.21.15.148:0/324352644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3d815e3c0 msgr2=0x7ff3d81556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:55.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.229+0000 7ff3df063640 1 --2- 172.21.15.148:0/324352644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3d815e3c0 0x7ff3d81556e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc00c4d0 tx=0x7ff3cc00c990 comp rx=0 tx=0).stop 2024-03-29T04:10:55.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.229+0000 7ff3df063640 1 -- 172.21.15.148:0/324352644 shutdown_connections 2024-03-29T04:10:55.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.229+0000 7ff3df063640 1 --2- 172.21.15.148:0/324352644 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff3c406c560 0x7ff3c406ea20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:55.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.229+0000 7ff3df063640 1 --2- 172.21.15.148:0/324352644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3d8155c20 0x7ff3d81f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:55.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.229+0000 7ff3df063640 1 --2- 172.21.15.148:0/324352644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3d815e3c0 0x7ff3d81556e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:55.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.229+0000 7ff3df063640 1 --2- 172.21.15.148:0/324352644 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3d80b3430 0x7ff3d815fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:55.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.229+0000 7ff3df063640 1 -- 172.21.15.148:0/324352644 >> 172.21.15.148:0/324352644 conn(0x7ff3d80b6430 msgr2=0x7ff3d80d0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:55.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.229+0000 7ff3df063640 1 -- 172.21.15.148:0/324352644 shutdown_connections 2024-03-29T04:10:55.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.229+0000 7ff3df063640 1 -- 172.21.15.148:0/324352644 wait complete. 2024-03-29T04:10:55.220 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:10:55.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.345+0000 7f29ddf5f640 1 Processor -- start 2024-03-29T04:10:55.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.345+0000 7f29ddf5f640 1 -- start start 2024-03-29T04:10:55.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.345+0000 7f29ddf5f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29d80b2f00 0x7f29d8162680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:55.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.345+0000 7f29ddf5f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f29d80b3430 0x7f29d806b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:55.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.345+0000 7f29ddf5f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29d80cf330 con 0x7f29d815e3c0 2024-03-29T04:10:55.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.345+0000 7f29ddf5f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f29d80cf960 con 0x7f29d80b3430 2024-03-29T04:10:55.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.345+0000 7f29ddf5f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f29d8162cf0 con 0x7f29d80b2f00 2024-03-29T04:10:55.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.346+0000 7f29d7fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f29d80b3430 0x7f29d806b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:55.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.346+0000 7f29d77fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f29d815e3c0 0x7f29d8161f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50428/0 (socket says 172.21.15.148:50428) 2024-03-29T04:10:55.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.346+0000 7f29d7fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f29d80b3430 0x7f29d806b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53562/0 (socket says 172.21.15.148:53562) 2024-03-29T04:10:55.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.346+0000 7f29d77fe640 1 -- 172.21.15.148:0/954060313 learned_addr learned my addr 172.21.15.148:0/954060313 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:55.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.346+0000 7f29d6ffd640 1 --2- 172.21.15.148:0/954060313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29d80b2f00 0x7f29d8162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:55.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.346+0000 7f29d7fff640 1 -- 172.21.15.148:0/954060313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29d80b2f00 msgr2=0x7f29d8162680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:55.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.346+0000 7f29d7fff640 1 --2- 172.21.15.148:0/954060313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29d80b2f00 0x7f29d8162680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:55.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.346+0000 7f29d7fff640 1 -- 172.21.15.148:0/954060313 >> v1:172.21.15.148:6789/0 conn(0x7f29d815e3c0 legacy=0x7f29d8161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:55.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.346+0000 7f29d7fff640 1 -- 172.21.15.148:0/954060313 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d80cf5f0 con 0x7f29d80b3430 2024-03-29T04:10:55.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.346+0000 7f29d6ffd640 1 --2- 172.21.15.148:0/954060313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29d80b2f00 0x7f29d8162680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:55.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.347+0000 7f29d7fff640 1 --2- 172.21.15.148:0/954060313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29d80b3430 0x7f29d806b7f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009d90 tx=0x7f29d002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd896446182e8b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:55.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.347+0000 7f29d67fc640 1 -- 172.21.15.148:0/954060313 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d0039070 con 0x7f29d80b3430 2024-03-29T04:10:55.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.347+0000 7f29d67fc640 1 -- 172.21.15.148:0/954060313 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29d002cbc0 con 0x7f29d80b3430 2024-03-29T04:10:55.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.347+0000 7f29d67fc640 1 -- 172.21.15.148:0/954060313 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d0034440 con 0x7f29d80b3430 2024-03-29T04:10:55.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.347+0000 7f29ddf5f640 1 -- 172.21.15.148:0/954060313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29d80b3430 msgr2=0x7f29d806b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:55.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.347+0000 7f29ddf5f640 1 --2- 172.21.15.148:0/954060313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29d80b3430 0x7f29d806b7f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009d90 tx=0x7f29d002be10 comp rx=0 tx=0).stop 2024-03-29T04:10:55.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.347+0000 7f29ddf5f640 1 -- 172.21.15.148:0/954060313 shutdown_connections 2024-03-29T04:10:55.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.347+0000 7f29ddf5f640 1 --2- 172.21.15.148:0/954060313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29d80b3430 0x7f29d806b7f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:55.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.347+0000 7f29ddf5f640 1 --2- 172.21.15.148:0/954060313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29d80b2f00 0x7f29d8162680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:55.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.347+0000 7f29ddf5f640 1 -- 172.21.15.148:0/954060313 >> 172.21.15.148:0/954060313 conn(0x7f29d80b6430 msgr2=0x7f29d80d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:55.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.347+0000 7f29ddf5f640 1 -- 172.21.15.148:0/954060313 shutdown_connections 2024-03-29T04:10:55.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.347+0000 7f29ddf5f640 1 -- 172.21.15.148:0/954060313 wait complete. 2024-03-29T04:10:55.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.348+0000 7f29ddf5f640 1 Processor -- start 2024-03-29T04:10:55.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.349+0000 7f29ddf5f640 1 -- start start 2024-03-29T04:10:55.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.349+0000 7f29ddf5f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29d80b3430 0x7f29d8161020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:55.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.349+0000 7f29ddf5f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29d815e3c0 0x7f29d814c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:55.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.349+0000 7f29ddf5f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29d8149ee0 0x7f29d814a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:55.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.349+0000 7f29ddf5f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f29d80b3b20 con 0x7f29d80b3430 2024-03-29T04:10:55.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.349+0000 7f29ddf5f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f29d80b39a0 con 0x7f29d8149ee0 2024-03-29T04:10:55.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.349+0000 7f29ddf5f640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f29d806bd30 con 0x7f29d815e3c0 2024-03-29T04:10:55.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.349+0000 7f29d77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29d80b3430 0x7f29d8161020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:55.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.350+0000 7f29d77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29d80b3430 0x7f29d8161020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53566/0 (socket says 172.21.15.148:53566) 2024-03-29T04:10:55.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.350+0000 7f29d77fe640 1 -- 172.21.15.148:0/3000918033 learned_addr learned my addr 172.21.15.148:0/3000918033 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:55.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.350+0000 7f29d6ffd640 1 --2- 172.21.15.148:0/3000918033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29d815e3c0 0x7f29d814c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:55.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.350+0000 7f29d7fff640 1 --2- 172.21.15.148:0/3000918033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29d8149ee0 0x7f29d814a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:55.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.350+0000 7f29d77fe640 1 -- 172.21.15.148:0/3000918033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29d815e3c0 msgr2=0x7f29d814c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:55.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.350+0000 7f29d77fe640 1 --2- 172.21.15.148:0/3000918033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29d815e3c0 0x7f29d814c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:55.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.350+0000 7f29d77fe640 1 -- 172.21.15.148:0/3000918033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29d8149ee0 msgr2=0x7f29d814a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:55.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.350+0000 7f29d77fe640 1 --2- 172.21.15.148:0/3000918033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29d8149ee0 0x7f29d814a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:55.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.350+0000 7f29d77fe640 1 -- 172.21.15.148:0/3000918033 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d815cbf0 con 0x7f29d80b3430 2024-03-29T04:10:55.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.350+0000 7f29d77fe640 1 --2- 172.21.15.148:0/3000918033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29d80b3430 0x7f29d8161020 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f29c800c3c0 tx=0x7f29c800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:55.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.350+0000 7f29bffff640 1 -- 172.21.15.148:0/3000918033 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c8017070 con 0x7f29d80b3430 2024-03-29T04:10:55.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.350+0000 7f29bffff640 1 -- 172.21.15.148:0/3000918033 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29c8005ce0 con 0x7f29d80b3430 2024-03-29T04:10:55.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.350+0000 7f29ddf5f640 1 -- 172.21.15.148:0/3000918033 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d8160540 con 0x7f29d80b3430 2024-03-29T04:10:55.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.350+0000 7f29bffff640 1 -- 172.21.15.148:0/3000918033 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c80072f0 con 0x7f29d80b3430 2024-03-29T04:10:55.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.350+0000 7f29ddf5f640 1 -- 172.21.15.148:0/3000918033 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d8160b00 con 0x7f29d80b3430 2024-03-29T04:10:55.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.351+0000 7f29ddf5f640 1 -- 172.21.15.148:0/3000918033 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29d8068850 con 0x7f29d80b3430 2024-03-29T04:10:55.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.352+0000 7f29bffff640 1 -- 172.21.15.148:0/3000918033 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f29c801a070 con 0x7f29d80b3430 2024-03-29T04:10:55.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.352+0000 7f29bffff640 1 --2- 172.21.15.148:0/3000918033 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f29c406c560 0x7f29c406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:55.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.352+0000 7f29bffff640 1 -- 172.21.15.148:0/3000918033 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f29c808adc0 con 0x7f29d80b3430 2024-03-29T04:10:55.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.352+0000 7f29d6ffd640 1 --2- 172.21.15.148:0/3000918033 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f29c406c560 0x7f29c406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:55.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.353+0000 7f29d6ffd640 1 --2- 172.21.15.148:0/3000918033 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f29c406c560 0x7f29c406ea20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f29d8159f20 tx=0x7f29cc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:55.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.357+0000 7f29bffff640 1 -- 172.21.15.148:0/3000918033 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f29c8059070 con 0x7f29d80b3430 2024-03-29T04:10:55.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.537+0000 7f29ddf5f640 1 -- 172.21.15.148:0/3000918033 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f29d815d340 con 0x7f29d80b3430 2024-03-29T04:10:55.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.538+0000 7f29bffff640 1 -- 172.21.15.148:0/3000918033 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f29c8058970 con 0x7f29d80b3430 2024-03-29T04:10:55.517 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-29T04:10:55.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.540+0000 7f29ddf5f640 1 -- 172.21.15.148:0/3000918033 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f29c406c560 msgr2=0x7f29c406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:55.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.540+0000 7f29ddf5f640 1 --2- 172.21.15.148:0/3000918033 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f29c406c560 0x7f29c406ea20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f29d8159f20 tx=0x7f29cc008040 comp rx=0 tx=0).stop 2024-03-29T04:10:55.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.540+0000 7f29ddf5f640 1 -- 172.21.15.148:0/3000918033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29d80b3430 msgr2=0x7f29d8161020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:55.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.540+0000 7f29ddf5f640 1 --2- 172.21.15.148:0/3000918033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29d80b3430 0x7f29d8161020 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f29c800c3c0 tx=0x7f29c800c880 comp rx=0 tx=0).stop 2024-03-29T04:10:55.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.540+0000 7f29ddf5f640 1 -- 172.21.15.148:0/3000918033 shutdown_connections 2024-03-29T04:10:55.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.540+0000 7f29ddf5f640 1 --2- 172.21.15.148:0/3000918033 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f29c406c560 0x7f29c406ea20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:55.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.540+0000 7f29ddf5f640 1 --2- 172.21.15.148:0/3000918033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29d8149ee0 0x7f29d814a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:55.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.540+0000 7f29ddf5f640 1 --2- 172.21.15.148:0/3000918033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29d815e3c0 0x7f29d814c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:55.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.540+0000 7f29ddf5f640 1 --2- 172.21.15.148:0/3000918033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29d80b3430 0x7f29d8161020 secure :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f29c800c3c0 tx=0x7f29c800c880 comp rx=0 tx=0).stop 2024-03-29T04:10:55.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.540+0000 7f29ddf5f640 1 -- 172.21.15.148:0/3000918033 >> 172.21.15.148:0/3000918033 conn(0x7f29d80b6430 msgr2=0x7f29d80d0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:55.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.540+0000 7f29ddf5f640 1 -- 172.21.15.148:0/3000918033 shutdown_connections 2024-03-29T04:10:55.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.540+0000 7f29ddf5f640 1 -- 172.21.15.148:0/3000918033 wait complete. 2024-03-29T04:10:55.531 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:10:55.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.657+0000 7fc78fc22640 1 Processor -- start 2024-03-29T04:10:55.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.657+0000 7fc78fc22640 1 -- start start 2024-03-29T04:10:55.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.659+0000 7fc78fc22640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc788070400 0x7fc7880ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:55.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.659+0000 7fc78fc22640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc78806efe0 0x7fc788160230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:55.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.659+0000 7fc78fc22640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc78815cd40 con 0x7fc7880d1cd0 2024-03-29T04:10:55.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.659+0000 7fc78fc22640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fc78815d020 con 0x7fc788070400 2024-03-29T04:10:55.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.659+0000 7fc78fc22640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc78815d300 con 0x7fc78806efe0 2024-03-29T04:10:55.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.659+0000 7fc78d9be640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc788070400 0x7fc7880ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:55.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.659+0000 7fc78d1bd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc7880d1cd0 0x7fc78806e8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39912/0 (socket says 172.21.15.148:39912) 2024-03-29T04:10:55.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.659+0000 7fc78d1bd640 1 -- 172.21.15.148:0/940593713 learned_addr learned my addr 172.21.15.148:0/940593713 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:55.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.659+0000 7fc78e1bf640 1 --2- 172.21.15.148:0/940593713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc78806efe0 0x7fc788160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:55.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.659+0000 7fc78d9be640 1 -- 172.21.15.148:0/940593713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc78806efe0 msgr2=0x7fc788160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:55.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.659+0000 7fc78d9be640 1 --2- 172.21.15.148:0/940593713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc78806efe0 0x7fc788160230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:55.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.659+0000 7fc78d9be640 1 -- 172.21.15.148:0/940593713 >> v1:172.21.15.148:6789/0 conn(0x7fc7880d1cd0 legacy=0x7fc78806e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:55.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.659+0000 7fc78d9be640 1 -- 172.21.15.148:0/940593713 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78815d5e0 con 0x7fc788070400 2024-03-29T04:10:55.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.659+0000 7fc78c9bc640 1 -- 172.21.15.148:0/940593713 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3272035869 0 0) 0x7fc78815cd40 con 0x7fc7880d1cd0 2024-03-29T04:10:55.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.660+0000 7fc78d9be640 1 --2- 172.21.15.148:0/940593713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc788070400 0x7fc7880ce340 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc7780097a0 tx=0x7fc77802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb7d5583f500a560 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:55.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.660+0000 7fc78c9bc640 1 -- 172.21.15.148:0/940593713 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778035020 con 0x7fc788070400 2024-03-29T04:10:55.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.660+0000 7fc78c9bc640 1 -- 172.21.15.148:0/940593713 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc778031830 con 0x7fc788070400 2024-03-29T04:10:55.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.660+0000 7fc78c9bc640 1 -- 172.21.15.148:0/940593713 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778031c00 con 0x7fc788070400 2024-03-29T04:10:55.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.661+0000 7fc78fc22640 1 -- 172.21.15.148:0/940593713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc788070400 msgr2=0x7fc7880ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:55.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.661+0000 7fc78fc22640 1 --2- 172.21.15.148:0/940593713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc788070400 0x7fc7880ce340 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc7780097a0 tx=0x7fc77802b8d0 comp rx=0 tx=0).stop 2024-03-29T04:10:55.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.661+0000 7fc78fc22640 1 -- 172.21.15.148:0/940593713 shutdown_connections 2024-03-29T04:10:55.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.661+0000 7fc78fc22640 1 --2- 172.21.15.148:0/940593713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc78806efe0 0x7fc788160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:55.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.661+0000 7fc78fc22640 1 --2- 172.21.15.148:0/940593713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc788070400 0x7fc7880ce340 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:55.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.661+0000 7fc78fc22640 1 -- 172.21.15.148:0/940593713 >> 172.21.15.148:0/940593713 conn(0x7fc7880b3ce0 msgr2=0x7fc7880b40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:55.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.661+0000 7fc78fc22640 1 -- 172.21.15.148:0/940593713 shutdown_connections 2024-03-29T04:10:55.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.661+0000 7fc78fc22640 1 -- 172.21.15.148:0/940593713 wait complete. 2024-03-29T04:10:55.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.662+0000 7fc78fc22640 1 Processor -- start 2024-03-29T04:10:55.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.662+0000 7fc78fc22640 1 -- start start 2024-03-29T04:10:55.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.663+0000 7fc78fc22640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc78806efe0 0x7fc788151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:55.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.663+0000 7fc78fc22640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc788070400 0x7fc7880b3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:55.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.663+0000 7fc78fc22640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7880d1cd0 0x7fc7880b3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:55.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.663+0000 7fc78fc22640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc788070960 con 0x7fc7880d1cd0 2024-03-29T04:10:55.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.663+0000 7fc78fc22640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc7880707e0 con 0x7fc78806efe0 2024-03-29T04:10:55.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.663+0000 7fc78fc22640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc78815c860 con 0x7fc788070400 2024-03-29T04:10:55.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.663+0000 7fc78e1bf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7880d1cd0 0x7fc7880b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:55.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.663+0000 7fc78e1bf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7880d1cd0 0x7fc7880b3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55930/0 (socket says 172.21.15.148:55930) 2024-03-29T04:10:55.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.663+0000 7fc78e1bf640 1 -- 172.21.15.148:0/1252453027 learned_addr learned my addr 172.21.15.148:0/1252453027 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:55.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.663+0000 7fc78d9be640 1 --2- 172.21.15.148:0/1252453027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc78806efe0 0x7fc788151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:55.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.663+0000 7fc78d1bd640 1 --2- 172.21.15.148:0/1252453027 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc788070400 0x7fc7880b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:55.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.663+0000 7fc78e1bf640 1 -- 172.21.15.148:0/1252453027 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc788070400 msgr2=0x7fc7880b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:55.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.663+0000 7fc78e1bf640 1 --2- 172.21.15.148:0/1252453027 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc788070400 0x7fc7880b3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:55.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.663+0000 7fc78e1bf640 1 -- 172.21.15.148:0/1252453027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc78806efe0 msgr2=0x7fc788151840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:55.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.663+0000 7fc78e1bf640 1 --2- 172.21.15.148:0/1252453027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc78806efe0 0x7fc788151840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:55.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.663+0000 7fc78e1bf640 1 -- 172.21.15.148:0/1252453027 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc788153fa0 con 0x7fc7880d1cd0 2024-03-29T04:10:55.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.663+0000 7fc78e1bf640 1 --2- 172.21.15.148:0/1252453027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7880d1cd0 0x7fc7880b3540 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc780007e60 tx=0x7fc78000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:55.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.663+0000 7fc77e7fc640 1 -- 172.21.15.148:0/1252453027 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc780014620 con 0x7fc7880d1cd0 2024-03-29T04:10:55.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.663+0000 7fc77e7fc640 1 -- 172.21.15.148:0/1252453027 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc780019070 con 0x7fc7880d1cd0 2024-03-29T04:10:55.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.663+0000 7fc78fc22640 1 -- 172.21.15.148:0/1252453027 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc788154280 con 0x7fc7880d1cd0 2024-03-29T04:10:55.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.663+0000 7fc77e7fc640 1 -- 172.21.15.148:0/1252453027 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc780014a00 con 0x7fc7880d1cd0 2024-03-29T04:10:55.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.663+0000 7fc78fc22640 1 -- 172.21.15.148:0/1252453027 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc788155cb0 con 0x7fc7880d1cd0 2024-03-29T04:10:55.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.664+0000 7fc78fc22640 1 -- 172.21.15.148:0/1252453027 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc788068850 con 0x7fc7880d1cd0 2024-03-29T04:10:55.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.665+0000 7fc77e7fc640 1 -- 172.21.15.148:0/1252453027 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc780014ba0 con 0x7fc7880d1cd0 2024-03-29T04:10:55.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.665+0000 7fc77e7fc640 1 --2- 172.21.15.148:0/1252453027 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc76c06c560 0x7fc76c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:55.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.665+0000 7fc77e7fc640 1 -- 172.21.15.148:0/1252453027 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc78008c1e0 con 0x7fc7880d1cd0 2024-03-29T04:10:55.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.665+0000 7fc78d9be640 1 --2- 172.21.15.148:0/1252453027 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc76c06c560 0x7fc76c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:55.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.666+0000 7fc78d9be640 1 --2- 172.21.15.148:0/1252453027 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc76c06c560 0x7fc76c06ea20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc778036000 tx=0x7fc77803d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:55.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.670+0000 7fc77e7fc640 1 -- 172.21.15.148:0/1252453027 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc780059e40 con 0x7fc7880d1cd0 2024-03-29T04:10:55.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.851+0000 7fc78fc22640 1 -- 172.21.15.148:0/1252453027 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc7880b1670 con 0x7fc7880d1cd0 2024-03-29T04:10:55.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.851+0000 7fc77e7fc640 1 -- 172.21.15.148:0/1252453027 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fc78005e140 con 0x7fc7880d1cd0 2024-03-29T04:10:55.830 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:10:55.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.853+0000 7fc78fc22640 1 -- 172.21.15.148:0/1252453027 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc76c06c560 msgr2=0x7fc76c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:55.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.853+0000 7fc78fc22640 1 --2- 172.21.15.148:0/1252453027 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc76c06c560 0x7fc76c06ea20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc778036000 tx=0x7fc77803d040 comp rx=0 tx=0).stop 2024-03-29T04:10:55.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.853+0000 7fc78fc22640 1 -- 172.21.15.148:0/1252453027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7880d1cd0 msgr2=0x7fc7880b3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:55.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.853+0000 7fc78fc22640 1 --2- 172.21.15.148:0/1252453027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7880d1cd0 0x7fc7880b3540 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc780007e60 tx=0x7fc78000e4b0 comp rx=0 tx=0).stop 2024-03-29T04:10:55.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.853+0000 7fc78fc22640 1 -- 172.21.15.148:0/1252453027 shutdown_connections 2024-03-29T04:10:55.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.853+0000 7fc78fc22640 1 --2- 172.21.15.148:0/1252453027 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc76c06c560 0x7fc76c06ea20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:55.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.853+0000 7fc78fc22640 1 --2- 172.21.15.148:0/1252453027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7880d1cd0 0x7fc7880b3540 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:55.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.853+0000 7fc78fc22640 1 --2- 172.21.15.148:0/1252453027 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc788070400 0x7fc7880b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:55.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.853+0000 7fc78fc22640 1 --2- 172.21.15.148:0/1252453027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc78806efe0 0x7fc788151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:55.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.853+0000 7fc78fc22640 1 -- 172.21.15.148:0/1252453027 >> 172.21.15.148:0/1252453027 conn(0x7fc7880b3ce0 msgr2=0x7fc78806f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:55.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.853+0000 7fc78fc22640 1 -- 172.21.15.148:0/1252453027 shutdown_connections 2024-03-29T04:10:55.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.853+0000 7fc78fc22640 1 -- 172.21.15.148:0/1252453027 wait complete. 2024-03-29T04:10:55.846 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:10:55.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.970+0000 7efd60b11640 1 Processor -- start 2024-03-29T04:10:55.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.970+0000 7efd60b11640 1 -- start start 2024-03-29T04:10:55.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.971+0000 7efd60b11640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd5c0ba030 0x7efd5c0d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:55.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.971+0000 7efd60b11640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efd5c0d10d0 0x7efd5c0d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:55.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.971+0000 7efd60b11640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd5c15e7c0 con 0x7efd5c15e3e0 2024-03-29T04:10:55.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.971+0000 7efd60b11640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7efd5c0d3840 con 0x7efd5c0d10d0 2024-03-29T04:10:55.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.971+0000 7efd60b11640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efd5c0d2c30 con 0x7efd5c0ba030 2024-03-29T04:10:55.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.971+0000 7efd5ad9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efd5c0d10d0 0x7efd5c0d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:55.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.971+0000 7efd5ad9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efd5c0d10d0 0x7efd5c0d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55940/0 (socket says 172.21.15.148:55940) 2024-03-29T04:10:55.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.971+0000 7efd5ad9d640 1 -- 172.21.15.148:0/1780955098 learned_addr learned my addr 172.21.15.148:0/1780955098 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:55.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.972+0000 7efd53fff640 1 --2- 172.21.15.148:0/1780955098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd5c0ba030 0x7efd5c0d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:55.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.972+0000 7efd5ad9d640 1 -- 172.21.15.148:0/1780955098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd5c0ba030 msgr2=0x7efd5c0d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:55.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.972+0000 7efd5ad9d640 1 --2- 172.21.15.148:0/1780955098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd5c0ba030 0x7efd5c0d3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:55.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.972+0000 7efd5ad9d640 1 -- 172.21.15.148:0/1780955098 >> v1:172.21.15.157:6789/0 conn(0x7efd5c15e3e0 legacy=0x7efd5c0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:55.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.972+0000 7efd5ad9d640 1 -- 172.21.15.148:0/1780955098 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5c0d2f10 con 0x7efd5c0d10d0 2024-03-29T04:10:55.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.972+0000 7efd5ad9d640 1 --2- 172.21.15.148:0/1780955098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd5c0d10d0 0x7efd5c0d1580 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efd54009e50 tx=0x7efd5402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a23f1c8531f950d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:55.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.972+0000 7efd59d9b640 1 -- 172.21.15.148:0/1780955098 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5402cb00 con 0x7efd5c0d10d0 2024-03-29T04:10:55.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.973+0000 7efd59d9b640 1 -- 172.21.15.148:0/1780955098 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd5402cca0 con 0x7efd5c0d10d0 2024-03-29T04:10:55.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.973+0000 7efd59d9b640 1 -- 172.21.15.148:0/1780955098 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54034610 con 0x7efd5c0d10d0 2024-03-29T04:10:55.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.973+0000 7efd60b11640 1 -- 172.21.15.148:0/1780955098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd5c0d10d0 msgr2=0x7efd5c0d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:55.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.973+0000 7efd60b11640 1 --2- 172.21.15.148:0/1780955098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd5c0d10d0 0x7efd5c0d1580 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efd54009e50 tx=0x7efd5402bf60 comp rx=0 tx=0).stop 2024-03-29T04:10:55.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.973+0000 7efd60b11640 1 -- 172.21.15.148:0/1780955098 shutdown_connections 2024-03-29T04:10:55.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.973+0000 7efd60b11640 1 --2- 172.21.15.148:0/1780955098 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd5c0ba030 0x7efd5c0d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:55.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.973+0000 7efd60b11640 1 --2- 172.21.15.148:0/1780955098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd5c0d10d0 0x7efd5c0d1580 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:55.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.973+0000 7efd60b11640 1 -- 172.21.15.148:0/1780955098 >> 172.21.15.148:0/1780955098 conn(0x7efd5c0b6210 msgr2=0x7efd5c0b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:55.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.973+0000 7efd60b11640 1 -- 172.21.15.148:0/1780955098 shutdown_connections 2024-03-29T04:10:55.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.973+0000 7efd60b11640 1 -- 172.21.15.148:0/1780955098 wait complete. 2024-03-29T04:10:55.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.975+0000 7efd60b11640 1 Processor -- start 2024-03-29T04:10:55.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.975+0000 7efd60b11640 1 -- start start 2024-03-29T04:10:55.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.976+0000 7efd60b11640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd5c0ba030 0x7efd5c155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:55.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.976+0000 7efd60b11640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd5c0d10d0 0x7efd5c1f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:55.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.976+0000 7efd60b11640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd5c15e3e0 0x7efd5c0ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:55.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.976+0000 7efd60b11640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c167f50 con 0x7efd5c0ba030 2024-03-29T04:10:55.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.976+0000 7efd60b11640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c0d34d0 con 0x7efd5c15e3e0 2024-03-29T04:10:55.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.976+0000 7efd60b11640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efd5c1680d0 con 0x7efd5c0d10d0 2024-03-29T04:10:55.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.976+0000 7efd5a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd5c0ba030 0x7efd5c155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:55.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.976+0000 7efd5a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd5c0ba030 0x7efd5c155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55944/0 (socket says 172.21.15.148:55944) 2024-03-29T04:10:55.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.976+0000 7efd5a59c640 1 -- 172.21.15.148:0/226656264 learned_addr learned my addr 172.21.15.148:0/226656264 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:55.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.977+0000 7efd5ad9d640 1 --2- 172.21.15.148:0/226656264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd5c15e3e0 0x7efd5c0ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:55.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.977+0000 7efd53fff640 1 --2- 172.21.15.148:0/226656264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd5c0d10d0 0x7efd5c1f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:55.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.977+0000 7efd5a59c640 1 -- 172.21.15.148:0/226656264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd5c0d10d0 msgr2=0x7efd5c1f7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:55.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.977+0000 7efd5a59c640 1 --2- 172.21.15.148:0/226656264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd5c0d10d0 0x7efd5c1f7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:55.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.977+0000 7efd5a59c640 1 -- 172.21.15.148:0/226656264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd5c15e3e0 msgr2=0x7efd5c0ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:55.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.977+0000 7efd5a59c640 1 --2- 172.21.15.148:0/226656264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd5c15e3e0 0x7efd5c0ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:55.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.977+0000 7efd5a59c640 1 -- 172.21.15.148:0/226656264 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5c159940 con 0x7efd5c0ba030 2024-03-29T04:10:55.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.977+0000 7efd5a59c640 1 --2- 172.21.15.148:0/226656264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd5c0ba030 0x7efd5c155930 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efd4c00b420 tx=0x7efd4c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:55.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.977+0000 7efd52ffd640 1 -- 172.21.15.148:0/226656264 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c011830 con 0x7efd5c0ba030 2024-03-29T04:10:55.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.977+0000 7efd52ffd640 1 -- 172.21.15.148:0/226656264 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd4c016070 con 0x7efd5c0ba030 2024-03-29T04:10:55.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.977+0000 7efd60b11640 1 -- 172.21.15.148:0/226656264 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd5c159c20 con 0x7efd5c0ba030 2024-03-29T04:10:55.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.977+0000 7efd52ffd640 1 -- 172.21.15.148:0/226656264 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c011c10 con 0x7efd5c0ba030 2024-03-29T04:10:55.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.977+0000 7efd60b11640 1 -- 172.21.15.148:0/226656264 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd5c15a1e0 con 0x7efd5c0ba030 2024-03-29T04:10:55.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.978+0000 7efd60b11640 1 -- 172.21.15.148:0/226656264 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd5c0687e0 con 0x7efd5c0ba030 2024-03-29T04:10:55.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.978+0000 7efd52ffd640 1 -- 172.21.15.148:0/226656264 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efd4c01b460 con 0x7efd5c0ba030 2024-03-29T04:10:55.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.979+0000 7efd52ffd640 1 --2- 172.21.15.148:0/226656264 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efd3c06c530 0x7efd3c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:55.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.979+0000 7efd52ffd640 1 -- 172.21.15.148:0/226656264 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7efd4c08b4c0 con 0x7efd5c0ba030 2024-03-29T04:10:55.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.979+0000 7efd53fff640 1 --2- 172.21.15.148:0/226656264 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efd3c06c530 0x7efd3c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:55.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.980+0000 7efd53fff640 1 --2- 172.21.15.148:0/226656264 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efd3c06c530 0x7efd3c06e9f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0ca000 tx=0x7efd44008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:55.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:55.983+0000 7efd52ffd640 1 -- 172.21.15.148:0/226656264 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efd4c020020 con 0x7efd5c0ba030 2024-03-29T04:10:56.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.201+0000 7efd60b11640 1 -- 172.21.15.148:0/226656264 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efd5c15d700 con 0x7efd5c0ba030 2024-03-29T04:10:56.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.201+0000 7efd52ffd640 1 -- 172.21.15.148:0/226656264 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7efd4c059090 con 0x7efd5c0ba030 2024-03-29T04:10:56.180 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-03-29T04:10:56.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.203+0000 7efd60b11640 1 -- 172.21.15.148:0/226656264 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efd3c06c530 msgr2=0x7efd3c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:56.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.203+0000 7efd60b11640 1 --2- 172.21.15.148:0/226656264 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efd3c06c530 0x7efd3c06e9f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0ca000 tx=0x7efd44008040 comp rx=0 tx=0).stop 2024-03-29T04:10:56.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.203+0000 7efd60b11640 1 -- 172.21.15.148:0/226656264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd5c0ba030 msgr2=0x7efd5c155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:56.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.203+0000 7efd60b11640 1 --2- 172.21.15.148:0/226656264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd5c0ba030 0x7efd5c155930 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efd4c00b420 tx=0x7efd4c00b8e0 comp rx=0 tx=0).stop 2024-03-29T04:10:56.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.203+0000 7efd60b11640 1 -- 172.21.15.148:0/226656264 shutdown_connections 2024-03-29T04:10:56.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.203+0000 7efd60b11640 1 --2- 172.21.15.148:0/226656264 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efd3c06c530 0x7efd3c06e9f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:56.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.203+0000 7efd60b11640 1 --2- 172.21.15.148:0/226656264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd5c15e3e0 0x7efd5c0ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:56.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.203+0000 7efd60b11640 1 --2- 172.21.15.148:0/226656264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd5c0d10d0 0x7efd5c1f7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:56.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.203+0000 7efd60b11640 1 --2- 172.21.15.148:0/226656264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd5c0ba030 0x7efd5c155930 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:56.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.203+0000 7efd60b11640 1 -- 172.21.15.148:0/226656264 >> 172.21.15.148:0/226656264 conn(0x7efd5c0b6210 msgr2=0x7efd5c0b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:56.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.203+0000 7efd60b11640 1 -- 172.21.15.148:0/226656264 shutdown_connections 2024-03-29T04:10:56.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.203+0000 7efd60b11640 1 -- 172.21.15.148:0/226656264 wait complete. 2024-03-29T04:10:56.193 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:10:56.193 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:10:56.209 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:56.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.371+0000 7f6065d59640 1 Processor -- start 2024-03-29T04:10:56.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.371+0000 7f6065d59640 1 -- start start 2024-03-29T04:10:56.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.372+0000 7f6065d59640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6060147c80 0x7f606006ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:56.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.372+0000 7f6065d59640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60600d0350 0x7f606015fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:56.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.372+0000 7f6065d59640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f606006df40 con 0x7f606015eb90 2024-03-29T04:10:56.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.372+0000 7f6065d59640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f606006d3c0 con 0x7f6060147c80 2024-03-29T04:10:56.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.372+0000 7f6065d59640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f606006d6a0 con 0x7f60600d0350 2024-03-29T04:10:56.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.372+0000 7f605f7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6060147c80 0x7f606006ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:56.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.372+0000 7f605ffff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60600d0350 0x7f606015fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:56.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.372+0000 7f6056ffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f606015eb90 0x7f60600cfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.148:49330/0 (socket says 172.21.15.148:49330) 2024-03-29T04:10:56.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.372+0000 7f6056ffd640 1 -- 172.21.15.148:0/660495781 learned_addr learned my addr 172.21.15.148:0/660495781 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:56.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.373+0000 7f605f7fe640 1 -- 172.21.15.148:0/660495781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60600d0350 msgr2=0x7f606015fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:56.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.373+0000 7f605f7fe640 1 --2- 172.21.15.148:0/660495781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60600d0350 0x7f606015fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:56.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.373+0000 7f605f7fe640 1 -- 172.21.15.148:0/660495781 >> v1:172.21.15.157:6789/0 conn(0x7f606015eb90 legacy=0x7f60600cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:56.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.373+0000 7f605f7fe640 1 -- 172.21.15.148:0/660495781 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606006d980 con 0x7f6060147c80 2024-03-29T04:10:56.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.373+0000 7f605ffff640 1 --2- 172.21.15.148:0/660495781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60600d0350 0x7f606015fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:56.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.373+0000 7f605f7fe640 1 --2- 172.21.15.148:0/660495781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6060147c80 0x7f606006ca50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f60500097a0 tx=0x7f605002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6d31fe9e25ec05d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:56.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.374+0000 7f605effd640 1 -- 172.21.15.148:0/660495781 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6050035020 con 0x7f6060147c80 2024-03-29T04:10:56.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.374+0000 7f605effd640 1 -- 172.21.15.148:0/660495781 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6050031830 con 0x7f6060147c80 2024-03-29T04:10:56.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.374+0000 7f605effd640 1 -- 172.21.15.148:0/660495781 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6050031c00 con 0x7f6060147c80 2024-03-29T04:10:56.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.374+0000 7f6065d59640 1 -- 172.21.15.148:0/660495781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6060147c80 msgr2=0x7f606006ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:56.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.374+0000 7f6065d59640 1 --2- 172.21.15.148:0/660495781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6060147c80 0x7f606006ca50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f60500097a0 tx=0x7f605002b8d0 comp rx=0 tx=0).stop 2024-03-29T04:10:56.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.374+0000 7f6065d59640 1 -- 172.21.15.148:0/660495781 shutdown_connections 2024-03-29T04:10:56.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.374+0000 7f6065d59640 1 --2- 172.21.15.148:0/660495781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60600d0350 0x7f606015fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:56.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.374+0000 7f6065d59640 1 --2- 172.21.15.148:0/660495781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6060147c80 0x7f606006ca50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:56.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.374+0000 7f6065d59640 1 -- 172.21.15.148:0/660495781 >> 172.21.15.148:0/660495781 conn(0x7f60600b3e30 msgr2=0x7f60600b4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:56.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.374+0000 7f6065d59640 1 -- 172.21.15.148:0/660495781 shutdown_connections 2024-03-29T04:10:56.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.374+0000 7f6065d59640 1 -- 172.21.15.148:0/660495781 wait complete. 2024-03-29T04:10:56.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.375+0000 7f6065d59640 1 Processor -- start 2024-03-29T04:10:56.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.375+0000 7f6065d59640 1 -- start start 2024-03-29T04:10:56.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.376+0000 7f6065d59640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60600d0350 0x7f60601f40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:56.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.376+0000 7f6065d59640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6060147c80 0x7f60600cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:56.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.376+0000 7f6065d59640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f606015eb90 0x7f60600b6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:56.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.376+0000 7f6065d59640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f606006cf90 con 0x7f606015eb90 2024-03-29T04:10:56.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.376+0000 7f6065d59640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6060148060 con 0x7f60600d0350 2024-03-29T04:10:56.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.376+0000 7f6065d59640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f606006d110 con 0x7f6060147c80 2024-03-29T04:10:56.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.376+0000 7f605ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f606015eb90 0x7f60600b6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:56.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.376+0000 7f605f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60600d0350 0x7f60601f40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:56.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.376+0000 7f6056ffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6060147c80 0x7f60600cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:56.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.376+0000 7f605f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60600d0350 0x7f60601f40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:56506/0 (socket says 172.21.15.148:56506) 2024-03-29T04:10:56.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.376+0000 7f605f7fe640 1 -- 172.21.15.148:0/1787759949 learned_addr learned my addr 172.21.15.148:0/1787759949 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:56.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.376+0000 7f605ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f606015eb90 0x7f60600b6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55960/0 (socket says 172.21.15.148:55960) 2024-03-29T04:10:56.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.377+0000 7f605f7fe640 1 -- 172.21.15.148:0/1787759949 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6060147c80 msgr2=0x7f60600cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:56.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.377+0000 7f605f7fe640 1 --2- 172.21.15.148:0/1787759949 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6060147c80 0x7f60600cc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:56.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.377+0000 7f605f7fe640 1 -- 172.21.15.148:0/1787759949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f606015eb90 msgr2=0x7f60600b6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:56.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.377+0000 7f605f7fe640 1 --2- 172.21.15.148:0/1787759949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f606015eb90 0x7f60600b6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:56.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.377+0000 7f605f7fe640 1 -- 172.21.15.148:0/1787759949 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60601f4720 con 0x7f60600d0350 2024-03-29T04:10:56.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.377+0000 7f605ffff640 1 --2- 172.21.15.148:0/1787759949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f606015eb90 0x7f60600b6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:56.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.377+0000 7f605f7fe640 1 --2- 172.21.15.148:0/1787759949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60600d0350 0x7f60601f40d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f605002bee0 tx=0x7f60500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:56.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.377+0000 7f605cff9640 1 -- 172.21.15.148:0/1787759949 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6050035040 con 0x7f60600d0350 2024-03-29T04:10:56.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.377+0000 7f605cff9640 1 -- 172.21.15.148:0/1787759949 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f605003b860 con 0x7f60600d0350 2024-03-29T04:10:56.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.377+0000 7f605cff9640 1 -- 172.21.15.148:0/1787759949 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605003bc50 con 0x7f60600d0350 2024-03-29T04:10:56.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.377+0000 7f6065d59640 1 -- 172.21.15.148:0/1787759949 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60601f4a00 con 0x7f60600d0350 2024-03-29T04:10:56.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.377+0000 7f6065d59640 1 -- 172.21.15.148:0/1787759949 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60600cc6f0 con 0x7f60600d0350 2024-03-29T04:10:56.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.378+0000 7f6065d59640 1 -- 172.21.15.148:0/1787759949 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60600687e0 con 0x7f60600d0350 2024-03-29T04:10:56.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.379+0000 7f605cff9640 1 -- 172.21.15.148:0/1787759949 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6050038020 con 0x7f60600d0350 2024-03-29T04:10:56.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.379+0000 7f605cff9640 1 --2- 172.21.15.148:0/1787759949 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f604006c560 0x7f604006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:56.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.379+0000 7f605cff9640 1 -- 172.21.15.148:0/1787759949 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f60500ade50 con 0x7f60600d0350 2024-03-29T04:10:56.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.379+0000 7f6056ffd640 1 --2- 172.21.15.148:0/1787759949 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f604006c560 0x7f604006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:56.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.380+0000 7f6056ffd640 1 --2- 172.21.15.148:0/1787759949 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f604006c560 0x7f604006ea20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f60600ceae0 tx=0x7f6048007e80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:56.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.383+0000 7f605cff9640 1 -- 172.21.15.148:0/1787759949 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f605007c100 con 0x7f60600d0350 2024-03-29T04:10:56.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.566+0000 7f6065d59640 1 -- 172.21.15.148:0/1787759949 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6060143130 con 0x7f60600d0350 2024-03-29T04:10:56.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.567+0000 7f605cff9640 1 -- 172.21.15.148:0/1787759949 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18858 (secure 0 0 0) 0x7f6050031800 con 0x7f60600d0350 2024-03-29T04:10:56.547 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:56.547 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:41.662934+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:56.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.570+0000 7f6065d59640 1 -- 172.21.15.148:0/1787759949 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f604006c560 msgr2=0x7f604006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:56.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.570+0000 7f6065d59640 1 --2- 172.21.15.148:0/1787759949 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f604006c560 0x7f604006ea20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f60600ceae0 tx=0x7f6048007e80 comp rx=0 tx=0).stop 2024-03-29T04:10:56.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.570+0000 7f6065d59640 1 -- 172.21.15.148:0/1787759949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60600d0350 msgr2=0x7f60601f40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:56.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.570+0000 7f6065d59640 1 --2- 172.21.15.148:0/1787759949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60600d0350 0x7f60601f40d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f605002bee0 tx=0x7f60500046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:56.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.570+0000 7f6065d59640 1 -- 172.21.15.148:0/1787759949 shutdown_connections 2024-03-29T04:10:56.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.570+0000 7f6065d59640 1 --2- 172.21.15.148:0/1787759949 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f604006c560 0x7f604006ea20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:56.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.570+0000 7f6065d59640 1 --2- 172.21.15.148:0/1787759949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f606015eb90 0x7f60600b6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:56.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.570+0000 7f6065d59640 1 --2- 172.21.15.148:0/1787759949 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6060147c80 0x7f60600cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:56.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.570+0000 7f6065d59640 1 --2- 172.21.15.148:0/1787759949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60600d0350 0x7f60601f40d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:56.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.570+0000 7f6065d59640 1 -- 172.21.15.148:0/1787759949 >> 172.21.15.148:0/1787759949 conn(0x7f60600b3e30 msgr2=0x7f6060161000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:56.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.571+0000 7f6065d59640 1 -- 172.21.15.148:0/1787759949 shutdown_connections 2024-03-29T04:10:56.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.571+0000 7f6065d59640 1 -- 172.21.15.148:0/1787759949 wait complete. 2024-03-29T04:10:56.565 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:47.898351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:57.922313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:58.829405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:10:02.551763+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:10:56.565 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:10:56.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.691+0000 7f9a7dd7c640 1 Processor -- start 2024-03-29T04:10:56.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.691+0000 7f9a7dd7c640 1 -- start start 2024-03-29T04:10:56.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.691+0000 7f9a7dd7c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9a780b2f00 0x7f9a78162680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:56.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.691+0000 7f9a7dd7c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9a780b3430 0x7f9a7806b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:56.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.691+0000 7f9a7dd7c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a780cf330 con 0x7f9a7815e3c0 2024-03-29T04:10:56.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.691+0000 7f9a7dd7c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f9a780cf960 con 0x7f9a780b2f00 2024-03-29T04:10:56.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.691+0000 7f9a7dd7c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9a78162cf0 con 0x7f9a780b3430 2024-03-29T04:10:56.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.692+0000 7f9a76ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9a780b2f00 0x7f9a78162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:56.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.692+0000 7f9a76ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9a780b2f00 0x7f9a78162680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55976/0 (socket says 172.21.15.148:55976) 2024-03-29T04:10:56.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.692+0000 7f9a76ffd640 1 -- 172.21.15.148:0/613789702 learned_addr learned my addr 172.21.15.148:0/613789702 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:56.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.692+0000 7f9a77fff640 1 --2- 172.21.15.148:0/613789702 >> v2:172.21.15.157:3300/0 conn(0x7f9a780b3430 0x7f9a7806b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:56.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.692+0000 7f9a76ffd640 1 -- 172.21.15.148:0/613789702 >> v2:172.21.15.157:3300/0 conn(0x7f9a780b3430 msgr2=0x7f9a7806b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:56.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.692+0000 7f9a76ffd640 1 --2- 172.21.15.148:0/613789702 >> v2:172.21.15.157:3300/0 conn(0x7f9a780b3430 0x7f9a7806b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:56.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.692+0000 7f9a76ffd640 1 -- 172.21.15.148:0/613789702 >> v1:172.21.15.157:6789/0 conn(0x7f9a7815e3c0 legacy=0x7f9a78161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:56.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.692+0000 7f9a76ffd640 1 -- 172.21.15.148:0/613789702 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a780cf5f0 con 0x7f9a780b2f00 2024-03-29T04:10:56.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.692+0000 7f9a77fff640 1 --2- 172.21.15.148:0/613789702 >> v2:172.21.15.157:3300/0 conn(0x7f9a780b3430 0x7f9a7806b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:56.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.693+0000 7f9a76ffd640 1 --2- 172.21.15.148:0/613789702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a780b2f00 0x7f9a78162680 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9a780cf8d0 tx=0x7f9a6c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad692356ed9fdec server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:56.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.693+0000 7f9a767fc640 1 -- 172.21.15.148:0/613789702 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c035020 con 0x7f9a780b2f00 2024-03-29T04:10:56.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.693+0000 7f9a767fc640 1 -- 172.21.15.148:0/613789702 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a6c031870 con 0x7f9a780b2f00 2024-03-29T04:10:56.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.693+0000 7f9a767fc640 1 -- 172.21.15.148:0/613789702 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c031c40 con 0x7f9a780b2f00 2024-03-29T04:10:56.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.693+0000 7f9a7dd7c640 1 -- 172.21.15.148:0/613789702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a780b2f00 msgr2=0x7f9a78162680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:56.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.693+0000 7f9a7dd7c640 1 --2- 172.21.15.148:0/613789702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a780b2f00 0x7f9a78162680 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9a780cf8d0 tx=0x7f9a6c02b910 comp rx=0 tx=0).stop 2024-03-29T04:10:56.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.693+0000 7f9a7dd7c640 1 -- 172.21.15.148:0/613789702 shutdown_connections 2024-03-29T04:10:56.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.693+0000 7f9a7dd7c640 1 --2- 172.21.15.148:0/613789702 >> v2:172.21.15.157:3300/0 conn(0x7f9a780b3430 0x7f9a7806b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:56.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.693+0000 7f9a7dd7c640 1 --2- 172.21.15.148:0/613789702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a780b2f00 0x7f9a78162680 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:56.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.693+0000 7f9a7dd7c640 1 -- 172.21.15.148:0/613789702 >> 172.21.15.148:0/613789702 conn(0x7f9a780b6430 msgr2=0x7f9a780d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:56.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.693+0000 7f9a7dd7c640 1 -- 172.21.15.148:0/613789702 shutdown_connections 2024-03-29T04:10:56.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.693+0000 7f9a7dd7c640 1 -- 172.21.15.148:0/613789702 wait complete. 2024-03-29T04:10:56.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.695+0000 7f9a7dd7c640 1 Processor -- start 2024-03-29T04:10:56.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.695+0000 7f9a7dd7c640 1 -- start start 2024-03-29T04:10:56.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.695+0000 7f9a7dd7c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a780b3430 0x7f9a780c7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:56.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.695+0000 7f9a7dd7c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a7815e3c0 0x7f9a7815e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:56.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.695+0000 7f9a7dd7c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a7815eef0 0x7f9a78160c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:56.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.695+0000 7f9a7dd7c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9a780b3b20 con 0x7f9a7815e3c0 2024-03-29T04:10:56.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.695+0000 7f9a7dd7c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9a780b39a0 con 0x7f9a7815eef0 2024-03-29T04:10:56.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.695+0000 7f9a7dd7c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9a7806bd30 con 0x7f9a780b3430 2024-03-29T04:10:56.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.696+0000 7f9a76ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a7815e3c0 0x7f9a7815e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:56.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.696+0000 7f9a76ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a7815e3c0 0x7f9a7815e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55982/0 (socket says 172.21.15.148:55982) 2024-03-29T04:10:56.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.696+0000 7f9a76ffd640 1 -- 172.21.15.148:0/586021372 learned_addr learned my addr 172.21.15.148:0/586021372 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:56.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.696+0000 7f9a77fff640 1 --2- 172.21.15.148:0/586021372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a7815eef0 0x7f9a78160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:56.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.696+0000 7f9a777fe640 1 --2- 172.21.15.148:0/586021372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a780b3430 0x7f9a780c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:56.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.696+0000 7f9a76ffd640 1 -- 172.21.15.148:0/586021372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a780b3430 msgr2=0x7f9a780c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:56.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.696+0000 7f9a76ffd640 1 --2- 172.21.15.148:0/586021372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a780b3430 0x7f9a780c7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:56.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.696+0000 7f9a76ffd640 1 -- 172.21.15.148:0/586021372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a7815eef0 msgr2=0x7f9a78160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:56.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.696+0000 7f9a76ffd640 1 --2- 172.21.15.148:0/586021372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a7815eef0 0x7f9a78160c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:56.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.696+0000 7f9a76ffd640 1 -- 172.21.15.148:0/586021372 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a780c7e00 con 0x7f9a7815e3c0 2024-03-29T04:10:56.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.696+0000 7f9a77fff640 1 --2- 172.21.15.148:0/586021372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a7815eef0 0x7f9a78160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:56.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.696+0000 7f9a76ffd640 1 --2- 172.21.15.148:0/586021372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a7815e3c0 0x7f9a7815e9b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c02b950 tx=0x7f9a6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:56.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.696+0000 7f9a5ffff640 1 -- 172.21.15.148:0/586021372 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c040070 con 0x7f9a7815e3c0 2024-03-29T04:10:56.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.696+0000 7f9a5ffff640 1 -- 172.21.15.148:0/586021372 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a6c030d00 con 0x7f9a7815e3c0 2024-03-29T04:10:56.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.696+0000 7f9a7dd7c640 1 -- 172.21.15.148:0/586021372 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a780c80e0 con 0x7f9a7815e3c0 2024-03-29T04:10:56.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.696+0000 7f9a5ffff640 1 -- 172.21.15.148:0/586021372 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c035040 con 0x7f9a7815e3c0 2024-03-29T04:10:56.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.696+0000 7f9a7dd7c640 1 -- 172.21.15.148:0/586021372 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a780c86a0 con 0x7f9a7815e3c0 2024-03-29T04:10:56.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.697+0000 7f9a7dd7c640 1 -- 172.21.15.148:0/586021372 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a7815f300 con 0x7f9a7815e3c0 2024-03-29T04:10:56.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.698+0000 7f9a5ffff640 1 -- 172.21.15.148:0/586021372 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9a6c04b060 con 0x7f9a7815e3c0 2024-03-29T04:10:56.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.698+0000 7f9a5ffff640 1 --2- 172.21.15.148:0/586021372 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9a6406c530 0x7f9a6406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:56.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.698+0000 7f9a5ffff640 1 -- 172.21.15.148:0/586021372 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f9a6c0ad930 con 0x7f9a7815e3c0 2024-03-29T04:10:56.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.698+0000 7f9a777fe640 1 --2- 172.21.15.148:0/586021372 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9a6406c530 0x7f9a6406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:56.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.699+0000 7f9a777fe640 1 --2- 172.21.15.148:0/586021372 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9a6406c530 0x7f9a6406e9f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9a780cf040 tx=0x7f9a68009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:56.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:56.703+0000 7f9a5ffff640 1 -- 172.21.15.148:0/586021372 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9a7815f300 con 0x7f9a7815e3c0 2024-03-29T04:10:56.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.004+0000 7f9a7dd7c640 1 -- 172.21.15.148:0/586021372 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9a78068850 con 0x7f9a7815e3c0 2024-03-29T04:10:56.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.004+0000 7f9a5ffff640 1 -- 172.21.15.148:0/586021372 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f9a6c07bb50 con 0x7f9a7815e3c0 2024-03-29T04:10:56.983 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-29T04:10:56.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.006+0000 7f9a7dd7c640 1 -- 172.21.15.148:0/586021372 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9a6406c530 msgr2=0x7f9a6406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:56.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.006+0000 7f9a7dd7c640 1 --2- 172.21.15.148:0/586021372 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9a6406c530 0x7f9a6406e9f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9a780cf040 tx=0x7f9a68009040 comp rx=0 tx=0).stop 2024-03-29T04:10:56.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.006+0000 7f9a7dd7c640 1 -- 172.21.15.148:0/586021372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a7815e3c0 msgr2=0x7f9a7815e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:56.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.006+0000 7f9a7dd7c640 1 --2- 172.21.15.148:0/586021372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a7815e3c0 0x7f9a7815e9b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c02b950 tx=0x7f9a6c0046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:56.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.006+0000 7f9a7dd7c640 1 -- 172.21.15.148:0/586021372 shutdown_connections 2024-03-29T04:10:56.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.006+0000 7f9a7dd7c640 1 --2- 172.21.15.148:0/586021372 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9a6406c530 0x7f9a6406e9f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:56.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.006+0000 7f9a7dd7c640 1 --2- 172.21.15.148:0/586021372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a7815eef0 0x7f9a78160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:56.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.006+0000 7f9a7dd7c640 1 --2- 172.21.15.148:0/586021372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a7815e3c0 0x7f9a7815e9b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:56.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.006+0000 7f9a7dd7c640 1 --2- 172.21.15.148:0/586021372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a780b3430 0x7f9a780c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:56.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.006+0000 7f9a7dd7c640 1 -- 172.21.15.148:0/586021372 >> 172.21.15.148:0/586021372 conn(0x7f9a780b6430 msgr2=0x7f9a780b5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:56.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.007+0000 7f9a7dd7c640 1 -- 172.21.15.148:0/586021372 shutdown_connections 2024-03-29T04:10:56.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.007+0000 7f9a7dd7c640 1 -- 172.21.15.148:0/586021372 wait complete. 2024-03-29T04:10:56.997 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:10:57.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.122+0000 7f4bd856d640 1 Processor -- start 2024-03-29T04:10:57.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.122+0000 7f4bd856d640 1 -- start start 2024-03-29T04:10:57.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.123+0000 7f4bd856d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4bd00b3430 0x7f4bd0162680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:57.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.123+0000 7f4bd856d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bd00cf330 con 0x7f4bd00b2f00 2024-03-29T04:10:57.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.123+0000 7f4bd856d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bd0162cf0 con 0x7f4bd015e3c0 2024-03-29T04:10:57.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.123+0000 7f4bd856d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f4bd00cf960 con 0x7f4bd00b3430 2024-03-29T04:10:57.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.123+0000 7f4bd6b0a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4bd00b3430 0x7f4bd0162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:57.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.123+0000 7f4bd5b08640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4bd00b2f00 0x7f4bd006b7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39916/0 (socket says 172.21.15.148:39916) 2024-03-29T04:10:57.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.123+0000 7f4bd6b0a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4bd00b3430 0x7f4bd0162680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55984/0 (socket says 172.21.15.148:55984) 2024-03-29T04:10:57.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.123+0000 7f4bd5b08640 1 -- 172.21.15.148:0/591963572 learned_addr learned my addr 172.21.15.148:0/591963572 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:57.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.124+0000 7f4bd6b0a640 1 -- 172.21.15.148:0/591963572 >> v1:172.21.15.157:6789/0 conn(0x7f4bd015e3c0 legacy=0x7f4bd0161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:57.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.124+0000 7f4bd6b0a640 1 -- 172.21.15.148:0/591963572 >> v1:172.21.15.148:6789/0 conn(0x7f4bd00b2f00 legacy=0x7f4bd006b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:57.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.124+0000 7f4bd6b0a640 1 -- 172.21.15.148:0/591963572 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd00cf5f0 con 0x7f4bd00b3430 2024-03-29T04:10:57.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.124+0000 7f4bd5307640 1 -- 172.21.15.148:0/591963572 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 890777061 0 0) 0x7f4bd00cf330 con 0x7f4bd00b2f00 2024-03-29T04:10:57.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.124+0000 7f4bd6b0a640 1 --2- 172.21.15.148:0/591963572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd00b3430 0x7f4bd0162680 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009d90 tx=0x7f4bc802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec50bf9da1a2b52 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:57.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.125+0000 7f4bd5307640 1 -- 172.21.15.148:0/591963572 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8039070 con 0x7f4bd00b3430 2024-03-29T04:10:57.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.125+0000 7f4bd5307640 1 -- 172.21.15.148:0/591963572 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bc802cbd0 con 0x7f4bd00b3430 2024-03-29T04:10:57.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.125+0000 7f4bd5307640 1 -- 172.21.15.148:0/591963572 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8034440 con 0x7f4bd00b3430 2024-03-29T04:10:57.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.125+0000 7f4bd856d640 1 -- 172.21.15.148:0/591963572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd00b3430 msgr2=0x7f4bd0162680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:57.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.125+0000 7f4bd856d640 1 --2- 172.21.15.148:0/591963572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd00b3430 0x7f4bd0162680 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009d90 tx=0x7f4bc802be20 comp rx=0 tx=0).stop 2024-03-29T04:10:57.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.125+0000 7f4bd856d640 1 -- 172.21.15.148:0/591963572 shutdown_connections 2024-03-29T04:10:57.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.125+0000 7f4bd856d640 1 --2- 172.21.15.148:0/591963572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd00b3430 0x7f4bd0162680 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:57.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.125+0000 7f4bd856d640 1 -- 172.21.15.148:0/591963572 >> 172.21.15.148:0/591963572 conn(0x7f4bd00b6430 msgr2=0x7f4bd00d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:57.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.125+0000 7f4bd856d640 1 -- 172.21.15.148:0/591963572 shutdown_connections 2024-03-29T04:10:57.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.125+0000 7f4bd856d640 1 -- 172.21.15.148:0/591963572 wait complete. 2024-03-29T04:10:57.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.126+0000 7f4bd856d640 1 Processor -- start 2024-03-29T04:10:57.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.126+0000 7f4bd856d640 1 -- start start 2024-03-29T04:10:57.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.127+0000 7f4bd856d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bd00b3430 0x7f4bd0152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:57.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.127+0000 7f4bd856d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bd015e3c0 0x7f4bd00cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:57.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.127+0000 7f4bd856d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd00c87f0 0x7f4bd015a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:57.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.127+0000 7f4bd856d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd00b3b20 con 0x7f4bd00c87f0 2024-03-29T04:10:57.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.127+0000 7f4bd856d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd00b39a0 con 0x7f4bd015e3c0 2024-03-29T04:10:57.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.127+0000 7f4bd856d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4bd006bf00 con 0x7f4bd00b3430 2024-03-29T04:10:57.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.127+0000 7f4bd6b0a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd00c87f0 0x7f4bd015a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:57.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.127+0000 7f4bd6b0a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd00c87f0 0x7f4bd015a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55998/0 (socket says 172.21.15.148:55998) 2024-03-29T04:10:57.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.127+0000 7f4bd6b0a640 1 -- 172.21.15.148:0/4231204726 learned_addr learned my addr 172.21.15.148:0/4231204726 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:57.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.127+0000 7f4bd6309640 1 --2- 172.21.15.148:0/4231204726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bd00b3430 0x7f4bd0152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:57.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.128+0000 7f4bd5b08640 1 --2- 172.21.15.148:0/4231204726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bd015e3c0 0x7f4bd00cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:57.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.128+0000 7f4bd6b0a640 1 -- 172.21.15.148:0/4231204726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bd00b3430 msgr2=0x7f4bd0152da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:57.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.128+0000 7f4bd6b0a640 1 --2- 172.21.15.148:0/4231204726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bd00b3430 0x7f4bd0152da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:57.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.128+0000 7f4bd6b0a640 1 -- 172.21.15.148:0/4231204726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bd015e3c0 msgr2=0x7f4bd00cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:57.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.128+0000 7f4bd6b0a640 1 --2- 172.21.15.148:0/4231204726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bd015e3c0 0x7f4bd00cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:57.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.128+0000 7f4bd6b0a640 1 -- 172.21.15.148:0/4231204726 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd00cd1c0 con 0x7f4bd00c87f0 2024-03-29T04:10:57.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.128+0000 7f4bd5b08640 1 --2- 172.21.15.148:0/4231204726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bd015e3c0 0x7f4bd00cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:57.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.128+0000 7f4bd6b0a640 1 --2- 172.21.15.148:0/4231204726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd00c87f0 0x7f4bd015a8b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009d90 tx=0x7f4bc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:57.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.128+0000 7f4bc6ffd640 1 -- 172.21.15.148:0/4231204726 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8044020 con 0x7f4bd00c87f0 2024-03-29T04:10:57.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.128+0000 7f4bc6ffd640 1 -- 172.21.15.148:0/4231204726 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bc803f470 con 0x7f4bd00c87f0 2024-03-29T04:10:57.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.128+0000 7f4bc6ffd640 1 -- 172.21.15.148:0/4231204726 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8039040 con 0x7f4bd00c87f0 2024-03-29T04:10:57.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.128+0000 7f4bd856d640 1 -- 172.21.15.148:0/4231204726 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd00cd4a0 con 0x7f4bd00c87f0 2024-03-29T04:10:57.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.128+0000 7f4bd856d640 1 -- 172.21.15.148:0/4231204726 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd00cda60 con 0x7f4bd00c87f0 2024-03-29T04:10:57.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.129+0000 7f4bd856d640 1 -- 172.21.15.148:0/4231204726 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bd0068850 con 0x7f4bd00c87f0 2024-03-29T04:10:57.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.129+0000 7f4bc6ffd640 1 -- 172.21.15.148:0/4231204726 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4bc8047020 con 0x7f4bd00c87f0 2024-03-29T04:10:57.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.130+0000 7f4bc6ffd640 1 --2- 172.21.15.148:0/4231204726 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4bb406c530 0x7f4bb406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:57.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.130+0000 7f4bc6ffd640 1 -- 172.21.15.148:0/4231204726 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f4bc80ae750 con 0x7f4bd00c87f0 2024-03-29T04:10:57.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.130+0000 7f4bd6309640 1 --2- 172.21.15.148:0/4231204726 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4bb406c530 0x7f4bb406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:57.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.130+0000 7f4bd6309640 1 --2- 172.21.15.148:0/4231204726 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4bb406c530 0x7f4bb406e9f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc006fd0 tx=0x7f4bcc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:57.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.135+0000 7f4bc6ffd640 1 -- 172.21.15.148:0/4231204726 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4bc804e030 con 0x7f4bd00c87f0 2024-03-29T04:10:57.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.315+0000 7f4bd856d640 1 -- 172.21.15.148:0/4231204726 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4bd00d2af0 con 0x7f4bd00c87f0 2024-03-29T04:10:57.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.316+0000 7f4bc6ffd640 1 -- 172.21.15.148:0/4231204726 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f4bc807c970 con 0x7f4bd00c87f0 2024-03-29T04:10:57.295 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:10:57.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.318+0000 7f4bd856d640 1 -- 172.21.15.148:0/4231204726 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4bb406c530 msgr2=0x7f4bb406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:57.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.318+0000 7f4bd856d640 1 --2- 172.21.15.148:0/4231204726 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4bb406c530 0x7f4bb406e9f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc006fd0 tx=0x7f4bcc008040 comp rx=0 tx=0).stop 2024-03-29T04:10:57.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.318+0000 7f4bd856d640 1 -- 172.21.15.148:0/4231204726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd00c87f0 msgr2=0x7f4bd015a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:57.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.318+0000 7f4bd856d640 1 --2- 172.21.15.148:0/4231204726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd00c87f0 0x7f4bd015a8b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009d90 tx=0x7f4bc8004a40 comp rx=0 tx=0).stop 2024-03-29T04:10:57.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.318+0000 7f4bd856d640 1 -- 172.21.15.148:0/4231204726 shutdown_connections 2024-03-29T04:10:57.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.318+0000 7f4bd856d640 1 --2- 172.21.15.148:0/4231204726 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4bb406c530 0x7f4bb406e9f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:57.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.318+0000 7f4bd856d640 1 --2- 172.21.15.148:0/4231204726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd00c87f0 0x7f4bd015a8b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:57.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.318+0000 7f4bd856d640 1 --2- 172.21.15.148:0/4231204726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bd015e3c0 0x7f4bd00cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:57.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.318+0000 7f4bd856d640 1 --2- 172.21.15.148:0/4231204726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bd00b3430 0x7f4bd0152da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:57.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.318+0000 7f4bd856d640 1 -- 172.21.15.148:0/4231204726 >> 172.21.15.148:0/4231204726 conn(0x7f4bd00b6430 msgr2=0x7f4bd00d32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:57.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.318+0000 7f4bd856d640 1 -- 172.21.15.148:0/4231204726 shutdown_connections 2024-03-29T04:10:57.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.318+0000 7f4bd856d640 1 -- 172.21.15.148:0/4231204726 wait complete. 2024-03-29T04:10:57.311 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:10:57.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.439+0000 7f8830900640 1 Processor -- start 2024-03-29T04:10:57.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.439+0000 7f8830900640 1 -- start start 2024-03-29T04:10:57.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.440+0000 7f8830900640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f882c0afac0 0x7f882c070170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:57.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.440+0000 7f8830900640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f882c15d250 0x7f882c06ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:57.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.440+0000 7f8830900640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f882c0afea0 con 0x7f882c06e410 2024-03-29T04:10:57.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.440+0000 7f8830900640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f882c06e860 con 0x7f882c15d250 2024-03-29T04:10:57.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.440+0000 7f8830900640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f882c06c5c0 con 0x7f882c0afac0 2024-03-29T04:10:57.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.440+0000 7f8829d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f882c15d250 0x7f882c06ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:57.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.440+0000 7f882a59c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f882c0afac0 0x7f882c070170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:57.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.440+0000 7f8829d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f882c15d250 0x7f882c06ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:56558/0 (socket says 172.21.15.148:56558) 2024-03-29T04:10:57.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.440+0000 7f882ad9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f882c06e410 0x7f882c15fdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.148:49356/0 (socket says 172.21.15.148:49356) 2024-03-29T04:10:57.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.440+0000 7f8829d9b640 1 -- 172.21.15.148:0/975657161 learned_addr learned my addr 172.21.15.148:0/975657161 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:57.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.441+0000 7f882a59c640 1 -- 172.21.15.148:0/975657161 >> v2:172.21.15.157:3300/0 conn(0x7f882c15d250 msgr2=0x7f882c06ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:57.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.441+0000 7f882a59c640 1 --2- 172.21.15.148:0/975657161 >> v2:172.21.15.157:3300/0 conn(0x7f882c15d250 0x7f882c06ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:57.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.441+0000 7f882a59c640 1 -- 172.21.15.148:0/975657161 >> v1:172.21.15.157:6789/0 conn(0x7f882c06e410 legacy=0x7f882c15fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:57.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.441+0000 7f882959a640 1 -- 172.21.15.148:0/975657161 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 634752445 0 0) 0x7f882c0afea0 con 0x7f882c06e410 2024-03-29T04:10:57.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.441+0000 7f882a59c640 1 -- 172.21.15.148:0/975657161 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f882c0cb0e0 con 0x7f882c0afac0 2024-03-29T04:10:57.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.441+0000 7f8829d9b640 1 --2- 172.21.15.148:0/975657161 >> v2:172.21.15.157:3300/0 conn(0x7f882c15d250 0x7f882c06ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:57.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.442+0000 7f882a59c640 1 --2- 172.21.15.148:0/975657161 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f882c0afac0 0x7f882c070170 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f88200097a0 tx=0x7f882002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3aafc2bee7b34cc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:57.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.442+0000 7f882959a640 1 -- 172.21.15.148:0/975657161 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8820035020 con 0x7f882c0afac0 2024-03-29T04:10:57.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.442+0000 7f882959a640 1 -- 172.21.15.148:0/975657161 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8820031830 con 0x7f882c0afac0 2024-03-29T04:10:57.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.442+0000 7f882959a640 1 -- 172.21.15.148:0/975657161 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8820031c00 con 0x7f882c0afac0 2024-03-29T04:10:57.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.442+0000 7f8830900640 1 -- 172.21.15.148:0/975657161 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f882c0afac0 msgr2=0x7f882c070170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:57.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.442+0000 7f8830900640 1 --2- 172.21.15.148:0/975657161 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f882c0afac0 0x7f882c070170 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f88200097a0 tx=0x7f882002b8d0 comp rx=0 tx=0).stop 2024-03-29T04:10:57.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.442+0000 7f8830900640 1 -- 172.21.15.148:0/975657161 shutdown_connections 2024-03-29T04:10:57.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.442+0000 7f8830900640 1 --2- 172.21.15.148:0/975657161 >> v2:172.21.15.157:3300/0 conn(0x7f882c15d250 0x7f882c06ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:57.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.442+0000 7f8830900640 1 --2- 172.21.15.148:0/975657161 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f882c0afac0 0x7f882c070170 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:57.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.442+0000 7f8830900640 1 -- 172.21.15.148:0/975657161 >> 172.21.15.148:0/975657161 conn(0x7f882c0cd160 msgr2=0x7f882c0cd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:57.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.442+0000 7f8830900640 1 -- 172.21.15.148:0/975657161 shutdown_connections 2024-03-29T04:10:57.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.443+0000 7f8830900640 1 -- 172.21.15.148:0/975657161 wait complete. 2024-03-29T04:10:57.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.444+0000 7f8830900640 1 Processor -- start 2024-03-29T04:10:57.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.444+0000 7f8830900640 1 -- start start 2024-03-29T04:10:57.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.445+0000 7f8830900640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f882c06e410 0x7f882c0cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:57.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.445+0000 7f8830900640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f882c0afac0 0x7f882c14a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:57.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.445+0000 7f8830900640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f882c15d250 0x7f882c0cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:57.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.445+0000 7f8830900640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f882c0cb980 con 0x7f882c06e410 2024-03-29T04:10:57.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.445+0000 7f8830900640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f882c0cb6a0 con 0x7f882c0afac0 2024-03-29T04:10:57.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.445+0000 7f8830900640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f882c0cbc60 con 0x7f882c15d250 2024-03-29T04:10:57.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.445+0000 7f882a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f882c06e410 0x7f882c0cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:57.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.445+0000 7f882a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f882c06e410 0x7f882c0cedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56000/0 (socket says 172.21.15.148:56000) 2024-03-29T04:10:57.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.445+0000 7f882a59c640 1 -- 172.21.15.148:0/4103458782 learned_addr learned my addr 172.21.15.148:0/4103458782 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:57.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.445+0000 7f8829d9b640 1 --2- 172.21.15.148:0/4103458782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f882c0afac0 0x7f882c14a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:57.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.445+0000 7f882ad9d640 1 --2- 172.21.15.148:0/4103458782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f882c15d250 0x7f882c0cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:57.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.445+0000 7f882a59c640 1 -- 172.21.15.148:0/4103458782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f882c15d250 msgr2=0x7f882c0cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:57.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.445+0000 7f882a59c640 1 --2- 172.21.15.148:0/4103458782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f882c15d250 0x7f882c0cf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:57.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.445+0000 7f882a59c640 1 -- 172.21.15.148:0/4103458782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f882c0afac0 msgr2=0x7f882c14a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:57.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.445+0000 7f882a59c640 1 --2- 172.21.15.148:0/4103458782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f882c0afac0 0x7f882c14a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:57.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.445+0000 7f882a59c640 1 -- 172.21.15.148:0/4103458782 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f882c0ce350 con 0x7f882c06e410 2024-03-29T04:10:57.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.445+0000 7f882a59c640 1 --2- 172.21.15.148:0/4103458782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f882c06e410 0x7f882c0cedc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8820000c00 tx=0x7f88200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:57.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.446+0000 7f881affd640 1 -- 172.21.15.148:0/4103458782 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8820035070 con 0x7f882c06e410 2024-03-29T04:10:57.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.446+0000 7f881affd640 1 -- 172.21.15.148:0/4103458782 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8820004230 con 0x7f882c06e410 2024-03-29T04:10:57.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.446+0000 7f8830900640 1 -- 172.21.15.148:0/4103458782 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f882c0cff00 con 0x7f882c06e410 2024-03-29T04:10:57.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.446+0000 7f881affd640 1 -- 172.21.15.148:0/4103458782 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882003a630 con 0x7f882c06e410 2024-03-29T04:10:57.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.446+0000 7f8830900640 1 -- 172.21.15.148:0/4103458782 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f882c0d01e0 con 0x7f882c06e410 2024-03-29T04:10:57.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.447+0000 7f8830900640 1 -- 172.21.15.148:0/4103458782 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f882c068850 con 0x7f882c06e410 2024-03-29T04:10:57.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.447+0000 7f881affd640 1 -- 172.21.15.148:0/4103458782 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8820038020 con 0x7f882c06e410 2024-03-29T04:10:57.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.448+0000 7f881affd640 1 --2- 172.21.15.148:0/4103458782 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f880c06c560 0x7f880c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:57.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.448+0000 7f881affd640 1 -- 172.21.15.148:0/4103458782 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f88200ad610 con 0x7f882c06e410 2024-03-29T04:10:57.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.448+0000 7f8829d9b640 1 --2- 172.21.15.148:0/4103458782 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f880c06c560 0x7f880c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:57.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.448+0000 7f8829d9b640 1 --2- 172.21.15.148:0/4103458782 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f880c06c560 0x7f880c06ea20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f882c0ca6a0 tx=0x7f881c009240 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:57.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.452+0000 7f881affd640 1 -- 172.21.15.148:0/4103458782 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8820040070 con 0x7f882c06e410 2024-03-29T04:10:57.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.649+0000 7f8830900640 1 -- 172.21.15.148:0/4103458782 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f882c15cca0 con 0x7f882c06e410 2024-03-29T04:10:57.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.650+0000 7f881affd640 1 -- 172.21.15.148:0/4103458782 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f882007b1c0 con 0x7f882c06e410 2024-03-29T04:10:57.629 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-29T04:10:57.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.652+0000 7f8830900640 1 -- 172.21.15.148:0/4103458782 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f880c06c560 msgr2=0x7f880c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:57.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.652+0000 7f8830900640 1 --2- 172.21.15.148:0/4103458782 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f880c06c560 0x7f880c06ea20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f882c0ca6a0 tx=0x7f881c009240 comp rx=0 tx=0).stop 2024-03-29T04:10:57.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.652+0000 7f8830900640 1 -- 172.21.15.148:0/4103458782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f882c06e410 msgr2=0x7f882c0cedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:57.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.652+0000 7f8830900640 1 --2- 172.21.15.148:0/4103458782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f882c06e410 0x7f882c0cedc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8820000c00 tx=0x7f88200046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:57.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.652+0000 7f8830900640 1 -- 172.21.15.148:0/4103458782 shutdown_connections 2024-03-29T04:10:57.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.652+0000 7f8830900640 1 --2- 172.21.15.148:0/4103458782 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f880c06c560 0x7f880c06ea20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:57.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.652+0000 7f8830900640 1 --2- 172.21.15.148:0/4103458782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f882c15d250 0x7f882c0cf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:57.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.652+0000 7f8830900640 1 --2- 172.21.15.148:0/4103458782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f882c0afac0 0x7f882c14a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:57.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.652+0000 7f8830900640 1 --2- 172.21.15.148:0/4103458782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f882c06e410 0x7f882c0cedc0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:57.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.652+0000 7f8830900640 1 -- 172.21.15.148:0/4103458782 >> 172.21.15.148:0/4103458782 conn(0x7f882c0cd160 msgr2=0x7f882c155450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:57.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.652+0000 7f8830900640 1 -- 172.21.15.148:0/4103458782 shutdown_connections 2024-03-29T04:10:57.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.652+0000 7f8830900640 1 -- 172.21.15.148:0/4103458782 wait complete. 2024-03-29T04:10:57.642 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:10:57.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.769+0000 7fb70afbd640 1 Processor -- start 2024-03-29T04:10:57.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.769+0000 7fb70afbd640 1 -- start start 2024-03-29T04:10:57.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.769+0000 7fb70afbd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb7040ccec0 0x7fb7041693b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:57.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.769+0000 7fb70afbd640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7040b56c0 con 0x7fb70415f200 2024-03-29T04:10:57.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.769+0000 7fb70afbd640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7040cfdd0 con 0x7fb70415f5e0 2024-03-29T04:10:57.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.769+0000 7fb70afbd640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fb7040d09e0 con 0x7fb7040ccec0 2024-03-29T04:10:57.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.770+0000 7fb70955a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb7040ccec0 0x7fb7041693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:57.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.770+0000 7fb70955a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb7040ccec0 0x7fb7041693b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56008/0 (socket says 172.21.15.148:56008) 2024-03-29T04:10:57.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.770+0000 7fb70955a640 1 -- 172.21.15.148:0/1673370198 learned_addr learned my addr 172.21.15.148:0/1673370198 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:57.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.770+0000 7fb708d59640 1 --1- 172.21.15.148:0/1673370198 >> v1:172.21.15.148:6789/0 conn(0x7fb70415f200 0x7fb7040b4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39920/0 (socket says 172.21.15.148:39920) 2024-03-29T04:10:57.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.770+0000 7fb70955a640 1 -- 172.21.15.148:0/1673370198 >> v1:172.21.15.157:6789/0 conn(0x7fb70415f5e0 legacy=0x7fb7040cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:57.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.770+0000 7fb70955a640 1 -- 172.21.15.148:0/1673370198 >> v1:172.21.15.148:6789/0 conn(0x7fb70415f200 legacy=0x7fb7040b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:57.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.770+0000 7fb7037fe640 1 -- 172.21.15.148:0/1673370198 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3788841069 0 0) 0x7fb7040b56c0 con 0x7fb70415f200 2024-03-29T04:10:57.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.770+0000 7fb70955a640 1 -- 172.21.15.148:0/1673370198 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7040d00b0 con 0x7fb7040ccec0 2024-03-29T04:10:57.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.771+0000 7fb70955a640 1 --2- 172.21.15.148:0/1673370198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7040ccec0 0x7fb7041693b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009e50 tx=0x7fb6fc02c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d1f3f7348bf94e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:57.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.771+0000 7fb7037fe640 1 -- 172.21.15.148:0/1673370198 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc02cbb0 con 0x7fb7040ccec0 2024-03-29T04:10:57.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.771+0000 7fb7037fe640 1 -- 172.21.15.148:0/1673370198 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6fc02cd50 con 0x7fb7040ccec0 2024-03-29T04:10:57.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.771+0000 7fb7037fe640 1 -- 172.21.15.148:0/1673370198 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc034610 con 0x7fb7040ccec0 2024-03-29T04:10:57.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.771+0000 7fb70afbd640 1 -- 172.21.15.148:0/1673370198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7040ccec0 msgr2=0x7fb7041693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:57.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.771+0000 7fb70afbd640 1 --2- 172.21.15.148:0/1673370198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7040ccec0 0x7fb7041693b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009e50 tx=0x7fb6fc02c010 comp rx=0 tx=0).stop 2024-03-29T04:10:57.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.771+0000 7fb70afbd640 1 -- 172.21.15.148:0/1673370198 shutdown_connections 2024-03-29T04:10:57.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.771+0000 7fb70afbd640 1 --2- 172.21.15.148:0/1673370198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7040ccec0 0x7fb7041693b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:57.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.771+0000 7fb70afbd640 1 -- 172.21.15.148:0/1673370198 >> 172.21.15.148:0/1673370198 conn(0x7fb7040b3c70 msgr2=0x7fb7040b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:57.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.771+0000 7fb70afbd640 1 -- 172.21.15.148:0/1673370198 shutdown_connections 2024-03-29T04:10:57.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.771+0000 7fb70afbd640 1 -- 172.21.15.148:0/1673370198 wait complete. 2024-03-29T04:10:57.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.773+0000 7fb70afbd640 1 Processor -- start 2024-03-29T04:10:57.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.773+0000 7fb70afbd640 1 -- start start 2024-03-29T04:10:57.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.773+0000 7fb70afbd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7040ccec0 0x7fb70414ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:57.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.773+0000 7fb70afbd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb70415f200 0x7fb70414cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:57.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.773+0000 7fb70afbd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb70415f5e0 0x7fb7040c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:57.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.773+0000 7fb70afbd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb7040b7b40 con 0x7fb70415f5e0 2024-03-29T04:10:57.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.773+0000 7fb70afbd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb7040d0670 con 0x7fb70415f200 2024-03-29T04:10:57.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.773+0000 7fb70afbd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb7040b7cc0 con 0x7fb7040ccec0 2024-03-29T04:10:57.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.774+0000 7fb70955a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb70415f5e0 0x7fb7040c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:57.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.774+0000 7fb70955a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb70415f5e0 0x7fb7040c73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56016/0 (socket says 172.21.15.148:56016) 2024-03-29T04:10:57.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.774+0000 7fb70955a640 1 -- 172.21.15.148:0/691745914 learned_addr learned my addr 172.21.15.148:0/691745914 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:57.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.774+0000 7fb708d59640 1 --2- 172.21.15.148:0/691745914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7040ccec0 0x7fb70414ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:57.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.774+0000 7fb703fff640 1 --2- 172.21.15.148:0/691745914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb70415f200 0x7fb70414cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:57.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.774+0000 7fb70955a640 1 -- 172.21.15.148:0/691745914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7040ccec0 msgr2=0x7fb70414ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:57.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.774+0000 7fb70955a640 1 --2- 172.21.15.148:0/691745914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7040ccec0 0x7fb70414ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:57.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.774+0000 7fb70955a640 1 -- 172.21.15.148:0/691745914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb70415f200 msgr2=0x7fb70414cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:57.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.774+0000 7fb70955a640 1 --2- 172.21.15.148:0/691745914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb70415f200 0x7fb70414cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:57.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.774+0000 7fb70955a640 1 -- 172.21.15.148:0/691745914 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7041518f0 con 0x7fb70415f5e0 2024-03-29T04:10:57.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.774+0000 7fb70955a640 1 --2- 172.21.15.148:0/691745914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb70415f5e0 0x7fb7040c73d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009e50 tx=0x7fb6fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:57.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.774+0000 7fb7017fa640 1 -- 172.21.15.148:0/691745914 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc043030 con 0x7fb70415f5e0 2024-03-29T04:10:57.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.774+0000 7fb7017fa640 1 -- 172.21.15.148:0/691745914 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6fc03e450 con 0x7fb70415f5e0 2024-03-29T04:10:57.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.774+0000 7fb70afbd640 1 -- 172.21.15.148:0/691745914 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb704151bd0 con 0x7fb70415f5e0 2024-03-29T04:10:57.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.774+0000 7fb7017fa640 1 -- 172.21.15.148:0/691745914 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc009510 con 0x7fb70415f5e0 2024-03-29T04:10:57.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.774+0000 7fb70afbd640 1 -- 172.21.15.148:0/691745914 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7040c7910 con 0x7fb70415f5e0 2024-03-29T04:10:57.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.775+0000 7fb70afbd640 1 -- 172.21.15.148:0/691745914 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7040758d0 con 0x7fb70415f5e0 2024-03-29T04:10:57.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.776+0000 7fb7017fa640 1 -- 172.21.15.148:0/691745914 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb6fc046070 con 0x7fb70415f5e0 2024-03-29T04:10:57.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.776+0000 7fb7017fa640 1 --2- 172.21.15.148:0/691745914 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb6e406c560 0x7fb6e406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:57.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.776+0000 7fb7017fa640 1 -- 172.21.15.148:0/691745914 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb6fc0ade90 con 0x7fb70415f5e0 2024-03-29T04:10:57.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.776+0000 7fb708d59640 1 --2- 172.21.15.148:0/691745914 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb6e406c560 0x7fb6e406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:57.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.777+0000 7fb708d59640 1 --2- 172.21.15.148:0/691745914 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb6e406c560 0x7fb6e406ea20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4007f70 tx=0x7fb6f4007f00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:57.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.781+0000 7fb7017fa640 1 -- 172.21.15.148:0/691745914 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb6fc04d070 con 0x7fb70415f5e0 2024-03-29T04:10:57.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.964+0000 7fb70afbd640 1 -- 172.21.15.148:0/691745914 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb704068850 con 0x7fb70415f5e0 2024-03-29T04:10:57.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.964+0000 7fb7017fa640 1 -- 172.21.15.148:0/691745914 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fb6fc07ba60 con 0x7fb70415f5e0 2024-03-29T04:10:57.943 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:10:57.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.966+0000 7fb70afbd640 1 -- 172.21.15.148:0/691745914 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb6e406c560 msgr2=0x7fb6e406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:57.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.966+0000 7fb70afbd640 1 --2- 172.21.15.148:0/691745914 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb6e406c560 0x7fb6e406ea20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4007f70 tx=0x7fb6f4007f00 comp rx=0 tx=0).stop 2024-03-29T04:10:57.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.966+0000 7fb70afbd640 1 -- 172.21.15.148:0/691745914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb70415f5e0 msgr2=0x7fb7040c73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:57.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.966+0000 7fb70afbd640 1 --2- 172.21.15.148:0/691745914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb70415f5e0 0x7fb7040c73d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009e50 tx=0x7fb6fc004a40 comp rx=0 tx=0).stop 2024-03-29T04:10:57.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.966+0000 7fb70afbd640 1 -- 172.21.15.148:0/691745914 shutdown_connections 2024-03-29T04:10:57.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.966+0000 7fb70afbd640 1 --2- 172.21.15.148:0/691745914 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb6e406c560 0x7fb6e406ea20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:57.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.966+0000 7fb70afbd640 1 --2- 172.21.15.148:0/691745914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb70415f5e0 0x7fb7040c73d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:57.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.966+0000 7fb70afbd640 1 --2- 172.21.15.148:0/691745914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb70415f200 0x7fb70414cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:57.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.966+0000 7fb70afbd640 1 --2- 172.21.15.148:0/691745914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7040ccec0 0x7fb70414ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:57.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.966+0000 7fb70afbd640 1 -- 172.21.15.148:0/691745914 >> 172.21.15.148:0/691745914 conn(0x7fb7040b3c70 msgr2=0x7fb7040cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:57.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.966+0000 7fb70afbd640 1 -- 172.21.15.148:0/691745914 shutdown_connections 2024-03-29T04:10:57.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:57.966+0000 7fb70afbd640 1 -- 172.21.15.148:0/691745914 wait complete. 2024-03-29T04:10:57.957 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:10:58.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.080+0000 7f3cb474d640 1 Processor -- start 2024-03-29T04:10:58.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.080+0000 7f3cb474d640 1 -- start start 2024-03-29T04:10:58.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.081+0000 7f3cb474d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3cac0ba030 0x7f3cac0d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:58.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.081+0000 7f3cb474d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cac15e7c0 con 0x7f3cac0d10d0 2024-03-29T04:10:58.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.081+0000 7f3cb474d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cac0d2c30 con 0x7f3cac15e3e0 2024-03-29T04:10:58.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.081+0000 7f3cb474d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f3cac0d3840 con 0x7f3cac0ba030 2024-03-29T04:10:58.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.081+0000 7f3cb2cea640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3cac0d10d0 0x7f3cac167f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39930/0 (socket says 172.21.15.148:39930) 2024-03-29T04:10:58.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.081+0000 7f3cb2cea640 1 -- 172.21.15.148:0/70094695 learned_addr learned my addr 172.21.15.148:0/70094695 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:58.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.082+0000 7f3cb1ce8640 1 --2- 172.21.15.148:0/70094695 >> v2:172.21.15.157:3300/0 conn(0x7f3cac0ba030 0x7f3cac0d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:58.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.082+0000 7f3cb14e7640 1 -- 172.21.15.148:0/70094695 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 594370696 0 0) 0x7f3cac15e7c0 con 0x7f3cac0d10d0 2024-03-29T04:10:58.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.082+0000 7f3cb14e7640 1 -- 172.21.15.148:0/70094695 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c98003520 con 0x7f3cac0d10d0 2024-03-29T04:10:58.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.082+0000 7f3cb14e7640 1 -- 172.21.15.148:0/70094695 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3378631037 0 0) 0x7f3c98003520 con 0x7f3cac0d10d0 2024-03-29T04:10:58.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.082+0000 7f3cb14e7640 1 -- 172.21.15.148:0/70094695 >> v2:172.21.15.157:3300/0 conn(0x7f3cac0ba030 msgr2=0x7f3cac0d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:58.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.082+0000 7f3cb14e7640 1 --2- 172.21.15.148:0/70094695 >> v2:172.21.15.157:3300/0 conn(0x7f3cac0ba030 0x7f3cac0d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:58.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.082+0000 7f3cb14e7640 1 -- 172.21.15.148:0/70094695 >> v1:172.21.15.157:6789/0 conn(0x7f3cac15e3e0 legacy=0x7f3cac0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:58.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.082+0000 7f3cb14e7640 1 -- 172.21.15.148:0/70094695 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cac0d2f10 con 0x7f3cac0d10d0 2024-03-29T04:10:58.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.082+0000 7f3cb14e7640 1 -- 172.21.15.148:0/70094695 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f3ca40019f0 con 0x7f3cac0d10d0 2024-03-29T04:10:58.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.082+0000 7f3cb14e7640 1 -- 172.21.15.148:0/70094695 >> v1:172.21.15.148:6789/0 conn(0x7f3cac0d10d0 legacy=0x7f3cac167f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:58.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.082+0000 7f3cb14e7640 1 --2- 172.21.15.148:0/70094695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c980039b0 0x7f3c98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:58.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.082+0000 7f3cb14e7640 1 --2- 172.21.15.148:0/70094695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c98004800 0x7f3c98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:58.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.082+0000 7f3cb14e7640 1 --2- 172.21.15.148:0/70094695 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c980051c0 0x7f3c9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:58.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.082+0000 7f3cb14e7640 1 -- 172.21.15.148:0/70094695 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac0d3840 con 0x7f3c980039b0 2024-03-29T04:10:58.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.082+0000 7f3cb14e7640 1 -- 172.21.15.148:0/70094695 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9800c070 con 0x7f3c98004800 2024-03-29T04:10:58.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.082+0000 7f3cb14e7640 1 -- 172.21.15.148:0/70094695 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3c9800c270 con 0x7f3c980051c0 2024-03-29T04:10:58.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.082+0000 7f3cb24e9640 1 --2- 172.21.15.148:0/70094695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c980039b0 0x7f3c98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:58.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.082+0000 7f3cb1ce8640 1 --2- 172.21.15.148:0/70094695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c98004800 0x7f3c98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:58.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.082+0000 7f3cb2cea640 1 --2- 172.21.15.148:0/70094695 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c980051c0 0x7f3c9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:58.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.082+0000 7f3cb24e9640 1 -- 172.21.15.148:0/70094695 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c980051c0 msgr2=0x7f3c9800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:58.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.082+0000 7f3cb24e9640 1 --2- 172.21.15.148:0/70094695 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c980051c0 0x7f3c9800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:58.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.082+0000 7f3cb24e9640 1 -- 172.21.15.148:0/70094695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c98004800 msgr2=0x7f3c98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:58.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.082+0000 7f3cb24e9640 1 --2- 172.21.15.148:0/70094695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c98004800 0x7f3c98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:58.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.082+0000 7f3cb2cea640 1 -- 172.21.15.148:0/70094695 reap_dead start 2024-03-29T04:10:58.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.082+0000 7f3cb24e9640 1 -- 172.21.15.148:0/70094695 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c98003520 con 0x7f3c980039b0 2024-03-29T04:10:58.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.083+0000 7f3cb24e9640 1 --2- 172.21.15.148:0/70094695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c980039b0 0x7f3c98003dd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c001750 tx=0x7f3c98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93105421de05e78 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:58.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.083+0000 7f3cb14e7640 1 -- 172.21.15.148:0/70094695 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c001370 con 0x7f3c980039b0 2024-03-29T04:10:58.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.083+0000 7f3cb14e7640 1 -- 172.21.15.148:0/70094695 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c9c0052d0 con 0x7f3c980039b0 2024-03-29T04:10:58.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.083+0000 7f3cb14e7640 1 -- 172.21.15.148:0/70094695 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c003830 con 0x7f3c980039b0 2024-03-29T04:10:58.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.084+0000 7f3cb474d640 1 -- 172.21.15.148:0/70094695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c980039b0 msgr2=0x7f3c98003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:58.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.084+0000 7f3cb474d640 1 --2- 172.21.15.148:0/70094695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c980039b0 0x7f3c98003dd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c001750 tx=0x7f3c98005830 comp rx=0 tx=0).stop 2024-03-29T04:10:58.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.084+0000 7f3cb474d640 1 -- 172.21.15.148:0/70094695 shutdown_connections 2024-03-29T04:10:58.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.084+0000 7f3cb474d640 1 --2- 172.21.15.148:0/70094695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c980039b0 0x7f3c98003dd0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:58.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.084+0000 7f3cb474d640 1 -- 172.21.15.148:0/70094695 >> 172.21.15.148:0/70094695 conn(0x7f3cac0b6210 msgr2=0x7f3cac0b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:58.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.084+0000 7f3cb474d640 1 -- 172.21.15.148:0/70094695 shutdown_connections 2024-03-29T04:10:58.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.084+0000 7f3cb474d640 1 -- 172.21.15.148:0/70094695 wait complete. 2024-03-29T04:10:58.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.085+0000 7f3cb474d640 1 Processor -- start 2024-03-29T04:10:58.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.085+0000 7f3cb474d640 1 -- start start 2024-03-29T04:10:58.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.086+0000 7f3cb474d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c980039b0 0x7f3cac151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:58.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.086+0000 7f3cb474d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cac1503b0 0x7f3cac0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:58.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.086+0000 7f3cb474d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cac150830 0x7f3cac0cd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:58.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.086+0000 7f3cb474d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac0d3cd0 con 0x7f3c980039b0 2024-03-29T04:10:58.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.086+0000 7f3cb474d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac0d34d0 con 0x7f3cac150830 2024-03-29T04:10:58.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.086+0000 7f3cb474d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3cac0d3e50 con 0x7f3cac1503b0 2024-03-29T04:10:58.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.086+0000 7f3cb24e9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c980039b0 0x7f3cac151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:58.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.086+0000 7f3cb24e9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c980039b0 0x7f3cac151020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56030/0 (socket says 172.21.15.148:56030) 2024-03-29T04:10:58.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.086+0000 7f3cb24e9640 1 -- 172.21.15.148:0/570280943 learned_addr learned my addr 172.21.15.148:0/570280943 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:58.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.086+0000 7f3cb2cea640 1 --2- 172.21.15.148:0/570280943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cac150830 0x7f3cac0cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:58.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.086+0000 7f3cb1ce8640 1 --2- 172.21.15.148:0/570280943 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cac1503b0 0x7f3cac0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:58.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.086+0000 7f3cb24e9640 1 -- 172.21.15.148:0/570280943 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cac1503b0 msgr2=0x7f3cac0cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:58.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.086+0000 7f3cb24e9640 1 --2- 172.21.15.148:0/570280943 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cac1503b0 0x7f3cac0cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:58.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.086+0000 7f3cb24e9640 1 -- 172.21.15.148:0/570280943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cac150830 msgr2=0x7f3cac0cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:58.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.086+0000 7f3cb24e9640 1 --2- 172.21.15.148:0/570280943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cac150830 0x7f3cac0cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:58.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.086+0000 7f3cb24e9640 1 -- 172.21.15.148:0/570280943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cac15fc90 con 0x7f3c980039b0 2024-03-29T04:10:58.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.086+0000 7f3cb2cea640 1 --2- 172.21.15.148:0/570280943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cac150830 0x7f3cac0cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:58.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.086+0000 7f3cb24e9640 1 --2- 172.21.15.148:0/570280943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c980039b0 0x7f3cac151020 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c000c00 tx=0x7f3c9c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:58.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.087+0000 7f3ca2ffd640 1 -- 172.21.15.148:0/570280943 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c041070 con 0x7f3c980039b0 2024-03-29T04:10:58.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.087+0000 7f3ca2ffd640 1 -- 172.21.15.148:0/570280943 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c9c005480 con 0x7f3c980039b0 2024-03-29T04:10:58.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.087+0000 7f3cb474d640 1 -- 172.21.15.148:0/570280943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cac0cc1b0 con 0x7f3c980039b0 2024-03-29T04:10:58.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.087+0000 7f3ca2ffd640 1 -- 172.21.15.148:0/570280943 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c001370 con 0x7f3c980039b0 2024-03-29T04:10:58.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.087+0000 7f3cb474d640 1 -- 172.21.15.148:0/570280943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cac15ff30 con 0x7f3c980039b0 2024-03-29T04:10:58.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.088+0000 7f3cb474d640 1 -- 172.21.15.148:0/570280943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cac0cc6d0 con 0x7f3c980039b0 2024-03-29T04:10:58.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.088+0000 7f3ca2ffd640 1 -- 172.21.15.148:0/570280943 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3c9c044070 con 0x7f3c980039b0 2024-03-29T04:10:58.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.089+0000 7f3ca2ffd640 1 --2- 172.21.15.148:0/570280943 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3c9006c530 0x7f3c9006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:58.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.089+0000 7f3cb1ce8640 1 --2- 172.21.15.148:0/570280943 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3c9006c530 0x7f3c9006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:58.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.089+0000 7f3ca2ffd640 1 -- 172.21.15.148:0/570280943 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f3c9c0ad4f0 con 0x7f3c980039b0 2024-03-29T04:10:58.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.089+0000 7f3cb1ce8640 1 --2- 172.21.15.148:0/570280943 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3c9006c530 0x7f3c9006e9f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3cac0cc8d0 tx=0x7f3ca8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:58.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.093+0000 7f3ca2ffd640 1 -- 172.21.15.148:0/570280943 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3cac0cc6d0 con 0x7f3c980039b0 2024-03-29T04:10:58.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.285+0000 7f3cb474d640 1 -- 172.21.15.148:0/570280943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3cac0687e0 con 0x7f3c980039b0 2024-03-29T04:10:58.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.286+0000 7f3ca2ffd640 1 -- 172.21.15.148:0/570280943 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f3c9c07b710 con 0x7f3c980039b0 2024-03-29T04:10:58.265 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-03-29T04:10:58.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.288+0000 7f3cb474d640 1 -- 172.21.15.148:0/570280943 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3c9006c530 msgr2=0x7f3c9006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:58.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.288+0000 7f3cb474d640 1 --2- 172.21.15.148:0/570280943 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3c9006c530 0x7f3c9006e9f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3cac0cc8d0 tx=0x7f3ca8008040 comp rx=0 tx=0).stop 2024-03-29T04:10:58.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.288+0000 7f3cb474d640 1 -- 172.21.15.148:0/570280943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c980039b0 msgr2=0x7f3cac151020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:58.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.288+0000 7f3cb474d640 1 --2- 172.21.15.148:0/570280943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c980039b0 0x7f3cac151020 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c000c00 tx=0x7f3c9c001120 comp rx=0 tx=0).stop 2024-03-29T04:10:58.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.288+0000 7f3cb474d640 1 -- 172.21.15.148:0/570280943 shutdown_connections 2024-03-29T04:10:58.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.288+0000 7f3cb474d640 1 --2- 172.21.15.148:0/570280943 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3c9006c530 0x7f3c9006e9f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:58.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.288+0000 7f3cb474d640 1 --2- 172.21.15.148:0/570280943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cac150830 0x7f3cac0cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:58.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.288+0000 7f3cb474d640 1 --2- 172.21.15.148:0/570280943 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cac1503b0 0x7f3cac0cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:58.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.288+0000 7f3cb474d640 1 --2- 172.21.15.148:0/570280943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c980039b0 0x7f3cac151020 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:58.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.288+0000 7f3cb474d640 1 -- 172.21.15.148:0/570280943 >> 172.21.15.148:0/570280943 conn(0x7f3cac0b6210 msgr2=0x7f3cac06ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:58.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.289+0000 7f3cb474d640 1 -- 172.21.15.148:0/570280943 shutdown_connections 2024-03-29T04:10:58.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.289+0000 7f3cb474d640 1 -- 172.21.15.148:0/570280943 wait complete. 2024-03-29T04:10:58.278 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi148.front.sepia.ceph.com... 2024-03-29T04:10:58.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:58.279 DEBUG:teuthology.orchestra.run.smithi148:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T04:10:58.306 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:10:58.306 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-03-29T04:10:58.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:58.307 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T04:10:58.361 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:10:58.362 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:10:58.381 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:10:58.381 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:10:58.440 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:58.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.563+0000 7f3cbdb3d640 1 Processor -- start 2024-03-29T04:10:58.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.563+0000 7f3cbdb3d640 1 -- start start 2024-03-29T04:10:58.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.563+0000 7f3cbdb3d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cb815e3e0 0x7f3cb80d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:58.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.563+0000 7f3cbdb3d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3cb80ba030 0x7f3cb80d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:58.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.563+0000 7f3cbdb3d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3cb80d10d0 0x7f3cb80d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:58.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.563+0000 7f3cbdb3d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f3cb80d3840 con 0x7f3cb80d10d0 2024-03-29T04:10:58.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.563+0000 7f3cbdb3d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f3cb815e7c0 con 0x7f3cb80ba030 2024-03-29T04:10:58.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.563+0000 7f3cbdb3d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3cb80d2c30 con 0x7f3cb815e3e0 2024-03-29T04:10:58.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.564+0000 7f3cb7fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3cb80d10d0 0x7f3cb80d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:58.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.564+0000 7f3cb7fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3cb80d10d0 0x7f3cb80d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56044/0 (socket says 172.21.15.148:56044) 2024-03-29T04:10:58.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.564+0000 7f3cb7fff640 1 -- 172.21.15.148:0/2820186902 learned_addr learned my addr 172.21.15.148:0/2820186902 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:58.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.564+0000 7f3cb6ffd640 1 --2- 172.21.15.148:0/2820186902 >> v2:172.21.15.157:3300/0 conn(0x7f3cb80ba030 0x7f3cb80d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:58.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.564+0000 7f3cb77fe640 1 --2- 172.21.15.148:0/2820186902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cb815e3e0 0x7f3cb80d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:58.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.564+0000 7f3cb7fff640 1 -- 172.21.15.148:0/2820186902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cb815e3e0 msgr2=0x7f3cb80d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:58.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.564+0000 7f3cb7fff640 1 --2- 172.21.15.148:0/2820186902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cb815e3e0 0x7f3cb80d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:58.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.564+0000 7f3cb7fff640 1 -- 172.21.15.148:0/2820186902 >> v2:172.21.15.157:3300/0 conn(0x7f3cb80ba030 msgr2=0x7f3cb80d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:58.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.564+0000 7f3cb7fff640 1 --2- 172.21.15.148:0/2820186902 >> v2:172.21.15.157:3300/0 conn(0x7f3cb80ba030 0x7f3cb80d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:58.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.564+0000 7f3cb7fff640 1 -- 172.21.15.148:0/2820186902 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb80d2f10 con 0x7f3cb80d10d0 2024-03-29T04:10:58.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.564+0000 7f3cb6ffd640 1 --2- 172.21.15.148:0/2820186902 >> v2:172.21.15.157:3300/0 conn(0x7f3cb80ba030 0x7f3cb80d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:58.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.565+0000 7f3cb7fff640 1 --2- 172.21.15.148:0/2820186902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb80d10d0 0x7f3cb80d1580 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80d1c80 tx=0x7f3cb002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128b2d58946fb9c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:58.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.565+0000 7f3cb67fc640 1 -- 172.21.15.148:0/2820186902 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb002cb00 con 0x7f3cb80d10d0 2024-03-29T04:10:58.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.565+0000 7f3cb67fc640 1 -- 172.21.15.148:0/2820186902 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cb002cca0 con 0x7f3cb80d10d0 2024-03-29T04:10:58.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.565+0000 7f3cb67fc640 1 -- 172.21.15.148:0/2820186902 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0034610 con 0x7f3cb80d10d0 2024-03-29T04:10:58.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.566+0000 7f3cbdb3d640 1 -- 172.21.15.148:0/2820186902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb80d10d0 msgr2=0x7f3cb80d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:58.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.566+0000 7f3cbdb3d640 1 --2- 172.21.15.148:0/2820186902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb80d10d0 0x7f3cb80d1580 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80d1c80 tx=0x7f3cb002bf60 comp rx=0 tx=0).stop 2024-03-29T04:10:58.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.566+0000 7f3cbdb3d640 1 -- 172.21.15.148:0/2820186902 shutdown_connections 2024-03-29T04:10:58.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.566+0000 7f3cbdb3d640 1 --2- 172.21.15.148:0/2820186902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb80d10d0 0x7f3cb80d1580 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:58.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.566+0000 7f3cbdb3d640 1 --2- 172.21.15.148:0/2820186902 >> v2:172.21.15.157:3300/0 conn(0x7f3cb80ba030 0x7f3cb80d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:58.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.566+0000 7f3cbdb3d640 1 --2- 172.21.15.148:0/2820186902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cb815e3e0 0x7f3cb80d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:58.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.566+0000 7f3cbdb3d640 1 -- 172.21.15.148:0/2820186902 >> 172.21.15.148:0/2820186902 conn(0x7f3cb80b6210 msgr2=0x7f3cb80b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:58.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.566+0000 7f3cbdb3d640 1 -- 172.21.15.148:0/2820186902 shutdown_connections 2024-03-29T04:10:58.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.566+0000 7f3cbdb3d640 1 -- 172.21.15.148:0/2820186902 wait complete. 2024-03-29T04:10:58.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.567+0000 7f3cbdb3d640 1 Processor -- start 2024-03-29T04:10:58.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.567+0000 7f3cbdb3d640 1 -- start start 2024-03-29T04:10:58.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.568+0000 7f3cbdb3d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cb80ba030 0x7f3cb81f70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:58.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.568+0000 7f3cbdb3d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb80d10d0 0x7f3cb8160f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:58.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.568+0000 7f3cbdb3d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cb815e3e0 0x7f3cb81614b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:58.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.568+0000 7f3cbdb3d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb80d2870 con 0x7f3cb80d10d0 2024-03-29T04:10:58.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.568+0000 7f3cbdb3d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb80d34d0 con 0x7f3cb815e3e0 2024-03-29T04:10:58.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.568+0000 7f3cbdb3d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3cb80d29f0 con 0x7f3cb80ba030 2024-03-29T04:10:58.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.568+0000 7f3cb6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb80d10d0 0x7f3cb8160f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:58.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.568+0000 7f3cb6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb80d10d0 0x7f3cb8160f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56050/0 (socket says 172.21.15.148:56050) 2024-03-29T04:10:58.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.568+0000 7f3cb6ffd640 1 -- 172.21.15.148:0/1387784345 learned_addr learned my addr 172.21.15.148:0/1387784345 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:58.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.568+0000 7f3cb77fe640 1 --2- 172.21.15.148:0/1387784345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cb80ba030 0x7f3cb81f70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:58.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.568+0000 7f3cb7fff640 1 --2- 172.21.15.148:0/1387784345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cb815e3e0 0x7f3cb81614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:58.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.568+0000 7f3cb6ffd640 1 -- 172.21.15.148:0/1387784345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cb80ba030 msgr2=0x7f3cb81f70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:58.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.568+0000 7f3cb6ffd640 1 --2- 172.21.15.148:0/1387784345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cb80ba030 0x7f3cb81f70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:58.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.568+0000 7f3cb6ffd640 1 -- 172.21.15.148:0/1387784345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cb815e3e0 msgr2=0x7f3cb81614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:58.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.568+0000 7f3cb6ffd640 1 --2- 172.21.15.148:0/1387784345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cb815e3e0 0x7f3cb81614b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:58.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.568+0000 7f3cb6ffd640 1 -- 172.21.15.148:0/1387784345 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb81f75e0 con 0x7f3cb80d10d0 2024-03-29T04:10:58.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.569+0000 7f3cb77fe640 1 --2- 172.21.15.148:0/1387784345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cb80ba030 0x7f3cb81f70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:58.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.569+0000 7f3cb7fff640 1 --2- 172.21.15.148:0/1387784345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cb815e3e0 0x7f3cb81614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:58.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.569+0000 7f3cb6ffd640 1 --2- 172.21.15.148:0/1387784345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb80d10d0 0x7f3cb8160f70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00e810 tx=0x7f3cac00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:58.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.569+0000 7f3c9ffff640 1 -- 172.21.15.148:0/1387784345 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac00f040 con 0x7f3cb80d10d0 2024-03-29T04:10:58.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.569+0000 7f3c9ffff640 1 -- 172.21.15.148:0/1387784345 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cac004030 con 0x7f3cb80d10d0 2024-03-29T04:10:58.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.569+0000 7f3cbdb3d640 1 -- 172.21.15.148:0/1387784345 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cb815f690 con 0x7f3cb80d10d0 2024-03-29T04:10:58.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.569+0000 7f3c9ffff640 1 -- 172.21.15.148:0/1387784345 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac004400 con 0x7f3cb80d10d0 2024-03-29T04:10:58.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.569+0000 7f3cbdb3d640 1 -- 172.21.15.148:0/1387784345 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cb815fc50 con 0x7f3cb80d10d0 2024-03-29T04:10:58.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.570+0000 7f3cbdb3d640 1 -- 172.21.15.148:0/1387784345 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cb80687e0 con 0x7f3cb80d10d0 2024-03-29T04:10:58.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.570+0000 7f3c9ffff640 1 -- 172.21.15.148:0/1387784345 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3cac00c6d0 con 0x7f3cb80d10d0 2024-03-29T04:10:58.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.571+0000 7f3c9ffff640 1 --2- 172.21.15.148:0/1387784345 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3ca406c530 0x7f3ca406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:58.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.571+0000 7f3cb77fe640 1 --2- 172.21.15.148:0/1387784345 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3ca406c530 0x7f3ca406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:58.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.571+0000 7f3c9ffff640 1 -- 172.21.15.148:0/1387784345 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f3cac08b760 con 0x7f3cb80d10d0 2024-03-29T04:10:58.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.571+0000 7f3cb77fe640 1 --2- 172.21.15.148:0/1387784345 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3ca406c530 0x7f3ca406e9f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8006fd0 tx=0x7f3ca8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:58.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.575+0000 7f3c9ffff640 1 -- 172.21.15.148:0/1387784345 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3cac017020 con 0x7f3cb80d10d0 2024-03-29T04:10:58.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.757+0000 7f3cbdb3d640 1 -- 172.21.15.148:0/1387784345 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3cb80d3cc0 con 0x7f3cb80d10d0 2024-03-29T04:10:58.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.759+0000 7f3c9ffff640 1 -- 172.21.15.148:0/1387784345 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18858 (secure 0 0 0) 0x7f3cac059a10 con 0x7f3cb80d10d0 2024-03-29T04:10:58.738 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:10:58.738 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:41.662934+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:58.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.761+0000 7f3cbdb3d640 1 -- 172.21.15.148:0/1387784345 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3ca406c530 msgr2=0x7f3ca406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:58.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.761+0000 7f3cbdb3d640 1 --2- 172.21.15.148:0/1387784345 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3ca406c530 0x7f3ca406e9f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8006fd0 tx=0x7f3ca8008040 comp rx=0 tx=0).stop 2024-03-29T04:10:58.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.761+0000 7f3cbdb3d640 1 -- 172.21.15.148:0/1387784345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb80d10d0 msgr2=0x7f3cb8160f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:58.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.761+0000 7f3cbdb3d640 1 --2- 172.21.15.148:0/1387784345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb80d10d0 0x7f3cb8160f70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00e810 tx=0x7f3cac00ecd0 comp rx=0 tx=0).stop 2024-03-29T04:10:58.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.761+0000 7f3cbdb3d640 1 -- 172.21.15.148:0/1387784345 shutdown_connections 2024-03-29T04:10:58.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.761+0000 7f3cbdb3d640 1 --2- 172.21.15.148:0/1387784345 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3ca406c530 0x7f3ca406e9f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:58.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.761+0000 7f3cbdb3d640 1 --2- 172.21.15.148:0/1387784345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cb815e3e0 0x7f3cb81614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:58.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.761+0000 7f3cbdb3d640 1 --2- 172.21.15.148:0/1387784345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb80d10d0 0x7f3cb8160f70 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:58.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.761+0000 7f3cbdb3d640 1 --2- 172.21.15.148:0/1387784345 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cb80ba030 0x7f3cb81f70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:58.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.761+0000 7f3cbdb3d640 1 -- 172.21.15.148:0/1387784345 >> 172.21.15.148:0/1387784345 conn(0x7f3cb80b6210 msgr2=0x7f3cb80b7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:58.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.761+0000 7f3cbdb3d640 1 -- 172.21.15.148:0/1387784345 shutdown_connections 2024-03-29T04:10:58.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.761+0000 7f3cbdb3d640 1 -- 172.21.15.148:0/1387784345 wait complete. 2024-03-29T04:10:58.752 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:47.898351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:57.922313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:58.829405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:10:02.551763+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:10:58.752 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:10:58.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.878+0000 7f5f19235640 1 Processor -- start 2024-03-29T04:10:58.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.878+0000 7f5f19235640 1 -- start start 2024-03-29T04:10:58.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.879+0000 7f5f19235640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5f1415e3c0 0x7f5f14161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:58.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.879+0000 7f5f19235640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f140cf330 con 0x7f5f140b2f00 2024-03-29T04:10:58.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.879+0000 7f5f19235640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f14162cf0 con 0x7f5f140b3430 2024-03-29T04:10:58.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.879+0000 7f5f19235640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5f140cf960 con 0x7f5f1415e3c0 2024-03-29T04:10:58.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.879+0000 7f5f1259c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5f140b2f00 0x7f5f141624b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39944/0 (socket says 172.21.15.148:39944) 2024-03-29T04:10:58.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.879+0000 7f5f1259c640 1 -- 172.21.15.148:0/3624302487 learned_addr learned my addr 172.21.15.148:0/3624302487 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:58.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.879+0000 7f5f11d9b640 1 -- 172.21.15.148:0/3624302487 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 554708281 0 0) 0x7f5f140cf330 con 0x7f5f140b2f00 2024-03-29T04:10:58.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.879+0000 7f5f12d9d640 1 --2- 172.21.15.148:0/3624302487 >> v2:172.21.15.157:3300/0 conn(0x7f5f1415e3c0 0x7f5f14161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:58.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.879+0000 7f5f11d9b640 1 -- 172.21.15.148:0/3624302487 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f00003520 con 0x7f5f140b2f00 2024-03-29T04:10:58.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.880+0000 7f5f11d9b640 1 -- 172.21.15.148:0/3624302487 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3563524574 0 0) 0x7f5f00003520 con 0x7f5f140b2f00 2024-03-29T04:10:58.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.880+0000 7f5f11d9b640 1 -- 172.21.15.148:0/3624302487 >> v2:172.21.15.157:3300/0 conn(0x7f5f1415e3c0 msgr2=0x7f5f14161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:58.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.880+0000 7f5f11d9b640 1 --2- 172.21.15.148:0/3624302487 >> v2:172.21.15.157:3300/0 conn(0x7f5f1415e3c0 0x7f5f14161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:58.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.880+0000 7f5f11d9b640 1 -- 172.21.15.148:0/3624302487 >> v1:172.21.15.157:6789/0 conn(0x7f5f140b3430 legacy=0x7f5f1406b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:58.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.880+0000 7f5f11d9b640 1 -- 172.21.15.148:0/3624302487 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f140cf5f0 con 0x7f5f140b2f00 2024-03-29T04:10:58.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.880+0000 7f5f11d9b640 1 -- 172.21.15.148:0/3624302487 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f5f04001670 con 0x7f5f140b2f00 2024-03-29T04:10:58.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.880+0000 7f5f11d9b640 1 -- 172.21.15.148:0/3624302487 >> v1:172.21.15.148:6789/0 conn(0x7f5f140b2f00 legacy=0x7f5f141624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:58.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.880+0000 7f5f11d9b640 1 --2- 172.21.15.148:0/3624302487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f000039b0 0x7f5f00003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:58.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.880+0000 7f5f11d9b640 1 --2- 172.21.15.148:0/3624302487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f00004800 0x7f5f00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:58.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.880+0000 7f5f11d9b640 1 --2- 172.21.15.148:0/3624302487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f000051c0 0x7f5f0000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:58.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.880+0000 7f5f11d9b640 1 -- 172.21.15.148:0/3624302487 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5f140cf960 con 0x7f5f000051c0 2024-03-29T04:10:58.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.880+0000 7f5f11d9b640 1 -- 172.21.15.148:0/3624302487 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0000c160 con 0x7f5f00004800 2024-03-29T04:10:58.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.880+0000 7f5f11d9b640 1 -- 172.21.15.148:0/3624302487 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5f0000c360 con 0x7f5f000039b0 2024-03-29T04:10:58.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.880+0000 7f5f1359e640 1 --2- 172.21.15.148:0/3624302487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f000051c0 0x7f5f0000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:58.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.880+0000 7f5f12d9d640 1 --2- 172.21.15.148:0/3624302487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f000039b0 0x7f5f00003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:58.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.880+0000 7f5f1259c640 1 --2- 172.21.15.148:0/3624302487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f00004800 0x7f5f00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:58.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.880+0000 7f5f1359e640 1 -- 172.21.15.148:0/3624302487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f000039b0 msgr2=0x7f5f00003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:58.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.880+0000 7f5f1359e640 1 --2- 172.21.15.148:0/3624302487 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f000039b0 0x7f5f00003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:58.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.880+0000 7f5f1359e640 1 -- 172.21.15.148:0/3624302487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f00004800 msgr2=0x7f5f00004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:58.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.880+0000 7f5f1359e640 1 --2- 172.21.15.148:0/3624302487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f00004800 0x7f5f00004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:58.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.880+0000 7f5f1359e640 1 -- 172.21.15.148:0/3624302487 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f00003520 con 0x7f5f000051c0 2024-03-29T04:10:58.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.881+0000 7f5f1359e640 1 -- 172.21.15.148:0/3624302487 reap_dead start 2024-03-29T04:10:58.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.881+0000 7f5f1359e640 1 --2- 172.21.15.148:0/3624302487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f000051c0 0x7f5f0000b9c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c001e50 tx=0x7f5f00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd7d4d4864fa89c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:58.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.881+0000 7f5f11d9b640 1 -- 172.21.15.148:0/3624302487 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c001a70 con 0x7f5f000051c0 2024-03-29T04:10:58.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.881+0000 7f5f11d9b640 1 -- 172.21.15.148:0/3624302487 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f0c003bb0 con 0x7f5f000051c0 2024-03-29T04:10:58.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.881+0000 7f5f11d9b640 1 -- 172.21.15.148:0/3624302487 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c0050d0 con 0x7f5f000051c0 2024-03-29T04:10:58.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.882+0000 7f5f19235640 1 -- 172.21.15.148:0/3624302487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f000051c0 msgr2=0x7f5f0000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:58.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.882+0000 7f5f19235640 1 --2- 172.21.15.148:0/3624302487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f000051c0 0x7f5f0000b9c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c001e50 tx=0x7f5f00005830 comp rx=0 tx=0).stop 2024-03-29T04:10:58.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.882+0000 7f5f19235640 1 -- 172.21.15.148:0/3624302487 shutdown_connections 2024-03-29T04:10:58.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.882+0000 7f5f19235640 1 --2- 172.21.15.148:0/3624302487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f000051c0 0x7f5f0000b9c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:58.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.882+0000 7f5f19235640 1 -- 172.21.15.148:0/3624302487 >> 172.21.15.148:0/3624302487 conn(0x7f5f140b6430 msgr2=0x7f5f140d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:58.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.882+0000 7f5f19235640 1 -- 172.21.15.148:0/3624302487 shutdown_connections 2024-03-29T04:10:58.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.882+0000 7f5f19235640 1 -- 172.21.15.148:0/3624302487 wait complete. 2024-03-29T04:10:58.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.883+0000 7f5f19235640 1 Processor -- start 2024-03-29T04:10:58.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.883+0000 7f5f19235640 1 -- start start 2024-03-29T04:10:58.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.884+0000 7f5f19235640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f000051c0 0x7f5f1415ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:58.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.884+0000 7f5f19235640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f1415d570 0x7f5f141530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:58.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.884+0000 7f5f19235640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f140cd790 0x7f5f140cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:58.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.884+0000 7f5f19235640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5f140b3b20 con 0x7f5f000051c0 2024-03-29T04:10:58.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.884+0000 7f5f19235640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5f140b39a0 con 0x7f5f1415d570 2024-03-29T04:10:58.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.884+0000 7f5f19235640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5f14161f70 con 0x7f5f140cd790 2024-03-29T04:10:58.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.884+0000 7f5f12d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f000051c0 0x7f5f1415ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:58.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.884+0000 7f5f12d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f000051c0 0x7f5f1415ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56060/0 (socket says 172.21.15.148:56060) 2024-03-29T04:10:58.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.884+0000 7f5f12d9d640 1 -- 172.21.15.148:0/3336192706 learned_addr learned my addr 172.21.15.148:0/3336192706 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:58.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.884+0000 7f5f1259c640 1 --2- 172.21.15.148:0/3336192706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f1415d570 0x7f5f141530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:58.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.884+0000 7f5f1359e640 1 --2- 172.21.15.148:0/3336192706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f140cd790 0x7f5f140cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:58.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.884+0000 7f5f12d9d640 1 -- 172.21.15.148:0/3336192706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f140cd790 msgr2=0x7f5f140cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:58.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.884+0000 7f5f12d9d640 1 --2- 172.21.15.148:0/3336192706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f140cd790 0x7f5f140cdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:58.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.884+0000 7f5f12d9d640 1 -- 172.21.15.148:0/3336192706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f1415d570 msgr2=0x7f5f141530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:58.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.884+0000 7f5f12d9d640 1 --2- 172.21.15.148:0/3336192706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f1415d570 0x7f5f141530c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:58.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.884+0000 7f5f12d9d640 1 -- 172.21.15.148:0/3336192706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f140ccb50 con 0x7f5f000051c0 2024-03-29T04:10:58.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.884+0000 7f5f12d9d640 1 --2- 172.21.15.148:0/3336192706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f000051c0 0x7f5f1415ff40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5efc011940 tx=0x7f5efc011d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:58.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.885+0000 7f5f0b7fe640 1 -- 172.21.15.148:0/3336192706 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc00f9d0 con 0x7f5f000051c0 2024-03-29T04:10:58.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.885+0000 7f5f0b7fe640 1 -- 172.21.15.148:0/3336192706 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5efc00fb70 con 0x7f5f000051c0 2024-03-29T04:10:58.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.885+0000 7f5f0b7fe640 1 -- 172.21.15.148:0/3336192706 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc014440 con 0x7f5f000051c0 2024-03-29T04:10:58.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.885+0000 7f5f19235640 1 -- 172.21.15.148:0/3336192706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f140cc870 con 0x7f5f000051c0 2024-03-29T04:10:58.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.885+0000 7f5f19235640 1 -- 172.21.15.148:0/3336192706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f1415f740 con 0x7f5f000051c0 2024-03-29T04:10:58.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.886+0000 7f5f19235640 1 -- 172.21.15.148:0/3336192706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f1415d950 con 0x7f5f000051c0 2024-03-29T04:10:58.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.886+0000 7f5f0b7fe640 1 -- 172.21.15.148:0/3336192706 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5efc012040 con 0x7f5f000051c0 2024-03-29T04:10:58.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.887+0000 7f5f0b7fe640 1 --2- 172.21.15.148:0/3336192706 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5ef406c530 0x7f5ef406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:58.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.887+0000 7f5f0b7fe640 1 -- 172.21.15.148:0/3336192706 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f5efc08bff0 con 0x7f5f000051c0 2024-03-29T04:10:58.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.887+0000 7f5f1259c640 1 --2- 172.21.15.148:0/3336192706 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5ef406c530 0x7f5ef406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:58.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.887+0000 7f5f1259c640 1 --2- 172.21.15.148:0/3336192706 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5ef406c530 0x7f5ef406e9f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5f140ccfc0 tx=0x7f5f040078f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:58.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:58.895+0000 7f5f0b7fe640 1 -- 172.21.15.148:0/3336192706 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5efc019030 con 0x7f5f000051c0 2024-03-29T04:10:59.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.078+0000 7f5f19235640 1 -- 172.21.15.148:0/3336192706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5f14068850 con 0x7f5f000051c0 2024-03-29T04:10:59.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.078+0000 7f5f0b7fe640 1 -- 172.21.15.148:0/3336192706 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f5efc05a210 con 0x7f5f000051c0 2024-03-29T04:10:59.057 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-29T04:10:59.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.081+0000 7f5f19235640 1 -- 172.21.15.148:0/3336192706 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5ef406c530 msgr2=0x7f5ef406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:59.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.081+0000 7f5f19235640 1 --2- 172.21.15.148:0/3336192706 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5ef406c530 0x7f5ef406e9f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5f140ccfc0 tx=0x7f5f040078f0 comp rx=0 tx=0).stop 2024-03-29T04:10:59.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.081+0000 7f5f19235640 1 -- 172.21.15.148:0/3336192706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f000051c0 msgr2=0x7f5f1415ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:59.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.081+0000 7f5f19235640 1 --2- 172.21.15.148:0/3336192706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f000051c0 0x7f5f1415ff40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5efc011940 tx=0x7f5efc011d40 comp rx=0 tx=0).stop 2024-03-29T04:10:59.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.081+0000 7f5f19235640 1 -- 172.21.15.148:0/3336192706 shutdown_connections 2024-03-29T04:10:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.081+0000 7f5f19235640 1 --2- 172.21.15.148:0/3336192706 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5ef406c530 0x7f5ef406e9f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.081+0000 7f5f19235640 1 --2- 172.21.15.148:0/3336192706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f140cd790 0x7f5f140cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.081+0000 7f5f19235640 1 --2- 172.21.15.148:0/3336192706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f1415d570 0x7f5f141530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.081+0000 7f5f19235640 1 --2- 172.21.15.148:0/3336192706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f000051c0 0x7f5f1415ff40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.081+0000 7f5f19235640 1 -- 172.21.15.148:0/3336192706 >> 172.21.15.148:0/3336192706 conn(0x7f5f140b6430 msgr2=0x7f5f140b54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.081+0000 7f5f19235640 1 -- 172.21.15.148:0/3336192706 shutdown_connections 2024-03-29T04:10:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.081+0000 7f5f19235640 1 -- 172.21.15.148:0/3336192706 wait complete. 2024-03-29T04:10:59.075 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:10:59.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.201+0000 7f909a693640 1 Processor -- start 2024-03-29T04:10:59.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.201+0000 7f909a693640 1 -- start start 2024-03-29T04:10:59.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.201+0000 7f909a693640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f90940ca2e0 0x7f909406fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:59.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.201+0000 7f909a693640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9094144f10 con 0x7f90940d02f0 2024-03-29T04:10:59.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.201+0000 7f909a693640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90940ca9c0 con 0x7f9094144b30 2024-03-29T04:10:59.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.201+0000 7f909a693640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f90940702c0 con 0x7f90940ca2e0 2024-03-29T04:10:59.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.202+0000 7f90937fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f90940ca2e0 0x7f909406fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:59.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.202+0000 7f9098c30640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f90940d02f0 0x7f90940ceaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39956/0 (socket says 172.21.15.148:39956) 2024-03-29T04:10:59.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.202+0000 7f90937fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f90940ca2e0 0x7f909406fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56070/0 (socket says 172.21.15.148:56070) 2024-03-29T04:10:59.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.202+0000 7f9098c30640 1 -- 172.21.15.148:0/2057455484 learned_addr learned my addr 172.21.15.148:0/2057455484 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:59.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.202+0000 7f9092ffd640 1 -- 172.21.15.148:0/2057455484 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2349766137 0 0) 0x7f9094144f10 con 0x7f90940d02f0 2024-03-29T04:10:59.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.202+0000 7f9092ffd640 1 -- 172.21.15.148:0/2057455484 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f907c003520 con 0x7f90940d02f0 2024-03-29T04:10:59.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.202+0000 7f90937fe640 1 -- 172.21.15.148:0/2057455484 >> v1:172.21.15.157:6789/0 conn(0x7f9094144b30 legacy=0x7f909415f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:59.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.202+0000 7f90937fe640 1 -- 172.21.15.148:0/2057455484 >> v1:172.21.15.148:6789/0 conn(0x7f90940d02f0 legacy=0x7f90940ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:59.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.202+0000 7f90937fe640 1 -- 172.21.15.148:0/2057455484 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90940caca0 con 0x7f90940ca2e0 2024-03-29T04:10:59.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.203+0000 7f90937fe640 1 --2- 172.21.15.148:0/2057455484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90940ca2e0 0x7f909406fd80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f90880097a0 tx=0x7f908802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8c022a0f957448 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:59.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.203+0000 7f9092ffd640 1 -- 172.21.15.148:0/2057455484 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088034070 con 0x7f90940ca2e0 2024-03-29T04:10:59.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.203+0000 7f9092ffd640 1 -- 172.21.15.148:0/2057455484 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9088030610 con 0x7f90940ca2e0 2024-03-29T04:10:59.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.203+0000 7f9092ffd640 1 -- 172.21.15.148:0/2057455484 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90880309e0 con 0x7f90940ca2e0 2024-03-29T04:10:59.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.204+0000 7f909a693640 1 -- 172.21.15.148:0/2057455484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90940ca2e0 msgr2=0x7f909406fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:59.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.204+0000 7f909a693640 1 --2- 172.21.15.148:0/2057455484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90940ca2e0 0x7f909406fd80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f90880097a0 tx=0x7f908802b960 comp rx=0 tx=0).stop 2024-03-29T04:10:59.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.204+0000 7f909a693640 1 -- 172.21.15.148:0/2057455484 shutdown_connections 2024-03-29T04:10:59.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.204+0000 7f909a693640 1 --2- 172.21.15.148:0/2057455484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90940ca2e0 0x7f909406fd80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:59.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.204+0000 7f909a693640 1 -- 172.21.15.148:0/2057455484 >> 172.21.15.148:0/2057455484 conn(0x7f90940b2c60 msgr2=0x7f90940b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:59.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.204+0000 7f909a693640 1 -- 172.21.15.148:0/2057455484 shutdown_connections 2024-03-29T04:10:59.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.204+0000 7f909a693640 1 -- 172.21.15.148:0/2057455484 wait complete. 2024-03-29T04:10:59.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.205+0000 7f909a693640 1 Processor -- start 2024-03-29T04:10:59.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.205+0000 7f909a693640 1 -- start start 2024-03-29T04:10:59.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.206+0000 7f909a693640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90940ca2e0 0x7f909413da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:59.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.206+0000 7f909a693640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90940d02f0 0x7f9094160ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:59.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.206+0000 7f909a693640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9094144b30 0x7f9094160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:59.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.206+0000 7f909a693640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f90940ca770 con 0x7f90940ca2e0 2024-03-29T04:10:59.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.206+0000 7f909a693640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f909415f090 con 0x7f90940d02f0 2024-03-29T04:10:59.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.206+0000 7f909a693640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f909415eac0 con 0x7f9094144b30 2024-03-29T04:10:59.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.206+0000 7f9093fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90940ca2e0 0x7f909413da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:59.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.206+0000 7f9093fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90940ca2e0 0x7f909413da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56078/0 (socket says 172.21.15.148:56078) 2024-03-29T04:10:59.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.206+0000 7f9093fff640 1 -- 172.21.15.148:0/1289608551 learned_addr learned my addr 172.21.15.148:0/1289608551 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:59.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.206+0000 7f90937fe640 1 --2- 172.21.15.148:0/1289608551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90940d02f0 0x7f9094160ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:59.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.206+0000 7f9098c30640 1 --2- 172.21.15.148:0/1289608551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9094144b30 0x7f9094160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:59.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.206+0000 7f9093fff640 1 -- 172.21.15.148:0/1289608551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9094144b30 msgr2=0x7f9094160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:59.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.206+0000 7f9093fff640 1 --2- 172.21.15.148:0/1289608551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9094144b30 0x7f9094160ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:59.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.206+0000 7f9093fff640 1 -- 172.21.15.148:0/1289608551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90940d02f0 msgr2=0x7f9094160ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:59.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.206+0000 7f9093fff640 1 --2- 172.21.15.148:0/1289608551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90940d02f0 0x7f9094160ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:59.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.206+0000 7f9093fff640 1 -- 172.21.15.148:0/1289608551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90941f4ba0 con 0x7f90940ca2e0 2024-03-29T04:10:59.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.206+0000 7f9093fff640 1 --2- 172.21.15.148:0/1289608551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90940ca2e0 0x7f909413da30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f908400c3c0 tx=0x7f908400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:59.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.207+0000 7f9090ff9640 1 -- 172.21.15.148:0/1289608551 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90840052d0 con 0x7f90940ca2e0 2024-03-29T04:10:59.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.207+0000 7f9090ff9640 1 -- 172.21.15.148:0/1289608551 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9084005490 con 0x7f90940ca2e0 2024-03-29T04:10:59.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.207+0000 7f9090ff9640 1 -- 172.21.15.148:0/1289608551 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9084007260 con 0x7f90940ca2e0 2024-03-29T04:10:59.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.207+0000 7f909a693640 1 -- 172.21.15.148:0/1289608551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9094150750 con 0x7f90940ca2e0 2024-03-29T04:10:59.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.207+0000 7f909a693640 1 -- 172.21.15.148:0/1289608551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f909413df70 con 0x7f90940ca2e0 2024-03-29T04:10:59.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.208+0000 7f909a693640 1 -- 172.21.15.148:0/1289608551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9094150540 con 0x7f90940ca2e0 2024-03-29T04:10:59.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.208+0000 7f9090ff9640 1 -- 172.21.15.148:0/1289608551 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9084007400 con 0x7f90940ca2e0 2024-03-29T04:10:59.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.209+0000 7f9090ff9640 1 --2- 172.21.15.148:0/1289608551 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f908006c530 0x7f908006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:59.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.209+0000 7f90937fe640 1 --2- 172.21.15.148:0/1289608551 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f908006c530 0x7f908006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:59.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.209+0000 7f9090ff9640 1 -- 172.21.15.148:0/1289608551 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f908408af10 con 0x7f90940ca2e0 2024-03-29T04:10:59.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.209+0000 7f90937fe640 1 --2- 172.21.15.148:0/1289608551 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f908006c530 0x7f908006e9f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9088036000 tx=0x7f908803b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:59.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.213+0000 7f9090ff9640 1 -- 172.21.15.148:0/1289608551 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f90840591c0 con 0x7f90940ca2e0 2024-03-29T04:10:59.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.421+0000 7f909a693640 1 -- 172.21.15.148:0/1289608551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f90941618d0 con 0x7f90940ca2e0 2024-03-29T04:10:59.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.421+0000 7f9090ff9640 1 -- 172.21.15.148:0/1289608551 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f9084058ac0 con 0x7f90940ca2e0 2024-03-29T04:10:59.401 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:10:59.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.423+0000 7f909a693640 1 -- 172.21.15.148:0/1289608551 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f908006c530 msgr2=0x7f908006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:59.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.423+0000 7f909a693640 1 --2- 172.21.15.148:0/1289608551 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f908006c530 0x7f908006e9f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9088036000 tx=0x7f908803b040 comp rx=0 tx=0).stop 2024-03-29T04:10:59.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.423+0000 7f909a693640 1 -- 172.21.15.148:0/1289608551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90940ca2e0 msgr2=0x7f909413da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:59.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.423+0000 7f909a693640 1 --2- 172.21.15.148:0/1289608551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90940ca2e0 0x7f909413da30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f908400c3c0 tx=0x7f908400c880 comp rx=0 tx=0).stop 2024-03-29T04:10:59.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.423+0000 7f909a693640 1 -- 172.21.15.148:0/1289608551 shutdown_connections 2024-03-29T04:10:59.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.423+0000 7f909a693640 1 --2- 172.21.15.148:0/1289608551 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f908006c530 0x7f908006e9f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:59.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.423+0000 7f909a693640 1 --2- 172.21.15.148:0/1289608551 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9094144b30 0x7f9094160ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:59.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.423+0000 7f909a693640 1 --2- 172.21.15.148:0/1289608551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90940d02f0 0x7f9094160ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:59.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.423+0000 7f909a693640 1 --2- 172.21.15.148:0/1289608551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90940ca2e0 0x7f909413da30 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:59.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.423+0000 7f909a693640 1 -- 172.21.15.148:0/1289608551 >> 172.21.15.148:0/1289608551 conn(0x7f90940b2c60 msgr2=0x7f90940b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:59.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.423+0000 7f909a693640 1 -- 172.21.15.148:0/1289608551 shutdown_connections 2024-03-29T04:10:59.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.423+0000 7f909a693640 1 -- 172.21.15.148:0/1289608551 wait complete. 2024-03-29T04:10:59.413 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:10:59.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.538+0000 7fc1310a5640 1 Processor -- start 2024-03-29T04:10:59.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.538+0000 7fc1310a5640 1 -- start start 2024-03-29T04:10:59.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.539+0000 7fc1310a5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc12c15e3c0 0x7fc12c161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:59.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.539+0000 7fc1310a5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc12c0b3430 0x7fc12c06b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:59.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.539+0000 7fc1310a5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc12c0cf330 con 0x7fc12c0b2f00 2024-03-29T04:10:59.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.539+0000 7fc1310a5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc12c0cf960 con 0x7fc12c0b3430 2024-03-29T04:10:59.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.539+0000 7fc1310a5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc12c162cf0 con 0x7fc12c15e3c0 2024-03-29T04:10:59.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.540+0000 7fc12b59e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc12c0b3430 0x7fc12c06b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:59.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.540+0000 7fc12ad9d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc12c15e3c0 0x7fc12c161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:59.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.540+0000 7fc12b59e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc12c0b3430 0x7fc12c06b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:56682/0 (socket says 172.21.15.148:56682) 2024-03-29T04:10:59.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.540+0000 7fc12b59e640 1 -- 172.21.15.148:0/4120606389 learned_addr learned my addr 172.21.15.148:0/4120606389 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:59.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.540+0000 7fc12ad9d640 1 -- 172.21.15.148:0/4120606389 >> v2:172.21.15.157:3300/0 conn(0x7fc12c0b3430 msgr2=0x7fc12c06b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:59.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.540+0000 7fc12ad9d640 1 --2- 172.21.15.148:0/4120606389 >> v2:172.21.15.157:3300/0 conn(0x7fc12c0b3430 0x7fc12c06b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:59.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.540+0000 7fc12ad9d640 1 -- 172.21.15.148:0/4120606389 >> v1:172.21.15.157:6789/0 conn(0x7fc12c0b2f00 legacy=0x7fc12c1624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:59.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.540+0000 7fc12ad9d640 1 -- 172.21.15.148:0/4120606389 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12c0cf5f0 con 0x7fc12c15e3c0 2024-03-29T04:10:59.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.540+0000 7fc12b59e640 1 --2- 172.21.15.148:0/4120606389 >> v2:172.21.15.157:3300/0 conn(0x7fc12c0b3430 0x7fc12c06b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:59.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.541+0000 7fc12ad9d640 1 --2- 172.21.15.148:0/4120606389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc12c15e3c0 0x7fc12c161f70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc1200097a0 tx=0x7fc12002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccfaeb0126ccd409 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:59.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.541+0000 7fc129d9b640 1 -- 172.21.15.148:0/4120606389 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1200315a0 con 0x7fc12c15e3c0 2024-03-29T04:10:59.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.541+0000 7fc129d9b640 1 -- 172.21.15.148:0/4120606389 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc120037070 con 0x7fc12c15e3c0 2024-03-29T04:10:59.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.541+0000 7fc129d9b640 1 -- 172.21.15.148:0/4120606389 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc120031960 con 0x7fc12c15e3c0 2024-03-29T04:10:59.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.541+0000 7fc1310a5640 1 -- 172.21.15.148:0/4120606389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc12c15e3c0 msgr2=0x7fc12c161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:59.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.541+0000 7fc1310a5640 1 --2- 172.21.15.148:0/4120606389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc12c15e3c0 0x7fc12c161f70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc1200097a0 tx=0x7fc12002b840 comp rx=0 tx=0).stop 2024-03-29T04:10:59.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.541+0000 7fc1310a5640 1 -- 172.21.15.148:0/4120606389 shutdown_connections 2024-03-29T04:10:59.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.541+0000 7fc1310a5640 1 --2- 172.21.15.148:0/4120606389 >> v2:172.21.15.157:3300/0 conn(0x7fc12c0b3430 0x7fc12c06b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:59.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.541+0000 7fc1310a5640 1 --2- 172.21.15.148:0/4120606389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc12c15e3c0 0x7fc12c161f70 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:59.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.541+0000 7fc1310a5640 1 -- 172.21.15.148:0/4120606389 >> 172.21.15.148:0/4120606389 conn(0x7fc12c0b6430 msgr2=0x7fc12c0d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:59.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.542+0000 7fc1310a5640 1 -- 172.21.15.148:0/4120606389 shutdown_connections 2024-03-29T04:10:59.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.542+0000 7fc1310a5640 1 -- 172.21.15.148:0/4120606389 wait complete. 2024-03-29T04:10:59.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.543+0000 7fc1310a5640 1 Processor -- start 2024-03-29T04:10:59.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.543+0000 7fc1310a5640 1 -- start start 2024-03-29T04:10:59.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.544+0000 7fc1310a5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc12c0b3430 0x7fc12c15e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:59.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.544+0000 7fc1310a5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc12c15e3c0 0x7fc12c0b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:59.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.544+0000 7fc1310a5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc12c14ca00 0x7fc12c14bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:59.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.544+0000 7fc1310a5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c0b3b20 con 0x7fc12c0b3430 2024-03-29T04:10:59.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.544+0000 7fc1310a5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c0b39a0 con 0x7fc12c14ca00 2024-03-29T04:10:59.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.544+0000 7fc1310a5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc12c06bd30 con 0x7fc12c15e3c0 2024-03-29T04:10:59.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.544+0000 7fc12ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc12c0b3430 0x7fc12c15e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:59.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.544+0000 7fc12ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc12c0b3430 0x7fc12c15e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56088/0 (socket says 172.21.15.148:56088) 2024-03-29T04:10:59.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.544+0000 7fc12ad9d640 1 -- 172.21.15.148:0/4005438378 learned_addr learned my addr 172.21.15.148:0/4005438378 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:59.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.544+0000 7fc12a59c640 1 --2- 172.21.15.148:0/4005438378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc12c15e3c0 0x7fc12c0b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:59.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.544+0000 7fc12b59e640 1 --2- 172.21.15.148:0/4005438378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc12c14ca00 0x7fc12c14bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:59.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.544+0000 7fc12ad9d640 1 -- 172.21.15.148:0/4005438378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc12c15e3c0 msgr2=0x7fc12c0b4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:59.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.544+0000 7fc12ad9d640 1 --2- 172.21.15.148:0/4005438378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc12c15e3c0 0x7fc12c0b4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:59.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.544+0000 7fc12ad9d640 1 -- 172.21.15.148:0/4005438378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc12c14ca00 msgr2=0x7fc12c14bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:59.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.544+0000 7fc12ad9d640 1 --2- 172.21.15.148:0/4005438378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc12c14ca00 0x7fc12c14bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:59.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.544+0000 7fc12ad9d640 1 -- 172.21.15.148:0/4005438378 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12c15f2b0 con 0x7fc12c0b3430 2024-03-29T04:10:59.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.544+0000 7fc12ad9d640 1 --2- 172.21.15.148:0/4005438378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc12c0b3430 0x7fc12c15e9b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc120000c00 tx=0x7fc1200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:59.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.544+0000 7fc1137fe640 1 -- 172.21.15.148:0/4005438378 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc120032630 con 0x7fc12c0b3430 2024-03-29T04:10:59.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.545+0000 7fc1137fe640 1 -- 172.21.15.148:0/4005438378 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1200327f0 con 0x7fc12c0b3430 2024-03-29T04:10:59.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.545+0000 7fc1310a5640 1 -- 172.21.15.148:0/4005438378 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc12c15f590 con 0x7fc12c0b3430 2024-03-29T04:10:59.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.545+0000 7fc1137fe640 1 -- 172.21.15.148:0/4005438378 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc120032be0 con 0x7fc12c0b3430 2024-03-29T04:10:59.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.545+0000 7fc1310a5640 1 -- 172.21.15.148:0/4005438378 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc12c15fb50 con 0x7fc12c0b3430 2024-03-29T04:10:59.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.546+0000 7fc1310a5640 1 -- 172.21.15.148:0/4005438378 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc12c068850 con 0x7fc12c0b3430 2024-03-29T04:10:59.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.546+0000 7fc1137fe640 1 -- 172.21.15.148:0/4005438378 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc12004d3f0 con 0x7fc12c0b3430 2024-03-29T04:10:59.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.546+0000 7fc1137fe640 1 --2- 172.21.15.148:0/4005438378 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc11806c530 0x7fc11806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:59.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.547+0000 7fc12a59c640 1 --2- 172.21.15.148:0/4005438378 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc11806c530 0x7fc11806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:59.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.547+0000 7fc1137fe640 1 -- 172.21.15.148:0/4005438378 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc12002f030 con 0x7fc12c0b3430 2024-03-29T04:10:59.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.547+0000 7fc12a59c640 1 --2- 172.21.15.148:0/4005438378 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc11806c530 0x7fc11806e9f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc12c0cb2d0 tx=0x7fc11c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:59.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.551+0000 7fc1137fe640 1 -- 172.21.15.148:0/4005438378 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc120040070 con 0x7fc12c0b3430 2024-03-29T04:10:59.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.730+0000 7fc1310a5640 1 -- 172.21.15.148:0/4005438378 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc12c14a330 con 0x7fc12c0b3430 2024-03-29T04:10:59.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.731+0000 7fc1137fe640 1 -- 172.21.15.148:0/4005438378 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fc12007b100 con 0x7fc12c0b3430 2024-03-29T04:10:59.710 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-29T04:10:59.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.733+0000 7fc1310a5640 1 -- 172.21.15.148:0/4005438378 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc11806c530 msgr2=0x7fc11806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:59.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.733+0000 7fc1310a5640 1 --2- 172.21.15.148:0/4005438378 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc11806c530 0x7fc11806e9f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc12c0cb2d0 tx=0x7fc11c009040 comp rx=0 tx=0).stop 2024-03-29T04:10:59.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.733+0000 7fc1310a5640 1 -- 172.21.15.148:0/4005438378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc12c0b3430 msgr2=0x7fc12c15e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:59.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.733+0000 7fc1310a5640 1 --2- 172.21.15.148:0/4005438378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc12c0b3430 0x7fc12c15e9b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc120000c00 tx=0x7fc1200046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:59.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.733+0000 7fc1310a5640 1 -- 172.21.15.148:0/4005438378 shutdown_connections 2024-03-29T04:10:59.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.733+0000 7fc1310a5640 1 --2- 172.21.15.148:0/4005438378 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc11806c530 0x7fc11806e9f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:59.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.733+0000 7fc1310a5640 1 --2- 172.21.15.148:0/4005438378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc12c14ca00 0x7fc12c14bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:59.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.733+0000 7fc1310a5640 1 --2- 172.21.15.148:0/4005438378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc12c15e3c0 0x7fc12c0b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:59.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.733+0000 7fc1310a5640 1 --2- 172.21.15.148:0/4005438378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc12c0b3430 0x7fc12c15e9b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:59.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.733+0000 7fc1310a5640 1 -- 172.21.15.148:0/4005438378 >> 172.21.15.148:0/4005438378 conn(0x7fc12c0b6430 msgr2=0x7fc12c0b5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:59.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.733+0000 7fc1310a5640 1 -- 172.21.15.148:0/4005438378 shutdown_connections 2024-03-29T04:10:59.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.733+0000 7fc1310a5640 1 -- 172.21.15.148:0/4005438378 wait complete. 2024-03-29T04:10:59.723 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:10:59.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.849+0000 7f68aab90640 1 Processor -- start 2024-03-29T04:10:59.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.849+0000 7f68aab90640 1 -- start start 2024-03-29T04:10:59.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.850+0000 7f68aab90640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f68a40ce110 0x7f68a40b7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:59.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.850+0000 7f68aab90640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68a40ce540 con 0x7f68a40cff50 2024-03-29T04:10:59.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.850+0000 7f68aab90640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68a40b5030 con 0x7f68a40cc6a0 2024-03-29T04:10:59.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.850+0000 7f68aab90640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f68a40ce7d0 con 0x7f68a40ce110 2024-03-29T04:10:59.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.850+0000 7f68a3fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f68a40cff50 0x7f68a415f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39960/0 (socket says 172.21.15.148:39960) 2024-03-29T04:10:59.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.850+0000 7f68a3fff640 1 -- 172.21.15.148:0/2767707826 learned_addr learned my addr 172.21.15.148:0/2767707826 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:10:59.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.850+0000 7f68a912d640 1 --2- 172.21.15.148:0/2767707826 >> v2:172.21.15.157:3300/0 conn(0x7f68a40ce110 0x7f68a40b7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:59.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.851+0000 7f68a37fe640 1 -- 172.21.15.148:0/2767707826 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3984303133 0 0) 0x7f68a40ce540 con 0x7f68a40cff50 2024-03-29T04:10:59.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.851+0000 7f68a37fe640 1 -- 172.21.15.148:0/2767707826 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f688c003520 con 0x7f68a40cff50 2024-03-29T04:10:59.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.851+0000 7f68a37fe640 1 -- 172.21.15.148:0/2767707826 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4113579651 0 0) 0x7f68a40b5030 con 0x7f68a40cc6a0 2024-03-29T04:10:59.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.851+0000 7f68a37fe640 1 -- 172.21.15.148:0/2767707826 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f68a40ce540 con 0x7f68a40cc6a0 2024-03-29T04:10:59.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.851+0000 7f68a912d640 1 -- 172.21.15.148:0/2767707826 >> v1:172.21.15.157:6789/0 conn(0x7f68a40cc6a0 legacy=0x7f68a40cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:59.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.851+0000 7f68a37fe640 1 -- 172.21.15.148:0/2767707826 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 692821735 0 0) 0x7f688c003520 con 0x7f68a40cff50 2024-03-29T04:10:59.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.851+0000 7f68a912d640 1 -- 172.21.15.148:0/2767707826 >> v1:172.21.15.148:6789/0 conn(0x7f68a40cff50 legacy=0x7f68a415f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:59.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.851+0000 7f68a912d640 1 -- 172.21.15.148:0/2767707826 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a40b5310 con 0x7f68a40ce110 2024-03-29T04:10:59.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.852+0000 7f68a912d640 1 --2- 172.21.15.148:0/2767707826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a40ce110 0x7f68a40b7220 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f68940097a0 tx=0x7f689402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d3d45d40e73b5a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:59.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.852+0000 7f68a37fe640 1 -- 172.21.15.148:0/2767707826 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6894037070 con 0x7f68a40ce110 2024-03-29T04:10:59.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.852+0000 7f68a37fe640 1 -- 172.21.15.148:0/2767707826 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6894030510 con 0x7f68a40ce110 2024-03-29T04:10:59.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.852+0000 7f68a37fe640 1 -- 172.21.15.148:0/2767707826 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6898001670 con 0x7f68a40ce110 2024-03-29T04:10:59.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.852+0000 7f68aab90640 1 -- 172.21.15.148:0/2767707826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a40ce110 msgr2=0x7f68a40b7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:59.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.852+0000 7f68aab90640 1 --2- 172.21.15.148:0/2767707826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a40ce110 0x7f68a40b7220 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f68940097a0 tx=0x7f689402b790 comp rx=0 tx=0).stop 2024-03-29T04:10:59.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.852+0000 7f68aab90640 1 -- 172.21.15.148:0/2767707826 shutdown_connections 2024-03-29T04:10:59.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.852+0000 7f68aab90640 1 --2- 172.21.15.148:0/2767707826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a40ce110 0x7f68a40b7220 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:59.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.852+0000 7f68aab90640 1 -- 172.21.15.148:0/2767707826 >> 172.21.15.148:0/2767707826 conn(0x7f68a40cead0 msgr2=0x7f68a40ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:59.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.852+0000 7f68aab90640 1 -- 172.21.15.148:0/2767707826 shutdown_connections 2024-03-29T04:10:59.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.853+0000 7f68aab90640 1 -- 172.21.15.148:0/2767707826 wait complete. 2024-03-29T04:10:59.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.854+0000 7f68aab90640 1 Processor -- start 2024-03-29T04:10:59.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.854+0000 7f68aab90640 1 -- start start 2024-03-29T04:10:59.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.856+0000 7f68aab90640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68a40cc6a0 0x7f68a4155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:59.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.856+0000 7f68aab90640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68a40ce110 0x7f68a40af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:59.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.856+0000 7f68aab90640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a40cff50 0x7f68a40afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:59.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.856+0000 7f68aab90640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f68a4158ee0 con 0x7f68a40ce110 2024-03-29T04:10:59.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.856+0000 7f68aab90640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f68a40b6d90 con 0x7f68a40cff50 2024-03-29T04:10:59.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.856+0000 7f68aab90640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f68a4159060 con 0x7f68a40cc6a0 2024-03-29T04:10:59.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.856+0000 7f68a3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68a40ce110 0x7f68a40af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:59.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.856+0000 7f68a3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68a40ce110 0x7f68a40af7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56092/0 (socket says 172.21.15.148:56092) 2024-03-29T04:10:59.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.856+0000 7f68a3fff640 1 -- 172.21.15.148:0/1157882586 learned_addr learned my addr 172.21.15.148:0/1157882586 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:10:59.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.856+0000 7f68a912d640 1 --2- 172.21.15.148:0/1157882586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a40cff50 0x7f68a40afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:59.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.856+0000 7f68a892c640 1 --2- 172.21.15.148:0/1157882586 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68a40cc6a0 0x7f68a4155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:59.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.856+0000 7f68a3fff640 1 -- 172.21.15.148:0/1157882586 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68a40cc6a0 msgr2=0x7f68a4155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:59.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.856+0000 7f68a3fff640 1 --2- 172.21.15.148:0/1157882586 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68a40cc6a0 0x7f68a4155380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:59.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.856+0000 7f68a3fff640 1 -- 172.21.15.148:0/1157882586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a40cff50 msgr2=0x7f68a40afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:59.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.856+0000 7f68a3fff640 1 --2- 172.21.15.148:0/1157882586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a40cff50 0x7f68a40afd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:59.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.856+0000 7f68a3fff640 1 -- 172.21.15.148:0/1157882586 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a40c9c60 con 0x7f68a40ce110 2024-03-29T04:10:59.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.856+0000 7f68a3fff640 1 --2- 172.21.15.148:0/1157882586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68a40ce110 0x7f68a40af7f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f689800c750 tx=0x7f689800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:59.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.856+0000 7f68a17fa640 1 -- 172.21.15.148:0/1157882586 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6898005ae0 con 0x7f68a40ce110 2024-03-29T04:10:59.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.857+0000 7f68a17fa640 1 -- 172.21.15.148:0/1157882586 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6898005c80 con 0x7f68a40ce110 2024-03-29T04:10:59.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.857+0000 7f68a17fa640 1 -- 172.21.15.148:0/1157882586 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68980072b0 con 0x7f68a40ce110 2024-03-29T04:10:59.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.857+0000 7f68aab90640 1 -- 172.21.15.148:0/1157882586 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a40c9f40 con 0x7f68a40ce110 2024-03-29T04:10:59.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.857+0000 7f68aab90640 1 -- 172.21.15.148:0/1157882586 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a40caa20 con 0x7f68a40ce110 2024-03-29T04:10:59.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.858+0000 7f68aab90640 1 -- 172.21.15.148:0/1157882586 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68a4068850 con 0x7f68a40ce110 2024-03-29T04:10:59.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.858+0000 7f68a17fa640 1 -- 172.21.15.148:0/1157882586 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6898007450 con 0x7f68a40ce110 2024-03-29T04:10:59.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.859+0000 7f68a17fa640 1 --2- 172.21.15.148:0/1157882586 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f689006c530 0x7f689006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:59.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.859+0000 7f68a892c640 1 --2- 172.21.15.148:0/1157882586 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f689006c530 0x7f689006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:59.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.859+0000 7f68a17fa640 1 -- 172.21.15.148:0/1157882586 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f689808b960 con 0x7f68a40ce110 2024-03-29T04:10:59.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.859+0000 7f68a892c640 1 --2- 172.21.15.148:0/1157882586 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f689006c530 0x7f689006e9f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f689c001c20 tx=0x7f689c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:59.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:10:59.863+0000 7f68a17fa640 1 -- 172.21.15.148:0/1157882586 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6898059cd0 con 0x7f68a40ce110 2024-03-29T04:11:00.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.040+0000 7f68aab90640 1 -- 172.21.15.148:0/1157882586 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f68a40d1cb0 con 0x7f68a40ce110 2024-03-29T04:11:00.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.040+0000 7f68a17fa640 1 -- 172.21.15.148:0/1157882586 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f68980595d0 con 0x7f68a40ce110 2024-03-29T04:11:00.019 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:11:00.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.042+0000 7f68aab90640 1 -- 172.21.15.148:0/1157882586 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f689006c530 msgr2=0x7f689006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:00.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.042+0000 7f68aab90640 1 --2- 172.21.15.148:0/1157882586 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f689006c530 0x7f689006e9f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f689c001c20 tx=0x7f689c009040 comp rx=0 tx=0).stop 2024-03-29T04:11:00.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.042+0000 7f68aab90640 1 -- 172.21.15.148:0/1157882586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68a40ce110 msgr2=0x7f68a40af7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:00.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.042+0000 7f68aab90640 1 --2- 172.21.15.148:0/1157882586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68a40ce110 0x7f68a40af7f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f689800c750 tx=0x7f689800cc10 comp rx=0 tx=0).stop 2024-03-29T04:11:00.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.042+0000 7f68aab90640 1 -- 172.21.15.148:0/1157882586 shutdown_connections 2024-03-29T04:11:00.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.042+0000 7f68aab90640 1 --2- 172.21.15.148:0/1157882586 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f689006c530 0x7f689006e9f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:00.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.042+0000 7f68aab90640 1 --2- 172.21.15.148:0/1157882586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a40cff50 0x7f68a40afd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:00.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.042+0000 7f68aab90640 1 --2- 172.21.15.148:0/1157882586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68a40ce110 0x7f68a40af7f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:00.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.042+0000 7f68aab90640 1 --2- 172.21.15.148:0/1157882586 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68a40cc6a0 0x7f68a4155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:00.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.042+0000 7f68aab90640 1 -- 172.21.15.148:0/1157882586 >> 172.21.15.148:0/1157882586 conn(0x7f68a40cead0 msgr2=0x7f68a40ceeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:00.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.042+0000 7f68aab90640 1 -- 172.21.15.148:0/1157882586 shutdown_connections 2024-03-29T04:11:00.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.042+0000 7f68aab90640 1 -- 172.21.15.148:0/1157882586 wait complete. 2024-03-29T04:11:00.032 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:11:00.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.157+0000 7efcd5d90640 1 Processor -- start 2024-03-29T04:11:00.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.157+0000 7efcd5d90640 1 -- start start 2024-03-29T04:11:00.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.158+0000 7efcd5d90640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcd015fd60 0x7efcd015eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:00.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.158+0000 7efcd5d90640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcd00cfe50 con 0x7efcd0070e20 2024-03-29T04:11:00.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.158+0000 7efcd5d90640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcd006dd30 con 0x7efcd0162b90 2024-03-29T04:11:00.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.158+0000 7efcd5d90640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efcd0160210 con 0x7efcd015fd60 2024-03-29T04:11:00.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.158+0000 7efccf7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7efcd0070e20 0x7efcd0071200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39964/0 (socket says 172.21.15.148:39964) 2024-03-29T04:11:00.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.158+0000 7efccf7fe640 1 -- 172.21.15.148:0/1286247021 learned_addr learned my addr 172.21.15.148:0/1286247021 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:00.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.158+0000 7efccffff640 1 --2- 172.21.15.148:0/1286247021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcd015fd60 0x7efcd015eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:00.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.158+0000 7efcce7fc640 1 -- 172.21.15.148:0/1286247021 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2679870721 0 0) 0x7efcd00cfe50 con 0x7efcd0070e20 2024-03-29T04:11:00.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.158+0000 7efcce7fc640 1 -- 172.21.15.148:0/1286247021 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcb8003520 con 0x7efcd0070e20 2024-03-29T04:11:00.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.158+0000 7efcce7fc640 1 -- 172.21.15.148:0/1286247021 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 844523425 0 0) 0x7efcd006dd30 con 0x7efcd0162b90 2024-03-29T04:11:00.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.158+0000 7efcce7fc640 1 -- 172.21.15.148:0/1286247021 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcd00cfe50 con 0x7efcd0162b90 2024-03-29T04:11:00.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.158+0000 7efccffff640 1 -- 172.21.15.148:0/1286247021 >> v1:172.21.15.157:6789/0 conn(0x7efcd0162b90 legacy=0x7efcd00d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:00.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.158+0000 7efccffff640 1 -- 172.21.15.148:0/1286247021 >> v1:172.21.15.148:6789/0 conn(0x7efcd0070e20 legacy=0x7efcd0071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:00.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.158+0000 7efccffff640 1 -- 172.21.15.148:0/1286247021 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd006e010 con 0x7efcd015fd60 2024-03-29T04:11:00.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.159+0000 7efccffff640 1 --2- 172.21.15.148:0/1286247021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcd015fd60 0x7efcd015eca0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efcc8009dc0 tx=0x7efcc802bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53ed9dd75a76437b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:00.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.159+0000 7efcce7fc640 1 -- 172.21.15.148:0/1286247021 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc8039070 con 0x7efcd015fd60 2024-03-29T04:11:00.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.159+0000 7efcce7fc640 1 -- 172.21.15.148:0/1286247021 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcc802ccd0 con 0x7efcd015fd60 2024-03-29T04:11:00.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.160+0000 7efcce7fc640 1 -- 172.21.15.148:0/1286247021 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc8034660 con 0x7efcd015fd60 2024-03-29T04:11:00.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.160+0000 7efcd5d90640 1 -- 172.21.15.148:0/1286247021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcd015fd60 msgr2=0x7efcd015eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:00.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.160+0000 7efcd5d90640 1 --2- 172.21.15.148:0/1286247021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcd015fd60 0x7efcd015eca0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efcc8009dc0 tx=0x7efcc802bf20 comp rx=0 tx=0).stop 2024-03-29T04:11:00.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.160+0000 7efcd5d90640 1 -- 172.21.15.148:0/1286247021 shutdown_connections 2024-03-29T04:11:00.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.160+0000 7efcd5d90640 1 --2- 172.21.15.148:0/1286247021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcd015fd60 0x7efcd015eca0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:00.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.160+0000 7efcd5d90640 1 -- 172.21.15.148:0/1286247021 >> 172.21.15.148:0/1286247021 conn(0x7efcd00d2100 msgr2=0x7efcd00d2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:00.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.160+0000 7efcd5d90640 1 -- 172.21.15.148:0/1286247021 shutdown_connections 2024-03-29T04:11:00.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.160+0000 7efcd5d90640 1 -- 172.21.15.148:0/1286247021 wait complete. 2024-03-29T04:11:00.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.161+0000 7efcd5d90640 1 Processor -- start 2024-03-29T04:11:00.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.161+0000 7efcd5d90640 1 -- start start 2024-03-29T04:11:00.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.162+0000 7efcd5d90640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efcd0070e20 0x7efcd00b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:00.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.162+0000 7efcd5d90640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcd015fd60 0x7efcd00c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:00.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.162+0000 7efcd5d90640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcd0162b90 0x7efcd00c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:00.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.162+0000 7efcd5d90640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efcd00d0570 con 0x7efcd0162b90 2024-03-29T04:11:00.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.162+0000 7efcd5d90640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efcd00d03f0 con 0x7efcd0070e20 2024-03-29T04:11:00.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.162+0000 7efcd5d90640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efcd015c280 con 0x7efcd015fd60 2024-03-29T04:11:00.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.162+0000 7efccffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcd0162b90 0x7efcd00c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:00.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.162+0000 7efccffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcd0162b90 0x7efcd00c83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56108/0 (socket says 172.21.15.148:56108) 2024-03-29T04:11:00.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.162+0000 7efccffff640 1 -- 172.21.15.148:0/1179099705 learned_addr learned my addr 172.21.15.148:0/1179099705 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:00.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.162+0000 7efccf7fe640 1 --2- 172.21.15.148:0/1179099705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efcd0070e20 0x7efcd00b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:00.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.162+0000 7efcceffd640 1 --2- 172.21.15.148:0/1179099705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcd015fd60 0x7efcd00c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:00.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.163+0000 7efccffff640 1 -- 172.21.15.148:0/1179099705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcd015fd60 msgr2=0x7efcd00c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:00.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.163+0000 7efccffff640 1 --2- 172.21.15.148:0/1179099705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcd015fd60 0x7efcd00c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:00.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.163+0000 7efccffff640 1 -- 172.21.15.148:0/1179099705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efcd0070e20 msgr2=0x7efcd00b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:00.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.163+0000 7efccffff640 1 --2- 172.21.15.148:0/1179099705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efcd0070e20 0x7efcd00b53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:00.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.163+0000 7efccffff640 1 -- 172.21.15.148:0/1179099705 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd006f410 con 0x7efcd0162b90 2024-03-29T04:11:00.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.163+0000 7efcceffd640 1 --2- 172.21.15.148:0/1179099705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcd015fd60 0x7efcd00c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:00.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.163+0000 7efccffff640 1 --2- 172.21.15.148:0/1179099705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcd0162b90 0x7efcd00c83c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7efcc8037040 tx=0x7efcc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:00.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.163+0000 7efcb7fff640 1 -- 172.21.15.148:0/1179099705 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc8039050 con 0x7efcd0162b90 2024-03-29T04:11:00.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.163+0000 7efcb7fff640 1 -- 172.21.15.148:0/1179099705 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcc8045070 con 0x7efcd0162b90 2024-03-29T04:11:00.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.163+0000 7efcd5d90640 1 -- 172.21.15.148:0/1179099705 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd006f6f0 con 0x7efcd0162b90 2024-03-29T04:11:00.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.163+0000 7efcb7fff640 1 -- 172.21.15.148:0/1179099705 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc8009450 con 0x7efcd0162b90 2024-03-29T04:11:00.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.163+0000 7efcd5d90640 1 -- 172.21.15.148:0/1179099705 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd006fcb0 con 0x7efcd0162b90 2024-03-29T04:11:00.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.164+0000 7efcd5d90640 1 -- 172.21.15.148:0/1179099705 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcd00687e0 con 0x7efcd0162b90 2024-03-29T04:11:00.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.164+0000 7efcb7fff640 1 -- 172.21.15.148:0/1179099705 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efcc803e850 con 0x7efcd0162b90 2024-03-29T04:11:00.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.165+0000 7efcb7fff640 1 --2- 172.21.15.148:0/1179099705 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efcbc06c530 0x7efcbc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:00.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.165+0000 7efcb7fff640 1 -- 172.21.15.148:0/1179099705 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7efcc80ae9a0 con 0x7efcd0162b90 2024-03-29T04:11:00.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.165+0000 7efccf7fe640 1 --2- 172.21.15.148:0/1179099705 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efcbc06c530 0x7efcbc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:00.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.165+0000 7efccf7fe640 1 --2- 172.21.15.148:0/1179099705 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efcbc06c530 0x7efcbc06e9f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efcc0007fd0 tx=0x7efcc0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:00.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.169+0000 7efcb7fff640 1 -- 172.21.15.148:0/1179099705 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efcc807cd70 con 0x7efcd0162b90 2024-03-29T04:11:00.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.494+0000 7efcd5d90640 1 -- 172.21.15.148:0/1179099705 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efcd01626f0 con 0x7efcd0162b90 2024-03-29T04:11:00.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.494+0000 7efcb7fff640 1 -- 172.21.15.148:0/1179099705 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7efcc807c670 con 0x7efcd0162b90 2024-03-29T04:11:00.473 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-03-29T04:11:00.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.496+0000 7efcd5d90640 1 -- 172.21.15.148:0/1179099705 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efcbc06c530 msgr2=0x7efcbc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:00.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.496+0000 7efcd5d90640 1 --2- 172.21.15.148:0/1179099705 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efcbc06c530 0x7efcbc06e9f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efcc0007fd0 tx=0x7efcc0009040 comp rx=0 tx=0).stop 2024-03-29T04:11:00.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.496+0000 7efcd5d90640 1 -- 172.21.15.148:0/1179099705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcd0162b90 msgr2=0x7efcd00c83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:00.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.496+0000 7efcd5d90640 1 --2- 172.21.15.148:0/1179099705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcd0162b90 0x7efcd00c83c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7efcc8037040 tx=0x7efcc8004a40 comp rx=0 tx=0).stop 2024-03-29T04:11:00.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.496+0000 7efcd5d90640 1 -- 172.21.15.148:0/1179099705 shutdown_connections 2024-03-29T04:11:00.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.496+0000 7efcd5d90640 1 --2- 172.21.15.148:0/1179099705 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efcbc06c530 0x7efcbc06e9f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:00.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.496+0000 7efcd5d90640 1 --2- 172.21.15.148:0/1179099705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcd0162b90 0x7efcd00c83c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:00.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.496+0000 7efcd5d90640 1 --2- 172.21.15.148:0/1179099705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efcd015fd60 0x7efcd00c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:00.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.496+0000 7efcd5d90640 1 --2- 172.21.15.148:0/1179099705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efcd0070e20 0x7efcd00b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:00.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.496+0000 7efcd5d90640 1 -- 172.21.15.148:0/1179099705 >> 172.21.15.148:0/1179099705 conn(0x7efcd00d2100 msgr2=0x7efcd015c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:00.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.496+0000 7efcd5d90640 1 -- 172.21.15.148:0/1179099705 shutdown_connections 2024-03-29T04:11:00.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.496+0000 7efcd5d90640 1 -- 172.21.15.148:0/1179099705 wait complete. 2024-03-29T04:11:00.486 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:11:00.486 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:11:00.506 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:11:00.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.631+0000 7f1ef9750640 1 Processor -- start 2024-03-29T04:11:00.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.631+0000 7f1ef9750640 1 -- start start 2024-03-29T04:11:00.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.631+0000 7f1ef9750640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1ef415f460 0x7f1ef406bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:00.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.631+0000 7f1ef9750640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ef415f8d0 con 0x7f1ef406ec50 2024-03-29T04:11:00.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.631+0000 7f1ef9750640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ef40725c0 con 0x7f1ef40b6140 2024-03-29T04:11:00.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.631+0000 7f1ef9750640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f1ef40728a0 con 0x7f1ef415f460 2024-03-29T04:11:00.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.632+0000 7f1ef27fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f1ef406ec50 0x7f1ef40b4690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39966/0 (socket says 172.21.15.148:39966) 2024-03-29T04:11:00.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.632+0000 7f1ef27fc640 1 -- 172.21.15.148:0/4184168468 learned_addr learned my addr 172.21.15.148:0/4184168468 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:00.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.632+0000 7f1ef37fe640 1 --2- 172.21.15.148:0/4184168468 >> v2:172.21.15.157:3300/0 conn(0x7f1ef415f460 0x7f1ef406bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:00.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.632+0000 7f1ef1ffb640 1 -- 172.21.15.148:0/4184168468 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 259040434 0 0) 0x7f1ef415f8d0 con 0x7f1ef406ec50 2024-03-29T04:11:00.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.632+0000 7f1ef1ffb640 1 -- 172.21.15.148:0/4184168468 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ee0003520 con 0x7f1ef406ec50 2024-03-29T04:11:00.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.632+0000 7f1ef1ffb640 1 -- 172.21.15.148:0/4184168468 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1044415489 0 0) 0x7f1ee0003520 con 0x7f1ef406ec50 2024-03-29T04:11:00.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.632+0000 7f1ef1ffb640 1 -- 172.21.15.148:0/4184168468 >> v2:172.21.15.157:3300/0 conn(0x7f1ef415f460 msgr2=0x7f1ef406bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:00.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.632+0000 7f1ef1ffb640 1 --2- 172.21.15.148:0/4184168468 >> v2:172.21.15.157:3300/0 conn(0x7f1ef415f460 0x7f1ef406bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:00.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.632+0000 7f1ef1ffb640 1 -- 172.21.15.148:0/4184168468 >> v1:172.21.15.157:6789/0 conn(0x7f1ef40b6140 legacy=0x7f1ef406e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:00.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.632+0000 7f1ef1ffb640 1 -- 172.21.15.148:0/4184168468 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef4072b80 con 0x7f1ef406ec50 2024-03-29T04:11:00.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.632+0000 7f1ef1ffb640 1 -- 172.21.15.148:0/4184168468 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f1edc001690 con 0x7f1ef406ec50 2024-03-29T04:11:00.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.632+0000 7f1ef1ffb640 1 -- 172.21.15.148:0/4184168468 >> v1:172.21.15.148:6789/0 conn(0x7f1ef406ec50 legacy=0x7f1ef40b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:00.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.632+0000 7f1ef1ffb640 1 --2- 172.21.15.148:0/4184168468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ee00039b0 0x7f1ee0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:00.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.632+0000 7f1ef1ffb640 1 --2- 172.21.15.148:0/4184168468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ee0004800 0x7f1ee0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:00.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.632+0000 7f1ef1ffb640 1 --2- 172.21.15.148:0/4184168468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ee00051c0 0x7f1ee000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:00.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.632+0000 7f1ef1ffb640 1 -- 172.21.15.148:0/4184168468 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef40728a0 con 0x7f1ee00039b0 2024-03-29T04:11:00.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.632+0000 7f1ef1ffb640 1 -- 172.21.15.148:0/4184168468 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee000c160 con 0x7f1ee00051c0 2024-03-29T04:11:00.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.632+0000 7f1ef1ffb640 1 -- 172.21.15.148:0/4184168468 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1ee000c360 con 0x7f1ee0004800 2024-03-29T04:11:00.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.632+0000 7f1ef2ffd640 1 --2- 172.21.15.148:0/4184168468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ee00039b0 0x7f1ee0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:00.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.633+0000 7f1ef37fe640 1 --2- 172.21.15.148:0/4184168468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ee00051c0 0x7f1ee000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:00.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.633+0000 7f1ef27fc640 1 --2- 172.21.15.148:0/4184168468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ee0004800 0x7f1ee0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:00.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.633+0000 7f1ef2ffd640 1 -- 172.21.15.148:0/4184168468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ee0004800 msgr2=0x7f1ee0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:00.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.633+0000 7f1ef2ffd640 1 --2- 172.21.15.148:0/4184168468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ee0004800 0x7f1ee0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:00.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.633+0000 7f1ef2ffd640 1 -- 172.21.15.148:0/4184168468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ee00051c0 msgr2=0x7f1ee000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:00.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.633+0000 7f1ef2ffd640 1 --2- 172.21.15.148:0/4184168468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ee00051c0 0x7f1ee000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:00.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.633+0000 7f1ef2ffd640 1 -- 172.21.15.148:0/4184168468 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee0003520 con 0x7f1ee00039b0 2024-03-29T04:11:00.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.633+0000 7f1ef37fe640 1 --2- 172.21.15.148:0/4184168468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ee00051c0 0x7f1ee000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:00.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.633+0000 7f1ef37fe640 1 -- 172.21.15.148:0/4184168468 reap_dead start 2024-03-29T04:11:00.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.633+0000 7f1ef2ffd640 1 --2- 172.21.15.148:0/4184168468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ee00039b0 0x7f1ee0003dd0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1eec001e50 tx=0x7f1ee0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1335d13c1b3651 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:00.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.634+0000 7f1ef1ffb640 1 -- 172.21.15.148:0/4184168468 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eec001a70 con 0x7f1ee00039b0 2024-03-29T04:11:00.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.634+0000 7f1ef1ffb640 1 -- 172.21.15.148:0/4184168468 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eec003b70 con 0x7f1ee00039b0 2024-03-29T04:11:00.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.634+0000 7f1ef1ffb640 1 -- 172.21.15.148:0/4184168468 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eec0050d0 con 0x7f1ee00039b0 2024-03-29T04:11:00.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.634+0000 7f1ef9750640 1 -- 172.21.15.148:0/4184168468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ee00039b0 msgr2=0x7f1ee0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:00.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.634+0000 7f1ef9750640 1 --2- 172.21.15.148:0/4184168468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ee00039b0 0x7f1ee0003dd0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1eec001e50 tx=0x7f1ee0005830 comp rx=0 tx=0).stop 2024-03-29T04:11:00.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.634+0000 7f1ef9750640 1 -- 172.21.15.148:0/4184168468 shutdown_connections 2024-03-29T04:11:00.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.634+0000 7f1ef9750640 1 --2- 172.21.15.148:0/4184168468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ee00039b0 0x7f1ee0003dd0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:00.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.634+0000 7f1ef9750640 1 -- 172.21.15.148:0/4184168468 >> 172.21.15.148:0/4184168468 conn(0x7f1ef40d11b0 msgr2=0x7f1ef40d15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:00.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.634+0000 7f1ef9750640 1 -- 172.21.15.148:0/4184168468 shutdown_connections 2024-03-29T04:11:00.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.634+0000 7f1ef9750640 1 -- 172.21.15.148:0/4184168468 wait complete. 2024-03-29T04:11:00.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.635+0000 7f1ef9750640 1 Processor -- start 2024-03-29T04:11:00.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.635+0000 7f1ef9750640 1 -- start start 2024-03-29T04:11:00.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.636+0000 7f1ef9750640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ee00039b0 0x7f1ef40cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:00.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.636+0000 7f1ef9750640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ef4156780 0x7f1ef41f4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:00.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.636+0000 7f1ef9750640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ef41f49a0 0x7f1ef41e9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:00.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.636+0000 7f1ef9750640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef406c670 con 0x7f1ef4156780 2024-03-29T04:11:00.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.636+0000 7f1ef9750640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef4073380 con 0x7f1ee00039b0 2024-03-29T04:11:00.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.636+0000 7f1ef9750640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1ef406c7f0 con 0x7f1ef41f49a0 2024-03-29T04:11:00.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.636+0000 7f1ef27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ef4156780 0x7f1ef41f4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:00.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.636+0000 7f1ef27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ef4156780 0x7f1ef41f4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56118/0 (socket says 172.21.15.148:56118) 2024-03-29T04:11:00.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.636+0000 7f1ef27fc640 1 -- 172.21.15.148:0/2231649069 learned_addr learned my addr 172.21.15.148:0/2231649069 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:00.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.636+0000 7f1ef2ffd640 1 --2- 172.21.15.148:0/2231649069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ee00039b0 0x7f1ef40cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:00.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.636+0000 7f1ef37fe640 1 --2- 172.21.15.148:0/2231649069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ef41f49a0 0x7f1ef41e9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:00.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.637+0000 7f1ef27fc640 1 -- 172.21.15.148:0/2231649069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ef41f49a0 msgr2=0x7f1ef41e9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:00.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.637+0000 7f1ef27fc640 1 --2- 172.21.15.148:0/2231649069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ef41f49a0 0x7f1ef41e9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:00.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.637+0000 7f1ef27fc640 1 -- 172.21.15.148:0/2231649069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ee00039b0 msgr2=0x7f1ef40cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:00.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.637+0000 7f1ef27fc640 1 --2- 172.21.15.148:0/2231649069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ee00039b0 0x7f1ef40cffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:00.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.637+0000 7f1ef27fc640 1 -- 172.21.15.148:0/2231649069 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef40cd2c0 con 0x7f1ef4156780 2024-03-29T04:11:00.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.637+0000 7f1ef27fc640 1 --2- 172.21.15.148:0/2231649069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ef4156780 0x7f1ef41f4460 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1edc001880 tx=0x7f1edc00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:00.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.637+0000 7f1eeb7fe640 1 -- 172.21.15.148:0/2231649069 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc00ee60 con 0x7f1ef4156780 2024-03-29T04:11:00.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.637+0000 7f1eeb7fe640 1 -- 172.21.15.148:0/2231649069 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1edc018070 con 0x7f1ef4156780 2024-03-29T04:11:00.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.637+0000 7f1ef9750640 1 -- 172.21.15.148:0/2231649069 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ef40cd5a0 con 0x7f1ef4156780 2024-03-29T04:11:00.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.637+0000 7f1eeb7fe640 1 -- 172.21.15.148:0/2231649069 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc00ca90 con 0x7f1ef4156780 2024-03-29T04:11:00.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.637+0000 7f1ef9750640 1 -- 172.21.15.148:0/2231649069 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ef40cdb60 con 0x7f1ef4156780 2024-03-29T04:11:00.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.638+0000 7f1eeb7fe640 1 -- 172.21.15.148:0/2231649069 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1edc00cc30 con 0x7f1ef4156780 2024-03-29T04:11:00.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.638+0000 7f1ef9750640 1 -- 172.21.15.148:0/2231649069 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ef40687e0 con 0x7f1ef4156780 2024-03-29T04:11:00.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.639+0000 7f1eeb7fe640 1 --2- 172.21.15.148:0/2231649069 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1ed406c560 0x7f1ed406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:00.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.639+0000 7f1eeb7fe640 1 -- 172.21.15.148:0/2231649069 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f1edc08b490 con 0x7f1ef4156780 2024-03-29T04:11:00.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.639+0000 7f1ef2ffd640 1 --2- 172.21.15.148:0/2231649069 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1ed406c560 0x7f1ed406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:00.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.639+0000 7f1ef2ffd640 1 --2- 172.21.15.148:0/2231649069 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1ed406c560 0x7f1ed406ea20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1eec034000 tx=0x7f1eec03b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:00.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.643+0000 7f1eeb7fe640 1 -- 172.21.15.148:0/2231649069 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1edc059060 con 0x7f1ef4156780 2024-03-29T04:11:00.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.829+0000 7f1ef9750640 1 -- 172.21.15.148:0/2231649069 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ef406e6b0 con 0x7f1ef4156780 2024-03-29T04:11:00.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.831+0000 7f1eeb7fe640 1 -- 172.21.15.148:0/2231649069 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18858 (secure 0 0 0) 0x7f1edc05d360 con 0x7f1ef4156780 2024-03-29T04:11:00.810 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:11:00.810 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:41.662934+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:11:00.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.832+0000 7f1ef9750640 1 -- 172.21.15.148:0/2231649069 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1ed406c560 msgr2=0x7f1ed406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:00.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.832+0000 7f1ef9750640 1 --2- 172.21.15.148:0/2231649069 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1ed406c560 0x7f1ed406ea20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1eec034000 tx=0x7f1eec03b040 comp rx=0 tx=0).stop 2024-03-29T04:11:00.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.832+0000 7f1ef9750640 1 -- 172.21.15.148:0/2231649069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ef4156780 msgr2=0x7f1ef41f4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:00.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.832+0000 7f1ef9750640 1 --2- 172.21.15.148:0/2231649069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ef4156780 0x7f1ef41f4460 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1edc001880 tx=0x7f1edc00ec10 comp rx=0 tx=0).stop 2024-03-29T04:11:00.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.833+0000 7f1ef9750640 1 -- 172.21.15.148:0/2231649069 shutdown_connections 2024-03-29T04:11:00.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.833+0000 7f1ef9750640 1 --2- 172.21.15.148:0/2231649069 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1ed406c560 0x7f1ed406ea20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:00.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.833+0000 7f1ef9750640 1 --2- 172.21.15.148:0/2231649069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ef41f49a0 0x7f1ef41e9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:00.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.833+0000 7f1ef9750640 1 --2- 172.21.15.148:0/2231649069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ef4156780 0x7f1ef41f4460 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:00.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.833+0000 7f1ef9750640 1 --2- 172.21.15.148:0/2231649069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ee00039b0 0x7f1ef40cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:00.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.833+0000 7f1ef9750640 1 -- 172.21.15.148:0/2231649069 >> 172.21.15.148:0/2231649069 conn(0x7f1ef40d11b0 msgr2=0x7f1ef40b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:00.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.833+0000 7f1ef9750640 1 -- 172.21.15.148:0/2231649069 shutdown_connections 2024-03-29T04:11:00.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.833+0000 7f1ef9750640 1 -- 172.21.15.148:0/2231649069 wait complete. 2024-03-29T04:11:00.823 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:47.898351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:57.922313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:58.829405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:10:02.551763+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:11:00.824 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:11:00.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.950+0000 7f64f099d640 1 Processor -- start 2024-03-29T04:11:00.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.950+0000 7f64f099d640 1 -- start start 2024-03-29T04:11:00.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.951+0000 7f64f099d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64ec0b9cb0 0x7f64ec0ced60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:00.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.951+0000 7f64f099d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f64ec0cf2a0 0x7f64ec06bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:00.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.951+0000 7f64f099d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64ec0b7220 con 0x7f64ec15e1f0 2024-03-29T04:11:00.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.951+0000 7f64f099d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f64ec06b5b0 con 0x7f64ec0cf2a0 2024-03-29T04:11:00.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.951+0000 7f64f099d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f64ec0d36c0 con 0x7f64ec0b9cb0 2024-03-29T04:11:00.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.951+0000 7f64eaffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64ec0b9cb0 0x7f64ec0ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:00.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.951+0000 7f64ebfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f64ec0cf2a0 0x7f64ec06bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:00.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.952+0000 7f64eaffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64ec0b9cb0 0x7f64ec0ced60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.148:34392/0 (socket says 172.21.15.148:34392) 2024-03-29T04:11:00.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.952+0000 7f64eaffd640 1 -- 172.21.15.148:0/84067788 learned_addr learned my addr 172.21.15.148:0/84067788 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:00.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.952+0000 7f64eb7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f64ec15e1f0 0x7f64ec0d2e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.148:49450/0 (socket says 172.21.15.148:49450) 2024-03-29T04:11:00.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.952+0000 7f64ea7fc640 1 -- 172.21.15.148:0/84067788 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 466747064 0 0) 0x7f64dc001110 con 0x7f64ec15e1f0 2024-03-29T04:11:00.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.952+0000 7f64ea7fc640 1 -- 172.21.15.148:0/84067788 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64d4003520 con 0x7f64ec15e1f0 2024-03-29T04:11:00.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.952+0000 7f64ebfff640 1 -- 172.21.15.148:0/84067788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64ec0b9cb0 msgr2=0x7f64ec0ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:00.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.952+0000 7f64ebfff640 1 --2- 172.21.15.148:0/84067788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64ec0b9cb0 0x7f64ec0ced60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:00.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.952+0000 7f64ebfff640 1 -- 172.21.15.148:0/84067788 >> v1:172.21.15.157:6789/0 conn(0x7f64ec15e1f0 legacy=0x7f64ec0d2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:00.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.952+0000 7f64ebfff640 1 -- 172.21.15.148:0/84067788 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ec0d39a0 con 0x7f64ec0cf2a0 2024-03-29T04:11:00.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.952+0000 7f64eaffd640 1 --2- 172.21.15.148:0/84067788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64ec0b9cb0 0x7f64ec0ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:11:00.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.953+0000 7f64ebfff640 1 --2- 172.21.15.148:0/84067788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ec0cf2a0 0x7f64ec06bd70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009d90 tx=0x7f64e402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bfa3b0c828ed23d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:00.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.953+0000 7f64ea7fc640 1 -- 172.21.15.148:0/84067788 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4039070 con 0x7f64ec0cf2a0 2024-03-29T04:11:00.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.953+0000 7f64ea7fc640 1 -- 172.21.15.148:0/84067788 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64e402cc50 con 0x7f64ec0cf2a0 2024-03-29T04:11:00.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.953+0000 7f64ea7fc640 1 -- 172.21.15.148:0/84067788 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4034440 con 0x7f64ec0cf2a0 2024-03-29T04:11:00.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.954+0000 7f64f099d640 1 -- 172.21.15.148:0/84067788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ec0cf2a0 msgr2=0x7f64ec06bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:00.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.954+0000 7f64f099d640 1 --2- 172.21.15.148:0/84067788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ec0cf2a0 0x7f64ec06bd70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009d90 tx=0x7f64e402bea0 comp rx=0 tx=0).stop 2024-03-29T04:11:00.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.954+0000 7f64f099d640 1 -- 172.21.15.148:0/84067788 shutdown_connections 2024-03-29T04:11:00.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.954+0000 7f64f099d640 1 --2- 172.21.15.148:0/84067788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ec0cf2a0 0x7f64ec06bd70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:00.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.954+0000 7f64f099d640 1 --2- 172.21.15.148:0/84067788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64ec0b9cb0 0x7f64ec0ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:00.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.954+0000 7f64f099d640 1 -- 172.21.15.148:0/84067788 >> 172.21.15.148:0/84067788 conn(0x7f64ec0d4090 msgr2=0x7f64ec0d44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:00.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.954+0000 7f64f099d640 1 -- 172.21.15.148:0/84067788 shutdown_connections 2024-03-29T04:11:00.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.954+0000 7f64f099d640 1 -- 172.21.15.148:0/84067788 wait complete. 2024-03-29T04:11:00.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.955+0000 7f64f099d640 1 Processor -- start 2024-03-29T04:11:00.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.955+0000 7f64f099d640 1 -- start start 2024-03-29T04:11:00.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.956+0000 7f64f099d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64ec0b9cb0 0x7f64ec0c79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:00.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.956+0000 7f64f099d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64ec0cf2a0 0x7f64ec1f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:00.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.956+0000 7f64f099d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ec15e1f0 0x7f64ec144bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:00.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.956+0000 7f64f099d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec15e5d0 con 0x7f64ec0b9cb0 2024-03-29T04:11:00.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.956+0000 7f64f099d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec06b850 con 0x7f64ec15e1f0 2024-03-29T04:11:00.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.956+0000 7f64f099d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f64ec06c2b0 con 0x7f64ec0cf2a0 2024-03-29T04:11:00.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.956+0000 7f64eb7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64ec0b9cb0 0x7f64ec0c79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:00.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.956+0000 7f64eb7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64ec0b9cb0 0x7f64ec0c79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56128/0 (socket says 172.21.15.148:56128) 2024-03-29T04:11:00.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.956+0000 7f64eb7fe640 1 -- 172.21.15.148:0/986745637 learned_addr learned my addr 172.21.15.148:0/986745637 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:00.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.956+0000 7f64ebfff640 1 --2- 172.21.15.148:0/986745637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ec15e1f0 0x7f64ec144bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:00.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.956+0000 7f64eaffd640 1 --2- 172.21.15.148:0/986745637 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64ec0cf2a0 0x7f64ec1f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:00.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.956+0000 7f64eb7fe640 1 -- 172.21.15.148:0/986745637 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64ec0cf2a0 msgr2=0x7f64ec1f3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:00.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.956+0000 7f64eb7fe640 1 --2- 172.21.15.148:0/986745637 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64ec0cf2a0 0x7f64ec1f3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:00.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.956+0000 7f64eb7fe640 1 -- 172.21.15.148:0/986745637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ec15e1f0 msgr2=0x7f64ec144bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:00.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.956+0000 7f64eb7fe640 1 --2- 172.21.15.148:0/986745637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ec15e1f0 0x7f64ec144bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:00.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.956+0000 7f64eb7fe640 1 -- 172.21.15.148:0/986745637 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ec0ca8d0 con 0x7f64ec0b9cb0 2024-03-29T04:11:00.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.957+0000 7f64eb7fe640 1 --2- 172.21.15.148:0/986745637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64ec0b9cb0 0x7f64ec0c79f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00c3c0 tx=0x7f64dc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:00.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.957+0000 7f64d3fff640 1 -- 172.21.15.148:0/986745637 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc0074e0 con 0x7f64ec0b9cb0 2024-03-29T04:11:00.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.957+0000 7f64d3fff640 1 -- 172.21.15.148:0/986745637 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64dc0047e0 con 0x7f64ec0b9cb0 2024-03-29T04:11:00.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.957+0000 7f64f099d640 1 -- 172.21.15.148:0/986745637 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64ec156740 con 0x7f64ec0b9cb0 2024-03-29T04:11:00.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.957+0000 7f64d3fff640 1 -- 172.21.15.148:0/986745637 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc004bd0 con 0x7f64ec0b9cb0 2024-03-29T04:11:00.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.957+0000 7f64f099d640 1 -- 172.21.15.148:0/986745637 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64ec156d00 con 0x7f64ec0b9cb0 2024-03-29T04:11:00.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.958+0000 7f64f099d640 1 -- 172.21.15.148:0/986745637 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64ec15a350 con 0x7f64ec0b9cb0 2024-03-29T04:11:00.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.958+0000 7f64d3fff640 1 -- 172.21.15.148:0/986745637 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f64dc01a020 con 0x7f64ec0b9cb0 2024-03-29T04:11:00.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.959+0000 7f64d3fff640 1 --2- 172.21.15.148:0/986745637 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f64d806c530 0x7f64d806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:00.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.959+0000 7f64eaffd640 1 --2- 172.21.15.148:0/986745637 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f64d806c530 0x7f64d806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:00.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.959+0000 7f64d3fff640 1 -- 172.21.15.148:0/986745637 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f64dc08b220 con 0x7f64ec0b9cb0 2024-03-29T04:11:00.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.959+0000 7f64eaffd640 1 --2- 172.21.15.148:0/986745637 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f64d806c530 0x7f64d806e9f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f64ec0b94f0 tx=0x7f64e0009240 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:00.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:00.963+0000 7f64d3fff640 1 -- 172.21.15.148:0/986745637 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f64ec15a350 con 0x7f64ec0b9cb0 2024-03-29T04:11:01.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.143+0000 7f64f099d640 1 -- 172.21.15.148:0/986745637 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f64ec0685c0 con 0x7f64ec0b9cb0 2024-03-29T04:11:01.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.144+0000 7f64d3fff640 1 -- 172.21.15.148:0/986745637 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f64ec0b7220 con 0x7f64ec0b9cb0 2024-03-29T04:11:01.123 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-29T04:11:01.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.146+0000 7f64f099d640 1 -- 172.21.15.148:0/986745637 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f64d806c530 msgr2=0x7f64d806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:01.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.146+0000 7f64f099d640 1 --2- 172.21.15.148:0/986745637 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f64d806c530 0x7f64d806e9f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f64ec0b94f0 tx=0x7f64e0009240 comp rx=0 tx=0).stop 2024-03-29T04:11:01.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.146+0000 7f64f099d640 1 -- 172.21.15.148:0/986745637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64ec0b9cb0 msgr2=0x7f64ec0c79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:01.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.146+0000 7f64f099d640 1 --2- 172.21.15.148:0/986745637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64ec0b9cb0 0x7f64ec0c79f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00c3c0 tx=0x7f64dc00c880 comp rx=0 tx=0).stop 2024-03-29T04:11:01.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.146+0000 7f64f099d640 1 -- 172.21.15.148:0/986745637 shutdown_connections 2024-03-29T04:11:01.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.146+0000 7f64f099d640 1 --2- 172.21.15.148:0/986745637 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f64d806c530 0x7f64d806e9f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.146+0000 7f64f099d640 1 --2- 172.21.15.148:0/986745637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ec15e1f0 0x7f64ec144bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.146+0000 7f64f099d640 1 --2- 172.21.15.148:0/986745637 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64ec0cf2a0 0x7f64ec1f3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.146+0000 7f64f099d640 1 --2- 172.21.15.148:0/986745637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64ec0b9cb0 0x7f64ec0c79f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.146+0000 7f64f099d640 1 -- 172.21.15.148:0/986745637 >> 172.21.15.148:0/986745637 conn(0x7f64ec0d4090 msgr2=0x7f64ec158260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:01.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.146+0000 7f64f099d640 1 -- 172.21.15.148:0/986745637 shutdown_connections 2024-03-29T04:11:01.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.146+0000 7f64f099d640 1 -- 172.21.15.148:0/986745637 wait complete. 2024-03-29T04:11:01.139 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:11:01.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.295+0000 7f730f96d640 1 Processor -- start 2024-03-29T04:11:01.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.296+0000 7f730f96d640 1 -- start start 2024-03-29T04:11:01.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.296+0000 7f730f96d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f730815e3c0 0x7f7308161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:01.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.296+0000 7f730f96d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73080b3430 0x7f730806b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:01.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.296+0000 7f730f96d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73080cf330 con 0x7f73080b2f00 2024-03-29T04:11:01.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.296+0000 7f730f96d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f73080cf960 con 0x7f730815e3c0 2024-03-29T04:11:01.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.296+0000 7f730f96d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7308162cf0 con 0x7f73080b3430 2024-03-29T04:11:01.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.297+0000 7f730cf08640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f73080b2f00 0x7f73081624b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39976/0 (socket says 172.21.15.148:39976) 2024-03-29T04:11:01.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.297+0000 7f730cf08640 1 -- 172.21.15.148:0/4184999975 learned_addr learned my addr 172.21.15.148:0/4184999975 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:01.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.297+0000 7f730d709640 1 --2- 172.21.15.148:0/4184999975 >> v2:172.21.15.157:3300/0 conn(0x7f730815e3c0 0x7f7308161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:01.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.297+0000 7f730df0a640 1 --2- 172.21.15.148:0/4184999975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73080b3430 0x7f730806b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:01.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.297+0000 7f72fbfff640 1 -- 172.21.15.148:0/4184999975 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2379654384 0 0) 0x7f73080cf330 con 0x7f73080b2f00 2024-03-29T04:11:01.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.297+0000 7f72fbfff640 1 -- 172.21.15.148:0/4184999975 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72f4003520 con 0x7f73080b2f00 2024-03-29T04:11:01.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.297+0000 7f730d709640 1 -- 172.21.15.148:0/4184999975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73080b3430 msgr2=0x7f730806b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:01.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.297+0000 7f730d709640 1 --2- 172.21.15.148:0/4184999975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73080b3430 0x7f730806b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.297+0000 7f730d709640 1 -- 172.21.15.148:0/4184999975 >> v1:172.21.15.148:6789/0 conn(0x7f73080b2f00 legacy=0x7f73081624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:01.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.297+0000 7f730d709640 1 -- 172.21.15.148:0/4184999975 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73080cf5f0 con 0x7f730815e3c0 2024-03-29T04:11:01.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.298+0000 7f730d709640 1 --2- 172.21.15.148:0/4184999975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f730815e3c0 0x7f7308161f70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0097a0 tx=0x7f72fc02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75378590cb678f65 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:01.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.298+0000 7f72fbfff640 1 -- 172.21.15.148:0/4184999975 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc0315a0 con 0x7f730815e3c0 2024-03-29T04:11:01.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.298+0000 7f72fbfff640 1 -- 172.21.15.148:0/4184999975 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72fc037070 con 0x7f730815e3c0 2024-03-29T04:11:01.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.298+0000 7f72fbfff640 1 -- 172.21.15.148:0/4184999975 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc031960 con 0x7f730815e3c0 2024-03-29T04:11:01.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.298+0000 7f730f96d640 1 -- 172.21.15.148:0/4184999975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f730815e3c0 msgr2=0x7f7308161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:01.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.298+0000 7f730f96d640 1 --2- 172.21.15.148:0/4184999975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f730815e3c0 0x7f7308161f70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0097a0 tx=0x7f72fc02b840 comp rx=0 tx=0).stop 2024-03-29T04:11:01.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.298+0000 7f730f96d640 1 -- 172.21.15.148:0/4184999975 shutdown_connections 2024-03-29T04:11:01.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.298+0000 7f730f96d640 1 --2- 172.21.15.148:0/4184999975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73080b3430 0x7f730806b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.298+0000 7f730f96d640 1 --2- 172.21.15.148:0/4184999975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f730815e3c0 0x7f7308161f70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.298+0000 7f730f96d640 1 -- 172.21.15.148:0/4184999975 >> 172.21.15.148:0/4184999975 conn(0x7f73080b6430 msgr2=0x7f73080d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:01.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.299+0000 7f730f96d640 1 -- 172.21.15.148:0/4184999975 shutdown_connections 2024-03-29T04:11:01.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.299+0000 7f730f96d640 1 -- 172.21.15.148:0/4184999975 wait complete. 2024-03-29T04:11:01.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.300+0000 7f730f96d640 1 Processor -- start 2024-03-29T04:11:01.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.300+0000 7f730f96d640 1 -- start start 2024-03-29T04:11:01.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.300+0000 7f730f96d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73080b3430 0x7f73081f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:01.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.300+0000 7f730f96d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f730815e3c0 0x7f73080b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:01.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.300+0000 7f730f96d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73080b5090 0x7f73080c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:01.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.300+0000 7f730f96d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f73080b3b20 con 0x7f73080b3430 2024-03-29T04:11:01.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.300+0000 7f730f96d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f73080b39a0 con 0x7f73080b5090 2024-03-29T04:11:01.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.301+0000 7f730f96d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f730806bd30 con 0x7f730815e3c0 2024-03-29T04:11:01.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.301+0000 7f730d709640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73080b3430 0x7f73081f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:01.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.301+0000 7f730d709640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73080b3430 0x7f73081f6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56138/0 (socket says 172.21.15.148:56138) 2024-03-29T04:11:01.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.301+0000 7f730d709640 1 -- 172.21.15.148:0/2591677218 learned_addr learned my addr 172.21.15.148:0/2591677218 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:01.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.301+0000 7f730d709640 1 -- 172.21.15.148:0/2591677218 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f730815e3c0 msgr2=0x7f73080b4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:01.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.301+0000 7f730d709640 1 --2- 172.21.15.148:0/2591677218 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f730815e3c0 0x7f73080b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.301+0000 7f730d709640 1 -- 172.21.15.148:0/2591677218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73080b5090 msgr2=0x7f73080c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:01.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.301+0000 7f730d709640 1 --2- 172.21.15.148:0/2591677218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73080b5090 0x7f73080c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.301+0000 7f730d709640 1 -- 172.21.15.148:0/2591677218 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73080c9fa0 con 0x7f73080b3430 2024-03-29T04:11:01.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.301+0000 7f730d709640 1 --2- 172.21.15.148:0/2591677218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73080b3430 0x7f73081f6d40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0098c0 tx=0x7f72fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:01.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.301+0000 7f72f9ffb640 1 -- 172.21.15.148:0/2591677218 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc032630 con 0x7f73080b3430 2024-03-29T04:11:01.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.301+0000 7f72f9ffb640 1 -- 172.21.15.148:0/2591677218 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72fc0327f0 con 0x7f73080b3430 2024-03-29T04:11:01.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.301+0000 7f730f96d640 1 -- 172.21.15.148:0/2591677218 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73080ca280 con 0x7f73080b3430 2024-03-29T04:11:01.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.301+0000 7f72f9ffb640 1 -- 172.21.15.148:0/2591677218 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc032be0 con 0x7f73080b3430 2024-03-29T04:11:01.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.301+0000 7f730f96d640 1 -- 172.21.15.148:0/2591677218 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73080ca840 con 0x7f73080b3430 2024-03-29T04:11:01.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.303+0000 7f730f96d640 1 -- 172.21.15.148:0/2591677218 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7308068850 con 0x7f73080b3430 2024-03-29T04:11:01.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.303+0000 7f72f9ffb640 1 -- 172.21.15.148:0/2591677218 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f72fc04e430 con 0x7f73080b3430 2024-03-29T04:11:01.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.303+0000 7f72f9ffb640 1 --2- 172.21.15.148:0/2591677218 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f72ec06c530 0x7f72ec06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:01.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.303+0000 7f72f9ffb640 1 -- 172.21.15.148:0/2591677218 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f72fc02f030 con 0x7f73080b3430 2024-03-29T04:11:01.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.303+0000 7f730cf08640 1 --2- 172.21.15.148:0/2591677218 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f72ec06c530 0x7f72ec06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:01.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.304+0000 7f730cf08640 1 --2- 172.21.15.148:0/2591677218 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f72ec06c530 0x7f72ec06e9f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7304000f60 tx=0x7f73040078f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:01.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.308+0000 7f72f9ffb640 1 -- 172.21.15.148:0/2591677218 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f72fc040070 con 0x7f73080b3430 2024-03-29T04:11:01.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.490+0000 7f730f96d640 1 -- 172.21.15.148:0/2591677218 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f73080cf110 con 0x7f73080b3430 2024-03-29T04:11:01.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.490+0000 7f72f9ffb640 1 -- 172.21.15.148:0/2591677218 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f72fc07bbc0 con 0x7f73080b3430 2024-03-29T04:11:01.469 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:11:01.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.492+0000 7f730f96d640 1 -- 172.21.15.148:0/2591677218 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f72ec06c530 msgr2=0x7f72ec06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:01.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.493+0000 7f730f96d640 1 --2- 172.21.15.148:0/2591677218 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f72ec06c530 0x7f72ec06e9f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7304000f60 tx=0x7f73040078f0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.493+0000 7f730f96d640 1 -- 172.21.15.148:0/2591677218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73080b3430 msgr2=0x7f73081f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:01.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.493+0000 7f730f96d640 1 --2- 172.21.15.148:0/2591677218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73080b3430 0x7f73081f6d40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0098c0 tx=0x7f72fc0046c0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.493+0000 7f730f96d640 1 -- 172.21.15.148:0/2591677218 shutdown_connections 2024-03-29T04:11:01.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.493+0000 7f730f96d640 1 --2- 172.21.15.148:0/2591677218 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f72ec06c530 0x7f72ec06e9f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.493+0000 7f730f96d640 1 --2- 172.21.15.148:0/2591677218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73080b5090 0x7f73080c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.493+0000 7f730f96d640 1 --2- 172.21.15.148:0/2591677218 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f730815e3c0 0x7f73080b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.493+0000 7f730f96d640 1 --2- 172.21.15.148:0/2591677218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73080b3430 0x7f73081f6d40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.493+0000 7f730f96d640 1 -- 172.21.15.148:0/2591677218 >> 172.21.15.148:0/2591677218 conn(0x7f73080b6430 msgr2=0x7f730815d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:01.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.493+0000 7f730f96d640 1 -- 172.21.15.148:0/2591677218 shutdown_connections 2024-03-29T04:11:01.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.493+0000 7f730f96d640 1 -- 172.21.15.148:0/2591677218 wait complete. 2024-03-29T04:11:01.483 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:11:01.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.617+0000 7f70573f4640 1 Processor -- start 2024-03-29T04:11:01.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.617+0000 7f70573f4640 1 -- start start 2024-03-29T04:11:01.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.618+0000 7f70573f4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7050073120 0x7f70500b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:01.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.618+0000 7f70573f4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f70501412d0 0x7f705006c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:01.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.618+0000 7f70573f4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70500b3d60 con 0x7f7050072d40 2024-03-29T04:11:01.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.618+0000 7f70573f4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f705015ee60 con 0x7f70501412d0 2024-03-29T04:11:01.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.618+0000 7f70573f4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f70500b34c0 con 0x7f7050073120 2024-03-29T04:11:01.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.618+0000 7f7055991640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f70501412d0 0x7f705006c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:01.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.618+0000 7f7055190640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7050072d40 0x7f705015b6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39978/0 (socket says 172.21.15.148:39978) 2024-03-29T04:11:01.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.618+0000 7f7055190640 1 -- 172.21.15.148:0/4165890439 learned_addr learned my addr 172.21.15.148:0/4165890439 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:01.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.618+0000 7f705498f640 1 --2- 172.21.15.148:0/4165890439 >> v2:172.21.15.157:3300/0 conn(0x7f7050073120 0x7f70500b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:01.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.618+0000 7f7055991640 1 -- 172.21.15.148:0/4165890439 >> v2:172.21.15.157:3300/0 conn(0x7f7050073120 msgr2=0x7f70500b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:01.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.618+0000 7f7055991640 1 --2- 172.21.15.148:0/4165890439 >> v2:172.21.15.157:3300/0 conn(0x7f7050073120 0x7f70500b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.618+0000 7f7055991640 1 -- 172.21.15.148:0/4165890439 >> v1:172.21.15.148:6789/0 conn(0x7f7050072d40 legacy=0x7f705015b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:01.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.618+0000 7f7055991640 1 -- 172.21.15.148:0/4165890439 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70500b37a0 con 0x7f70501412d0 2024-03-29T04:11:01.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.618+0000 7f7047fff640 1 -- 172.21.15.148:0/4165890439 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1890062787 0 0) 0x7f70500b3d60 con 0x7f7050072d40 2024-03-29T04:11:01.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.619+0000 7f7055991640 1 --2- 172.21.15.148:0/4165890439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70501412d0 0x7f705006c5c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f705015efe0 tx=0x7f704802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da72e13fee7dd70 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:01.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.619+0000 7f7047fff640 1 -- 172.21.15.148:0/4165890439 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048039070 con 0x7f70501412d0 2024-03-29T04:11:01.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.619+0000 7f7047fff640 1 -- 172.21.15.148:0/4165890439 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f704802cc60 con 0x7f70501412d0 2024-03-29T04:11:01.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.619+0000 7f7047fff640 1 -- 172.21.15.148:0/4165890439 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048034660 con 0x7f70501412d0 2024-03-29T04:11:01.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.619+0000 7f70573f4640 1 -- 172.21.15.148:0/4165890439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70501412d0 msgr2=0x7f705006c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:01.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.619+0000 7f70573f4640 1 --2- 172.21.15.148:0/4165890439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70501412d0 0x7f705006c5c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f705015efe0 tx=0x7f704802beb0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.620+0000 7f70573f4640 1 -- 172.21.15.148:0/4165890439 shutdown_connections 2024-03-29T04:11:01.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.620+0000 7f70573f4640 1 --2- 172.21.15.148:0/4165890439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70501412d0 0x7f705006c5c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.620+0000 7f70573f4640 1 --2- 172.21.15.148:0/4165890439 >> v2:172.21.15.157:3300/0 conn(0x7f7050073120 0x7f70500b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.620+0000 7f70573f4640 1 -- 172.21.15.148:0/4165890439 >> 172.21.15.148:0/4165890439 conn(0x7f70500d00a0 msgr2=0x7f70500d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:01.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.620+0000 7f70573f4640 1 -- 172.21.15.148:0/4165890439 shutdown_connections 2024-03-29T04:11:01.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.620+0000 7f70573f4640 1 -- 172.21.15.148:0/4165890439 wait complete. 2024-03-29T04:11:01.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.621+0000 7f70573f4640 1 Processor -- start 2024-03-29T04:11:01.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.621+0000 7f70573f4640 1 -- start start 2024-03-29T04:11:01.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.622+0000 7f70573f4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7050072d40 0x7f70500cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:01.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.622+0000 7f70573f4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7050073120 0x7f70500ad990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:01.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.622+0000 7f70573f4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70501412d0 0x7f70500aded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:01.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.622+0000 7f70573f4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7050141830 con 0x7f7050073120 2024-03-29T04:11:01.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.622+0000 7f70573f4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f70501416b0 con 0x7f7050072d40 2024-03-29T04:11:01.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.622+0000 7f70573f4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f705015bdf0 con 0x7f70501412d0 2024-03-29T04:11:01.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.622+0000 7f705498f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7050073120 0x7f70500ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:01.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.622+0000 7f705498f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7050073120 0x7f70500ad990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56150/0 (socket says 172.21.15.148:56150) 2024-03-29T04:11:01.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.622+0000 7f705498f640 1 -- 172.21.15.148:0/167814833 learned_addr learned my addr 172.21.15.148:0/167814833 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:01.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.622+0000 7f705498f640 1 -- 172.21.15.148:0/167814833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70501412d0 msgr2=0x7f70500aded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:01.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.622+0000 7f705498f640 1 --2- 172.21.15.148:0/167814833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70501412d0 0x7f70500aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.622+0000 7f705498f640 1 -- 172.21.15.148:0/167814833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7050072d40 msgr2=0x7f70500cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:01.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.622+0000 7f705498f640 1 --2- 172.21.15.148:0/167814833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7050072d40 0x7f70500cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.622+0000 7f705498f640 1 -- 172.21.15.148:0/167814833 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70500ce670 con 0x7f7050073120 2024-03-29T04:11:01.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.622+0000 7f705498f640 1 --2- 172.21.15.148:0/167814833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7050073120 0x7f70500ad990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f704000d800 tx=0x7f704000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:01.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.622+0000 7f7045ffb640 1 -- 172.21.15.148:0/167814833 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704000b6c0 con 0x7f7050073120 2024-03-29T04:11:01.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.622+0000 7f7045ffb640 1 -- 172.21.15.148:0/167814833 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7040019070 con 0x7f7050073120 2024-03-29T04:11:01.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.622+0000 7f70573f4640 1 -- 172.21.15.148:0/167814833 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70500cd8b0 con 0x7f7050073120 2024-03-29T04:11:01.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.622+0000 7f7045ffb640 1 -- 172.21.15.148:0/167814833 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704000ba80 con 0x7f7050073120 2024-03-29T04:11:01.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.622+0000 7f70573f4640 1 -- 172.21.15.148:0/167814833 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70500cde70 con 0x7f7050073120 2024-03-29T04:11:01.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.624+0000 7f70573f4640 1 -- 172.21.15.148:0/167814833 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7050068850 con 0x7f7050073120 2024-03-29T04:11:01.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.624+0000 7f7045ffb640 1 -- 172.21.15.148:0/167814833 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f704000bc20 con 0x7f7050073120 2024-03-29T04:11:01.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.624+0000 7f7045ffb640 1 --2- 172.21.15.148:0/167814833 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f703406c530 0x7f703406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:01.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.624+0000 7f7045ffb640 1 -- 172.21.15.148:0/167814833 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f704008bf00 con 0x7f7050073120 2024-03-29T04:11:01.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.624+0000 7f7055190640 1 --2- 172.21.15.148:0/167814833 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f703406c530 0x7f703406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:01.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.625+0000 7f7055190640 1 --2- 172.21.15.148:0/167814833 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f703406c530 0x7f703406e9f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f704c007fd0 tx=0x7f704c007920 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:01.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.629+0000 7f7045ffb640 1 -- 172.21.15.148:0/167814833 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f704005a120 con 0x7f7050073120 2024-03-29T04:11:01.623 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:11:01.626 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:11:01.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.846+0000 7f70573f4640 1 -- 172.21.15.148:0/167814833 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f705006c6b0 con 0x7f7050073120 2024-03-29T04:11:01.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.846+0000 7f7045ffb640 1 -- 172.21.15.148:0/167814833 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f7040059a20 con 0x7f7050073120 2024-03-29T04:11:01.825 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-29T04:11:01.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.848+0000 7f70573f4640 1 -- 172.21.15.148:0/167814833 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f703406c530 msgr2=0x7f703406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:01.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.848+0000 7f70573f4640 1 --2- 172.21.15.148:0/167814833 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f703406c530 0x7f703406e9f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f704c007fd0 tx=0x7f704c007920 comp rx=0 tx=0).stop 2024-03-29T04:11:01.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.848+0000 7f70573f4640 1 -- 172.21.15.148:0/167814833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7050073120 msgr2=0x7f70500ad990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:01.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.848+0000 7f70573f4640 1 --2- 172.21.15.148:0/167814833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7050073120 0x7f70500ad990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f704000d800 tx=0x7f704000dcc0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.848+0000 7f70573f4640 1 -- 172.21.15.148:0/167814833 shutdown_connections 2024-03-29T04:11:01.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.848+0000 7f70573f4640 1 --2- 172.21.15.148:0/167814833 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f703406c530 0x7f703406e9f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.848+0000 7f70573f4640 1 --2- 172.21.15.148:0/167814833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70501412d0 0x7f70500aded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.848+0000 7f70573f4640 1 --2- 172.21.15.148:0/167814833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7050073120 0x7f70500ad990 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.848+0000 7f70573f4640 1 --2- 172.21.15.148:0/167814833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7050072d40 0x7f70500cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.848+0000 7f70573f4640 1 -- 172.21.15.148:0/167814833 >> 172.21.15.148:0/167814833 conn(0x7f70500d00a0 msgr2=0x7f70500b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:01.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.848+0000 7f70573f4640 1 -- 172.21.15.148:0/167814833 shutdown_connections 2024-03-29T04:11:01.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.848+0000 7f70573f4640 1 -- 172.21.15.148:0/167814833 wait complete. 2024-03-29T04:11:01.839 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:11:01.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.969+0000 7f57157c8640 1 Processor -- start 2024-03-29T04:11:01.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.970+0000 7f57157c8640 1 -- start start 2024-03-29T04:11:01.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.970+0000 7f57157c8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f571006efe0 0x7f57100700d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:01.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.970+0000 7f57157c8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f57100ce310 0x7f57100b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:01.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.970+0000 7f57157c8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f571015d460 con 0x7f571015a120 2024-03-29T04:11:01.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.970+0000 7f57157c8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f57100d1b20 con 0x7f571006efe0 2024-03-29T04:11:01.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.970+0000 7f57157c8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f57100d0f60 con 0x7f57100ce310 2024-03-29T04:11:01.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.970+0000 7f570e7fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f571006efe0 0x7f57100700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:01.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.970+0000 7f570e7fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f571006efe0 0x7f57100700d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56158/0 (socket says 172.21.15.148:56158) 2024-03-29T04:11:01.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.970+0000 7f570e7fc640 1 -- 172.21.15.148:0/3698348261 learned_addr learned my addr 172.21.15.148:0/3698348261 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:01.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.971+0000 7f570f7fe640 1 --2- 172.21.15.148:0/3698348261 >> v2:172.21.15.157:3300/0 conn(0x7f57100ce310 0x7f57100b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:01.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.971+0000 7f570e7fc640 1 -- 172.21.15.148:0/3698348261 >> v2:172.21.15.157:3300/0 conn(0x7f57100ce310 msgr2=0x7f57100b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:01.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.971+0000 7f570e7fc640 1 --2- 172.21.15.148:0/3698348261 >> v2:172.21.15.157:3300/0 conn(0x7f57100ce310 0x7f57100b6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.971+0000 7f570e7fc640 1 -- 172.21.15.148:0/3698348261 >> v1:172.21.15.157:6789/0 conn(0x7f571015a120 legacy=0x7f571006e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:01.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.971+0000 7f570e7fc640 1 -- 172.21.15.148:0/3698348261 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f571015d700 con 0x7f571006efe0 2024-03-29T04:11:01.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.972+0000 7f570e7fc640 1 --2- 172.21.15.148:0/3698348261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f571006efe0 0x7f57100700d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f57100cc630 tx=0x7f570402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d010e28dc8d38cf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:01.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.972+0000 7f570dffb640 1 -- 172.21.15.148:0/3698348261 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704034070 con 0x7f571006efe0 2024-03-29T04:11:01.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.972+0000 7f570dffb640 1 -- 172.21.15.148:0/3698348261 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5704030860 con 0x7f571006efe0 2024-03-29T04:11:01.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.972+0000 7f570dffb640 1 -- 172.21.15.148:0/3698348261 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704030c30 con 0x7f571006efe0 2024-03-29T04:11:01.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.973+0000 7f57157c8640 1 -- 172.21.15.148:0/3698348261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f571006efe0 msgr2=0x7f57100700d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:01.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.973+0000 7f57157c8640 1 --2- 172.21.15.148:0/3698348261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f571006efe0 0x7f57100700d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f57100cc630 tx=0x7f570402b9a0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.973+0000 7f57157c8640 1 -- 172.21.15.148:0/3698348261 shutdown_connections 2024-03-29T04:11:01.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.973+0000 7f57157c8640 1 --2- 172.21.15.148:0/3698348261 >> v2:172.21.15.157:3300/0 conn(0x7f57100ce310 0x7f57100b6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.973+0000 7f57157c8640 1 --2- 172.21.15.148:0/3698348261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f571006efe0 0x7f57100700d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.973+0000 7f57157c8640 1 -- 172.21.15.148:0/3698348261 >> 172.21.15.148:0/3698348261 conn(0x7f57100cead0 msgr2=0x7f57100ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:01.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.973+0000 7f57157c8640 1 -- 172.21.15.148:0/3698348261 shutdown_connections 2024-03-29T04:11:01.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.973+0000 7f57157c8640 1 -- 172.21.15.148:0/3698348261 wait complete. 2024-03-29T04:11:01.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.975+0000 7f57157c8640 1 Processor -- start 2024-03-29T04:11:01.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.975+0000 7f57157c8640 1 -- start start 2024-03-29T04:11:01.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.976+0000 7f57157c8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f571006efe0 0x7f57100cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:01.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.976+0000 7f57157c8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57100ce310 0x7f57101520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:01.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.976+0000 7f57157c8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f571015a120 0x7f57101530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:01.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.976+0000 7f57157c8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f571006f5c0 con 0x7f57100ce310 2024-03-29T04:11:01.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.976+0000 7f57157c8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f57100ce7d0 con 0x7f571006efe0 2024-03-29T04:11:01.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.976+0000 7f57157c8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f571006f740 con 0x7f571015a120 2024-03-29T04:11:01.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.976+0000 7f570e7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57100ce310 0x7f57101520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:01.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.976+0000 7f570e7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57100ce310 0x7f57101520a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56162/0 (socket says 172.21.15.148:56162) 2024-03-29T04:11:01.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.976+0000 7f570e7fc640 1 -- 172.21.15.148:0/65900468 learned_addr learned my addr 172.21.15.148:0/65900468 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:01.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.976+0000 7f570effd640 1 --2- 172.21.15.148:0/65900468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f571006efe0 0x7f57100cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:01.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.977+0000 7f570f7fe640 1 --2- 172.21.15.148:0/65900468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f571015a120 0x7f57101530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:01.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.977+0000 7f570e7fc640 1 -- 172.21.15.148:0/65900468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f571015a120 msgr2=0x7f57101530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:01.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.977+0000 7f570e7fc640 1 --2- 172.21.15.148:0/65900468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f571015a120 0x7f57101530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.977+0000 7f570e7fc640 1 -- 172.21.15.148:0/65900468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f571006efe0 msgr2=0x7f57100cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:01.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.977+0000 7f570e7fc640 1 --2- 172.21.15.148:0/65900468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f571006efe0 0x7f57100cbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:01.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.977+0000 7f570e7fc640 1 -- 172.21.15.148:0/65900468 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57100b4c00 con 0x7f57100ce310 2024-03-29T04:11:01.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.977+0000 7f570e7fc640 1 --2- 172.21.15.148:0/65900468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57100ce310 0x7f57101520a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f570402b9e0 tx=0x7f5704004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:01.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.977+0000 7f56f77fe640 1 -- 172.21.15.148:0/65900468 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704048070 con 0x7f57100ce310 2024-03-29T04:11:01.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.977+0000 7f56f77fe640 1 -- 172.21.15.148:0/65900468 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5704030660 con 0x7f57100ce310 2024-03-29T04:11:01.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.977+0000 7f56f77fe640 1 -- 172.21.15.148:0/65900468 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704034040 con 0x7f57100ce310 2024-03-29T04:11:01.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.977+0000 7f57157c8640 1 -- 172.21.15.148:0/65900468 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57101f2230 con 0x7f57100ce310 2024-03-29T04:11:01.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.977+0000 7f57157c8640 1 -- 172.21.15.148:0/65900468 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57101538c0 con 0x7f57100ce310 2024-03-29T04:11:01.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.978+0000 7f57157c8640 1 -- 172.21.15.148:0/65900468 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5710068850 con 0x7f57100ce310 2024-03-29T04:11:01.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.979+0000 7f56f77fe640 1 -- 172.21.15.148:0/65900468 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5704037070 con 0x7f57100ce310 2024-03-29T04:11:01.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.979+0000 7f56f77fe640 1 --2- 172.21.15.148:0/65900468 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f56fc06c530 0x7f56fc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:01.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.979+0000 7f570effd640 1 --2- 172.21.15.148:0/65900468 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f56fc06c530 0x7f56fc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:01.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.979+0000 7f56f77fe640 1 -- 172.21.15.148:0/65900468 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f57040ad770 con 0x7f57100ce310 2024-03-29T04:11:01.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.980+0000 7f570effd640 1 --2- 172.21.15.148:0/65900468 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f56fc06c530 0x7f56fc06e9f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f57000012e0 tx=0x7f5700009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:01.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:01.984+0000 7f56f77fe640 1 -- 172.21.15.148:0/65900468 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f570407b460 con 0x7f57100ce310 2024-03-29T04:11:02.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.320+0000 7f57157c8640 1 -- 172.21.15.148:0/65900468 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f571015f900 con 0x7f57100ce310 2024-03-29T04:11:02.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.321+0000 7f56f77fe640 1 -- 172.21.15.148:0/65900468 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f570407f760 con 0x7f57100ce310 2024-03-29T04:11:02.300 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:11:02.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.322+0000 7f57157c8640 1 -- 172.21.15.148:0/65900468 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f56fc06c530 msgr2=0x7f56fc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:02.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.322+0000 7f57157c8640 1 --2- 172.21.15.148:0/65900468 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f56fc06c530 0x7f56fc06e9f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f57000012e0 tx=0x7f5700009040 comp rx=0 tx=0).stop 2024-03-29T04:11:02.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.322+0000 7f57157c8640 1 -- 172.21.15.148:0/65900468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57100ce310 msgr2=0x7f57101520a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:02.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.322+0000 7f57157c8640 1 --2- 172.21.15.148:0/65900468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57100ce310 0x7f57101520a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f570402b9e0 tx=0x7f5704004730 comp rx=0 tx=0).stop 2024-03-29T04:11:02.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.322+0000 7f57157c8640 1 -- 172.21.15.148:0/65900468 shutdown_connections 2024-03-29T04:11:02.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.323+0000 7f57157c8640 1 --2- 172.21.15.148:0/65900468 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f56fc06c530 0x7f56fc06e9f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:02.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.323+0000 7f57157c8640 1 --2- 172.21.15.148:0/65900468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f571015a120 0x7f57101530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:02.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.323+0000 7f57157c8640 1 --2- 172.21.15.148:0/65900468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57100ce310 0x7f57101520a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:02.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.323+0000 7f57157c8640 1 --2- 172.21.15.148:0/65900468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f571006efe0 0x7f57100cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:02.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.323+0000 7f57157c8640 1 -- 172.21.15.148:0/65900468 >> 172.21.15.148:0/65900468 conn(0x7f57100cead0 msgr2=0x7f571015dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:02.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.323+0000 7f57157c8640 1 -- 172.21.15.148:0/65900468 shutdown_connections 2024-03-29T04:11:02.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.323+0000 7f57157c8640 1 -- 172.21.15.148:0/65900468 wait complete. 2024-03-29T04:11:02.313 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:11:02.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.437+0000 7f1bdc2c5640 1 Processor -- start 2024-03-29T04:11:02.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.437+0000 7f1bdc2c5640 1 -- start start 2024-03-29T04:11:02.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.438+0000 7f1bdc2c5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1bd40ba030 0x7f1bd40d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:02.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.438+0000 7f1bdc2c5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1bd40d10d0 0x7f1bd40d1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:02.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.438+0000 7f1bdc2c5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bd415e7c0 con 0x7f1bd415e3e0 2024-03-29T04:11:02.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.438+0000 7f1bdc2c5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f1bd40d1a70 con 0x7f1bd40d10d0 2024-03-29T04:11:02.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.438+0000 7f1bdc2c5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f1bd40d3840 con 0x7f1bd40ba030 2024-03-29T04:11:02.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.438+0000 7f1bda862640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1bd40d10d0 0x7f1bd40d1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:02.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.438+0000 7f1bda061640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f1bd415e3e0 0x7f1bd40d2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39988/0 (socket says 172.21.15.148:39988) 2024-03-29T04:11:02.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.438+0000 7f1bda061640 1 -- 172.21.15.148:0/3427889678 learned_addr learned my addr 172.21.15.148:0/3427889678 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:02.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.439+0000 7f1bd9860640 1 --2- 172.21.15.148:0/3427889678 >> v2:172.21.15.157:3300/0 conn(0x7f1bd40ba030 0x7f1bd40d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:02.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.439+0000 7f1bda862640 1 -- 172.21.15.148:0/3427889678 >> v2:172.21.15.157:3300/0 conn(0x7f1bd40ba030 msgr2=0x7f1bd40d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:02.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.439+0000 7f1bda862640 1 --2- 172.21.15.148:0/3427889678 >> v2:172.21.15.157:3300/0 conn(0x7f1bd40ba030 0x7f1bd40d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:02.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.439+0000 7f1bda862640 1 -- 172.21.15.148:0/3427889678 >> v1:172.21.15.148:6789/0 conn(0x7f1bd415e3e0 legacy=0x7f1bd40d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:02.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.439+0000 7f1bda862640 1 -- 172.21.15.148:0/3427889678 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd40d2c30 con 0x7f1bd40d10d0 2024-03-29T04:11:02.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.439+0000 7f1bd905f640 1 -- 172.21.15.148:0/3427889678 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1432969190 0 0) 0x7f1bd415e7c0 con 0x7f1bd415e3e0 2024-03-29T04:11:02.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.440+0000 7f1bda862640 1 --2- 172.21.15.148:0/3427889678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bd40d10d0 0x7f1bd40d1530 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1bd40d05d0 tx=0x7f1bcc02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431d23b9cfb58b8f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:02.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.440+0000 7f1bd905f640 1 -- 172.21.15.148:0/3427889678 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc039070 con 0x7f1bd40d10d0 2024-03-29T04:11:02.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.440+0000 7f1bd905f640 1 -- 172.21.15.148:0/3427889678 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bcc02cd00 con 0x7f1bd40d10d0 2024-03-29T04:11:02.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.440+0000 7f1bd905f640 1 -- 172.21.15.148:0/3427889678 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc034660 con 0x7f1bd40d10d0 2024-03-29T04:11:02.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.440+0000 7f1bdc2c5640 1 -- 172.21.15.148:0/3427889678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bd40d10d0 msgr2=0x7f1bd40d1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:02.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.440+0000 7f1bdc2c5640 1 --2- 172.21.15.148:0/3427889678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bd40d10d0 0x7f1bd40d1530 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1bd40d05d0 tx=0x7f1bcc02bf50 comp rx=0 tx=0).stop 2024-03-29T04:11:02.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.440+0000 7f1bdc2c5640 1 -- 172.21.15.148:0/3427889678 shutdown_connections 2024-03-29T04:11:02.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.440+0000 7f1bdc2c5640 1 --2- 172.21.15.148:0/3427889678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bd40d10d0 0x7f1bd40d1530 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:02.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.440+0000 7f1bdc2c5640 1 --2- 172.21.15.148:0/3427889678 >> v2:172.21.15.157:3300/0 conn(0x7f1bd40ba030 0x7f1bd40d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:02.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.440+0000 7f1bdc2c5640 1 -- 172.21.15.148:0/3427889678 >> 172.21.15.148:0/3427889678 conn(0x7f1bd40b6210 msgr2=0x7f1bd40b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:02.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.440+0000 7f1bdc2c5640 1 -- 172.21.15.148:0/3427889678 shutdown_connections 2024-03-29T04:11:02.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.440+0000 7f1bdc2c5640 1 -- 172.21.15.148:0/3427889678 wait complete. 2024-03-29T04:11:02.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.442+0000 7f1bdc2c5640 1 Processor -- start 2024-03-29T04:11:02.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.442+0000 7f1bdc2c5640 1 -- start start 2024-03-29T04:11:02.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.442+0000 7f1bdc2c5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1bd40ba030 0x7f1bd4158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:02.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.442+0000 7f1bdc2c5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bd40d10d0 0x7f1bd4158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:02.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.442+0000 7f1bdc2c5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bd415e3e0 0x7f1bd41549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:02.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.442+0000 7f1bdc2c5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd40d34d0 con 0x7f1bd415e3e0 2024-03-29T04:11:02.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.442+0000 7f1bdc2c5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd40d31f0 con 0x7f1bd40d10d0 2024-03-29T04:11:02.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.442+0000 7f1bdc2c5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1bd4167f50 con 0x7f1bd40ba030 2024-03-29T04:11:02.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.442+0000 7f1bda862640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bd415e3e0 0x7f1bd41549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:02.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.442+0000 7f1bd9860640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bd40d10d0 0x7f1bd4158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:02.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.442+0000 7f1bda862640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bd415e3e0 0x7f1bd41549e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56172/0 (socket says 172.21.15.148:56172) 2024-03-29T04:11:02.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.442+0000 7f1bda862640 1 -- 172.21.15.148:0/654942679 learned_addr learned my addr 172.21.15.148:0/654942679 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:02.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.443+0000 7f1bda061640 1 --2- 172.21.15.148:0/654942679 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1bd40ba030 0x7f1bd4158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:02.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.443+0000 7f1bda862640 1 -- 172.21.15.148:0/654942679 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1bd40ba030 msgr2=0x7f1bd4158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:02.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.443+0000 7f1bda862640 1 --2- 172.21.15.148:0/654942679 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1bd40ba030 0x7f1bd4158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:02.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.443+0000 7f1bda862640 1 -- 172.21.15.148:0/654942679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bd40d10d0 msgr2=0x7f1bd4158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:02.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.443+0000 7f1bda862640 1 --2- 172.21.15.148:0/654942679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bd40d10d0 0x7f1bd4158650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:02.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.443+0000 7f1bda862640 1 -- 172.21.15.148:0/654942679 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd415f1a0 con 0x7f1bd415e3e0 2024-03-29T04:11:02.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.443+0000 7f1bda862640 1 --2- 172.21.15.148:0/654942679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bd415e3e0 0x7f1bd41549e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009e80 tx=0x7f1bcc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:02.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.443+0000 7f1bcaffd640 1 -- 172.21.15.148:0/654942679 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc039050 con 0x7f1bd415e3e0 2024-03-29T04:11:02.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.443+0000 7f1bcaffd640 1 -- 172.21.15.148:0/654942679 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bcc045070 con 0x7f1bd415e3e0 2024-03-29T04:11:02.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.443+0000 7f1bdc2c5640 1 -- 172.21.15.148:0/654942679 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd415f480 con 0x7f1bd415e3e0 2024-03-29T04:11:02.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.443+0000 7f1bcaffd640 1 -- 172.21.15.148:0/654942679 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc009450 con 0x7f1bd415e3e0 2024-03-29T04:11:02.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.443+0000 7f1bdc2c5640 1 -- 172.21.15.148:0/654942679 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd415fa40 con 0x7f1bd415e3e0 2024-03-29T04:11:02.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.444+0000 7f1bdc2c5640 1 -- 172.21.15.148:0/654942679 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bd40687e0 con 0x7f1bd415e3e0 2024-03-29T04:11:02.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.444+0000 7f1bcaffd640 1 -- 172.21.15.148:0/654942679 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1bcc03e850 con 0x7f1bd415e3e0 2024-03-29T04:11:02.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.445+0000 7f1bcaffd640 1 --2- 172.21.15.148:0/654942679 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1bb806c530 0x7f1bb806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:02.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.445+0000 7f1bcaffd640 1 -- 172.21.15.148:0/654942679 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f1bcc0ae7f0 con 0x7f1bd415e3e0 2024-03-29T04:11:02.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.445+0000 7f1bda061640 1 --2- 172.21.15.148:0/654942679 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1bb806c530 0x7f1bb806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:02.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.446+0000 7f1bda061640 1 --2- 172.21.15.148:0/654942679 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1bb806c530 0x7f1bb806e9f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0007fd0 tx=0x7f1bd0007de0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:02.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.449+0000 7f1bcaffd640 1 -- 172.21.15.148:0/654942679 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1bcc07c4e0 con 0x7f1bd415e3e0 2024-03-29T04:11:02.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.637+0000 7f1bdc2c5640 1 -- 172.21.15.148:0/654942679 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1bd406d3b0 con 0x7f1bd415e3e0 2024-03-29T04:11:02.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.637+0000 7f1bcaffd640 1 -- 172.21.15.148:0/654942679 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f1bcc0807e0 con 0x7f1bd415e3e0 2024-03-29T04:11:02.616 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-03-29T04:11:02.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.639+0000 7f1bdc2c5640 1 -- 172.21.15.148:0/654942679 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1bb806c530 msgr2=0x7f1bb806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:02.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.639+0000 7f1bdc2c5640 1 --2- 172.21.15.148:0/654942679 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1bb806c530 0x7f1bb806e9f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0007fd0 tx=0x7f1bd0007de0 comp rx=0 tx=0).stop 2024-03-29T04:11:02.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.640+0000 7f1bdc2c5640 1 -- 172.21.15.148:0/654942679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bd415e3e0 msgr2=0x7f1bd41549e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:02.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.640+0000 7f1bdc2c5640 1 --2- 172.21.15.148:0/654942679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bd415e3e0 0x7f1bd41549e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009e80 tx=0x7f1bcc004a40 comp rx=0 tx=0).stop 2024-03-29T04:11:02.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.640+0000 7f1bdc2c5640 1 -- 172.21.15.148:0/654942679 shutdown_connections 2024-03-29T04:11:02.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.640+0000 7f1bdc2c5640 1 --2- 172.21.15.148:0/654942679 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1bb806c530 0x7f1bb806e9f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:02.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.640+0000 7f1bdc2c5640 1 --2- 172.21.15.148:0/654942679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bd415e3e0 0x7f1bd41549e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:02.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.640+0000 7f1bdc2c5640 1 --2- 172.21.15.148:0/654942679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bd40d10d0 0x7f1bd4158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:02.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.640+0000 7f1bdc2c5640 1 --2- 172.21.15.148:0/654942679 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1bd40ba030 0x7f1bd4158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:02.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.640+0000 7f1bdc2c5640 1 -- 172.21.15.148:0/654942679 >> 172.21.15.148:0/654942679 conn(0x7f1bd40b6210 msgr2=0x7f1bd4157110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:02.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.640+0000 7f1bdc2c5640 1 -- 172.21.15.148:0/654942679 shutdown_connections 2024-03-29T04:11:02.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.640+0000 7f1bdc2c5640 1 -- 172.21.15.148:0/654942679 wait complete. 2024-03-29T04:11:02.629 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2024-03-29T04:11:02.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:02.630 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-03-29T04:11:02.659 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:11:02.659 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-03-29T04:11:02.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:02.660 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-29T04:11:02.721 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:11:02.721 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:11:02.738 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:11:02.739 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:11:02.794 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:11:02.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.958+0000 7f61716c8640 1 Processor -- start 2024-03-29T04:11:02.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.958+0000 7f61716c8640 1 -- start start 2024-03-29T04:11:02.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.959+0000 7f61716c8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f616c0cd600 0x7f616c06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:02.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.959+0000 7f61716c8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f616c06c6c0 0x7f616c15c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:02.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.959+0000 7f61716c8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f616c0cd9e0 con 0x7f616c0b5280 2024-03-29T04:11:02.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.959+0000 7f61716c8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f616c06d800 con 0x7f616c06c6c0 2024-03-29T04:11:02.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.959+0000 7f61716c8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f616c15c680 con 0x7f616c0cd600 2024-03-29T04:11:02.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.959+0000 7f616a7fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f616c06c6c0 0x7f616c15c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:02.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.959+0000 7f616a7fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f616c06c6c0 0x7f616c15c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56184/0 (socket says 172.21.15.148:56184) 2024-03-29T04:11:02.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.959+0000 7f616a7fc640 1 -- 172.21.15.148:0/2945232263 learned_addr learned my addr 172.21.15.148:0/2945232263 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:02.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.959+0000 7f616affd640 1 --2- 172.21.15.148:0/2945232263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f616c0cd600 0x7f616c06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:02.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.959+0000 7f616a7fc640 1 -- 172.21.15.148:0/2945232263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f616c0cd600 msgr2=0x7f616c06dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:02.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.959+0000 7f616a7fc640 1 --2- 172.21.15.148:0/2945232263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f616c0cd600 0x7f616c06dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:02.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.959+0000 7f616a7fc640 1 -- 172.21.15.148:0/2945232263 >> v1:172.21.15.157:6789/0 conn(0x7f616c0b5280 legacy=0x7f616c15c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:02.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.959+0000 7f616a7fc640 1 -- 172.21.15.148:0/2945232263 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f616c06cbf0 con 0x7f616c06c6c0 2024-03-29T04:11:02.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.959+0000 7f6169ffb640 1 -- 172.21.15.148:0/2945232263 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3533513005 0 0) 0x7f616c0cd9e0 con 0x7f616c0b5280 2024-03-29T04:11:02.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.960+0000 7f616a7fc640 1 --2- 172.21.15.148:0/2945232263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f616c06c6c0 0x7f616c15c140 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6160009870 tx=0x7f616002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306f9caa1d6efc1f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:02.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.960+0000 7f6169ffb640 1 -- 172.21.15.148:0/2945232263 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6160034070 con 0x7f616c06c6c0 2024-03-29T04:11:02.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.960+0000 7f6169ffb640 1 -- 172.21.15.148:0/2945232263 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6160030840 con 0x7f616c06c6c0 2024-03-29T04:11:02.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.960+0000 7f6169ffb640 1 -- 172.21.15.148:0/2945232263 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6160030c10 con 0x7f616c06c6c0 2024-03-29T04:11:02.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.961+0000 7f61716c8640 1 -- 172.21.15.148:0/2945232263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f616c06c6c0 msgr2=0x7f616c15c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:02.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.961+0000 7f61716c8640 1 --2- 172.21.15.148:0/2945232263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f616c06c6c0 0x7f616c15c140 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6160009870 tx=0x7f616002b980 comp rx=0 tx=0).stop 2024-03-29T04:11:02.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.961+0000 7f61716c8640 1 -- 172.21.15.148:0/2945232263 shutdown_connections 2024-03-29T04:11:02.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.961+0000 7f61716c8640 1 --2- 172.21.15.148:0/2945232263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f616c06c6c0 0x7f616c15c140 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:02.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.961+0000 7f61716c8640 1 --2- 172.21.15.148:0/2945232263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f616c0cd600 0x7f616c06dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:02.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.961+0000 7f61716c8640 1 -- 172.21.15.148:0/2945232263 >> 172.21.15.148:0/2945232263 conn(0x7f616c0b2c60 msgr2=0x7f616c0b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:02.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.961+0000 7f61716c8640 1 -- 172.21.15.148:0/2945232263 shutdown_connections 2024-03-29T04:11:02.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.961+0000 7f61716c8640 1 -- 172.21.15.148:0/2945232263 wait complete. 2024-03-29T04:11:02.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.962+0000 7f61716c8640 1 Processor -- start 2024-03-29T04:11:02.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.962+0000 7f61716c8640 1 -- start start 2024-03-29T04:11:02.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.963+0000 7f61716c8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f616c06c6c0 0x7f616c0c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:02.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.963+0000 7f61716c8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f616c0b5280 0x7f616c1f01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:02.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.963+0000 7f61716c8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f616c0cd600 0x7f616c0d0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:02.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.963+0000 7f61716c8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f616c06d490 con 0x7f616c06c6c0 2024-03-29T04:11:02.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.963+0000 7f61716c8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f616c06d1b0 con 0x7f616c0b5280 2024-03-29T04:11:02.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.963+0000 7f61716c8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f616c0b56f0 con 0x7f616c0cd600 2024-03-29T04:11:02.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.963+0000 7f616affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f616c06c6c0 0x7f616c0c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:02.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.963+0000 7f616affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f616c06c6c0 0x7f616c0c68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56196/0 (socket says 172.21.15.148:56196) 2024-03-29T04:11:02.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.963+0000 7f616affd640 1 -- 172.21.15.148:0/1051601075 learned_addr learned my addr 172.21.15.148:0/1051601075 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:02.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.963+0000 7f616a7fc640 1 --2- 172.21.15.148:0/1051601075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f616c0b5280 0x7f616c1f01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:02.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.963+0000 7f616b7fe640 1 --2- 172.21.15.148:0/1051601075 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f616c0cd600 0x7f616c0d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:02.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.963+0000 7f616affd640 1 -- 172.21.15.148:0/1051601075 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f616c0cd600 msgr2=0x7f616c0d0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:02.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.963+0000 7f616affd640 1 --2- 172.21.15.148:0/1051601075 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f616c0cd600 0x7f616c0d0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:02.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.963+0000 7f616affd640 1 -- 172.21.15.148:0/1051601075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f616c0b5280 msgr2=0x7f616c1f01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:02.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.963+0000 7f616affd640 1 --2- 172.21.15.148:0/1051601075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f616c0b5280 0x7f616c1f01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:02.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.963+0000 7f616affd640 1 -- 172.21.15.148:0/1051601075 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f616c0c6ec0 con 0x7f616c06c6c0 2024-03-29T04:11:02.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.964+0000 7f616affd640 1 --2- 172.21.15.148:0/1051601075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f616c06c6c0 0x7f616c0c68b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f615c00e870 tx=0x7f615c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:02.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.964+0000 7f61537fe640 1 -- 172.21.15.148:0/1051601075 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c017070 con 0x7f616c06c6c0 2024-03-29T04:11:02.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.964+0000 7f61537fe640 1 -- 172.21.15.148:0/1051601075 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f615c00c940 con 0x7f616c06c6c0 2024-03-29T04:11:02.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.964+0000 7f61716c8640 1 -- 172.21.15.148:0/1051601075 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f616c0c71a0 con 0x7f616c06c6c0 2024-03-29T04:11:02.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.964+0000 7f61537fe640 1 -- 172.21.15.148:0/1051601075 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c00cd30 con 0x7f616c06c6c0 2024-03-29T04:11:02.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.964+0000 7f61716c8640 1 -- 172.21.15.148:0/1051601075 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f616c0c7760 con 0x7f616c06c6c0 2024-03-29T04:11:02.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.965+0000 7f61716c8640 1 -- 172.21.15.148:0/1051601075 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f616c0687e0 con 0x7f616c06c6c0 2024-03-29T04:11:02.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.965+0000 7f61537fe640 1 -- 172.21.15.148:0/1051601075 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f615c01a070 con 0x7f616c06c6c0 2024-03-29T04:11:02.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.966+0000 7f61537fe640 1 --2- 172.21.15.148:0/1051601075 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f615806c530 0x7f615806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:02.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.966+0000 7f616a7fc640 1 --2- 172.21.15.148:0/1051601075 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f615806c530 0x7f615806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:02.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.966+0000 7f61537fe640 1 -- 172.21.15.148:0/1051601075 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f615c08bd30 con 0x7f616c06c6c0 2024-03-29T04:11:02.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.966+0000 7f616a7fc640 1 --2- 172.21.15.148:0/1051601075 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f615806c530 0x7f615806e9f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6160036000 tx=0x7f6160008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:02.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:02.970+0000 7f61537fe640 1 -- 172.21.15.148:0/1051601075 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f615c059900 con 0x7f616c06c6c0 2024-03-29T04:11:03.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.183+0000 7f61716c8640 1 -- 172.21.15.148:0/1051601075 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f616c0d0790 con 0x7f616c06c6c0 2024-03-29T04:11:03.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.184+0000 7f61537fe640 1 -- 172.21.15.148:0/1051601075 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18858 (secure 0 0 0) 0x7f615c05dc00 con 0x7f616c06c6c0 2024-03-29T04:11:03.164 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:11:03.164 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:41.662934+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:11:03.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.187+0000 7f61716c8640 1 -- 172.21.15.148:0/1051601075 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f615806c530 msgr2=0x7f615806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:03.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.187+0000 7f61716c8640 1 --2- 172.21.15.148:0/1051601075 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f615806c530 0x7f615806e9f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6160036000 tx=0x7f6160008030 comp rx=0 tx=0).stop 2024-03-29T04:11:03.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.187+0000 7f61716c8640 1 -- 172.21.15.148:0/1051601075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f616c06c6c0 msgr2=0x7f616c0c68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:03.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.187+0000 7f61716c8640 1 --2- 172.21.15.148:0/1051601075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f616c06c6c0 0x7f616c0c68b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f615c00e870 tx=0x7f615c00ec70 comp rx=0 tx=0).stop 2024-03-29T04:11:03.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.187+0000 7f61716c8640 1 -- 172.21.15.148:0/1051601075 shutdown_connections 2024-03-29T04:11:03.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.187+0000 7f61716c8640 1 --2- 172.21.15.148:0/1051601075 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f615806c530 0x7f615806e9f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:03.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.187+0000 7f61716c8640 1 --2- 172.21.15.148:0/1051601075 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f616c0cd600 0x7f616c0d0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:03.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.187+0000 7f61716c8640 1 --2- 172.21.15.148:0/1051601075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f616c0b5280 0x7f616c1f01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:03.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.187+0000 7f61716c8640 1 --2- 172.21.15.148:0/1051601075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f616c06c6c0 0x7f616c0c68b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:03.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.187+0000 7f61716c8640 1 -- 172.21.15.148:0/1051601075 >> 172.21.15.148:0/1051601075 conn(0x7f616c0b2c60 msgr2=0x7f616c151c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:03.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.187+0000 7f61716c8640 1 -- 172.21.15.148:0/1051601075 shutdown_connections 2024-03-29T04:11:03.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.187+0000 7f61716c8640 1 -- 172.21.15.148:0/1051601075 wait complete. 2024-03-29T04:11:03.178 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:47.898351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:57.922313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:58.829405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:10:02.551763+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:11:03.178 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:11:03.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.303+0000 7f0485590640 1 Processor -- start 2024-03-29T04:11:03.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.303+0000 7f0485590640 1 -- start start 2024-03-29T04:11:03.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.304+0000 7f0485590640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04800b2f00 0x7f0480162680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:03.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.304+0000 7f0485590640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f04800b3430 0x7f048006b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:03.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.304+0000 7f0485590640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04800cf330 con 0x7f048015e3c0 2024-03-29T04:11:03.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.304+0000 7f0485590640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f04800cf960 con 0x7f04800b3430 2024-03-29T04:11:03.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.304+0000 7f0485590640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0480162cf0 con 0x7f04800b2f00 2024-03-29T04:11:03.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.304+0000 7f047f7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f04800b3430 0x7f048006b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:03.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.304+0000 7f0477fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04800b2f00 0x7f0480162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:03.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.304+0000 7f047effd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f048015e3c0 0x7f0480161f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.148:49482/0 (socket says 172.21.15.148:49482) 2024-03-29T04:11:03.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.304+0000 7f047f7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f04800b3430 0x7f048006b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:56826/0 (socket says 172.21.15.148:56826) 2024-03-29T04:11:03.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.304+0000 7f0477fff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04800b2f00 0x7f0480162680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.148:34472/0 (socket says 172.21.15.148:34472) 2024-03-29T04:11:03.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.304+0000 7f047effd640 1 -- 172.21.15.148:0/772668445 learned_addr learned my addr 172.21.15.148:0/772668445 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:03.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.305+0000 7f047f7fe640 1 -- 172.21.15.148:0/772668445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04800b2f00 msgr2=0x7f0480162680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:03.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.305+0000 7f047f7fe640 1 --2- 172.21.15.148:0/772668445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04800b2f00 0x7f0480162680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:03.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.305+0000 7f047f7fe640 1 -- 172.21.15.148:0/772668445 >> v1:172.21.15.157:6789/0 conn(0x7f048015e3c0 legacy=0x7f0480161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:03.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.305+0000 7f047f7fe640 1 -- 172.21.15.148:0/772668445 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04800cf5f0 con 0x7f04800b3430 2024-03-29T04:11:03.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.305+0000 7f047e7fc640 1 -- 172.21.15.148:0/772668445 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2568496514 0 0) 0x7f04800cf330 con 0x7f048015e3c0 2024-03-29T04:11:03.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.305+0000 7f0477fff640 1 --2- 172.21.15.148:0/772668445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04800b2f00 0x7f0480162680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:11:03.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.306+0000 7f047f7fe640 1 --2- 172.21.15.148:0/772668445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04800b3430 0x7f048006b7f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0478009d90 tx=0x7f047802be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4345d7232f4864b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:03.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.306+0000 7f047e7fc640 1 -- 172.21.15.148:0/772668445 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478039070 con 0x7f04800b3430 2024-03-29T04:11:03.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.306+0000 7f047e7fc640 1 -- 172.21.15.148:0/772668445 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f047802cbc0 con 0x7f04800b3430 2024-03-29T04:11:03.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.306+0000 7f047e7fc640 1 -- 172.21.15.148:0/772668445 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478034440 con 0x7f04800b3430 2024-03-29T04:11:03.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.306+0000 7f0485590640 1 -- 172.21.15.148:0/772668445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04800b3430 msgr2=0x7f048006b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:03.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.306+0000 7f0485590640 1 --2- 172.21.15.148:0/772668445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04800b3430 0x7f048006b7f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0478009d90 tx=0x7f047802be10 comp rx=0 tx=0).stop 2024-03-29T04:11:03.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.306+0000 7f0485590640 1 -- 172.21.15.148:0/772668445 shutdown_connections 2024-03-29T04:11:03.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.306+0000 7f0485590640 1 --2- 172.21.15.148:0/772668445 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04800b3430 0x7f048006b7f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:03.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.306+0000 7f0485590640 1 --2- 172.21.15.148:0/772668445 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04800b2f00 0x7f0480162680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:03.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.306+0000 7f0485590640 1 -- 172.21.15.148:0/772668445 >> 172.21.15.148:0/772668445 conn(0x7f04800b6430 msgr2=0x7f04800d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:03.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.306+0000 7f0485590640 1 -- 172.21.15.148:0/772668445 shutdown_connections 2024-03-29T04:11:03.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.306+0000 7f0485590640 1 -- 172.21.15.148:0/772668445 wait complete. 2024-03-29T04:11:03.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.307+0000 7f0485590640 1 Processor -- start 2024-03-29T04:11:03.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.307+0000 7f0485590640 1 -- start start 2024-03-29T04:11:03.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.308+0000 7f0485590640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04800b3430 0x7f048015e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:03.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.308+0000 7f0485590640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f048015e3c0 0x7f04800cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:03.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.308+0000 7f0485590640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f048015fd90 0x7f04800b8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:03.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.308+0000 7f0485590640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f04800b3b20 con 0x7f048015fd90 2024-03-29T04:11:03.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.308+0000 7f0485590640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f04800b39a0 con 0x7f04800b3430 2024-03-29T04:11:03.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.308+0000 7f0485590640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f048006bd30 con 0x7f048015e3c0 2024-03-29T04:11:03.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.308+0000 7f047f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f048015fd90 0x7f04800b8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:03.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.308+0000 7f047f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f048015fd90 0x7f04800b8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56210/0 (socket says 172.21.15.148:56210) 2024-03-29T04:11:03.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.308+0000 7f047f7fe640 1 -- 172.21.15.148:0/1622822900 learned_addr learned my addr 172.21.15.148:0/1622822900 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:03.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.308+0000 7f047effd640 1 --2- 172.21.15.148:0/1622822900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04800b3430 0x7f048015e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:03.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.308+0000 7f0477fff640 1 --2- 172.21.15.148:0/1622822900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f048015e3c0 0x7f04800cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:03.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.309+0000 7f047f7fe640 1 -- 172.21.15.148:0/1622822900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f048015e3c0 msgr2=0x7f04800cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:03.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.309+0000 7f047f7fe640 1 --2- 172.21.15.148:0/1622822900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f048015e3c0 0x7f04800cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:03.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.309+0000 7f047f7fe640 1 -- 172.21.15.148:0/1622822900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04800b3430 msgr2=0x7f048015e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:03.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.309+0000 7f047f7fe640 1 --2- 172.21.15.148:0/1622822900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04800b3430 0x7f048015e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:03.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.309+0000 7f047f7fe640 1 -- 172.21.15.148:0/1622822900 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0480161820 con 0x7f048015fd90 2024-03-29T04:11:03.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.309+0000 7f0477fff640 1 --2- 172.21.15.148:0/1622822900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f048015e3c0 0x7f04800cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:03.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.309+0000 7f047f7fe640 1 --2- 172.21.15.148:0/1622822900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f048015fd90 0x7f04800b8480 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0478009d90 tx=0x7f0478004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:03.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.309+0000 7f04777fe640 1 -- 172.21.15.148:0/1622822900 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478039050 con 0x7f048015fd90 2024-03-29T04:11:03.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.309+0000 7f0485590640 1 -- 172.21.15.148:0/1622822900 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04801f77d0 con 0x7f048015fd90 2024-03-29T04:11:03.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.309+0000 7f04777fe640 1 -- 172.21.15.148:0/1622822900 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0478045070 con 0x7f048015fd90 2024-03-29T04:11:03.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.309+0000 7f04777fe640 1 -- 172.21.15.148:0/1622822900 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478009450 con 0x7f048015fd90 2024-03-29T04:11:03.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.309+0000 7f0485590640 1 -- 172.21.15.148:0/1622822900 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04801f7d90 con 0x7f048015fd90 2024-03-29T04:11:03.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.310+0000 7f0485590640 1 -- 172.21.15.148:0/1622822900 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0480068850 con 0x7f048015fd90 2024-03-29T04:11:03.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.311+0000 7f04777fe640 1 -- 172.21.15.148:0/1622822900 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f047803e850 con 0x7f048015fd90 2024-03-29T04:11:03.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.311+0000 7f04777fe640 1 --2- 172.21.15.148:0/1622822900 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f046006c560 0x7f046006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:03.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.311+0000 7f04777fe640 1 -- 172.21.15.148:0/1622822900 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f04780aeaa0 con 0x7f048015fd90 2024-03-29T04:11:03.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.311+0000 7f047effd640 1 --2- 172.21.15.148:0/1622822900 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f046006c560 0x7f046006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:03.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.311+0000 7f047effd640 1 --2- 172.21.15.148:0/1622822900 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f046006c560 0x7f046006ea20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0470007fd0 tx=0x7f0470007de0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:03.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.315+0000 7f04777fe640 1 -- 172.21.15.148:0/1622822900 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f047807cd60 con 0x7f048015fd90 2024-03-29T04:11:03.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.493+0000 7f0485590640 1 -- 172.21.15.148:0/1622822900 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f04800cae40 con 0x7f048015fd90 2024-03-29T04:11:03.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.494+0000 7f04777fe640 1 -- 172.21.15.148:0/1622822900 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f047807c660 con 0x7f048015fd90 2024-03-29T04:11:03.473 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-29T04:11:03.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.496+0000 7f0485590640 1 -- 172.21.15.148:0/1622822900 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f046006c560 msgr2=0x7f046006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:03.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.496+0000 7f0485590640 1 --2- 172.21.15.148:0/1622822900 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f046006c560 0x7f046006ea20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0470007fd0 tx=0x7f0470007de0 comp rx=0 tx=0).stop 2024-03-29T04:11:03.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.496+0000 7f0485590640 1 -- 172.21.15.148:0/1622822900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f048015fd90 msgr2=0x7f04800b8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:03.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.496+0000 7f0485590640 1 --2- 172.21.15.148:0/1622822900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f048015fd90 0x7f04800b8480 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0478009d90 tx=0x7f0478004a40 comp rx=0 tx=0).stop 2024-03-29T04:11:03.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.496+0000 7f0485590640 1 -- 172.21.15.148:0/1622822900 shutdown_connections 2024-03-29T04:11:03.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.496+0000 7f0485590640 1 --2- 172.21.15.148:0/1622822900 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f046006c560 0x7f046006ea20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:03.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.496+0000 7f0485590640 1 --2- 172.21.15.148:0/1622822900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f048015fd90 0x7f04800b8480 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:03.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.496+0000 7f0485590640 1 --2- 172.21.15.148:0/1622822900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f048015e3c0 0x7f04800cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:03.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.496+0000 7f0485590640 1 --2- 172.21.15.148:0/1622822900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04800b3430 0x7f048015e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:03.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.496+0000 7f0485590640 1 -- 172.21.15.148:0/1622822900 >> 172.21.15.148:0/1622822900 conn(0x7f04800b6430 msgr2=0x7f048015c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:03.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.496+0000 7f0485590640 1 -- 172.21.15.148:0/1622822900 shutdown_connections 2024-03-29T04:11:03.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.496+0000 7f0485590640 1 -- 172.21.15.148:0/1622822900 wait complete. 2024-03-29T04:11:03.489 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:11:03.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.631+0000 7fe3a0021640 1 Processor -- start 2024-03-29T04:11:03.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.631+0000 7fe3a0021640 1 -- start start 2024-03-29T04:11:03.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.632+0000 7fe3a0021640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe3980b2f00 0x7fe398162680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:03.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.632+0000 7fe3a0021640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3980b3430 0x7fe39806b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:03.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.632+0000 7fe3a0021640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3980cf330 con 0x7fe39815e3c0 2024-03-29T04:11:03.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.632+0000 7fe3a0021640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe3980cf960 con 0x7fe3980b2f00 2024-03-29T04:11:03.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.632+0000 7fe3a0021640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe398162cf0 con 0x7fe3980b3430 2024-03-29T04:11:03.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.632+0000 7fe39ddbd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe39815e3c0 0x7fe398161f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39994/0 (socket says 172.21.15.148:39994) 2024-03-29T04:11:03.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.632+0000 7fe39ddbd640 1 -- 172.21.15.148:0/690515368 learned_addr learned my addr 172.21.15.148:0/690515368 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:03.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.632+0000 7fe39d5bc640 1 --2- 172.21.15.148:0/690515368 >> v2:172.21.15.157:3300/0 conn(0x7fe3980b2f00 0x7fe398162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:03.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.632+0000 7fe39e5be640 1 --2- 172.21.15.148:0/690515368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3980b3430 0x7fe39806b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:03.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.632+0000 7fe39cdbb640 1 -- 172.21.15.148:0/690515368 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3812774756 0 0) 0x7fe3980cf330 con 0x7fe39815e3c0 2024-03-29T04:11:03.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.632+0000 7fe39cdbb640 1 -- 172.21.15.148:0/690515368 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe384003520 con 0x7fe39815e3c0 2024-03-29T04:11:03.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.633+0000 7fe39d5bc640 1 -- 172.21.15.148:0/690515368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3980b3430 msgr2=0x7fe39806b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:03.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.633+0000 7fe39d5bc640 1 --2- 172.21.15.148:0/690515368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3980b3430 0x7fe39806b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:03.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.633+0000 7fe39d5bc640 1 -- 172.21.15.148:0/690515368 >> v1:172.21.15.148:6789/0 conn(0x7fe39815e3c0 legacy=0x7fe398161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:03.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.633+0000 7fe39cdbb640 1 -- 172.21.15.148:0/690515368 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 299136358 0 0) 0x7fe384003520 con 0x7fe39815e3c0 2024-03-29T04:11:03.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.633+0000 7fe39d5bc640 1 -- 172.21.15.148:0/690515368 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3980cf5f0 con 0x7fe3980b2f00 2024-03-29T04:11:03.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.633+0000 7fe39d5bc640 1 --2- 172.21.15.148:0/690515368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3980b2f00 0x7fe398162680 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe3880097a0 tx=0x7fe38802b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1536d91db2f57416 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:03.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.633+0000 7fe39cdbb640 1 -- 172.21.15.148:0/690515368 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe388037070 con 0x7fe3980b2f00 2024-03-29T04:11:03.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.634+0000 7fe39cdbb640 1 -- 172.21.15.148:0/690515368 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe388030510 con 0x7fe3980b2f00 2024-03-29T04:11:03.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.634+0000 7fe39cdbb640 1 -- 172.21.15.148:0/690515368 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390001670 con 0x7fe3980b2f00 2024-03-29T04:11:03.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.634+0000 7fe3a0021640 1 -- 172.21.15.148:0/690515368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3980b2f00 msgr2=0x7fe398162680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:03.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.634+0000 7fe3a0021640 1 --2- 172.21.15.148:0/690515368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3980b2f00 0x7fe398162680 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe3880097a0 tx=0x7fe38802b710 comp rx=0 tx=0).stop 2024-03-29T04:11:03.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.634+0000 7fe3a0021640 1 -- 172.21.15.148:0/690515368 shutdown_connections 2024-03-29T04:11:03.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.634+0000 7fe3a0021640 1 --2- 172.21.15.148:0/690515368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe3980b3430 0x7fe39806b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:03.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.634+0000 7fe3a0021640 1 --2- 172.21.15.148:0/690515368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3980b2f00 0x7fe398162680 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:03.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.634+0000 7fe3a0021640 1 -- 172.21.15.148:0/690515368 >> 172.21.15.148:0/690515368 conn(0x7fe3980b6430 msgr2=0x7fe3980d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:03.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.634+0000 7fe3a0021640 1 -- 172.21.15.148:0/690515368 shutdown_connections 2024-03-29T04:11:03.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.634+0000 7fe3a0021640 1 -- 172.21.15.148:0/690515368 wait complete. 2024-03-29T04:11:03.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.635+0000 7fe3a0021640 1 Processor -- start 2024-03-29T04:11:03.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.635+0000 7fe3a0021640 1 -- start start 2024-03-29T04:11:03.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.636+0000 7fe3a0021640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3980b3430 0x7fe398148580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:03.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.636+0000 7fe3a0021640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe39815e3c0 0x7fe3980c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:03.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.636+0000 7fe3a0021640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3980c8340 0x7fe3980d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:03.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.636+0000 7fe3a0021640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe3980b3b20 con 0x7fe3980c8340 2024-03-29T04:11:03.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.636+0000 7fe3a0021640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe3980b39a0 con 0x7fe3980b3430 2024-03-29T04:11:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.636+0000 7fe3a0021640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe39806bd30 con 0x7fe39815e3c0 2024-03-29T04:11:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.636+0000 7fe39e5be640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3980c8340 0x7fe3980d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.636+0000 7fe39e5be640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3980c8340 0x7fe3980d26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56218/0 (socket says 172.21.15.148:56218) 2024-03-29T04:11:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.636+0000 7fe39e5be640 1 -- 172.21.15.148:0/3402967035 learned_addr learned my addr 172.21.15.148:0/3402967035 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.636+0000 7fe39d5bc640 1 --2- 172.21.15.148:0/3402967035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe39815e3c0 0x7fe3980c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.636+0000 7fe39ddbd640 1 --2- 172.21.15.148:0/3402967035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3980b3430 0x7fe398148580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.636+0000 7fe39e5be640 1 -- 172.21.15.148:0/3402967035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe39815e3c0 msgr2=0x7fe3980c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.636+0000 7fe39e5be640 1 --2- 172.21.15.148:0/3402967035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe39815e3c0 0x7fe3980c7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.636+0000 7fe39e5be640 1 -- 172.21.15.148:0/3402967035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3980b3430 msgr2=0x7fe398148580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.636+0000 7fe39e5be640 1 --2- 172.21.15.148:0/3402967035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3980b3430 0x7fe398148580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.636+0000 7fe39e5be640 1 -- 172.21.15.148:0/3402967035 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39815f910 con 0x7fe3980c8340 2024-03-29T04:11:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.636+0000 7fe39d5bc640 1 --2- 172.21.15.148:0/3402967035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe39815e3c0 0x7fe3980c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.636+0000 7fe39ddbd640 1 --2- 172.21.15.148:0/3402967035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3980b3430 0x7fe398148580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.637+0000 7fe39e5be640 1 --2- 172.21.15.148:0/3402967035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3980c8340 0x7fe3980d26a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe39400efc0 tx=0x7fe39400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.637+0000 7fe38e7fc640 1 -- 172.21.15.148:0/3402967035 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe394018020 con 0x7fe3980c8340 2024-03-29T04:11:03.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.637+0000 7fe3a0021640 1 -- 172.21.15.148:0/3402967035 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe398160070 con 0x7fe3980c8340 2024-03-29T04:11:03.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.637+0000 7fe38e7fc640 1 -- 172.21.15.148:0/3402967035 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe394014480 con 0x7fe3980c8340 2024-03-29T04:11:03.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.637+0000 7fe38e7fc640 1 -- 172.21.15.148:0/3402967035 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe394014870 con 0x7fe3980c8340 2024-03-29T04:11:03.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.637+0000 7fe3a0021640 1 -- 172.21.15.148:0/3402967035 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe398160630 con 0x7fe3980c8340 2024-03-29T04:11:03.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.638+0000 7fe3a0021640 1 -- 172.21.15.148:0/3402967035 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe398068850 con 0x7fe3980c8340 2024-03-29T04:11:03.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.638+0000 7fe38e7fc640 1 -- 172.21.15.148:0/3402967035 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe39401b020 con 0x7fe3980c8340 2024-03-29T04:11:03.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.639+0000 7fe38e7fc640 1 --2- 172.21.15.148:0/3402967035 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe37c06c530 0x7fe37c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:03.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.639+0000 7fe39ddbd640 1 --2- 172.21.15.148:0/3402967035 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe37c06c530 0x7fe37c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:03.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.639+0000 7fe38e7fc640 1 -- 172.21.15.148:0/3402967035 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe39408c6e0 con 0x7fe3980c8340 2024-03-29T04:11:03.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.639+0000 7fe39ddbd640 1 --2- 172.21.15.148:0/3402967035 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe37c06c530 0x7fe37c06e9f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe390007a00 tx=0x7fe390009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:03.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.643+0000 7fe38e7fc640 1 -- 172.21.15.148:0/3402967035 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe394022030 con 0x7fe3980c8340 2024-03-29T04:11:03.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.825+0000 7fe3a0021640 1 -- 172.21.15.148:0/3402967035 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe3980cf110 con 0x7fe3980c8340 2024-03-29T04:11:03.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.826+0000 7fe38e7fc640 1 -- 172.21.15.148:0/3402967035 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fe394079020 con 0x7fe3980c8340 2024-03-29T04:11:03.805 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:11:03.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.827+0000 7fe3a0021640 1 -- 172.21.15.148:0/3402967035 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe37c06c530 msgr2=0x7fe37c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:03.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.828+0000 7fe3a0021640 1 --2- 172.21.15.148:0/3402967035 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe37c06c530 0x7fe37c06e9f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe390007a00 tx=0x7fe390009040 comp rx=0 tx=0).stop 2024-03-29T04:11:03.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.828+0000 7fe3a0021640 1 -- 172.21.15.148:0/3402967035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3980c8340 msgr2=0x7fe3980d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:03.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.828+0000 7fe3a0021640 1 --2- 172.21.15.148:0/3402967035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3980c8340 0x7fe3980d26a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe39400efc0 tx=0x7fe39400c490 comp rx=0 tx=0).stop 2024-03-29T04:11:03.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.828+0000 7fe3a0021640 1 -- 172.21.15.148:0/3402967035 shutdown_connections 2024-03-29T04:11:03.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.828+0000 7fe3a0021640 1 --2- 172.21.15.148:0/3402967035 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe37c06c530 0x7fe37c06e9f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:03.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.828+0000 7fe3a0021640 1 --2- 172.21.15.148:0/3402967035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3980c8340 0x7fe3980d26a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:03.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.828+0000 7fe3a0021640 1 --2- 172.21.15.148:0/3402967035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe39815e3c0 0x7fe3980c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:03.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.828+0000 7fe3a0021640 1 --2- 172.21.15.148:0/3402967035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3980b3430 0x7fe398148580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:03.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.828+0000 7fe3a0021640 1 -- 172.21.15.148:0/3402967035 >> 172.21.15.148:0/3402967035 conn(0x7fe3980b6430 msgr2=0x7fe3980b4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:03.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.828+0000 7fe3a0021640 1 -- 172.21.15.148:0/3402967035 shutdown_connections 2024-03-29T04:11:03.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.828+0000 7fe3a0021640 1 -- 172.21.15.148:0/3402967035 wait complete. 2024-03-29T04:11:03.817 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:11:03.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.966+0000 7fd98343c640 1 Processor -- start 2024-03-29T04:11:04.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.966+0000 7fd98343c640 1 -- start start 2024-03-29T04:11:04.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.967+0000 7fd98343c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd97c15f200 0x7fd97c15f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:04.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.967+0000 7fd98343c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd97c0b4fb0 0x7fd97c0cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:04.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.967+0000 7fd98343c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd97c0cfdd0 con 0x7fd97c0b5430 2024-03-29T04:11:04.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.967+0000 7fd98343c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd97c0d09e0 con 0x7fd97c15f200 2024-03-29T04:11:04.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.967+0000 7fd98343c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd97c0d00b0 con 0x7fd97c0b4fb0 2024-03-29T04:11:04.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.967+0000 7fd9819d9640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd97c0b5430 0x7fd97c162020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40000/0 (socket says 172.21.15.148:40000) 2024-03-29T04:11:04.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.967+0000 7fd9819d9640 1 -- 172.21.15.148:0/3723299506 learned_addr learned my addr 172.21.15.148:0/3723299506 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:04.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.968+0000 7fd9811d8640 1 --2- 172.21.15.148:0/3723299506 >> v2:172.21.15.157:3300/0 conn(0x7fd97c15f200 0x7fd97c15f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:04.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.968+0000 7fd9809d7640 1 --2- 172.21.15.148:0/3723299506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd97c0b4fb0 0x7fd97c0cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:04.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.968+0000 7fd973fff640 1 -- 172.21.15.148:0/3723299506 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 996721656 0 0) 0x7fd97c0cfdd0 con 0x7fd97c0b5430 2024-03-29T04:11:04.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.968+0000 7fd973fff640 1 -- 172.21.15.148:0/3723299506 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd968003520 con 0x7fd97c0b5430 2024-03-29T04:11:04.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.968+0000 7fd9811d8640 1 -- 172.21.15.148:0/3723299506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd97c0b4fb0 msgr2=0x7fd97c0cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:04.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.968+0000 7fd9811d8640 1 --2- 172.21.15.148:0/3723299506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd97c0b4fb0 0x7fd97c0cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.968+0000 7fd9811d8640 1 -- 172.21.15.148:0/3723299506 >> v1:172.21.15.148:6789/0 conn(0x7fd97c0b5430 legacy=0x7fd97c162020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:04.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.968+0000 7fd9811d8640 1 -- 172.21.15.148:0/3723299506 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97c0d0390 con 0x7fd97c15f200 2024-03-29T04:11:04.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.968+0000 7fd9809d7640 1 --2- 172.21.15.148:0/3723299506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd97c0b4fb0 0x7fd97c0cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:11:04.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.969+0000 7fd9811d8640 1 --2- 172.21.15.148:0/3723299506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd97c15f200 0x7fd97c15f5e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd9740097a0 tx=0x7fd97402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7250b8bed9e0e1b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:04.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.969+0000 7fd973fff640 1 -- 172.21.15.148:0/3723299506 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd974035020 con 0x7fd97c15f200 2024-03-29T04:11:04.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.969+0000 7fd973fff640 1 -- 172.21.15.148:0/3723299506 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd974031830 con 0x7fd97c15f200 2024-03-29T04:11:04.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.969+0000 7fd973fff640 1 -- 172.21.15.148:0/3723299506 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd974031c00 con 0x7fd97c15f200 2024-03-29T04:11:04.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.969+0000 7fd98343c640 1 -- 172.21.15.148:0/3723299506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd97c15f200 msgr2=0x7fd97c15f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:04.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.969+0000 7fd98343c640 1 --2- 172.21.15.148:0/3723299506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd97c15f200 0x7fd97c15f5e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd9740097a0 tx=0x7fd97402b8d0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.969+0000 7fd98343c640 1 -- 172.21.15.148:0/3723299506 shutdown_connections 2024-03-29T04:11:04.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.970+0000 7fd98343c640 1 --2- 172.21.15.148:0/3723299506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd97c0b4fb0 0x7fd97c0cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.970+0000 7fd98343c640 1 --2- 172.21.15.148:0/3723299506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd97c15f200 0x7fd97c15f5e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.970+0000 7fd98343c640 1 -- 172.21.15.148:0/3723299506 >> 172.21.15.148:0/3723299506 conn(0x7fd97c0b3c70 msgr2=0x7fd97c0b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:04.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.970+0000 7fd98343c640 1 -- 172.21.15.148:0/3723299506 shutdown_connections 2024-03-29T04:11:04.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.970+0000 7fd98343c640 1 -- 172.21.15.148:0/3723299506 wait complete. 2024-03-29T04:11:04.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.971+0000 7fd98343c640 1 Processor -- start 2024-03-29T04:11:04.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.971+0000 7fd98343c640 1 -- start start 2024-03-29T04:11:04.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.971+0000 7fd98343c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd97c0b4fb0 0x7fd97c1506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:04.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.971+0000 7fd98343c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd97c0b5430 0x7fd97c0c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:04.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.972+0000 7fd98343c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd97c15f200 0x7fd97c0c6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:04.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.972+0000 7fd98343c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c0b7cc0 con 0x7fd97c0b5430 2024-03-29T04:11:04.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.972+0000 7fd98343c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c0b7b40 con 0x7fd97c15f200 2024-03-29T04:11:04.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.972+0000 7fd98343c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd97c0b7e40 con 0x7fd97c0b4fb0 2024-03-29T04:11:04.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.972+0000 7fd9809d7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd97c0b5430 0x7fd97c0c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:04.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.972+0000 7fd9809d7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd97c0b5430 0x7fd97c0c8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56234/0 (socket says 172.21.15.148:56234) 2024-03-29T04:11:04.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.972+0000 7fd9809d7640 1 -- 172.21.15.148:0/1176991784 learned_addr learned my addr 172.21.15.148:0/1176991784 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:04.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.972+0000 7fd9819d9640 1 --2- 172.21.15.148:0/1176991784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd97c15f200 0x7fd97c0c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:04.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.972+0000 7fd9811d8640 1 --2- 172.21.15.148:0/1176991784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd97c0b4fb0 0x7fd97c1506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:04.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.972+0000 7fd9809d7640 1 -- 172.21.15.148:0/1176991784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd97c0b4fb0 msgr2=0x7fd97c1506e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:04.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.972+0000 7fd9809d7640 1 --2- 172.21.15.148:0/1176991784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd97c0b4fb0 0x7fd97c1506e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.972+0000 7fd9809d7640 1 -- 172.21.15.148:0/1176991784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd97c15f200 msgr2=0x7fd97c0c6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:04.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.972+0000 7fd9809d7640 1 --2- 172.21.15.148:0/1176991784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd97c15f200 0x7fd97c0c6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.972+0000 7fd9809d7640 1 -- 172.21.15.148:0/1176991784 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97c0c7460 con 0x7fd97c0b5430 2024-03-29T04:11:04.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.972+0000 7fd9819d9640 1 --2- 172.21.15.148:0/1176991784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd97c15f200 0x7fd97c0c6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:04.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.972+0000 7fd9811d8640 1 --2- 172.21.15.148:0/1176991784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd97c0b4fb0 0x7fd97c1506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:04.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.972+0000 7fd9809d7640 1 --2- 172.21.15.148:0/1176991784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd97c0b5430 0x7fd97c0c8500 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00e960 tx=0x7fd96c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:04.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.972+0000 7fd971ffb640 1 -- 172.21.15.148:0/1176991784 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c017070 con 0x7fd97c0b5430 2024-03-29T04:11:04.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.972+0000 7fd971ffb640 1 -- 172.21.15.148:0/1176991784 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd96c00ca00 con 0x7fd97c0b5430 2024-03-29T04:11:04.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.972+0000 7fd971ffb640 1 -- 172.21.15.148:0/1176991784 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c00cdd0 con 0x7fd97c0b5430 2024-03-29T04:11:04.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.972+0000 7fd98343c640 1 -- 172.21.15.148:0/1176991784 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd97c0c7740 con 0x7fd97c0b5430 2024-03-29T04:11:04.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.972+0000 7fd98343c640 1 -- 172.21.15.148:0/1176991784 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd97c0c7d00 con 0x7fd97c0b5430 2024-03-29T04:11:04.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.974+0000 7fd98343c640 1 -- 172.21.15.148:0/1176991784 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd97c068850 con 0x7fd97c0b5430 2024-03-29T04:11:04.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.975+0000 7fd971ffb640 1 -- 172.21.15.148:0/1176991784 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd96c01a070 con 0x7fd97c0b5430 2024-03-29T04:11:04.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.975+0000 7fd971ffb640 1 --2- 172.21.15.148:0/1176991784 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd96006c530 0x7fd96006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:04.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.975+0000 7fd971ffb640 1 -- 172.21.15.148:0/1176991784 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd96c08bd70 con 0x7fd97c0b5430 2024-03-29T04:11:04.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.975+0000 7fd9811d8640 1 --2- 172.21.15.148:0/1176991784 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd96006c530 0x7fd96006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:04.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.976+0000 7fd9811d8640 1 --2- 172.21.15.148:0/1176991784 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd96006c530 0x7fd96006e9f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd974036000 tx=0x7fd974008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:04.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:03.979+0000 7fd971ffb640 1 -- 172.21.15.148:0/1176991784 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd96c059f90 con 0x7fd97c0b5430 2024-03-29T04:11:04.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.173+0000 7fd98343c640 1 -- 172.21.15.148:0/1176991784 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd97c0b42a0 con 0x7fd97c0b5430 2024-03-29T04:11:04.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.173+0000 7fd971ffb640 1 -- 172.21.15.148:0/1176991784 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fd96c059890 con 0x7fd97c0b5430 2024-03-29T04:11:04.181 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-29T04:11:04.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.175+0000 7fd98343c640 1 -- 172.21.15.148:0/1176991784 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd96006c530 msgr2=0x7fd96006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:04.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.175+0000 7fd98343c640 1 --2- 172.21.15.148:0/1176991784 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd96006c530 0x7fd96006e9f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd974036000 tx=0x7fd974008030 comp rx=0 tx=0).stop 2024-03-29T04:11:04.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.175+0000 7fd98343c640 1 -- 172.21.15.148:0/1176991784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd97c0b5430 msgr2=0x7fd97c0c8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:04.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.175+0000 7fd98343c640 1 --2- 172.21.15.148:0/1176991784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd97c0b5430 0x7fd97c0c8500 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00e960 tx=0x7fd96c00ee20 comp rx=0 tx=0).stop 2024-03-29T04:11:04.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.175+0000 7fd98343c640 1 -- 172.21.15.148:0/1176991784 shutdown_connections 2024-03-29T04:11:04.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.175+0000 7fd98343c640 1 --2- 172.21.15.148:0/1176991784 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd96006c530 0x7fd96006e9f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.175+0000 7fd98343c640 1 --2- 172.21.15.148:0/1176991784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd97c15f200 0x7fd97c0c6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.175+0000 7fd98343c640 1 --2- 172.21.15.148:0/1176991784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd97c0b5430 0x7fd97c0c8500 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.175+0000 7fd98343c640 1 --2- 172.21.15.148:0/1176991784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd97c0b4fb0 0x7fd97c1506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.175+0000 7fd98343c640 1 -- 172.21.15.148:0/1176991784 >> 172.21.15.148:0/1176991784 conn(0x7fd97c0b3c70 msgr2=0x7fd97c154d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:04.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.175+0000 7fd98343c640 1 -- 172.21.15.148:0/1176991784 shutdown_connections 2024-03-29T04:11:04.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.175+0000 7fd98343c640 1 -- 172.21.15.148:0/1176991784 wait complete. 2024-03-29T04:11:04.184 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:11:04.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.311+0000 7fb5f0ae4640 1 Processor -- start 2024-03-29T04:11:04.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.311+0000 7fb5f0ae4640 1 -- start start 2024-03-29T04:11:04.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.313+0000 7fb5f0ae4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb5ec06ecd0 0x7fb5ec0aff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:04.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.313+0000 7fb5f0ae4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5ec0af980 0x7fb5ec06bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:04.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.313+0000 7fb5f0ae4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5ec0b47e0 con 0x7fb5ec15ab20 2024-03-29T04:11:04.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.313+0000 7fb5f0ae4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fb5ec06fa90 con 0x7fb5ec06ecd0 2024-03-29T04:11:04.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.313+0000 7fb5f0ae4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb5ec0b4ac0 con 0x7fb5ec0af980 2024-03-29T04:11:04.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.313+0000 7fb5ea59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb5ec06ecd0 0x7fb5ec0aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:04.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.313+0000 7fb5ea59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb5ec06ecd0 0x7fb5ec0aff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56238/0 (socket says 172.21.15.148:56238) 2024-03-29T04:11:04.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.313+0000 7fb5ea59c640 1 -- 172.21.15.148:0/929432247 learned_addr learned my addr 172.21.15.148:0/929432247 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:04.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.313+0000 7fb5ead9d640 1 --2- 172.21.15.148:0/929432247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5ec0af980 0x7fb5ec06bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:04.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.313+0000 7fb5ea59c640 1 -- 172.21.15.148:0/929432247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5ec0af980 msgr2=0x7fb5ec06bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:04.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.313+0000 7fb5ea59c640 1 --2- 172.21.15.148:0/929432247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5ec0af980 0x7fb5ec06bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.313+0000 7fb5ea59c640 1 -- 172.21.15.148:0/929432247 >> v1:172.21.15.157:6789/0 conn(0x7fb5ec15ab20 legacy=0x7fb5ec06f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:04.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.313+0000 7fb5ea59c640 1 -- 172.21.15.148:0/929432247 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5ec15b960 con 0x7fb5ec06ecd0 2024-03-29T04:11:04.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.314+0000 7fb5ea59c640 1 --2- 172.21.15.148:0/929432247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5ec06ecd0 0x7fb5ec0aff20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00aa10 tx=0x7fb5dc02cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc58b8eaf8491707 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:04.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.314+0000 7fb5e959a640 1 -- 172.21.15.148:0/929432247 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc03a070 con 0x7fb5ec06ecd0 2024-03-29T04:11:04.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.314+0000 7fb5e959a640 1 -- 172.21.15.148:0/929432247 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5dc02db80 con 0x7fb5ec06ecd0 2024-03-29T04:11:04.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.314+0000 7fb5e959a640 1 -- 172.21.15.148:0/929432247 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc00b040 con 0x7fb5ec06ecd0 2024-03-29T04:11:04.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.315+0000 7fb5f0ae4640 1 -- 172.21.15.148:0/929432247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5ec06ecd0 msgr2=0x7fb5ec0aff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:04.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.315+0000 7fb5f0ae4640 1 --2- 172.21.15.148:0/929432247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5ec06ecd0 0x7fb5ec0aff20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00aa10 tx=0x7fb5dc02cdd0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.315+0000 7fb5f0ae4640 1 -- 172.21.15.148:0/929432247 shutdown_connections 2024-03-29T04:11:04.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.315+0000 7fb5f0ae4640 1 --2- 172.21.15.148:0/929432247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5ec0af980 0x7fb5ec06bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.315+0000 7fb5f0ae4640 1 --2- 172.21.15.148:0/929432247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5ec06ecd0 0x7fb5ec0aff20 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.315+0000 7fb5f0ae4640 1 -- 172.21.15.148:0/929432247 >> 172.21.15.148:0/929432247 conn(0x7fb5ec0d0ca0 msgr2=0x7fb5ec0d10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:04.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.315+0000 7fb5f0ae4640 1 -- 172.21.15.148:0/929432247 shutdown_connections 2024-03-29T04:11:04.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.315+0000 7fb5f0ae4640 1 -- 172.21.15.148:0/929432247 wait complete. 2024-03-29T04:11:04.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.316+0000 7fb5f0ae4640 1 Processor -- start 2024-03-29T04:11:04.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.316+0000 7fb5f0ae4640 1 -- start start 2024-03-29T04:11:04.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.317+0000 7fb5f0ae4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5ec06ecd0 0x7fb5ec1554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:04.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.317+0000 7fb5f0ae4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5ec0af980 0x7fb5ec0b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:04.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.317+0000 7fb5f0ae4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5ec15ab20 0x7fb5ec0b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:04.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.317+0000 7fb5f0ae4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec067cd0 con 0x7fb5ec06ecd0 2024-03-29T04:11:04.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.317+0000 7fb5f0ae4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec067b50 con 0x7fb5ec0af980 2024-03-29T04:11:04.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.317+0000 7fb5f0ae4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb5ec067e50 con 0x7fb5ec15ab20 2024-03-29T04:11:04.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.317+0000 7fb5ea59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5ec06ecd0 0x7fb5ec1554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:04.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.317+0000 7fb5ea59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5ec06ecd0 0x7fb5ec1554f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56244/0 (socket says 172.21.15.148:56244) 2024-03-29T04:11:04.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.317+0000 7fb5ea59c640 1 -- 172.21.15.148:0/1449255317 learned_addr learned my addr 172.21.15.148:0/1449255317 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:04.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.317+0000 7fb5ead9d640 1 --2- 172.21.15.148:0/1449255317 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5ec15ab20 0x7fb5ec0b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:04.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.317+0000 7fb5e9d9b640 1 --2- 172.21.15.148:0/1449255317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5ec0af980 0x7fb5ec0b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:04.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.317+0000 7fb5ea59c640 1 -- 172.21.15.148:0/1449255317 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5ec15ab20 msgr2=0x7fb5ec0b09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:04.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.317+0000 7fb5ea59c640 1 --2- 172.21.15.148:0/1449255317 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5ec15ab20 0x7fb5ec0b09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.317+0000 7fb5ea59c640 1 -- 172.21.15.148:0/1449255317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5ec0af980 msgr2=0x7fb5ec0b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:04.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.317+0000 7fb5ea59c640 1 --2- 172.21.15.148:0/1449255317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5ec0af980 0x7fb5ec0b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.317+0000 7fb5ea59c640 1 -- 172.21.15.148:0/1449255317 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5ec0ae180 con 0x7fb5ec06ecd0 2024-03-29T04:11:04.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.318+0000 7fb5ea59c640 1 --2- 172.21.15.148:0/1449255317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5ec06ecd0 0x7fb5ec1554f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00aa10 tx=0x7fb5dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:04.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.318+0000 7fb5daffd640 1 -- 172.21.15.148:0/1449255317 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc03a030 con 0x7fb5ec06ecd0 2024-03-29T04:11:04.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.318+0000 7fb5daffd640 1 -- 172.21.15.148:0/1449255317 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5dc03c070 con 0x7fb5ec06ecd0 2024-03-29T04:11:04.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.318+0000 7fb5f0ae4640 1 -- 172.21.15.148:0/1449255317 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5ec0ae460 con 0x7fb5ec06ecd0 2024-03-29T04:11:04.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.318+0000 7fb5daffd640 1 -- 172.21.15.148:0/1449255317 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc033550 con 0x7fb5ec06ecd0 2024-03-29T04:11:04.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.318+0000 7fb5f0ae4640 1 -- 172.21.15.148:0/1449255317 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5ec0aea20 con 0x7fb5ec06ecd0 2024-03-29T04:11:04.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.319+0000 7fb5f0ae4640 1 -- 172.21.15.148:0/1449255317 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5ec068850 con 0x7fb5ec06ecd0 2024-03-29T04:11:04.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.319+0000 7fb5daffd640 1 -- 172.21.15.148:0/1449255317 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb5dc03e030 con 0x7fb5ec06ecd0 2024-03-29T04:11:04.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.320+0000 7fb5daffd640 1 --2- 172.21.15.148:0/1449255317 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb5cc06c530 0x7fb5cc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:04.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.320+0000 7fb5daffd640 1 -- 172.21.15.148:0/1449255317 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb5dc0acf70 con 0x7fb5ec06ecd0 2024-03-29T04:11:04.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.320+0000 7fb5e9d9b640 1 --2- 172.21.15.148:0/1449255317 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb5cc06c530 0x7fb5cc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:04.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.320+0000 7fb5e9d9b640 1 --2- 172.21.15.148:0/1449255317 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb5cc06c530 0x7fb5cc06e9f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0006fd0 tx=0x7fb5e0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:04.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.324+0000 7fb5daffd640 1 -- 172.21.15.148:0/1449255317 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb5dc07b190 con 0x7fb5ec06ecd0 2024-03-29T04:11:04.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.596+0000 7fb5f0ae4640 1 -- 172.21.15.148:0/1449255317 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb5ec0d1120 con 0x7fb5ec06ecd0 2024-03-29T04:11:04.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.596+0000 7fb5daffd640 1 -- 172.21.15.148:0/1449255317 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fb5dc07aa90 con 0x7fb5ec06ecd0 2024-03-29T04:11:04.575 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:11:04.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.598+0000 7fb5f0ae4640 1 -- 172.21.15.148:0/1449255317 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb5cc06c530 msgr2=0x7fb5cc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:04.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.598+0000 7fb5f0ae4640 1 --2- 172.21.15.148:0/1449255317 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb5cc06c530 0x7fb5cc06e9f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0006fd0 tx=0x7fb5e0008040 comp rx=0 tx=0).stop 2024-03-29T04:11:04.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.598+0000 7fb5f0ae4640 1 -- 172.21.15.148:0/1449255317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5ec06ecd0 msgr2=0x7fb5ec1554f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:04.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.598+0000 7fb5f0ae4640 1 --2- 172.21.15.148:0/1449255317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5ec06ecd0 0x7fb5ec1554f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00aa10 tx=0x7fb5dc004730 comp rx=0 tx=0).stop 2024-03-29T04:11:04.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.598+0000 7fb5f0ae4640 1 -- 172.21.15.148:0/1449255317 shutdown_connections 2024-03-29T04:11:04.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.598+0000 7fb5f0ae4640 1 --2- 172.21.15.148:0/1449255317 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb5cc06c530 0x7fb5cc06e9f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.598+0000 7fb5f0ae4640 1 --2- 172.21.15.148:0/1449255317 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5ec15ab20 0x7fb5ec0b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.598+0000 7fb5f0ae4640 1 --2- 172.21.15.148:0/1449255317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5ec0af980 0x7fb5ec0b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.598+0000 7fb5f0ae4640 1 --2- 172.21.15.148:0/1449255317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5ec06ecd0 0x7fb5ec1554f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.598+0000 7fb5f0ae4640 1 -- 172.21.15.148:0/1449255317 >> 172.21.15.148:0/1449255317 conn(0x7fb5ec0d0ca0 msgr2=0x7fb5ec159870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:04.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.598+0000 7fb5f0ae4640 1 -- 172.21.15.148:0/1449255317 shutdown_connections 2024-03-29T04:11:04.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.598+0000 7fb5f0ae4640 1 -- 172.21.15.148:0/1449255317 wait complete. 2024-03-29T04:11:04.592 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:11:04.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.715+0000 7f12c824a640 1 Processor -- start 2024-03-29T04:11:04.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.715+0000 7f12c824a640 1 -- start start 2024-03-29T04:11:04.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.716+0000 7f12c824a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f12c015c5f0 0x7f12c00cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:04.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.716+0000 7f12c824a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12c01608b0 0x7f12c015cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:04.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.716+0000 7f12c824a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12c00d19d0 con 0x7f12c00ceb50 2024-03-29T04:11:04.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.717+0000 7f12c824a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f12c00cf470 con 0x7f12c015c5f0 2024-03-29T04:11:04.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.717+0000 7f12c824a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f12c00ccde0 con 0x7f12c01608b0 2024-03-29T04:11:04.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.717+0000 7f12c57e5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f12c015c5f0 0x7f12c00cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:04.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.717+0000 7f12c5fe6640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f12c00ceb50 0x7f12c00cfd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40006/0 (socket says 172.21.15.148:40006) 2024-03-29T04:11:04.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.717+0000 7f12c5fe6640 1 -- 172.21.15.148:0/1507742835 learned_addr learned my addr 172.21.15.148:0/1507742835 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:04.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.717+0000 7f12c67e7640 1 --2- 172.21.15.148:0/1507742835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12c01608b0 0x7f12c015cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:04.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.717+0000 7f12c4fe4640 1 -- 172.21.15.148:0/1507742835 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4269062761 0 0) 0x7f12c00d19d0 con 0x7f12c00ceb50 2024-03-29T04:11:04.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.717+0000 7f12c4fe4640 1 -- 172.21.15.148:0/1507742835 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12ac003520 con 0x7f12c00ceb50 2024-03-29T04:11:04.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.717+0000 7f12c57e5640 1 -- 172.21.15.148:0/1507742835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12c01608b0 msgr2=0x7f12c015cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:04.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.717+0000 7f12c57e5640 1 --2- 172.21.15.148:0/1507742835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12c01608b0 0x7f12c015cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.717+0000 7f12c57e5640 1 -- 172.21.15.148:0/1507742835 >> v1:172.21.15.148:6789/0 conn(0x7f12c00ceb50 legacy=0x7f12c00cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:04.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.717+0000 7f12c57e5640 1 -- 172.21.15.148:0/1507742835 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c00ccb00 con 0x7f12c015c5f0 2024-03-29T04:11:04.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.718+0000 7f12c57e5640 1 --2- 172.21.15.148:0/1507742835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12c015c5f0 0x7f12c00cef30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f12b00097a0 tx=0x7f12b002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf23a442fd5e959e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:04.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.718+0000 7f12c4fe4640 1 -- 172.21.15.148:0/1507742835 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b0035020 con 0x7f12c015c5f0 2024-03-29T04:11:04.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.718+0000 7f12c4fe4640 1 -- 172.21.15.148:0/1507742835 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12b0031830 con 0x7f12c015c5f0 2024-03-29T04:11:04.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.718+0000 7f12c4fe4640 1 -- 172.21.15.148:0/1507742835 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b0031c00 con 0x7f12c015c5f0 2024-03-29T04:11:04.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.718+0000 7f12c824a640 1 -- 172.21.15.148:0/1507742835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12c015c5f0 msgr2=0x7f12c00cef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:04.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.718+0000 7f12c824a640 1 --2- 172.21.15.148:0/1507742835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12c015c5f0 0x7f12c00cef30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f12b00097a0 tx=0x7f12b002b8d0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.718+0000 7f12c824a640 1 -- 172.21.15.148:0/1507742835 shutdown_connections 2024-03-29T04:11:04.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.718+0000 7f12c824a640 1 --2- 172.21.15.148:0/1507742835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12c01608b0 0x7f12c015cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.718+0000 7f12c824a640 1 --2- 172.21.15.148:0/1507742835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12c015c5f0 0x7f12c00cef30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.718+0000 7f12c824a640 1 -- 172.21.15.148:0/1507742835 >> 172.21.15.148:0/1507742835 conn(0x7f12c00b3c70 msgr2=0x7f12c00b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:04.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.719+0000 7f12c824a640 1 -- 172.21.15.148:0/1507742835 shutdown_connections 2024-03-29T04:11:04.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.719+0000 7f12c824a640 1 -- 172.21.15.148:0/1507742835 wait complete. 2024-03-29T04:11:04.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.720+0000 7f12c824a640 1 Processor -- start 2024-03-29T04:11:04.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.720+0000 7f12c824a640 1 -- start start 2024-03-29T04:11:04.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.720+0000 7f12c824a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12c00ceb50 0x7f12c01559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:04.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.720+0000 7f12c824a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12c01608b0 0x7f12c01f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:04.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.720+0000 7f12c824a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12c00ca5c0 0x7f12c00caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:04.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.720+0000 7f12c824a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f12c00ce540 con 0x7f12c01608b0 2024-03-29T04:11:04.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.720+0000 7f12c824a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f12c00cc790 con 0x7f12c00ceb50 2024-03-29T04:11:04.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.720+0000 7f12c824a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f12c00ce6c0 con 0x7f12c00ca5c0 2024-03-29T04:11:04.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.720+0000 7f12c57e5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12c01608b0 0x7f12c01f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:04.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.720+0000 7f12c57e5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12c01608b0 0x7f12c01f50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56266/0 (socket says 172.21.15.148:56266) 2024-03-29T04:11:04.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.720+0000 7f12c57e5640 1 -- 172.21.15.148:0/201425148 learned_addr learned my addr 172.21.15.148:0/201425148 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:04.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.721+0000 7f12c5fe6640 1 --2- 172.21.15.148:0/201425148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12c00ceb50 0x7f12c01559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:04.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.721+0000 7f12c67e7640 1 --2- 172.21.15.148:0/201425148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12c00ca5c0 0x7f12c00caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:04.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.721+0000 7f12c57e5640 1 -- 172.21.15.148:0/201425148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12c00ca5c0 msgr2=0x7f12c00caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:04.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.721+0000 7f12c57e5640 1 --2- 172.21.15.148:0/201425148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12c00ca5c0 0x7f12c00caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.721+0000 7f12c57e5640 1 -- 172.21.15.148:0/201425148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12c00ceb50 msgr2=0x7f12c01559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:04.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.721+0000 7f12c57e5640 1 --2- 172.21.15.148:0/201425148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12c00ceb50 0x7f12c01559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.721+0000 7f12c57e5640 1 -- 172.21.15.148:0/201425148 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c01f5600 con 0x7f12c01608b0 2024-03-29T04:11:04.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.721+0000 7f12c67e7640 1 --2- 172.21.15.148:0/201425148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12c00ca5c0 0x7f12c00caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:04.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.721+0000 7f12c5fe6640 1 --2- 172.21.15.148:0/201425148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12c00ceb50 0x7f12c01559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:04.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.721+0000 7f12c57e5640 1 --2- 172.21.15.148:0/201425148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12c01608b0 0x7f12c01f50c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f12b002bee0 tx=0x7f12b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:04.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.721+0000 7f12b67fc640 1 -- 172.21.15.148:0/201425148 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b0035040 con 0x7f12c01608b0 2024-03-29T04:11:04.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.721+0000 7f12b67fc640 1 -- 172.21.15.148:0/201425148 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12b003b920 con 0x7f12c01608b0 2024-03-29T04:11:04.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.721+0000 7f12c824a640 1 -- 172.21.15.148:0/201425148 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12c00c9520 con 0x7f12c01608b0 2024-03-29T04:11:04.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.721+0000 7f12b67fc640 1 -- 172.21.15.148:0/201425148 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b003bd10 con 0x7f12c01608b0 2024-03-29T04:11:04.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.721+0000 7f12c824a640 1 -- 172.21.15.148:0/201425148 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12c00c9ae0 con 0x7f12c01608b0 2024-03-29T04:11:04.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.722+0000 7f12c824a640 1 -- 172.21.15.148:0/201425148 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12c00cc580 con 0x7f12c01608b0 2024-03-29T04:11:04.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.723+0000 7f12b67fc640 1 -- 172.21.15.148:0/201425148 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f12b0038020 con 0x7f12c01608b0 2024-03-29T04:11:04.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.723+0000 7f12b67fc640 1 --2- 172.21.15.148:0/201425148 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f12a406c560 0x7f12a406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:04.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.723+0000 7f12c5fe6640 1 --2- 172.21.15.148:0/201425148 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f12a406c560 0x7f12a406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:04.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.723+0000 7f12b67fc640 1 -- 172.21.15.148:0/201425148 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f12b00adc00 con 0x7f12c01608b0 2024-03-29T04:11:04.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.724+0000 7f12c5fe6640 1 --2- 172.21.15.148:0/201425148 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f12a406c560 0x7f12a406ea20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f12bc007fd0 tx=0x7f12bc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:04.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.728+0000 7f12b67fc640 1 -- 172.21.15.148:0/201425148 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f12b00b4080 con 0x7f12c01608b0 2024-03-29T04:11:04.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.922+0000 7f12c824a640 1 -- 172.21.15.148:0/201425148 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f12c00687e0 con 0x7f12c01608b0 2024-03-29T04:11:04.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.922+0000 7f12b67fc640 1 -- 172.21.15.148:0/201425148 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f12c01f5600 con 0x7f12c01608b0 2024-03-29T04:11:04.901 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-03-29T04:11:04.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.924+0000 7f12c824a640 1 -- 172.21.15.148:0/201425148 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f12a406c560 msgr2=0x7f12a406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:04.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.924+0000 7f12c824a640 1 --2- 172.21.15.148:0/201425148 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f12a406c560 0x7f12a406ea20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f12bc007fd0 tx=0x7f12bc009040 comp rx=0 tx=0).stop 2024-03-29T04:11:04.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.924+0000 7f12c824a640 1 -- 172.21.15.148:0/201425148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12c01608b0 msgr2=0x7f12c01f50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:04.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.924+0000 7f12c824a640 1 --2- 172.21.15.148:0/201425148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12c01608b0 0x7f12c01f50c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f12b002bee0 tx=0x7f12b00046c0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.924+0000 7f12c824a640 1 -- 172.21.15.148:0/201425148 shutdown_connections 2024-03-29T04:11:04.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.924+0000 7f12c824a640 1 --2- 172.21.15.148:0/201425148 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f12a406c560 0x7f12a406ea20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.924+0000 7f12c824a640 1 --2- 172.21.15.148:0/201425148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12c00ca5c0 0x7f12c00caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.924+0000 7f12c824a640 1 --2- 172.21.15.148:0/201425148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12c01608b0 0x7f12c01f50c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.924+0000 7f12c824a640 1 --2- 172.21.15.148:0/201425148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12c00ceb50 0x7f12c01559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:04.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.924+0000 7f12c824a640 1 -- 172.21.15.148:0/201425148 >> 172.21.15.148:0/201425148 conn(0x7f12c00b3c70 msgr2=0x7f12c00b5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:04.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.924+0000 7f12c824a640 1 -- 172.21.15.148:0/201425148 shutdown_connections 2024-03-29T04:11:04.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:04.924+0000 7f12c824a640 1 -- 172.21.15.148:0/201425148 wait complete. 2024-03-29T04:11:04.915 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:11:04.915 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:11:04.931 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:11:05.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.092+0000 7f18f40c5640 1 Processor -- start 2024-03-29T04:11:05.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.092+0000 7f18f40c5640 1 -- start start 2024-03-29T04:11:05.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.093+0000 7f18f40c5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f18ec15e3e0 0x7f18ec0d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:05.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.093+0000 7f18f40c5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18ec0d10d0 0x7f18ec0d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:05.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.093+0000 7f18f40c5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18ec15e7c0 con 0x7f18ec0ba030 2024-03-29T04:11:05.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.093+0000 7f18f40c5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f18ec0d3840 con 0x7f18ec15e3e0 2024-03-29T04:11:05.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.093+0000 7f18f40c5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f18ec0d2c30 con 0x7f18ec0d10d0 2024-03-29T04:11:05.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.093+0000 7f18f1660640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f18ec0ba030 0x7f18ec0d2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40014/0 (socket says 172.21.15.148:40014) 2024-03-29T04:11:05.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.093+0000 7f18f1660640 1 -- 172.21.15.148:0/2469294951 learned_addr learned my addr 172.21.15.148:0/2469294951 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:05.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.093+0000 7f18f2662640 1 --2- 172.21.15.148:0/2469294951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18ec0d10d0 0x7f18ec0d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:05.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.093+0000 7f18f1e61640 1 --2- 172.21.15.148:0/2469294951 >> v2:172.21.15.157:3300/0 conn(0x7f18ec15e3e0 0x7f18ec0d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:05.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.093+0000 7f18f0e5f640 1 -- 172.21.15.148:0/2469294951 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 277777742 0 0) 0x7f18ec15e7c0 con 0x7f18ec0ba030 2024-03-29T04:11:05.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.093+0000 7f18f0e5f640 1 -- 172.21.15.148:0/2469294951 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18d8003520 con 0x7f18ec0ba030 2024-03-29T04:11:05.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.093+0000 7f18f0e5f640 1 -- 172.21.15.148:0/2469294951 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1316083121 0 0) 0x7f18d8003520 con 0x7f18ec0ba030 2024-03-29T04:11:05.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.093+0000 7f18f0e5f640 1 -- 172.21.15.148:0/2469294951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18ec0d10d0 msgr2=0x7f18ec0d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:05.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.093+0000 7f18f0e5f640 1 --2- 172.21.15.148:0/2469294951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18ec0d10d0 0x7f18ec0d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.093+0000 7f18f0e5f640 1 -- 172.21.15.148:0/2469294951 >> v2:172.21.15.157:3300/0 conn(0x7f18ec15e3e0 msgr2=0x7f18ec0d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:05.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.093+0000 7f18f0e5f640 1 --2- 172.21.15.148:0/2469294951 >> v2:172.21.15.157:3300/0 conn(0x7f18ec15e3e0 0x7f18ec0d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.093+0000 7f18f0e5f640 1 -- 172.21.15.148:0/2469294951 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ec0d2f10 con 0x7f18ec0ba030 2024-03-29T04:11:05.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.093+0000 7f18f0e5f640 1 -- 172.21.15.148:0/2469294951 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f18dc001690 con 0x7f18ec0ba030 2024-03-29T04:11:05.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.093+0000 7f18f0e5f640 1 -- 172.21.15.148:0/2469294951 >> v1:172.21.15.148:6789/0 conn(0x7f18ec0ba030 legacy=0x7f18ec0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:05.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.093+0000 7f18f0e5f640 1 --2- 172.21.15.148:0/2469294951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18d80039b0 0x7f18d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:05.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.093+0000 7f18f0e5f640 1 --2- 172.21.15.148:0/2469294951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d80043a0 0x7f18d8004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:05.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.094+0000 7f18f0e5f640 1 --2- 172.21.15.148:0/2469294951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d8004ef0 0x7f18d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:05.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.094+0000 7f18f0e5f640 1 -- 172.21.15.148:0/2469294951 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec0d3840 con 0x7f18d80043a0 2024-03-29T04:11:05.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.094+0000 7f18f0e5f640 1 -- 172.21.15.148:0/2469294951 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec0d2c30 con 0x7f18d8004ef0 2024-03-29T04:11:05.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.094+0000 7f18f0e5f640 1 -- 172.21.15.148:0/2469294951 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f18d800bf00 con 0x7f18d80039b0 2024-03-29T04:11:05.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.094+0000 7f18f1e61640 1 --2- 172.21.15.148:0/2469294951 >> v2:172.21.15.157:3300/0 conn(0x7f18ec15e3e0 0x7f18ec0d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:11:05.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.094+0000 7f18f2662640 1 --2- 172.21.15.148:0/2469294951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18ec0d10d0 0x7f18ec0d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:11:05.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.094+0000 7f18f1660640 1 --2- 172.21.15.148:0/2469294951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d80043a0 0x7f18d8004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:05.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.094+0000 7f18f1e61640 1 --2- 172.21.15.148:0/2469294951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18d80039b0 0x7f18d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:05.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.094+0000 7f18f2662640 1 --2- 172.21.15.148:0/2469294951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d8004ef0 0x7f18d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:05.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.094+0000 7f18f1660640 1 -- 172.21.15.148:0/2469294951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18d80039b0 msgr2=0x7f18d8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:05.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.094+0000 7f18f1660640 1 --2- 172.21.15.148:0/2469294951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18d80039b0 0x7f18d8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.094+0000 7f18f1660640 1 -- 172.21.15.148:0/2469294951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d8004ef0 msgr2=0x7f18d800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:05.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.094+0000 7f18f1660640 1 --2- 172.21.15.148:0/2469294951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d8004ef0 0x7f18d800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.094+0000 7f18f1660640 1 -- 172.21.15.148:0/2469294951 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d8003520 con 0x7f18d80043a0 2024-03-29T04:11:05.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.094+0000 7f18f2662640 1 --2- 172.21.15.148:0/2469294951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d8004ef0 0x7f18d800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:05.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.095+0000 7f18f1660640 1 --2- 172.21.15.148:0/2469294951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d80043a0 0x7f18d8004880 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00c210 tx=0x7f18d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2da7a4bffbf029e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:05.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.095+0000 7f18f0e5f640 1 -- 172.21.15.148:0/2469294951 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc007d50 con 0x7f18d80043a0 2024-03-29T04:11:05.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.095+0000 7f18f0e5f640 1 -- 172.21.15.148:0/2469294951 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18dc030070 con 0x7f18d80043a0 2024-03-29T04:11:05.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.095+0000 7f18f0e5f640 1 -- 172.21.15.148:0/2469294951 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc005d10 con 0x7f18d80043a0 2024-03-29T04:11:05.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.095+0000 7f18f40c5640 1 -- 172.21.15.148:0/2469294951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d80043a0 msgr2=0x7f18d8004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:05.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.095+0000 7f18f40c5640 1 --2- 172.21.15.148:0/2469294951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d80043a0 0x7f18d8004880 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00c210 tx=0x7f18d8005830 comp rx=0 tx=0).stop 2024-03-29T04:11:05.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.095+0000 7f18f2662640 1 -- 172.21.15.148:0/2469294951 reap_dead start 2024-03-29T04:11:05.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.095+0000 7f18f40c5640 1 -- 172.21.15.148:0/2469294951 shutdown_connections 2024-03-29T04:11:05.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.095+0000 7f18f40c5640 1 -- 172.21.15.148:0/2469294951 >> 172.21.15.148:0/2469294951 conn(0x7f18ec0b6210 msgr2=0x7f18ec0b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:05.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.096+0000 7f18f40c5640 1 -- 172.21.15.148:0/2469294951 shutdown_connections 2024-03-29T04:11:05.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.096+0000 7f18f40c5640 1 -- 172.21.15.148:0/2469294951 wait complete. 2024-03-29T04:11:05.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.097+0000 7f18f40c5640 1 Processor -- start 2024-03-29T04:11:05.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.097+0000 7f18f40c5640 1 -- start start 2024-03-29T04:11:05.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.097+0000 7f18f40c5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d80043a0 0x7f18ec157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:05.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.097+0000 7f18f40c5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18ec15fa00 0x7f18ec15fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:05.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.097+0000 7f18f40c5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18ec1603a0 0x7f18ec15c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:05.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.097+0000 7f18f40c5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec0d1580 con 0x7f18ec15fa00 2024-03-29T04:11:05.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.097+0000 7f18f40c5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec0d34d0 con 0x7f18d80043a0 2024-03-29T04:11:05.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.097+0000 7f18f40c5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f18ec0d1700 con 0x7f18ec1603a0 2024-03-29T04:11:05.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.098+0000 7f18f1660640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18ec15fa00 0x7f18ec15fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:05.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.098+0000 7f18f1660640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18ec15fa00 0x7f18ec15fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56286/0 (socket says 172.21.15.148:56286) 2024-03-29T04:11:05.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.098+0000 7f18f1660640 1 -- 172.21.15.148:0/1001692196 learned_addr learned my addr 172.21.15.148:0/1001692196 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:05.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.098+0000 7f18f2662640 1 --2- 172.21.15.148:0/1001692196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18ec1603a0 0x7f18ec15c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:05.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.098+0000 7f18f1e61640 1 --2- 172.21.15.148:0/1001692196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d80043a0 0x7f18ec157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:05.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.098+0000 7f18f1660640 1 -- 172.21.15.148:0/1001692196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18ec1603a0 msgr2=0x7f18ec15c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:05.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.098+0000 7f18f1660640 1 --2- 172.21.15.148:0/1001692196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18ec1603a0 0x7f18ec15c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.098+0000 7f18f1660640 1 -- 172.21.15.148:0/1001692196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d80043a0 msgr2=0x7f18ec157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:05.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.098+0000 7f18f1660640 1 --2- 172.21.15.148:0/1001692196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d80043a0 0x7f18ec157e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.098+0000 7f18f1660640 1 -- 172.21.15.148:0/1001692196 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ec1f3eb0 con 0x7f18ec15fa00 2024-03-29T04:11:05.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.098+0000 7f18f1660640 1 --2- 172.21.15.148:0/1001692196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18ec15fa00 0x7f18ec15fe60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f18dc02c540 tx=0x7f18ec0d30e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:05.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.098+0000 7f18e27fc640 1 -- 172.21.15.148:0/1001692196 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc001c90 con 0x7f18ec15fa00 2024-03-29T04:11:05.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.098+0000 7f18e27fc640 1 -- 172.21.15.148:0/1001692196 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18dc004c10 con 0x7f18ec15fa00 2024-03-29T04:11:05.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.098+0000 7f18f40c5640 1 -- 172.21.15.148:0/1001692196 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18ec15a480 con 0x7f18ec15fa00 2024-03-29T04:11:05.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.098+0000 7f18e27fc640 1 -- 172.21.15.148:0/1001692196 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc007070 con 0x7f18ec15fa00 2024-03-29T04:11:05.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.098+0000 7f18f40c5640 1 -- 172.21.15.148:0/1001692196 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18ec15b990 con 0x7f18ec15fa00 2024-03-29T04:11:05.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.099+0000 7f18f40c5640 1 -- 172.21.15.148:0/1001692196 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18ec0687e0 con 0x7f18ec15fa00 2024-03-29T04:11:05.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.100+0000 7f18e27fc640 1 -- 172.21.15.148:0/1001692196 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f18dc03f070 con 0x7f18ec15fa00 2024-03-29T04:11:05.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.100+0000 7f18e27fc640 1 --2- 172.21.15.148:0/1001692196 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f18d006c560 0x7f18d006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:05.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.100+0000 7f18e27fc640 1 -- 172.21.15.148:0/1001692196 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f18dc0acc40 con 0x7f18ec15fa00 2024-03-29T04:11:05.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.100+0000 7f18f1e61640 1 --2- 172.21.15.148:0/1001692196 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f18d006c560 0x7f18d006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:05.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.101+0000 7f18f1e61640 1 --2- 172.21.15.148:0/1001692196 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f18d006c560 0x7f18d006ea20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f18e80093d0 tx=0x7f18e8009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:05.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.105+0000 7f18e27fc640 1 -- 172.21.15.148:0/1001692196 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f18dc07a8f0 con 0x7f18ec15fa00 2024-03-29T04:11:05.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.286+0000 7f18f40c5640 1 -- 172.21.15.148:0/1001692196 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18ec0cdf40 con 0x7f18ec15fa00 2024-03-29T04:11:05.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.287+0000 7f18e27fc640 1 -- 172.21.15.148:0/1001692196 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18858 (secure 0 0 0) 0x7f18ec0cdf40 con 0x7f18ec15fa00 2024-03-29T04:11:05.266 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:11:05.266 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:41.662934+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:11:05.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.289+0000 7f18f40c5640 1 -- 172.21.15.148:0/1001692196 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f18d006c560 msgr2=0x7f18d006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:05.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.289+0000 7f18f40c5640 1 --2- 172.21.15.148:0/1001692196 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f18d006c560 0x7f18d006ea20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f18e80093d0 tx=0x7f18e8009190 comp rx=0 tx=0).stop 2024-03-29T04:11:05.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.289+0000 7f18f40c5640 1 -- 172.21.15.148:0/1001692196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18ec15fa00 msgr2=0x7f18ec15fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:05.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.289+0000 7f18f40c5640 1 --2- 172.21.15.148:0/1001692196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18ec15fa00 0x7f18ec15fe60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f18dc02c540 tx=0x7f18ec0d30e0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.289+0000 7f18f40c5640 1 -- 172.21.15.148:0/1001692196 shutdown_connections 2024-03-29T04:11:05.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.289+0000 7f18f40c5640 1 --2- 172.21.15.148:0/1001692196 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f18d006c560 0x7f18d006ea20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.289+0000 7f18f40c5640 1 --2- 172.21.15.148:0/1001692196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18ec1603a0 0x7f18ec15c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.289+0000 7f18f40c5640 1 --2- 172.21.15.148:0/1001692196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18ec15fa00 0x7f18ec15fe60 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.289+0000 7f18f40c5640 1 --2- 172.21.15.148:0/1001692196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d80043a0 0x7f18ec157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.289+0000 7f18f40c5640 1 -- 172.21.15.148:0/1001692196 >> 172.21.15.148:0/1001692196 conn(0x7f18ec0b6210 msgr2=0x7f18ec0ca3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:05.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.289+0000 7f18f40c5640 1 -- 172.21.15.148:0/1001692196 shutdown_connections 2024-03-29T04:11:05.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.289+0000 7f18f40c5640 1 -- 172.21.15.148:0/1001692196 wait complete. 2024-03-29T04:11:05.280 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:47.898351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:57.922313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:58.829405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:10:02.551763+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:11:05.281 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:11:05.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.409+0000 7fce4e13d640 1 Processor -- start 2024-03-29T04:11:05.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.409+0000 7fce4e13d640 1 -- start start 2024-03-29T04:11:05.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.410+0000 7fce4e13d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fce4815e3c0 0x7fce48161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:05.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.410+0000 7fce4e13d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fce480b2f00 0x7fce481624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:05.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.410+0000 7fce4e13d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce480b3430 0x7fce480d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:05.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.410+0000 7fce4e13d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fce480cf960 con 0x7fce4815e3c0 2024-03-29T04:11:05.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.410+0000 7fce4e13d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fce480cf330 con 0x7fce480b2f00 2024-03-29T04:11:05.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.410+0000 7fce4e13d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fce48162cf0 con 0x7fce480b3430 2024-03-29T04:11:05.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.410+0000 7fce477fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fce4815e3c0 0x7fce48161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:05.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.410+0000 7fce477fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fce4815e3c0 0x7fce48161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56290/0 (socket says 172.21.15.148:56290) 2024-03-29T04:11:05.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.410+0000 7fce477fe640 1 -- 172.21.15.148:0/728373962 learned_addr learned my addr 172.21.15.148:0/728373962 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:05.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.410+0000 7fce477fe640 1 -- 172.21.15.148:0/728373962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce480b3430 msgr2=0x7fce480d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:05.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.410+0000 7fce477fe640 1 --2- 172.21.15.148:0/728373962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce480b3430 0x7fce480d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.410+0000 7fce477fe640 1 -- 172.21.15.148:0/728373962 >> v2:172.21.15.157:3300/0 conn(0x7fce480b2f00 msgr2=0x7fce481624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:05.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.410+0000 7fce477fe640 1 --2- 172.21.15.148:0/728373962 >> v2:172.21.15.157:3300/0 conn(0x7fce480b2f00 0x7fce481624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.410+0000 7fce477fe640 1 -- 172.21.15.148:0/728373962 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce480cf5f0 con 0x7fce4815e3c0 2024-03-29T04:11:05.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.411+0000 7fce477fe640 1 --2- 172.21.15.148:0/728373962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce4815e3c0 0x7fce48161f70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fce480c7be0 tx=0x7fce3802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739cc8fd262d4ef7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:05.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.411+0000 7fce46ffd640 1 -- 172.21.15.148:0/728373962 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce38035020 con 0x7fce4815e3c0 2024-03-29T04:11:05.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.411+0000 7fce46ffd640 1 -- 172.21.15.148:0/728373962 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce38031870 con 0x7fce4815e3c0 2024-03-29T04:11:05.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.411+0000 7fce46ffd640 1 -- 172.21.15.148:0/728373962 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce38031c40 con 0x7fce4815e3c0 2024-03-29T04:11:05.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.412+0000 7fce4e13d640 1 -- 172.21.15.148:0/728373962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce4815e3c0 msgr2=0x7fce48161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:05.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.412+0000 7fce4e13d640 1 --2- 172.21.15.148:0/728373962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce4815e3c0 0x7fce48161f70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fce480c7be0 tx=0x7fce3802b910 comp rx=0 tx=0).stop 2024-03-29T04:11:05.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.412+0000 7fce4e13d640 1 -- 172.21.15.148:0/728373962 shutdown_connections 2024-03-29T04:11:05.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.412+0000 7fce4e13d640 1 --2- 172.21.15.148:0/728373962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce480b3430 0x7fce480d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.412+0000 7fce4e13d640 1 --2- 172.21.15.148:0/728373962 >> v2:172.21.15.157:3300/0 conn(0x7fce480b2f00 0x7fce481624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.412+0000 7fce4e13d640 1 --2- 172.21.15.148:0/728373962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce4815e3c0 0x7fce48161f70 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.412+0000 7fce4e13d640 1 -- 172.21.15.148:0/728373962 >> 172.21.15.148:0/728373962 conn(0x7fce480b6430 msgr2=0x7fce480d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:05.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.412+0000 7fce4e13d640 1 -- 172.21.15.148:0/728373962 shutdown_connections 2024-03-29T04:11:05.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.412+0000 7fce4e13d640 1 -- 172.21.15.148:0/728373962 wait complete. 2024-03-29T04:11:05.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.413+0000 7fce4e13d640 1 Processor -- start 2024-03-29T04:11:05.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.413+0000 7fce4e13d640 1 -- start start 2024-03-29T04:11:05.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.414+0000 7fce4e13d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce480b3430 0x7fce4814c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:05.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.414+0000 7fce4e13d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce4815e3c0 0x7fce48160380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:05.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.414+0000 7fce4e13d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce481578d0 0x7fce480cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:05.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.414+0000 7fce4e13d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fce480b39a0 con 0x7fce480b3430 2024-03-29T04:11:05.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.414+0000 7fce4e13d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fce4815e7a0 con 0x7fce481578d0 2024-03-29T04:11:05.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.414+0000 7fce4e13d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fce480b3b20 con 0x7fce4815e3c0 2024-03-29T04:11:05.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.414+0000 7fce477fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce480b3430 0x7fce4814c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:05.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.414+0000 7fce477fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce480b3430 0x7fce4814c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56304/0 (socket says 172.21.15.148:56304) 2024-03-29T04:11:05.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.414+0000 7fce477fe640 1 -- 172.21.15.148:0/3606547749 learned_addr learned my addr 172.21.15.148:0/3606547749 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:05.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.414+0000 7fce3effd640 1 --2- 172.21.15.148:0/3606547749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce4815e3c0 0x7fce48160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:05.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.414+0000 7fce47fff640 1 --2- 172.21.15.148:0/3606547749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce481578d0 0x7fce480cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:05.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.414+0000 7fce477fe640 1 -- 172.21.15.148:0/3606547749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce4815e3c0 msgr2=0x7fce48160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:05.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.414+0000 7fce477fe640 1 --2- 172.21.15.148:0/3606547749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce4815e3c0 0x7fce48160380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.414+0000 7fce477fe640 1 -- 172.21.15.148:0/3606547749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce481578d0 msgr2=0x7fce480cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:05.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.414+0000 7fce477fe640 1 --2- 172.21.15.148:0/3606547749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce481578d0 0x7fce480cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.414+0000 7fce477fe640 1 -- 172.21.15.148:0/3606547749 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce48160c60 con 0x7fce480b3430 2024-03-29T04:11:05.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.414+0000 7fce47fff640 1 --2- 172.21.15.148:0/3606547749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce481578d0 0x7fce480cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:05.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.415+0000 7fce477fe640 1 --2- 172.21.15.148:0/3606547749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce480b3430 0x7fce4814c6f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fce3802b950 tx=0x7fce380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:05.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.415+0000 7fce44ff9640 1 -- 172.21.15.148:0/3606547749 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce38040070 con 0x7fce480b3430 2024-03-29T04:11:05.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.415+0000 7fce44ff9640 1 -- 172.21.15.148:0/3606547749 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce3803a420 con 0x7fce480b3430 2024-03-29T04:11:05.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.415+0000 7fce4e13d640 1 -- 172.21.15.148:0/3606547749 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce48160f40 con 0x7fce480b3430 2024-03-29T04:11:05.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.415+0000 7fce44ff9640 1 -- 172.21.15.148:0/3606547749 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce38035040 con 0x7fce480b3430 2024-03-29T04:11:05.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.415+0000 7fce4e13d640 1 -- 172.21.15.148:0/3606547749 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce480c7ea0 con 0x7fce480b3430 2024-03-29T04:11:05.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.416+0000 7fce4e13d640 1 -- 172.21.15.148:0/3606547749 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce48068850 con 0x7fce480b3430 2024-03-29T04:11:05.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.417+0000 7fce44ff9640 1 -- 172.21.15.148:0/3606547749 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fce38038020 con 0x7fce480b3430 2024-03-29T04:11:05.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.418+0000 7fce44ff9640 1 --2- 172.21.15.148:0/3606547749 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fce2806c560 0x7fce2806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:05.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.418+0000 7fce44ff9640 1 -- 172.21.15.148:0/3606547749 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fce380adb10 con 0x7fce480b3430 2024-03-29T04:11:05.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.418+0000 7fce3effd640 1 --2- 172.21.15.148:0/3606547749 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fce2806c560 0x7fce2806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:05.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.418+0000 7fce3effd640 1 --2- 172.21.15.148:0/3606547749 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fce2806c560 0x7fce2806ea20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fce481f7950 tx=0x7fce30008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:05.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.422+0000 7fce44ff9640 1 -- 172.21.15.148:0/3606547749 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fce3807bd30 con 0x7fce480b3430 2024-03-29T04:11:05.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.606+0000 7fce4e13d640 1 -- 172.21.15.148:0/3606547749 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fce480cf110 con 0x7fce480b3430 2024-03-29T04:11:05.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.607+0000 7fce44ff9640 1 -- 172.21.15.148:0/3606547749 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fce3807b630 con 0x7fce480b3430 2024-03-29T04:11:05.586 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-29T04:11:05.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.609+0000 7fce4e13d640 1 -- 172.21.15.148:0/3606547749 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fce2806c560 msgr2=0x7fce2806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:05.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.609+0000 7fce4e13d640 1 --2- 172.21.15.148:0/3606547749 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fce2806c560 0x7fce2806ea20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fce481f7950 tx=0x7fce30008040 comp rx=0 tx=0).stop 2024-03-29T04:11:05.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.609+0000 7fce4e13d640 1 -- 172.21.15.148:0/3606547749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce480b3430 msgr2=0x7fce4814c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:05.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.609+0000 7fce4e13d640 1 --2- 172.21.15.148:0/3606547749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce480b3430 0x7fce4814c6f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fce3802b950 tx=0x7fce380046c0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.609+0000 7fce4e13d640 1 -- 172.21.15.148:0/3606547749 shutdown_connections 2024-03-29T04:11:05.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.609+0000 7fce4e13d640 1 --2- 172.21.15.148:0/3606547749 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fce2806c560 0x7fce2806ea20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.609+0000 7fce4e13d640 1 --2- 172.21.15.148:0/3606547749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce481578d0 0x7fce480cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.609+0000 7fce4e13d640 1 --2- 172.21.15.148:0/3606547749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce4815e3c0 0x7fce48160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.609+0000 7fce4e13d640 1 --2- 172.21.15.148:0/3606547749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce480b3430 0x7fce4814c6f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.609+0000 7fce4e13d640 1 -- 172.21.15.148:0/3606547749 >> 172.21.15.148:0/3606547749 conn(0x7fce480b6430 msgr2=0x7fce480b5640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:05.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.609+0000 7fce4e13d640 1 -- 172.21.15.148:0/3606547749 shutdown_connections 2024-03-29T04:11:05.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.609+0000 7fce4e13d640 1 -- 172.21.15.148:0/3606547749 wait complete. 2024-03-29T04:11:05.603 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:11:05.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.734+0000 7ff81938d640 1 Processor -- start 2024-03-29T04:11:05.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.734+0000 7ff81938d640 1 -- start start 2024-03-29T04:11:05.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.735+0000 7ff81938d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff81406c5c0 0x7ff814071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:05.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.735+0000 7ff81938d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff81406c9a0 0x7ff81415f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:05.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.735+0000 7ff81938d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff81415f9b0 con 0x7ff81406d5c0 2024-03-29T04:11:05.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.735+0000 7ff81938d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7ff814071dc0 con 0x7ff81406c5c0 2024-03-29T04:11:05.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.735+0000 7ff81938d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff8140720a0 con 0x7ff81406c9a0 2024-03-29T04:11:05.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.735+0000 7ff812ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff81406c5c0 0x7ff814071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:05.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.735+0000 7ff8137fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff81406d5c0 0x7ff81406da50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35656/0 (socket says 172.21.15.148:35656) 2024-03-29T04:11:05.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.735+0000 7ff8137fe640 1 -- 172.21.15.148:0/2994475748 learned_addr learned my addr 172.21.15.148:0/2994475748 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:05.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.735+0000 7ff8127fc640 1 --2- 172.21.15.148:0/2994475748 >> v2:172.21.15.157:3300/0 conn(0x7ff81406c9a0 0x7ff81415f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:05.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.735+0000 7ff812ffd640 1 -- 172.21.15.148:0/2994475748 >> v2:172.21.15.157:3300/0 conn(0x7ff81406c9a0 msgr2=0x7ff81415f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:05.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.735+0000 7ff812ffd640 1 --2- 172.21.15.148:0/2994475748 >> v2:172.21.15.157:3300/0 conn(0x7ff81406c9a0 0x7ff81415f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.735+0000 7ff812ffd640 1 -- 172.21.15.148:0/2994475748 >> v1:172.21.15.148:6789/0 conn(0x7ff81406d5c0 legacy=0x7ff81406da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:05.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.735+0000 7ff811ffb640 1 -- 172.21.15.148:0/2994475748 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2605550730 0 0) 0x7ff81415f9b0 con 0x7ff81406d5c0 2024-03-29T04:11:05.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.735+0000 7ff812ffd640 1 -- 172.21.15.148:0/2994475748 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff814072380 con 0x7ff81406c5c0 2024-03-29T04:11:05.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.736+0000 7ff812ffd640 1 --2- 172.21.15.148:0/2994475748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff81406c5c0 0x7ff814071740 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff81406f230 tx=0x7ff80402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7e325850404326 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:05.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.736+0000 7ff811ffb640 1 -- 172.21.15.148:0/2994475748 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff804035020 con 0x7ff81406c5c0 2024-03-29T04:11:05.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.736+0000 7ff811ffb640 1 -- 172.21.15.148:0/2994475748 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff804031830 con 0x7ff81406c5c0 2024-03-29T04:11:05.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.736+0000 7ff811ffb640 1 -- 172.21.15.148:0/2994475748 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff804031c00 con 0x7ff81406c5c0 2024-03-29T04:11:05.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.737+0000 7ff81938d640 1 -- 172.21.15.148:0/2994475748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff81406c5c0 msgr2=0x7ff814071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:05.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.737+0000 7ff81938d640 1 --2- 172.21.15.148:0/2994475748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff81406c5c0 0x7ff814071740 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff81406f230 tx=0x7ff80402b8d0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.737+0000 7ff81938d640 1 -- 172.21.15.148:0/2994475748 shutdown_connections 2024-03-29T04:11:05.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.737+0000 7ff81938d640 1 --2- 172.21.15.148:0/2994475748 >> v2:172.21.15.157:3300/0 conn(0x7ff81406c9a0 0x7ff81415f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.737+0000 7ff81938d640 1 --2- 172.21.15.148:0/2994475748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff81406c5c0 0x7ff814071740 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.737+0000 7ff81938d640 1 -- 172.21.15.148:0/2994475748 >> 172.21.15.148:0/2994475748 conn(0x7ff8140d00a0 msgr2=0x7ff8140d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:05.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.737+0000 7ff81938d640 1 -- 172.21.15.148:0/2994475748 shutdown_connections 2024-03-29T04:11:05.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.737+0000 7ff81938d640 1 -- 172.21.15.148:0/2994475748 wait complete. 2024-03-29T04:11:05.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.738+0000 7ff81938d640 1 Processor -- start 2024-03-29T04:11:05.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.738+0000 7ff81938d640 1 -- start start 2024-03-29T04:11:05.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.739+0000 7ff81938d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff81406c5c0 0x7ff814143a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:05.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.739+0000 7ff81938d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff81406c9a0 0x7ff8141f7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:05.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.739+0000 7ff81938d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff81406d5c0 0x7ff8141f7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:05.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.739+0000 7ff81938d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff814158e90 con 0x7ff81406c5c0 2024-03-29T04:11:05.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.739+0000 7ff81938d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff814072940 con 0x7ff81406d5c0 2024-03-29T04:11:05.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.739+0000 7ff81938d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff81406ce20 con 0x7ff81406c9a0 2024-03-29T04:11:05.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.739+0000 7ff812ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff81406c5c0 0x7ff814143a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:05.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.739+0000 7ff812ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff81406c5c0 0x7ff814143a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43074/0 (socket says 172.21.15.148:43074) 2024-03-29T04:11:05.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.739+0000 7ff812ffd640 1 -- 172.21.15.148:0/646301714 learned_addr learned my addr 172.21.15.148:0/646301714 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:05.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.739+0000 7ff8137fe640 1 --2- 172.21.15.148:0/646301714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff81406d5c0 0x7ff8141f7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:05.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.739+0000 7ff8127fc640 1 --2- 172.21.15.148:0/646301714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff81406c9a0 0x7ff8141f7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:05.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.740+0000 7ff812ffd640 1 -- 172.21.15.148:0/646301714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff81406c9a0 msgr2=0x7ff8141f7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:05.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.740+0000 7ff812ffd640 1 --2- 172.21.15.148:0/646301714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff81406c9a0 0x7ff8141f7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.740+0000 7ff812ffd640 1 -- 172.21.15.148:0/646301714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff81406d5c0 msgr2=0x7ff8141f7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:05.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.740+0000 7ff812ffd640 1 --2- 172.21.15.148:0/646301714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff81406d5c0 0x7ff8141f7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.740+0000 7ff812ffd640 1 -- 172.21.15.148:0/646301714 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff814143f60 con 0x7ff81406c5c0 2024-03-29T04:11:05.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.740+0000 7ff8137fe640 1 --2- 172.21.15.148:0/646301714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff81406d5c0 0x7ff8141f7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:05.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.740+0000 7ff812ffd640 1 --2- 172.21.15.148:0/646301714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff81406c5c0 0x7ff814143a20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff804000c00 tx=0x7ff8040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:05.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.740+0000 7ff7fb7fe640 1 -- 172.21.15.148:0/646301714 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80403a970 con 0x7ff81406c5c0 2024-03-29T04:11:05.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.740+0000 7ff7fb7fe640 1 -- 172.21.15.148:0/646301714 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff80403ab30 con 0x7ff81406c5c0 2024-03-29T04:11:05.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.740+0000 7ff81938d640 1 -- 172.21.15.148:0/646301714 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8141f82c0 con 0x7ff81406c5c0 2024-03-29T04:11:05.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.740+0000 7ff7fb7fe640 1 -- 172.21.15.148:0/646301714 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8040443f0 con 0x7ff81406c5c0 2024-03-29T04:11:05.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.740+0000 7ff81938d640 1 -- 172.21.15.148:0/646301714 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8140cc310 con 0x7ff81406c5c0 2024-03-29T04:11:05.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.741+0000 7ff81938d640 1 -- 172.21.15.148:0/646301714 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8140d06b0 con 0x7ff81406c5c0 2024-03-29T04:11:05.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.741+0000 7ff7fb7fe640 1 -- 172.21.15.148:0/646301714 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff80404b060 con 0x7ff81406c5c0 2024-03-29T04:11:05.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.742+0000 7ff7fb7fe640 1 --2- 172.21.15.148:0/646301714 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff80006c530 0x7ff80006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:05.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.742+0000 7ff7fb7fe640 1 -- 172.21.15.148:0/646301714 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff8040ad680 con 0x7ff81406c5c0 2024-03-29T04:11:05.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.742+0000 7ff8127fc640 1 --2- 172.21.15.148:0/646301714 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff80006c530 0x7ff80006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:05.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.742+0000 7ff8127fc640 1 --2- 172.21.15.148:0/646301714 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff80006c530 0x7ff80006e9f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff81406e410 tx=0x7ff808008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:05.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:05.747+0000 7ff7fb7fe640 1 -- 172.21.15.148:0/646301714 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff8140d06b0 con 0x7ff81406c5c0 2024-03-29T04:11:05.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.013+0000 7ff81938d640 1 -- 172.21.15.148:0/646301714 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff814068850 con 0x7ff81406c5c0 2024-03-29T04:11:05.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.013+0000 7ff7fb7fe640 1 -- 172.21.15.148:0/646301714 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7ff80403f070 con 0x7ff81406c5c0 2024-03-29T04:11:05.992 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:11:05.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.015+0000 7ff81938d640 1 -- 172.21.15.148:0/646301714 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff80006c530 msgr2=0x7ff80006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:05.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.015+0000 7ff81938d640 1 --2- 172.21.15.148:0/646301714 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff80006c530 0x7ff80006e9f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff81406e410 tx=0x7ff808008040 comp rx=0 tx=0).stop 2024-03-29T04:11:05.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.015+0000 7ff81938d640 1 -- 172.21.15.148:0/646301714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff81406c5c0 msgr2=0x7ff814143a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:05.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.015+0000 7ff81938d640 1 --2- 172.21.15.148:0/646301714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff81406c5c0 0x7ff814143a20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff804000c00 tx=0x7ff8040046c0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.015+0000 7ff81938d640 1 -- 172.21.15.148:0/646301714 shutdown_connections 2024-03-29T04:11:05.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.015+0000 7ff81938d640 1 --2- 172.21.15.148:0/646301714 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff80006c530 0x7ff80006e9f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.015+0000 7ff81938d640 1 --2- 172.21.15.148:0/646301714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff81406d5c0 0x7ff8141f7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.015+0000 7ff81938d640 1 --2- 172.21.15.148:0/646301714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff81406c9a0 0x7ff8141f7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.015+0000 7ff81938d640 1 --2- 172.21.15.148:0/646301714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff81406c5c0 0x7ff814143a20 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:05.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.015+0000 7ff81938d640 1 -- 172.21.15.148:0/646301714 >> 172.21.15.148:0/646301714 conn(0x7ff8140d00a0 msgr2=0x7ff814161810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:05.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.016+0000 7ff81938d640 1 -- 172.21.15.148:0/646301714 shutdown_connections 2024-03-29T04:11:05.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.016+0000 7ff81938d640 1 -- 172.21.15.148:0/646301714 wait complete. 2024-03-29T04:11:06.006 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:11:06.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.136+0000 7f248473d640 1 Processor -- start 2024-03-29T04:11:06.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.136+0000 7f248473d640 1 -- start start 2024-03-29T04:11:06.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.138+0000 7f248473d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f247c15e3c0 0x7f247c161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:06.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.138+0000 7f248473d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f247c0b3430 0x7f247c06b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:06.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.138+0000 7f248473d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f247c0cf330 con 0x7f247c0b2f00 2024-03-29T04:11:06.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.138+0000 7f248473d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f247c0cf960 con 0x7f247c15e3c0 2024-03-29T04:11:06.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.138+0000 7f248473d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f247c162cf0 con 0x7f247c0b3430 2024-03-29T04:11:06.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.138+0000 7f24824d9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f247c15e3c0 0x7f247c161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:06.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.138+0000 7f2481cd8640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f247c0b2f00 0x7f247c1624b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35670/0 (socket says 172.21.15.148:35670) 2024-03-29T04:11:06.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.138+0000 7f2481cd8640 1 -- 172.21.15.148:0/2443402655 learned_addr learned my addr 172.21.15.148:0/2443402655 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:06.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.138+0000 7f24824d9640 1 -- 172.21.15.148:0/2443402655 >> v2:172.21.15.157:3300/0 conn(0x7f247c0b3430 msgr2=0x7f247c06b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:06.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.138+0000 7f24824d9640 1 --2- 172.21.15.148:0/2443402655 >> v2:172.21.15.157:3300/0 conn(0x7f247c0b3430 0x7f247c06b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:06.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.138+0000 7f24824d9640 1 -- 172.21.15.148:0/2443402655 >> v1:172.21.15.148:6789/0 conn(0x7f247c0b2f00 legacy=0x7f247c1624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:06.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.138+0000 7f24824d9640 1 -- 172.21.15.148:0/2443402655 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247c0cf5f0 con 0x7f247c15e3c0 2024-03-29T04:11:06.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.138+0000 7f24814d7640 1 -- 172.21.15.148:0/2443402655 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 975137383 0 0) 0x7f247c0cf330 con 0x7f247c0b2f00 2024-03-29T04:11:06.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.139+0000 7f24824d9640 1 --2- 172.21.15.148:0/2443402655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f247c15e3c0 0x7f247c161f70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f247c0cf8d0 tx=0x7f246c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b089342c3d5e5afd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:06.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.139+0000 7f24814d7640 1 -- 172.21.15.148:0/2443402655 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c0315a0 con 0x7f247c15e3c0 2024-03-29T04:11:06.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.139+0000 7f24814d7640 1 -- 172.21.15.148:0/2443402655 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f246c037070 con 0x7f247c15e3c0 2024-03-29T04:11:06.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.139+0000 7f24814d7640 1 -- 172.21.15.148:0/2443402655 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c031960 con 0x7f247c15e3c0 2024-03-29T04:11:06.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.140+0000 7f248473d640 1 -- 172.21.15.148:0/2443402655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f247c15e3c0 msgr2=0x7f247c161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:06.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.140+0000 7f248473d640 1 --2- 172.21.15.148:0/2443402655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f247c15e3c0 0x7f247c161f70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f247c0cf8d0 tx=0x7f246c02b840 comp rx=0 tx=0).stop 2024-03-29T04:11:06.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.140+0000 7f248473d640 1 -- 172.21.15.148:0/2443402655 shutdown_connections 2024-03-29T04:11:06.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.140+0000 7f248473d640 1 --2- 172.21.15.148:0/2443402655 >> v2:172.21.15.157:3300/0 conn(0x7f247c0b3430 0x7f247c06b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:06.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.140+0000 7f248473d640 1 --2- 172.21.15.148:0/2443402655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f247c15e3c0 0x7f247c161f70 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:06.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.140+0000 7f248473d640 1 -- 172.21.15.148:0/2443402655 >> 172.21.15.148:0/2443402655 conn(0x7f247c0b6430 msgr2=0x7f247c0d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:06.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.140+0000 7f248473d640 1 -- 172.21.15.148:0/2443402655 shutdown_connections 2024-03-29T04:11:06.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.140+0000 7f248473d640 1 -- 172.21.15.148:0/2443402655 wait complete. 2024-03-29T04:11:06.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.142+0000 7f248473d640 1 Processor -- start 2024-03-29T04:11:06.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.142+0000 7f248473d640 1 -- start start 2024-03-29T04:11:06.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.143+0000 7f248473d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f247c0b3430 0x7f247c0cab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:06.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.143+0000 7f248473d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f247c15e3c0 0x7f247c1586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:06.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.143+0000 7f248473d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f247c158c20 0x7f247c1609e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:06.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.143+0000 7f248473d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f247c0b3b20 con 0x7f247c158c20 2024-03-29T04:11:06.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.143+0000 7f248473d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f247c0b39a0 con 0x7f247c0b3430 2024-03-29T04:11:06.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.143+0000 7f248473d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f247c06bd30 con 0x7f247c15e3c0 2024-03-29T04:11:06.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.143+0000 7f2482cda640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f247c158c20 0x7f247c1609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:06.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.143+0000 7f2482cda640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f247c158c20 0x7f247c1609e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43090/0 (socket says 172.21.15.148:43090) 2024-03-29T04:11:06.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.143+0000 7f2482cda640 1 -- 172.21.15.148:0/154223852 learned_addr learned my addr 172.21.15.148:0/154223852 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:06.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.144+0000 7f2481cd8640 1 --2- 172.21.15.148:0/154223852 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f247c15e3c0 0x7f247c1586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:06.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.144+0000 7f24824d9640 1 --2- 172.21.15.148:0/154223852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f247c0b3430 0x7f247c0cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:06.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.144+0000 7f2482cda640 1 -- 172.21.15.148:0/154223852 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f247c15e3c0 msgr2=0x7f247c1586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:06.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.144+0000 7f2482cda640 1 --2- 172.21.15.148:0/154223852 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f247c15e3c0 0x7f247c1586e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:06.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.144+0000 7f2482cda640 1 -- 172.21.15.148:0/154223852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f247c0b3430 msgr2=0x7f247c0cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:06.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.144+0000 7f2482cda640 1 --2- 172.21.15.148:0/154223852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f247c0b3430 0x7f247c0cab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:06.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.144+0000 7f2482cda640 1 -- 172.21.15.148:0/154223852 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247c1f4490 con 0x7f247c158c20 2024-03-29T04:11:06.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.144+0000 7f2482cda640 1 --2- 172.21.15.148:0/154223852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f247c158c20 0x7f247c1609e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f247400b970 tx=0x7f247400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:06.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.144+0000 7f2472ffd640 1 -- 172.21.15.148:0/154223852 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474019070 con 0x7f247c158c20 2024-03-29T04:11:06.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.144+0000 7f2472ffd640 1 -- 172.21.15.148:0/154223852 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f247400cb60 con 0x7f247c158c20 2024-03-29T04:11:06.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.144+0000 7f248473d640 1 -- 172.21.15.148:0/154223852 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f247c1f4770 con 0x7f247c158c20 2024-03-29T04:11:06.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.144+0000 7f2472ffd640 1 -- 172.21.15.148:0/154223852 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474014440 con 0x7f247c158c20 2024-03-29T04:11:06.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.144+0000 7f248473d640 1 -- 172.21.15.148:0/154223852 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f247c1f4d30 con 0x7f247c158c20 2024-03-29T04:11:06.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.145+0000 7f248473d640 1 -- 172.21.15.148:0/154223852 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f247c0cb120 con 0x7f247c158c20 2024-03-29T04:11:06.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.146+0000 7f2472ffd640 1 -- 172.21.15.148:0/154223852 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f247401c070 con 0x7f247c158c20 2024-03-29T04:11:06.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.146+0000 7f2472ffd640 1 --2- 172.21.15.148:0/154223852 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f246006c530 0x7f246006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:06.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.146+0000 7f2472ffd640 1 -- 172.21.15.148:0/154223852 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f247408b750 con 0x7f247c158c20 2024-03-29T04:11:06.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.146+0000 7f24824d9640 1 --2- 172.21.15.148:0/154223852 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f246006c530 0x7f246006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:06.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.147+0000 7f24824d9640 1 --2- 172.21.15.148:0/154223852 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f246006c530 0x7f246006e9f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f246c036000 tx=0x7f246c008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:06.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.154+0000 7f2472ffd640 1 -- 172.21.15.148:0/154223852 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f247c0cb120 con 0x7f247c158c20 2024-03-29T04:11:06.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.436+0000 7f248473d640 1 -- 172.21.15.148:0/154223852 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f247c0c7750 con 0x7f247c158c20 2024-03-29T04:11:06.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.437+0000 7f2472ffd640 1 -- 172.21.15.148:0/154223852 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f2474059320 con 0x7f247c158c20 2024-03-29T04:11:06.416 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-29T04:11:06.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.439+0000 7f248473d640 1 -- 172.21.15.148:0/154223852 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f246006c530 msgr2=0x7f246006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:06.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.439+0000 7f248473d640 1 --2- 172.21.15.148:0/154223852 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f246006c530 0x7f246006e9f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f246c036000 tx=0x7f246c008030 comp rx=0 tx=0).stop 2024-03-29T04:11:06.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.439+0000 7f248473d640 1 -- 172.21.15.148:0/154223852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f247c158c20 msgr2=0x7f247c1609e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:06.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.439+0000 7f248473d640 1 --2- 172.21.15.148:0/154223852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f247c158c20 0x7f247c1609e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f247400b970 tx=0x7f247400be30 comp rx=0 tx=0).stop 2024-03-29T04:11:06.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.439+0000 7f248473d640 1 -- 172.21.15.148:0/154223852 shutdown_connections 2024-03-29T04:11:06.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.439+0000 7f248473d640 1 --2- 172.21.15.148:0/154223852 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f246006c530 0x7f246006e9f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:06.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.439+0000 7f248473d640 1 --2- 172.21.15.148:0/154223852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f247c158c20 0x7f247c1609e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:06.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.439+0000 7f248473d640 1 --2- 172.21.15.148:0/154223852 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f247c15e3c0 0x7f247c1586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:06.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.439+0000 7f248473d640 1 --2- 172.21.15.148:0/154223852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f247c0b3430 0x7f247c0cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:06.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.439+0000 7f248473d640 1 -- 172.21.15.148:0/154223852 >> 172.21.15.148:0/154223852 conn(0x7f247c0b6430 msgr2=0x7f247c0d0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:06.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.439+0000 7f248473d640 1 -- 172.21.15.148:0/154223852 shutdown_connections 2024-03-29T04:11:06.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.439+0000 7f248473d640 1 -- 172.21.15.148:0/154223852 wait complete. 2024-03-29T04:11:06.432 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:11:06.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.580+0000 7f16437af640 1 Processor -- start 2024-03-29T04:11:06.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.580+0000 7f16437af640 1 -- start start 2024-03-29T04:11:06.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.581+0000 7f16437af640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f163c0cf1d0 0x7f163c0cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:06.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.581+0000 7f16437af640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f163c0d1910 0x7f163c15fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:06.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.581+0000 7f16437af640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f163c15c9b0 con 0x7f163c06f470 2024-03-29T04:11:06.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.581+0000 7f16437af640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f163c15cc90 con 0x7f163c0d1910 2024-03-29T04:11:06.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.581+0000 7f16437af640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f163c15cf70 con 0x7f163c0cf1d0 2024-03-29T04:11:06.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.581+0000 7f1640d4a640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f163c06f470 0x7f163c0d1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35680/0 (socket says 172.21.15.148:35680) 2024-03-29T04:11:06.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.581+0000 7f1640d4a640 1 -- 172.21.15.148:0/272851303 learned_addr learned my addr 172.21.15.148:0/272851303 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:06.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.581+0000 7f1641d4c640 1 --2- 172.21.15.148:0/272851303 >> v2:172.21.15.157:3300/0 conn(0x7f163c0d1910 0x7f163c15fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:06.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.581+0000 7f164154b640 1 --2- 172.21.15.148:0/272851303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f163c0cf1d0 0x7f163c0cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:06.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.581+0000 7f1633fff640 1 -- 172.21.15.148:0/272851303 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 824831928 0 0) 0x7f163c15c9b0 con 0x7f163c06f470 2024-03-29T04:11:06.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.582+0000 7f1633fff640 1 -- 172.21.15.148:0/272851303 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1628003520 con 0x7f163c06f470 2024-03-29T04:11:06.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.582+0000 7f1641d4c640 1 -- 172.21.15.148:0/272851303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f163c0cf1d0 msgr2=0x7f163c0cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:06.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.582+0000 7f1641d4c640 1 --2- 172.21.15.148:0/272851303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f163c0cf1d0 0x7f163c0cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:06.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.582+0000 7f1641d4c640 1 -- 172.21.15.148:0/272851303 >> v1:172.21.15.148:6789/0 conn(0x7f163c06f470 legacy=0x7f163c0d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:06.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.582+0000 7f1641d4c640 1 -- 172.21.15.148:0/272851303 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163c15d250 con 0x7f163c0d1910 2024-03-29T04:11:06.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.583+0000 7f1641d4c640 1 --2- 172.21.15.148:0/272851303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f163c0d1910 0x7f163c15fe90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1634009d90 tx=0x7f163402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d61cd968a7a1271 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:06.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.583+0000 7f1633fff640 1 -- 172.21.15.148:0/272851303 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634039070 con 0x7f163c0d1910 2024-03-29T04:11:06.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.583+0000 7f1633fff640 1 -- 172.21.15.148:0/272851303 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f163402cc50 con 0x7f163c0d1910 2024-03-29T04:11:06.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.583+0000 7f1633fff640 1 -- 172.21.15.148:0/272851303 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634034440 con 0x7f163c0d1910 2024-03-29T04:11:06.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.584+0000 7f16437af640 1 -- 172.21.15.148:0/272851303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f163c0d1910 msgr2=0x7f163c15fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:06.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.584+0000 7f16437af640 1 --2- 172.21.15.148:0/272851303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f163c0d1910 0x7f163c15fe90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1634009d90 tx=0x7f163402bea0 comp rx=0 tx=0).stop 2024-03-29T04:11:06.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.584+0000 7f16437af640 1 -- 172.21.15.148:0/272851303 shutdown_connections 2024-03-29T04:11:06.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.584+0000 7f16437af640 1 --2- 172.21.15.148:0/272851303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f163c0d1910 0x7f163c15fe90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:06.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.584+0000 7f16437af640 1 --2- 172.21.15.148:0/272851303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f163c0cf1d0 0x7f163c0cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:06.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.584+0000 7f16437af640 1 -- 172.21.15.148:0/272851303 >> 172.21.15.148:0/272851303 conn(0x7f163c0b3c70 msgr2=0x7f163c0b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:06.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.584+0000 7f16437af640 1 -- 172.21.15.148:0/272851303 shutdown_connections 2024-03-29T04:11:06.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.584+0000 7f16437af640 1 -- 172.21.15.148:0/272851303 wait complete. 2024-03-29T04:11:06.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.586+0000 7f16437af640 1 Processor -- start 2024-03-29T04:11:06.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.586+0000 7f16437af640 1 -- start start 2024-03-29T04:11:06.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.587+0000 7f16437af640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f163c06f470 0x7f163c0ca910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:06.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.587+0000 7f16437af640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f163c0cf1d0 0x7f163c1f5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:06.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.587+0000 7f16437af640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f163c0d1910 0x7f163c1f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:06.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.587+0000 7f16437af640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f163c0ce530 con 0x7f163c06f470 2024-03-29T04:11:06.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.587+0000 7f16437af640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f163c0ce3b0 con 0x7f163c0cf1d0 2024-03-29T04:11:06.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.587+0000 7f16437af640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f163c0ce6b0 con 0x7f163c0d1910 2024-03-29T04:11:06.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.587+0000 7f164154b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f163c06f470 0x7f163c0ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:06.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.587+0000 7f164154b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f163c06f470 0x7f163c0ca910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43094/0 (socket says 172.21.15.148:43094) 2024-03-29T04:11:06.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.587+0000 7f164154b640 1 -- 172.21.15.148:0/2536216211 learned_addr learned my addr 172.21.15.148:0/2536216211 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:06.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.587+0000 7f1640d4a640 1 --2- 172.21.15.148:0/2536216211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f163c0cf1d0 0x7f163c1f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:06.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.587+0000 7f1641d4c640 1 --2- 172.21.15.148:0/2536216211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f163c0d1910 0x7f163c1f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:06.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.587+0000 7f164154b640 1 -- 172.21.15.148:0/2536216211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f163c0d1910 msgr2=0x7f163c1f57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:06.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.587+0000 7f164154b640 1 --2- 172.21.15.148:0/2536216211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f163c0d1910 0x7f163c1f57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:06.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.587+0000 7f164154b640 1 -- 172.21.15.148:0/2536216211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f163c0cf1d0 msgr2=0x7f163c1f5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:06.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.587+0000 7f164154b640 1 --2- 172.21.15.148:0/2536216211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f163c0cf1d0 0x7f163c1f5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:06.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.587+0000 7f164154b640 1 -- 172.21.15.148:0/2536216211 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163c0b5ec0 con 0x7f163c06f470 2024-03-29T04:11:06.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.587+0000 7f1641d4c640 1 --2- 172.21.15.148:0/2536216211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f163c0d1910 0x7f163c1f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:06.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.587+0000 7f164154b640 1 --2- 172.21.15.148:0/2536216211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f163c06f470 0x7f163c0ca910 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f162c00e870 tx=0x7f162c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:06.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.588+0000 7f1631ffb640 1 -- 172.21.15.148:0/2536216211 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c017070 con 0x7f163c06f470 2024-03-29T04:11:06.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.588+0000 7f16437af640 1 -- 172.21.15.148:0/2536216211 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f163c0b61a0 con 0x7f163c06f470 2024-03-29T04:11:06.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.588+0000 7f1631ffb640 1 -- 172.21.15.148:0/2536216211 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f162c00c940 con 0x7f163c06f470 2024-03-29T04:11:06.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.588+0000 7f1631ffb640 1 -- 172.21.15.148:0/2536216211 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c00cd30 con 0x7f163c06f470 2024-03-29T04:11:06.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.588+0000 7f16437af640 1 -- 172.21.15.148:0/2536216211 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f163c0b6760 con 0x7f163c06f470 2024-03-29T04:11:06.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.589+0000 7f16437af640 1 -- 172.21.15.148:0/2536216211 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f163c0758d0 con 0x7f163c06f470 2024-03-29T04:11:06.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.589+0000 7f1631ffb640 1 -- 172.21.15.148:0/2536216211 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f162c01a070 con 0x7f163c06f470 2024-03-29T04:11:06.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.590+0000 7f1631ffb640 1 --2- 172.21.15.148:0/2536216211 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f162006c530 0x7f162006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:06.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.590+0000 7f1631ffb640 1 -- 172.21.15.148:0/2536216211 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f162c08be00 con 0x7f163c06f470 2024-03-29T04:11:06.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.590+0000 7f1640d4a640 1 --2- 172.21.15.148:0/2536216211 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f162006c530 0x7f162006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:06.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.590+0000 7f1640d4a640 1 --2- 172.21.15.148:0/2536216211 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f162006c530 0x7f162006e9f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f163c0c8c10 tx=0x7f1638009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:06.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.594+0000 7f1631ffb640 1 -- 172.21.15.148:0/2536216211 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f162c059fa0 con 0x7f163c06f470 2024-03-29T04:11:06.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.778+0000 7f16437af640 1 -- 172.21.15.148:0/2536216211 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f163c068850 con 0x7f163c06f470 2024-03-29T04:11:06.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.778+0000 7f1631ffb640 1 -- 172.21.15.148:0/2536216211 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f162c0598a0 con 0x7f163c06f470 2024-03-29T04:11:06.758 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:11:06.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.781+0000 7f16437af640 1 -- 172.21.15.148:0/2536216211 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f162006c530 msgr2=0x7f162006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:06.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.781+0000 7f16437af640 1 --2- 172.21.15.148:0/2536216211 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f162006c530 0x7f162006e9f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f163c0c8c10 tx=0x7f1638009040 comp rx=0 tx=0).stop 2024-03-29T04:11:06.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.781+0000 7f16437af640 1 -- 172.21.15.148:0/2536216211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f163c06f470 msgr2=0x7f163c0ca910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:06.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.781+0000 7f16437af640 1 --2- 172.21.15.148:0/2536216211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f163c06f470 0x7f163c0ca910 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f162c00e870 tx=0x7f162c00ed30 comp rx=0 tx=0).stop 2024-03-29T04:11:06.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.781+0000 7f16437af640 1 -- 172.21.15.148:0/2536216211 shutdown_connections 2024-03-29T04:11:06.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.781+0000 7f16437af640 1 --2- 172.21.15.148:0/2536216211 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f162006c530 0x7f162006e9f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:06.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.781+0000 7f16437af640 1 --2- 172.21.15.148:0/2536216211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f163c0d1910 0x7f163c1f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:06.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.781+0000 7f16437af640 1 --2- 172.21.15.148:0/2536216211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f163c0cf1d0 0x7f163c1f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:06.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.781+0000 7f16437af640 1 --2- 172.21.15.148:0/2536216211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f163c06f470 0x7f163c0ca910 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:06.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.781+0000 7f16437af640 1 -- 172.21.15.148:0/2536216211 >> 172.21.15.148:0/2536216211 conn(0x7f163c0b3c70 msgr2=0x7f163c158f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:06.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.781+0000 7f16437af640 1 -- 172.21.15.148:0/2536216211 shutdown_connections 2024-03-29T04:11:06.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.781+0000 7f16437af640 1 -- 172.21.15.148:0/2536216211 wait complete. 2024-03-29T04:11:06.775 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:11:06.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.914+0000 7fa03dd8d640 1 Processor -- start 2024-03-29T04:11:06.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.914+0000 7fa03dd8d640 1 -- start start 2024-03-29T04:11:06.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.914+0000 7fa03dd8d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa0380b2ee0 0x7fa03815f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:06.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.915+0000 7fa03dd8d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0380b3630 con 0x7fa03806c5c0 2024-03-29T04:11:06.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.915+0000 7fa03dd8d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0380b3910 con 0x7fa03815b060 2024-03-29T04:11:06.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.915+0000 7fa03dd8d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa0380b3bf0 con 0x7fa0380b2ee0 2024-03-29T04:11:06.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.915+0000 7fa037fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa03806c5c0 0x7fa0380cd530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35682/0 (socket says 172.21.15.148:35682) 2024-03-29T04:11:06.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.915+0000 7fa037fff640 1 -- 172.21.15.148:0/3065270743 learned_addr learned my addr 172.21.15.148:0/3065270743 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:06.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.915+0000 7fa0367fc640 1 -- 172.21.15.148:0/3065270743 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2553960796 0 0) 0x7fa0380b3630 con 0x7fa03806c5c0 2024-03-29T04:11:06.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.915+0000 7fa0367fc640 1 -- 172.21.15.148:0/3065270743 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa020003520 con 0x7fa03806c5c0 2024-03-29T04:11:06.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.915+0000 7fa036ffd640 1 --2- 172.21.15.148:0/3065270743 >> v2:172.21.15.157:3300/0 conn(0x7fa0380b2ee0 0x7fa03815f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:06.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.915+0000 7fa0367fc640 1 -- 172.21.15.148:0/3065270743 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2821247617 0 0) 0x7fa020003520 con 0x7fa03806c5c0 2024-03-29T04:11:06.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.915+0000 7fa0367fc640 1 -- 172.21.15.148:0/3065270743 >> v2:172.21.15.157:3300/0 conn(0x7fa0380b2ee0 msgr2=0x7fa03815f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:06.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.915+0000 7fa0367fc640 1 --2- 172.21.15.148:0/3065270743 >> v2:172.21.15.157:3300/0 conn(0x7fa0380b2ee0 0x7fa03815f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:06.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.915+0000 7fa0367fc640 1 -- 172.21.15.148:0/3065270743 >> v1:172.21.15.157:6789/0 conn(0x7fa03815b060 legacy=0x7fa0380b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:06.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.915+0000 7fa0367fc640 1 -- 172.21.15.148:0/3065270743 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0380b3ed0 con 0x7fa03806c5c0 2024-03-29T04:11:06.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.915+0000 7fa0367fc640 1 -- 172.21.15.148:0/3065270743 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7fa0300019f0 con 0x7fa03806c5c0 2024-03-29T04:11:06.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.915+0000 7fa0367fc640 1 -- 172.21.15.148:0/3065270743 >> v1:172.21.15.148:6789/0 conn(0x7fa03806c5c0 legacy=0x7fa0380cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:06.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.915+0000 7fa0367fc640 1 --2- 172.21.15.148:0/3065270743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0200039b0 0x7fa020003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:06.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.915+0000 7fa0367fc640 1 --2- 172.21.15.148:0/3065270743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa020004800 0x7fa020004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:06.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.915+0000 7fa0367fc640 1 --2- 172.21.15.148:0/3065270743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0200052e0 0x7fa02000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:06.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.915+0000 7fa0367fc640 1 -- 172.21.15.148:0/3065270743 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa0380b3bf0 con 0x7fa020004800 2024-03-29T04:11:06.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.915+0000 7fa0367fc640 1 -- 172.21.15.148:0/3065270743 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa02000c1b0 con 0x7fa0200039b0 2024-03-29T04:11:06.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.915+0000 7fa0367fc640 1 -- 172.21.15.148:0/3065270743 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa02000c3b0 con 0x7fa0200052e0 2024-03-29T04:11:06.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.915+0000 7fa036ffd640 1 --2- 172.21.15.148:0/3065270743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa020004800 0x7fa020004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:06.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.915+0000 7fa0377fe640 1 --2- 172.21.15.148:0/3065270743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0200039b0 0x7fa020003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:06.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.916+0000 7fa037fff640 1 --2- 172.21.15.148:0/3065270743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0200052e0 0x7fa02000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:06.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.916+0000 7fa036ffd640 1 -- 172.21.15.148:0/3065270743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0200052e0 msgr2=0x7fa02000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:06.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.916+0000 7fa036ffd640 1 --2- 172.21.15.148:0/3065270743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0200052e0 0x7fa02000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:06.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.916+0000 7fa036ffd640 1 -- 172.21.15.148:0/3065270743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0200039b0 msgr2=0x7fa020003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:06.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.916+0000 7fa036ffd640 1 --2- 172.21.15.148:0/3065270743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0200039b0 0x7fa020003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:06.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.916+0000 7fa036ffd640 1 -- 172.21.15.148:0/3065270743 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa020003520 con 0x7fa020004800 2024-03-29T04:11:06.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.916+0000 7fa037fff640 1 --2- 172.21.15.148:0/3065270743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0200052e0 0x7fa02000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:06.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.916+0000 7fa037fff640 1 -- 172.21.15.148:0/3065270743 reap_dead start 2024-03-29T04:11:06.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.917+0000 7fa036ffd640 1 --2- 172.21.15.148:0/3065270743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa020004800 0x7fa020004c80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa02c009a90 tx=0x7fa020005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb08a7db7e68a4fd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:06.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.917+0000 7fa0367fc640 1 -- 172.21.15.148:0/3065270743 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c039070 con 0x7fa020004800 2024-03-29T04:11:06.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.917+0000 7fa0367fc640 1 -- 172.21.15.148:0/3065270743 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa02c02c680 con 0x7fa020004800 2024-03-29T04:11:06.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.917+0000 7fa0367fc640 1 -- 172.21.15.148:0/3065270743 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c009690 con 0x7fa020004800 2024-03-29T04:11:06.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.917+0000 7fa03dd8d640 1 -- 172.21.15.148:0/3065270743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa020004800 msgr2=0x7fa020004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:06.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.917+0000 7fa03dd8d640 1 --2- 172.21.15.148:0/3065270743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa020004800 0x7fa020004c80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa02c009a90 tx=0x7fa020005830 comp rx=0 tx=0).stop 2024-03-29T04:11:06.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.917+0000 7fa03dd8d640 1 -- 172.21.15.148:0/3065270743 shutdown_connections 2024-03-29T04:11:06.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.917+0000 7fa03dd8d640 1 --2- 172.21.15.148:0/3065270743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa020004800 0x7fa020004c80 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:06.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.917+0000 7fa03dd8d640 1 -- 172.21.15.148:0/3065270743 >> 172.21.15.148:0/3065270743 conn(0x7fa0380cfed0 msgr2=0x7fa0380d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:06.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.917+0000 7fa03dd8d640 1 -- 172.21.15.148:0/3065270743 shutdown_connections 2024-03-29T04:11:06.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.917+0000 7fa03dd8d640 1 -- 172.21.15.148:0/3065270743 wait complete. 2024-03-29T04:11:06.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.919+0000 7fa03dd8d640 1 Processor -- start 2024-03-29T04:11:06.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.919+0000 7fa03dd8d640 1 -- start start 2024-03-29T04:11:06.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.919+0000 7fa03dd8d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa020004800 0x7fa0380cb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:06.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.919+0000 7fa03dd8d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0381f01e0 0x7fa038159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:06.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.919+0000 7fa03dd8d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa038159d70 0x7fa038157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:06.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.919+0000 7fa03dd8d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa03815b440 con 0x7fa0381f01e0 2024-03-29T04:11:06.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.919+0000 7fa03dd8d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa0380b3340 con 0x7fa038159d70 2024-03-29T04:11:06.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.919+0000 7fa03dd8d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa03806ca50 con 0x7fa020004800 2024-03-29T04:11:06.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.920+0000 7fa036ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0381f01e0 0x7fa038159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:06.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.920+0000 7fa036ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0381f01e0 0x7fa038159830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43116/0 (socket says 172.21.15.148:43116) 2024-03-29T04:11:06.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.920+0000 7fa036ffd640 1 -- 172.21.15.148:0/718722658 learned_addr learned my addr 172.21.15.148:0/718722658 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:06.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.920+0000 7fa0377fe640 1 --2- 172.21.15.148:0/718722658 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa020004800 0x7fa0380cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:06.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.920+0000 7fa037fff640 1 --2- 172.21.15.148:0/718722658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa038159d70 0x7fa038157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:06.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.920+0000 7fa036ffd640 1 -- 172.21.15.148:0/718722658 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa020004800 msgr2=0x7fa0380cb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:06.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.920+0000 7fa036ffd640 1 --2- 172.21.15.148:0/718722658 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa020004800 0x7fa0380cb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:06.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.920+0000 7fa036ffd640 1 -- 172.21.15.148:0/718722658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa038159d70 msgr2=0x7fa038157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:06.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.920+0000 7fa036ffd640 1 --2- 172.21.15.148:0/718722658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa038159d70 0x7fa038157b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:06.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.920+0000 7fa036ffd640 1 -- 172.21.15.148:0/718722658 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0381e3c20 con 0x7fa0381f01e0 2024-03-29T04:11:06.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.920+0000 7fa036ffd640 1 --2- 172.21.15.148:0/718722658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0381f01e0 0x7fa038159830 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa02c037040 tx=0x7fa02c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:06.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.920+0000 7fa01ffff640 1 -- 172.21.15.148:0/718722658 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c042020 con 0x7fa0381f01e0 2024-03-29T04:11:06.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.920+0000 7fa01ffff640 1 -- 172.21.15.148:0/718722658 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa02c03a040 con 0x7fa0381f01e0 2024-03-29T04:11:06.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.920+0000 7fa03dd8d640 1 -- 172.21.15.148:0/718722658 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0381e3f00 con 0x7fa0381f01e0 2024-03-29T04:11:06.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.920+0000 7fa01ffff640 1 -- 172.21.15.148:0/718722658 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c039070 con 0x7fa0381f01e0 2024-03-29T04:11:06.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.920+0000 7fa03dd8d640 1 -- 172.21.15.148:0/718722658 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0381e44c0 con 0x7fa0381f01e0 2024-03-29T04:11:06.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.921+0000 7fa03dd8d640 1 -- 172.21.15.148:0/718722658 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0380687e0 con 0x7fa0381f01e0 2024-03-29T04:11:06.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.922+0000 7fa01ffff640 1 -- 172.21.15.148:0/718722658 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa02c045020 con 0x7fa0381f01e0 2024-03-29T04:11:06.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.923+0000 7fa01ffff640 1 --2- 172.21.15.148:0/718722658 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa02406c560 0x7fa02406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:06.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.923+0000 7fa01ffff640 1 -- 172.21.15.148:0/718722658 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa02c0ace60 con 0x7fa0381f01e0 2024-03-29T04:11:06.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.923+0000 7fa0377fe640 1 --2- 172.21.15.148:0/718722658 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa02406c560 0x7fa02406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:06.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.923+0000 7fa0377fe640 1 --2- 172.21.15.148:0/718722658 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa02406c560 0x7fa02406ea20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa028000c00 tx=0x7fa028009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:06.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:06.927+0000 7fa01ffff640 1 -- 172.21.15.148:0/718722658 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa02c04c070 con 0x7fa0381f01e0 2024-03-29T04:11:07.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.217+0000 7fa03dd8d640 1 -- 172.21.15.148:0/718722658 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa03815a150 con 0x7fa0381f01e0 2024-03-29T04:11:07.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.218+0000 7fa01ffff640 1 -- 172.21.15.148:0/718722658 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fa02c07aa30 con 0x7fa0381f01e0 2024-03-29T04:11:07.197 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-03-29T04:11:07.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.219+0000 7fa03dd8d640 1 -- 172.21.15.148:0/718722658 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa02406c560 msgr2=0x7fa02406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:07.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.219+0000 7fa03dd8d640 1 --2- 172.21.15.148:0/718722658 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa02406c560 0x7fa02406ea20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa028000c00 tx=0x7fa028009040 comp rx=0 tx=0).stop 2024-03-29T04:11:07.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.219+0000 7fa03dd8d640 1 -- 172.21.15.148:0/718722658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0381f01e0 msgr2=0x7fa038159830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:07.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.219+0000 7fa03dd8d640 1 --2- 172.21.15.148:0/718722658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0381f01e0 0x7fa038159830 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa02c037040 tx=0x7fa02c004730 comp rx=0 tx=0).stop 2024-03-29T04:11:07.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.220+0000 7fa03dd8d640 1 -- 172.21.15.148:0/718722658 shutdown_connections 2024-03-29T04:11:07.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.220+0000 7fa03dd8d640 1 --2- 172.21.15.148:0/718722658 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa02406c560 0x7fa02406ea20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:07.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.220+0000 7fa03dd8d640 1 --2- 172.21.15.148:0/718722658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa038159d70 0x7fa038157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:07.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.220+0000 7fa03dd8d640 1 --2- 172.21.15.148:0/718722658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0381f01e0 0x7fa038159830 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:07.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.220+0000 7fa03dd8d640 1 --2- 172.21.15.148:0/718722658 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa020004800 0x7fa0380cb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:07.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.220+0000 7fa03dd8d640 1 -- 172.21.15.148:0/718722658 >> 172.21.15.148:0/718722658 conn(0x7fa0380cfed0 msgr2=0x7fa03806e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:07.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.220+0000 7fa03dd8d640 1 -- 172.21.15.148:0/718722658 shutdown_connections 2024-03-29T04:11:07.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.220+0000 7fa03dd8d640 1 -- 172.21.15.148:0/718722658 wait complete. 2024-03-29T04:11:07.213 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi148.front.sepia.ceph.com... 2024-03-29T04:11:07.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:07.213 DEBUG:teuthology.orchestra.run.smithi148:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-03-29T04:11:07.241 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:11:07.241 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-03-29T04:11:07.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:07.242 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-29T04:11:07.297 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:11:07.297 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:11:07.317 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:11:07.317 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:11:07.376 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:11:07.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.498+0000 7fb4106df640 1 Processor -- start 2024-03-29T04:11:07.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.498+0000 7fb4106df640 1 -- start start 2024-03-29T04:11:07.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.499+0000 7fb4106df640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb4080d17e0 0x7fb4080d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:07.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.499+0000 7fb4106df640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb40815e7c0 con 0x7fb4080ba030 2024-03-29T04:11:07.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.499+0000 7fb4106df640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4080d2c30 con 0x7fb40815e3e0 2024-03-29T04:11:07.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.499+0000 7fb4106df640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fb4080d3840 con 0x7fb4080d17e0 2024-03-29T04:11:07.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.500+0000 7fb40ec7c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb4080d17e0 0x7fb4080d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:07.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.500+0000 7fb40ec7c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb4080d17e0 0x7fb4080d3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43132/0 (socket says 172.21.15.148:43132) 2024-03-29T04:11:07.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.500+0000 7fb40dc7a640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb4080ba030 0x7fb4080d10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35686/0 (socket says 172.21.15.148:35686) 2024-03-29T04:11:07.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.500+0000 7fb40ec7c640 1 -- 172.21.15.148:0/3882852654 learned_addr learned my addr 172.21.15.148:0/3882852654 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:07.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.500+0000 7fb40ec7c640 1 -- 172.21.15.148:0/3882852654 >> v1:172.21.15.157:6789/0 conn(0x7fb40815e3e0 legacy=0x7fb4080d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:07.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.500+0000 7fb40ec7c640 1 -- 172.21.15.148:0/3882852654 >> v1:172.21.15.148:6789/0 conn(0x7fb4080ba030 legacy=0x7fb4080d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:07.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.500+0000 7fb40ec7c640 1 -- 172.21.15.148:0/3882852654 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4080d2f10 con 0x7fb4080d17e0 2024-03-29T04:11:07.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.500+0000 7fb40d479640 1 -- 172.21.15.148:0/3882852654 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3608533775 0 0) 0x7fb40815e7c0 con 0x7fb4080ba030 2024-03-29T04:11:07.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.501+0000 7fb40ec7c640 1 --2- 172.21.15.148:0/3882852654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4080d17e0 0x7fb4080d3cd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb404009870 tx=0x7fb40402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb51e7c2ef498ae2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:07.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.501+0000 7fb40d479640 1 -- 172.21.15.148:0/3882852654 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40402f840 con 0x7fb4080d17e0 2024-03-29T04:11:07.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.501+0000 7fb40d479640 1 -- 172.21.15.148:0/3882852654 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb40402f9e0 con 0x7fb4080d17e0 2024-03-29T04:11:07.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.501+0000 7fb40d479640 1 -- 172.21.15.148:0/3882852654 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40402fdb0 con 0x7fb4080d17e0 2024-03-29T04:11:07.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.501+0000 7fb4106df640 1 -- 172.21.15.148:0/3882852654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4080d17e0 msgr2=0x7fb4080d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:07.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.501+0000 7fb4106df640 1 --2- 172.21.15.148:0/3882852654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4080d17e0 0x7fb4080d3cd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb404009870 tx=0x7fb40402ba20 comp rx=0 tx=0).stop 2024-03-29T04:11:07.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.501+0000 7fb4106df640 1 -- 172.21.15.148:0/3882852654 shutdown_connections 2024-03-29T04:11:07.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.501+0000 7fb4106df640 1 --2- 172.21.15.148:0/3882852654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4080d17e0 0x7fb4080d3cd0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:07.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.501+0000 7fb4106df640 1 -- 172.21.15.148:0/3882852654 >> 172.21.15.148:0/3882852654 conn(0x7fb4080b6210 msgr2=0x7fb4080b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:07.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.501+0000 7fb4106df640 1 -- 172.21.15.148:0/3882852654 shutdown_connections 2024-03-29T04:11:07.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.501+0000 7fb4106df640 1 -- 172.21.15.148:0/3882852654 wait complete. 2024-03-29T04:11:07.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.503+0000 7fb4106df640 1 Processor -- start 2024-03-29T04:11:07.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.503+0000 7fb4106df640 1 -- start start 2024-03-29T04:11:07.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.503+0000 7fb4106df640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4080ba030 0x7fb40814d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:07.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.503+0000 7fb4106df640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4080d17e0 0x7fb408156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:07.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.503+0000 7fb4106df640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb40815e3e0 0x7fb4081597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:07.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.503+0000 7fb4106df640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb408168f60 con 0x7fb4080ba030 2024-03-29T04:11:07.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.503+0000 7fb4106df640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb4080d34d0 con 0x7fb4080d17e0 2024-03-29T04:11:07.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.503+0000 7fb4106df640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb4081690e0 con 0x7fb40815e3e0 2024-03-29T04:11:07.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.503+0000 7fb40e47b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4080ba030 0x7fb40814d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:07.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.503+0000 7fb40e47b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4080ba030 0x7fb40814d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43142/0 (socket says 172.21.15.148:43142) 2024-03-29T04:11:07.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.503+0000 7fb40e47b640 1 -- 172.21.15.148:0/1629039849 learned_addr learned my addr 172.21.15.148:0/1629039849 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:07.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.503+0000 7fb40dc7a640 1 --2- 172.21.15.148:0/1629039849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4080d17e0 0x7fb408156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:07.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.504+0000 7fb40ec7c640 1 --2- 172.21.15.148:0/1629039849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb40815e3e0 0x7fb4081597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:07.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.504+0000 7fb40e47b640 1 -- 172.21.15.148:0/1629039849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb40815e3e0 msgr2=0x7fb4081597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:07.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.504+0000 7fb40e47b640 1 --2- 172.21.15.148:0/1629039849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb40815e3e0 0x7fb4081597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:07.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.504+0000 7fb40e47b640 1 -- 172.21.15.148:0/1629039849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4080d17e0 msgr2=0x7fb408156300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:07.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.504+0000 7fb40e47b640 1 --2- 172.21.15.148:0/1629039849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4080d17e0 0x7fb408156300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:07.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.504+0000 7fb40e47b640 1 -- 172.21.15.148:0/1629039849 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40815ed50 con 0x7fb4080ba030 2024-03-29T04:11:07.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.504+0000 7fb40e47b640 1 --2- 172.21.15.148:0/1629039849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4080ba030 0x7fb40814d960 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb40000ba50 tx=0x7fb40000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:07.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.504+0000 7fb3feffd640 1 -- 172.21.15.148:0/1629039849 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400019070 con 0x7fb4080ba030 2024-03-29T04:11:07.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.504+0000 7fb3feffd640 1 -- 172.21.15.148:0/1629039849 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb40000cc20 con 0x7fb4080ba030 2024-03-29T04:11:07.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.504+0000 7fb3feffd640 1 -- 172.21.15.148:0/1629039849 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400014440 con 0x7fb4080ba030 2024-03-29T04:11:07.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.504+0000 7fb4106df640 1 -- 172.21.15.148:0/1629039849 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb40815f030 con 0x7fb4080ba030 2024-03-29T04:11:07.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.504+0000 7fb4106df640 1 -- 172.21.15.148:0/1629039849 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb40815f5f0 con 0x7fb4080ba030 2024-03-29T04:11:07.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.505+0000 7fb4106df640 1 -- 172.21.15.148:0/1629039849 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4080687e0 con 0x7fb4080ba030 2024-03-29T04:11:07.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.506+0000 7fb3feffd640 1 -- 172.21.15.148:0/1629039849 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb40001c070 con 0x7fb4080ba030 2024-03-29T04:11:07.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.506+0000 7fb3feffd640 1 --2- 172.21.15.148:0/1629039849 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb3ec06c530 0x7fb3ec06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:07.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.506+0000 7fb3feffd640 1 -- 172.21.15.148:0/1629039849 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb40008b770 con 0x7fb4080ba030 2024-03-29T04:11:07.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.506+0000 7fb40dc7a640 1 --2- 172.21.15.148:0/1629039849 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb3ec06c530 0x7fb3ec06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:07.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.506+0000 7fb40dc7a640 1 --2- 172.21.15.148:0/1629039849 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb3ec06c530 0x7fb3ec06e9f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb408151c00 tx=0x7fb3f8007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:07.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.511+0000 7fb3feffd640 1 -- 172.21.15.148:0/1629039849 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb400059340 con 0x7fb4080ba030 2024-03-29T04:11:07.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.693+0000 7fb4106df640 1 -- 172.21.15.148:0/1629039849 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4080b6770 con 0x7fb4080ba030 2024-03-29T04:11:07.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.694+0000 7fb3feffd640 1 -- 172.21.15.148:0/1629039849 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18858 (secure 0 0 0) 0x7fb40005d640 con 0x7fb4080ba030 2024-03-29T04:11:07.674 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:11:07.674 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:41.662934+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:11:07.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.697+0000 7fb4106df640 1 -- 172.21.15.148:0/1629039849 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb3ec06c530 msgr2=0x7fb3ec06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:07.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.697+0000 7fb4106df640 1 --2- 172.21.15.148:0/1629039849 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb3ec06c530 0x7fb3ec06e9f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb408151c00 tx=0x7fb3f8007d50 comp rx=0 tx=0).stop 2024-03-29T04:11:07.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.697+0000 7fb4106df640 1 -- 172.21.15.148:0/1629039849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4080ba030 msgr2=0x7fb40814d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:07.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.697+0000 7fb4106df640 1 --2- 172.21.15.148:0/1629039849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4080ba030 0x7fb40814d960 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb40000ba50 tx=0x7fb40000bf10 comp rx=0 tx=0).stop 2024-03-29T04:11:07.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.697+0000 7fb4106df640 1 -- 172.21.15.148:0/1629039849 shutdown_connections 2024-03-29T04:11:07.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.697+0000 7fb4106df640 1 --2- 172.21.15.148:0/1629039849 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb3ec06c530 0x7fb3ec06e9f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:07.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.697+0000 7fb4106df640 1 --2- 172.21.15.148:0/1629039849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb40815e3e0 0x7fb4081597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:07.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.697+0000 7fb4106df640 1 --2- 172.21.15.148:0/1629039849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4080d17e0 0x7fb408156300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:07.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.697+0000 7fb4106df640 1 --2- 172.21.15.148:0/1629039849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4080ba030 0x7fb40814d960 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:07.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.697+0000 7fb4106df640 1 -- 172.21.15.148:0/1629039849 >> 172.21.15.148:0/1629039849 conn(0x7fb4080b6210 msgr2=0x7fb4080b83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:07.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.697+0000 7fb4106df640 1 -- 172.21.15.148:0/1629039849 shutdown_connections 2024-03-29T04:11:07.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.697+0000 7fb4106df640 1 -- 172.21.15.148:0/1629039849 wait complete. 2024-03-29T04:11:07.688 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:47.898351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:57.922313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:58.829405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:10:02.551763+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:11:07.688 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:11:07.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.814+0000 7fc5bf3cd640 1 Processor -- start 2024-03-29T04:11:07.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.815+0000 7fc5bf3cd640 1 -- start start 2024-03-29T04:11:07.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.815+0000 7fc5bf3cd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc5b815e3c0 0x7fc5b8161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:07.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.815+0000 7fc5bf3cd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc5b80b2f00 0x7fc5b81624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:07.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.815+0000 7fc5bf3cd640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5b80cf330 con 0x7fc5b80b3430 2024-03-29T04:11:07.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.815+0000 7fc5bf3cd640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fc5b80cf960 con 0x7fc5b80b2f00 2024-03-29T04:11:07.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.815+0000 7fc5bf3cd640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc5b8162cf0 con 0x7fc5b815e3c0 2024-03-29T04:11:07.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.815+0000 7fc5bc968640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc5b80b2f00 0x7fc5b81624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:07.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.816+0000 7fc5bc968640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc5b80b2f00 0x7fc5b81624b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43148/0 (socket says 172.21.15.148:43148) 2024-03-29T04:11:07.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.816+0000 7fc5bc968640 1 -- 172.21.15.148:0/67947185 learned_addr learned my addr 172.21.15.148:0/67947185 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:07.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.816+0000 7fc5bd169640 1 --2- 172.21.15.148:0/67947185 >> v2:172.21.15.157:3300/0 conn(0x7fc5b815e3c0 0x7fc5b8161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:07.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.816+0000 7fc5bc968640 1 -- 172.21.15.148:0/67947185 >> v2:172.21.15.157:3300/0 conn(0x7fc5b815e3c0 msgr2=0x7fc5b8161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:07.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.816+0000 7fc5bc968640 1 --2- 172.21.15.148:0/67947185 >> v2:172.21.15.157:3300/0 conn(0x7fc5b815e3c0 0x7fc5b8161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:07.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.816+0000 7fc5bc968640 1 -- 172.21.15.148:0/67947185 >> v1:172.21.15.157:6789/0 conn(0x7fc5b80b3430 legacy=0x7fc5b80d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:07.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.816+0000 7fc5bc968640 1 -- 172.21.15.148:0/67947185 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b80cf5f0 con 0x7fc5b80b2f00 2024-03-29T04:11:07.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.816+0000 7fc5bc968640 1 --2- 172.21.15.148:0/67947185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5b80b2f00 0x7fc5b81624b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80d2670 tx=0x7fc5a802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e6067935d1d7f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:07.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.817+0000 7fc5affff640 1 -- 172.21.15.148:0/67947185 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a8035020 con 0x7fc5b80b2f00 2024-03-29T04:11:07.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.817+0000 7fc5affff640 1 -- 172.21.15.148:0/67947185 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5a8031870 con 0x7fc5b80b2f00 2024-03-29T04:11:07.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.817+0000 7fc5affff640 1 -- 172.21.15.148:0/67947185 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a8031c40 con 0x7fc5b80b2f00 2024-03-29T04:11:07.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.817+0000 7fc5bf3cd640 1 -- 172.21.15.148:0/67947185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5b80b2f00 msgr2=0x7fc5b81624b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:07.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.817+0000 7fc5bf3cd640 1 --2- 172.21.15.148:0/67947185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5b80b2f00 0x7fc5b81624b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80d2670 tx=0x7fc5a802b910 comp rx=0 tx=0).stop 2024-03-29T04:11:07.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.817+0000 7fc5bf3cd640 1 -- 172.21.15.148:0/67947185 shutdown_connections 2024-03-29T04:11:07.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.817+0000 7fc5bf3cd640 1 --2- 172.21.15.148:0/67947185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5b80b2f00 0x7fc5b81624b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:07.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.817+0000 7fc5bf3cd640 1 --2- 172.21.15.148:0/67947185 >> v2:172.21.15.157:3300/0 conn(0x7fc5b815e3c0 0x7fc5b8161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:07.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.817+0000 7fc5bf3cd640 1 -- 172.21.15.148:0/67947185 >> 172.21.15.148:0/67947185 conn(0x7fc5b80b6430 msgr2=0x7fc5b80d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:07.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.817+0000 7fc5bf3cd640 1 -- 172.21.15.148:0/67947185 shutdown_connections 2024-03-29T04:11:07.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.817+0000 7fc5bf3cd640 1 -- 172.21.15.148:0/67947185 wait complete. 2024-03-29T04:11:07.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.818+0000 7fc5bf3cd640 1 Processor -- start 2024-03-29T04:11:07.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.818+0000 7fc5bf3cd640 1 -- start start 2024-03-29T04:11:07.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.819+0000 7fc5bf3cd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5b80b3430 0x7fc5b8149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:07.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.819+0000 7fc5bf3cd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5b815e3c0 0x7fc5b815eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:07.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.819+0000 7fc5bf3cd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5b815f020 0x7fc5b8160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:07.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.819+0000 7fc5bf3cd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b80b39a0 con 0x7fc5b815f020 2024-03-29T04:11:07.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.819+0000 7fc5bf3cd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b815e7a0 con 0x7fc5b815e3c0 2024-03-29T04:11:07.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.819+0000 7fc5bf3cd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc5b80b3b20 con 0x7fc5b80b3430 2024-03-29T04:11:07.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.819+0000 7fc5bd96a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5b815f020 0x7fc5b8160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:07.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.819+0000 7fc5bd96a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5b815f020 0x7fc5b8160ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43160/0 (socket says 172.21.15.148:43160) 2024-03-29T04:11:07.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.819+0000 7fc5bd96a640 1 -- 172.21.15.148:0/3846790339 learned_addr learned my addr 172.21.15.148:0/3846790339 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:07.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.819+0000 7fc5bc968640 1 --2- 172.21.15.148:0/3846790339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5b815e3c0 0x7fc5b815eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:07.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.819+0000 7fc5bd169640 1 --2- 172.21.15.148:0/3846790339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5b80b3430 0x7fc5b8149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:07.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.820+0000 7fc5bd96a640 1 -- 172.21.15.148:0/3846790339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5b80b3430 msgr2=0x7fc5b8149750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:07.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.820+0000 7fc5bd96a640 1 --2- 172.21.15.148:0/3846790339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5b80b3430 0x7fc5b8149750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:07.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.820+0000 7fc5bd96a640 1 -- 172.21.15.148:0/3846790339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5b815e3c0 msgr2=0x7fc5b815eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:07.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.820+0000 7fc5bd96a640 1 --2- 172.21.15.148:0/3846790339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5b815e3c0 0x7fc5b815eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:07.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.820+0000 7fc5bd96a640 1 -- 172.21.15.148:0/3846790339 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b80cb440 con 0x7fc5b815f020 2024-03-29T04:11:07.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.820+0000 7fc5bd169640 1 --2- 172.21.15.148:0/3846790339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5b80b3430 0x7fc5b8149750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:07.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.820+0000 7fc5bd96a640 1 --2- 172.21.15.148:0/3846790339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5b815f020 0x7fc5b8160ad0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000c7b0 tx=0x7fc5b000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:07.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.820+0000 7fc5adffb640 1 -- 172.21.15.148:0/3846790339 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0017070 con 0x7fc5b815f020 2024-03-29T04:11:07.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.820+0000 7fc5adffb640 1 -- 172.21.15.148:0/3846790339 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5b0003da0 con 0x7fc5b815f020 2024-03-29T04:11:07.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.820+0000 7fc5adffb640 1 -- 172.21.15.148:0/3846790339 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b00052b0 con 0x7fc5b815f020 2024-03-29T04:11:07.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.820+0000 7fc5bf3cd640 1 -- 172.21.15.148:0/3846790339 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b80cb6e0 con 0x7fc5b815f020 2024-03-29T04:11:07.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.820+0000 7fc5bf3cd640 1 -- 172.21.15.148:0/3846790339 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b81617e0 con 0x7fc5b815f020 2024-03-29T04:11:07.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.821+0000 7fc5bf3cd640 1 -- 172.21.15.148:0/3846790339 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5b8157d70 con 0x7fc5b815f020 2024-03-29T04:11:07.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.821+0000 7fc5adffb640 1 -- 172.21.15.148:0/3846790339 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc5b001a070 con 0x7fc5b815f020 2024-03-29T04:11:07.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.822+0000 7fc5adffb640 1 --2- 172.21.15.148:0/3846790339 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc59c06c530 0x7fc59c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:07.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.822+0000 7fc5adffb640 1 -- 172.21.15.148:0/3846790339 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc5b008afe0 con 0x7fc5b815f020 2024-03-29T04:11:07.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.822+0000 7fc5bd169640 1 --2- 172.21.15.148:0/3846790339 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc59c06c530 0x7fc59c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:07.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.822+0000 7fc5bd169640 1 --2- 172.21.15.148:0/3846790339 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc59c06c530 0x7fc59c06e9f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4006fd0 tx=0x7fc5b4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:07.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:07.826+0000 7fc5adffb640 1 -- 172.21.15.148:0/3846790339 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc5b8157d70 con 0x7fc5b815f020 2024-03-29T04:11:08.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.034+0000 7fc5bf3cd640 1 -- 172.21.15.148:0/3846790339 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc5b8068850 con 0x7fc5b815f020 2024-03-29T04:11:08.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.034+0000 7fc5adffb640 1 -- 172.21.15.148:0/3846790339 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fc5b0058c40 con 0x7fc5b815f020 2024-03-29T04:11:08.013 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-29T04:11:08.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.036+0000 7fc5bf3cd640 1 -- 172.21.15.148:0/3846790339 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc59c06c530 msgr2=0x7fc59c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:08.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.036+0000 7fc5bf3cd640 1 --2- 172.21.15.148:0/3846790339 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc59c06c530 0x7fc59c06e9f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4006fd0 tx=0x7fc5b4008040 comp rx=0 tx=0).stop 2024-03-29T04:11:08.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.036+0000 7fc5bf3cd640 1 -- 172.21.15.148:0/3846790339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5b815f020 msgr2=0x7fc5b8160ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:08.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.036+0000 7fc5bf3cd640 1 --2- 172.21.15.148:0/3846790339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5b815f020 0x7fc5b8160ad0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000c7b0 tx=0x7fc5b000cc70 comp rx=0 tx=0).stop 2024-03-29T04:11:08.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.036+0000 7fc5bf3cd640 1 -- 172.21.15.148:0/3846790339 shutdown_connections 2024-03-29T04:11:08.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.036+0000 7fc5bf3cd640 1 --2- 172.21.15.148:0/3846790339 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc59c06c530 0x7fc59c06e9f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.036+0000 7fc5bf3cd640 1 --2- 172.21.15.148:0/3846790339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5b815f020 0x7fc5b8160ad0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.036+0000 7fc5bf3cd640 1 --2- 172.21.15.148:0/3846790339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5b815e3c0 0x7fc5b815eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.036+0000 7fc5bf3cd640 1 --2- 172.21.15.148:0/3846790339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5b80b3430 0x7fc5b8149750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.036+0000 7fc5bf3cd640 1 -- 172.21.15.148:0/3846790339 >> 172.21.15.148:0/3846790339 conn(0x7fc5b80b6430 msgr2=0x7fc5b806c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:08.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.036+0000 7fc5bf3cd640 1 -- 172.21.15.148:0/3846790339 shutdown_connections 2024-03-29T04:11:08.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.036+0000 7fc5bf3cd640 1 -- 172.21.15.148:0/3846790339 wait complete. 2024-03-29T04:11:08.026 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:11:08.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.151+0000 7f18026f2640 1 Processor -- start 2024-03-29T04:11:08.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.151+0000 7f18026f2640 1 -- start start 2024-03-29T04:11:08.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.152+0000 7f18026f2640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f17fc15e3c0 0x7f17fc161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:08.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.152+0000 7f18026f2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17fc0b3430 0x7f17fc06b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:08.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.152+0000 7f18026f2640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17fc0cf330 con 0x7f17fc0b2f00 2024-03-29T04:11:08.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.152+0000 7f18026f2640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f17fc0cf960 con 0x7f17fc15e3c0 2024-03-29T04:11:08.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.152+0000 7f18026f2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f17fc162cf0 con 0x7f17fc0b3430 2024-03-29T04:11:08.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.152+0000 7f17fbfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f17fc15e3c0 0x7f17fc161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:08.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.152+0000 7f17fb7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f17fc0b2f00 0x7f17fc1624b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35702/0 (socket says 172.21.15.148:35702) 2024-03-29T04:11:08.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.152+0000 7f17fb7fe640 1 -- 172.21.15.148:0/3839551159 learned_addr learned my addr 172.21.15.148:0/3839551159 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:08.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.153+0000 7f1800c8f640 1 --2- 172.21.15.148:0/3839551159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17fc0b3430 0x7f17fc06b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:08.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.153+0000 7f17fbfff640 1 -- 172.21.15.148:0/3839551159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17fc0b3430 msgr2=0x7f17fc06b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:08.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.153+0000 7f17fbfff640 1 --2- 172.21.15.148:0/3839551159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17fc0b3430 0x7f17fc06b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.153+0000 7f17fbfff640 1 -- 172.21.15.148:0/3839551159 >> v1:172.21.15.148:6789/0 conn(0x7f17fc0b2f00 legacy=0x7f17fc1624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:08.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.153+0000 7f17fbfff640 1 -- 172.21.15.148:0/3839551159 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17fc0cf5f0 con 0x7f17fc15e3c0 2024-03-29T04:11:08.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.153+0000 7f17fbfff640 1 --2- 172.21.15.148:0/3839551159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17fc15e3c0 0x7f17fc161f70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f17ec0097a0 tx=0x7f17ec02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2fd80c7763014f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:08.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.154+0000 7f17faffd640 1 -- 172.21.15.148:0/3839551159 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec0315a0 con 0x7f17fc15e3c0 2024-03-29T04:11:08.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.154+0000 7f17faffd640 1 -- 172.21.15.148:0/3839551159 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17ec037070 con 0x7f17fc15e3c0 2024-03-29T04:11:08.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.154+0000 7f17faffd640 1 -- 172.21.15.148:0/3839551159 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec031960 con 0x7f17fc15e3c0 2024-03-29T04:11:08.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.154+0000 7f18026f2640 1 -- 172.21.15.148:0/3839551159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17fc15e3c0 msgr2=0x7f17fc161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:08.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.154+0000 7f18026f2640 1 --2- 172.21.15.148:0/3839551159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17fc15e3c0 0x7f17fc161f70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f17ec0097a0 tx=0x7f17ec02b840 comp rx=0 tx=0).stop 2024-03-29T04:11:08.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.154+0000 7f18026f2640 1 -- 172.21.15.148:0/3839551159 shutdown_connections 2024-03-29T04:11:08.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.154+0000 7f18026f2640 1 --2- 172.21.15.148:0/3839551159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17fc0b3430 0x7f17fc06b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.154+0000 7f18026f2640 1 --2- 172.21.15.148:0/3839551159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17fc15e3c0 0x7f17fc161f70 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.154+0000 7f18026f2640 1 -- 172.21.15.148:0/3839551159 >> 172.21.15.148:0/3839551159 conn(0x7f17fc0b6430 msgr2=0x7f17fc0d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:08.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.154+0000 7f18026f2640 1 -- 172.21.15.148:0/3839551159 shutdown_connections 2024-03-29T04:11:08.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.154+0000 7f18026f2640 1 -- 172.21.15.148:0/3839551159 wait complete. 2024-03-29T04:11:08.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.155+0000 7f18026f2640 1 Processor -- start 2024-03-29T04:11:08.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.156+0000 7f18026f2640 1 -- start start 2024-03-29T04:11:08.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.156+0000 7f18026f2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17fc0b3430 0x7f17fc161190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:08.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.156+0000 7f18026f2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17fc15e3c0 0x7f17fc0cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:08.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.156+0000 7f18026f2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17fc0cdde0 0x7f17fc1e7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:08.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.156+0000 7f18026f2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc0b3b20 con 0x7f17fc0cdde0 2024-03-29T04:11:08.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.156+0000 7f18026f2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc0b39a0 con 0x7f17fc0b3430 2024-03-29T04:11:08.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.156+0000 7f18026f2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f17fc06bd30 con 0x7f17fc15e3c0 2024-03-29T04:11:08.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.156+0000 7f1800c8f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17fc0cdde0 0x7f17fc1e7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:08.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.156+0000 7f1800c8f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17fc0cdde0 0x7f17fc1e7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43178/0 (socket says 172.21.15.148:43178) 2024-03-29T04:11:08.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.156+0000 7f1800c8f640 1 -- 172.21.15.148:0/3201518232 learned_addr learned my addr 172.21.15.148:0/3201518232 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:08.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.157+0000 7f17fb7fe640 1 --2- 172.21.15.148:0/3201518232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17fc15e3c0 0x7f17fc0cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:08.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.157+0000 7f17fbfff640 1 --2- 172.21.15.148:0/3201518232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17fc0b3430 0x7f17fc161190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:08.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.157+0000 7f1800c8f640 1 -- 172.21.15.148:0/3201518232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17fc15e3c0 msgr2=0x7f17fc0cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:08.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.157+0000 7f1800c8f640 1 --2- 172.21.15.148:0/3201518232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17fc15e3c0 0x7f17fc0cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.157+0000 7f1800c8f640 1 -- 172.21.15.148:0/3201518232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17fc0b3430 msgr2=0x7f17fc161190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:08.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.157+0000 7f1800c8f640 1 --2- 172.21.15.148:0/3201518232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17fc0b3430 0x7f17fc161190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.157+0000 7f1800c8f640 1 -- 172.21.15.148:0/3201518232 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17fc1e8a80 con 0x7f17fc0cdde0 2024-03-29T04:11:08.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.157+0000 7f1800c8f640 1 --2- 172.21.15.148:0/3201518232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17fc0cdde0 0x7f17fc1e7d70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f17f400efc0 tx=0x7f17f400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:08.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.157+0000 7f17f8ff9640 1 -- 172.21.15.148:0/3201518232 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f4018020 con 0x7f17fc0cdde0 2024-03-29T04:11:08.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.157+0000 7f17f8ff9640 1 -- 172.21.15.148:0/3201518232 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17f4014480 con 0x7f17fc0cdde0 2024-03-29T04:11:08.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.157+0000 7f17f8ff9640 1 -- 172.21.15.148:0/3201518232 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f4014870 con 0x7f17fc0cdde0 2024-03-29T04:11:08.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.157+0000 7f18026f2640 1 -- 172.21.15.148:0/3201518232 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17fc1e8d60 con 0x7f17fc0cdde0 2024-03-29T04:11:08.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.157+0000 7f18026f2640 1 -- 172.21.15.148:0/3201518232 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17fc1e9320 con 0x7f17fc0cdde0 2024-03-29T04:11:08.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.158+0000 7f18026f2640 1 -- 172.21.15.148:0/3201518232 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17fc068850 con 0x7f17fc0cdde0 2024-03-29T04:11:08.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.159+0000 7f17f8ff9640 1 -- 172.21.15.148:0/3201518232 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f17f4005d10 con 0x7f17fc0cdde0 2024-03-29T04:11:08.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.159+0000 7f17f8ff9640 1 --2- 172.21.15.148:0/3201518232 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f17dc06c530 0x7f17dc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:08.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.159+0000 7f17f8ff9640 1 -- 172.21.15.148:0/3201518232 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f17f408c810 con 0x7f17fc0cdde0 2024-03-29T04:11:08.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.159+0000 7f17fbfff640 1 --2- 172.21.15.148:0/3201518232 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f17dc06c530 0x7f17dc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:08.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.159+0000 7f17fbfff640 1 --2- 172.21.15.148:0/3201518232 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f17dc06c530 0x7f17dc06e9f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f17ec036000 tx=0x7f17ec03d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:08.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.163+0000 7f17f8ff9640 1 -- 172.21.15.148:0/3201518232 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f17f4022030 con 0x7f17fc0cdde0 2024-03-29T04:11:08.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.340+0000 7f18026f2640 1 -- 172.21.15.148:0/3201518232 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f17fc0cf110 con 0x7f17fc0cdde0 2024-03-29T04:11:08.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.341+0000 7f17f8ff9640 1 -- 172.21.15.148:0/3201518232 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f17f4010040 con 0x7f17fc0cdde0 2024-03-29T04:11:08.320 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:11:08.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.343+0000 7f18026f2640 1 -- 172.21.15.148:0/3201518232 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f17dc06c530 msgr2=0x7f17dc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:08.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.343+0000 7f18026f2640 1 --2- 172.21.15.148:0/3201518232 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f17dc06c530 0x7f17dc06e9f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f17ec036000 tx=0x7f17ec03d040 comp rx=0 tx=0).stop 2024-03-29T04:11:08.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.343+0000 7f18026f2640 1 -- 172.21.15.148:0/3201518232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17fc0cdde0 msgr2=0x7f17fc1e7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:08.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.343+0000 7f18026f2640 1 --2- 172.21.15.148:0/3201518232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17fc0cdde0 0x7f17fc1e7d70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f17f400efc0 tx=0x7f17f400c490 comp rx=0 tx=0).stop 2024-03-29T04:11:08.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.343+0000 7f18026f2640 1 -- 172.21.15.148:0/3201518232 shutdown_connections 2024-03-29T04:11:08.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.343+0000 7f18026f2640 1 --2- 172.21.15.148:0/3201518232 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f17dc06c530 0x7f17dc06e9f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.343+0000 7f18026f2640 1 --2- 172.21.15.148:0/3201518232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17fc0cdde0 0x7f17fc1e7d70 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.343+0000 7f18026f2640 1 --2- 172.21.15.148:0/3201518232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17fc15e3c0 0x7f17fc0cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.343+0000 7f18026f2640 1 --2- 172.21.15.148:0/3201518232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17fc0b3430 0x7f17fc161190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.343+0000 7f18026f2640 1 -- 172.21.15.148:0/3201518232 >> 172.21.15.148:0/3201518232 conn(0x7f17fc0b6430 msgr2=0x7f17fc0b5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:08.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.343+0000 7f18026f2640 1 -- 172.21.15.148:0/3201518232 shutdown_connections 2024-03-29T04:11:08.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.343+0000 7f18026f2640 1 -- 172.21.15.148:0/3201518232 wait complete. 2024-03-29T04:11:08.332 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:11:08.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.458+0000 7f5dd9f7c640 1 Processor -- start 2024-03-29T04:11:08.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.458+0000 7f5dd9f7c640 1 -- start start 2024-03-29T04:11:08.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.459+0000 7f5dd9f7c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5dd415e3c0 0x7f5dd4161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:08.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.459+0000 7f5dd9f7c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5dd40b3430 0x7f5dd406b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:08.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.459+0000 7f5dd9f7c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dd40cf330 con 0x7f5dd40b2f00 2024-03-29T04:11:08.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.459+0000 7f5dd9f7c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f5dd40cf960 con 0x7f5dd40b3430 2024-03-29T04:11:08.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.459+0000 7f5dd9f7c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5dd4162cf0 con 0x7f5dd415e3c0 2024-03-29T04:11:08.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.459+0000 7f5dd3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5dd40b3430 0x7f5dd406b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:08.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.459+0000 7f5dd3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5dd40b3430 0x7f5dd406b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43188/0 (socket says 172.21.15.148:43188) 2024-03-29T04:11:08.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.459+0000 7f5dd3fff640 1 -- 172.21.15.148:0/1811270356 learned_addr learned my addr 172.21.15.148:0/1811270356 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:08.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.459+0000 7f5dd37fe640 1 --2- 172.21.15.148:0/1811270356 >> v2:172.21.15.157:3300/0 conn(0x7f5dd415e3c0 0x7f5dd4161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:08.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.459+0000 7f5dd3fff640 1 -- 172.21.15.148:0/1811270356 >> v2:172.21.15.157:3300/0 conn(0x7f5dd415e3c0 msgr2=0x7f5dd4161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:08.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.459+0000 7f5dd3fff640 1 --2- 172.21.15.148:0/1811270356 >> v2:172.21.15.157:3300/0 conn(0x7f5dd415e3c0 0x7f5dd4161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.459+0000 7f5dd3fff640 1 -- 172.21.15.148:0/1811270356 >> v1:172.21.15.157:6789/0 conn(0x7f5dd40b2f00 legacy=0x7f5dd41624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:08.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.459+0000 7f5dd3fff640 1 -- 172.21.15.148:0/1811270356 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd40cf5f0 con 0x7f5dd40b3430 2024-03-29T04:11:08.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.460+0000 7f5dd3fff640 1 --2- 172.21.15.148:0/1811270356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dd40b3430 0x7f5dd406b7f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5dd40d2670 tx=0x7f5dcc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8361bbe4ffde61 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:08.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.460+0000 7f5dd27fc640 1 -- 172.21.15.148:0/1811270356 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc02ca80 con 0x7f5dd40b3430 2024-03-29T04:11:08.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.460+0000 7f5dd27fc640 1 -- 172.21.15.148:0/1811270356 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dcc02cc20 con 0x7f5dd40b3430 2024-03-29T04:11:08.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.460+0000 7f5dd27fc640 1 -- 172.21.15.148:0/1811270356 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc0343f0 con 0x7f5dd40b3430 2024-03-29T04:11:08.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.461+0000 7f5dd9f7c640 1 -- 172.21.15.148:0/1811270356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dd40b3430 msgr2=0x7f5dd406b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:08.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.461+0000 7f5dd9f7c640 1 --2- 172.21.15.148:0/1811270356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dd40b3430 0x7f5dd406b7f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5dd40d2670 tx=0x7f5dcc02bee0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.461+0000 7f5dd9f7c640 1 -- 172.21.15.148:0/1811270356 shutdown_connections 2024-03-29T04:11:08.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.461+0000 7f5dd9f7c640 1 --2- 172.21.15.148:0/1811270356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dd40b3430 0x7f5dd406b7f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.461+0000 7f5dd9f7c640 1 --2- 172.21.15.148:0/1811270356 >> v2:172.21.15.157:3300/0 conn(0x7f5dd415e3c0 0x7f5dd4161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.461+0000 7f5dd9f7c640 1 -- 172.21.15.148:0/1811270356 >> 172.21.15.148:0/1811270356 conn(0x7f5dd40b6430 msgr2=0x7f5dd40d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:08.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.461+0000 7f5dd9f7c640 1 -- 172.21.15.148:0/1811270356 shutdown_connections 2024-03-29T04:11:08.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.461+0000 7f5dd9f7c640 1 -- 172.21.15.148:0/1811270356 wait complete. 2024-03-29T04:11:08.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.462+0000 7f5dd9f7c640 1 Processor -- start 2024-03-29T04:11:08.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.462+0000 7f5dd9f7c640 1 -- start start 2024-03-29T04:11:08.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.463+0000 7f5dd9f7c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5dd40b3430 0x7f5dd41f75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:08.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.463+0000 7f5dd9f7c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dd415e3c0 0x7f5dd40cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:08.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.463+0000 7f5dd9f7c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dd40cb8e0 0x7f5dd40b88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:08.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.463+0000 7f5dd9f7c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd40b3b20 con 0x7f5dd40cb8e0 2024-03-29T04:11:08.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.463+0000 7f5dd9f7c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd40b39a0 con 0x7f5dd415e3c0 2024-03-29T04:11:08.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.463+0000 7f5dd9f7c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5dd406bd30 con 0x7f5dd40b3430 2024-03-29T04:11:08.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.463+0000 7f5dd3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dd40cb8e0 0x7f5dd40b88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:08.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.463+0000 7f5dd3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dd40cb8e0 0x7f5dd40b88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43190/0 (socket says 172.21.15.148:43190) 2024-03-29T04:11:08.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.463+0000 7f5dd3fff640 1 -- 172.21.15.148:0/2945573001 learned_addr learned my addr 172.21.15.148:0/2945573001 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:08.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.463+0000 7f5dd37fe640 1 --2- 172.21.15.148:0/2945573001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5dd40b3430 0x7f5dd41f75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:08.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.463+0000 7f5dd3fff640 1 -- 172.21.15.148:0/2945573001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5dd40b3430 msgr2=0x7f5dd41f75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:08.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.463+0000 7f5dd3fff640 1 --2- 172.21.15.148:0/2945573001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5dd40b3430 0x7f5dd41f75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.463+0000 7f5dd2ffd640 1 --2- 172.21.15.148:0/2945573001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dd415e3c0 0x7f5dd40cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:08.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.463+0000 7f5dd3fff640 1 -- 172.21.15.148:0/2945573001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dd415e3c0 msgr2=0x7f5dd40cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:08.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.463+0000 7f5dd3fff640 1 --2- 172.21.15.148:0/2945573001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dd415e3c0 0x7f5dd40cb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.463+0000 7f5dd3fff640 1 -- 172.21.15.148:0/2945573001 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd41e9910 con 0x7f5dd40cb8e0 2024-03-29T04:11:08.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.463+0000 7f5dd3fff640 1 --2- 172.21.15.148:0/2945573001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dd40cb8e0 0x7f5dd40b88b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc036040 tx=0x7f5dcc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:08.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.464+0000 7f5dbbfff640 1 -- 172.21.15.148:0/2945573001 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc043020 con 0x7f5dd40cb8e0 2024-03-29T04:11:08.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.464+0000 7f5dbbfff640 1 -- 172.21.15.148:0/2945573001 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dcc03e450 con 0x7f5dd40cb8e0 2024-03-29T04:11:08.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.464+0000 7f5dd9f7c640 1 -- 172.21.15.148:0/2945573001 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dd40cbf10 con 0x7f5dd40cb8e0 2024-03-29T04:11:08.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.464+0000 7f5dbbfff640 1 -- 172.21.15.148:0/2945573001 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc009510 con 0x7f5dd40cb8e0 2024-03-29T04:11:08.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.464+0000 7f5dd9f7c640 1 -- 172.21.15.148:0/2945573001 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dd40cc4d0 con 0x7f5dd40cb8e0 2024-03-29T04:11:08.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.465+0000 7f5dd9f7c640 1 -- 172.21.15.148:0/2945573001 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dd4068850 con 0x7f5dd40cb8e0 2024-03-29T04:11:08.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.465+0000 7f5dbbfff640 1 -- 172.21.15.148:0/2945573001 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5dcc046020 con 0x7f5dd40cb8e0 2024-03-29T04:11:08.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.466+0000 7f5dbbfff640 1 --2- 172.21.15.148:0/2945573001 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5dc006c530 0x7f5dc006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:08.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.466+0000 7f5dbbfff640 1 -- 172.21.15.148:0/2945573001 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f5dcc0ae2c0 con 0x7f5dd40cb8e0 2024-03-29T04:11:08.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.466+0000 7f5dd37fe640 1 --2- 172.21.15.148:0/2945573001 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5dc006c530 0x7f5dc006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:08.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.466+0000 7f5dd37fe640 1 --2- 172.21.15.148:0/2945573001 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5dc006c530 0x7f5dc006e9f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8006fd0 tx=0x7f5dc8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:08.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.470+0000 7f5dbbfff640 1 -- 172.21.15.148:0/2945573001 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5dcc04d030 con 0x7f5dd40cb8e0 2024-03-29T04:11:08.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.654+0000 7f5dd9f7c640 1 -- 172.21.15.148:0/2945573001 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5dd41e9da0 con 0x7f5dd40cb8e0 2024-03-29T04:11:08.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.655+0000 7f5dbbfff640 1 -- 172.21.15.148:0/2945573001 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f5dcc07c570 con 0x7f5dd40cb8e0 2024-03-29T04:11:08.634 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-29T04:11:08.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.656+0000 7f5dd9f7c640 1 -- 172.21.15.148:0/2945573001 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5dc006c530 msgr2=0x7f5dc006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:08.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.656+0000 7f5dd9f7c640 1 --2- 172.21.15.148:0/2945573001 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5dc006c530 0x7f5dc006e9f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8006fd0 tx=0x7f5dc8008040 comp rx=0 tx=0).stop 2024-03-29T04:11:08.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.656+0000 7f5dd9f7c640 1 -- 172.21.15.148:0/2945573001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dd40cb8e0 msgr2=0x7f5dd40b88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:08.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.656+0000 7f5dd9f7c640 1 --2- 172.21.15.148:0/2945573001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dd40cb8e0 0x7f5dd40b88b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc036040 tx=0x7f5dcc004a40 comp rx=0 tx=0).stop 2024-03-29T04:11:08.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.657+0000 7f5dd9f7c640 1 -- 172.21.15.148:0/2945573001 shutdown_connections 2024-03-29T04:11:08.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.657+0000 7f5dd9f7c640 1 --2- 172.21.15.148:0/2945573001 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5dc006c530 0x7f5dc006e9f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.657+0000 7f5dd9f7c640 1 --2- 172.21.15.148:0/2945573001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dd40cb8e0 0x7f5dd40b88b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.657+0000 7f5dd9f7c640 1 --2- 172.21.15.148:0/2945573001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dd415e3c0 0x7f5dd40cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.657+0000 7f5dd9f7c640 1 --2- 172.21.15.148:0/2945573001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5dd40b3430 0x7f5dd41f75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.657+0000 7f5dd9f7c640 1 -- 172.21.15.148:0/2945573001 >> 172.21.15.148:0/2945573001 conn(0x7f5dd40b6430 msgr2=0x7f5dd40d0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:08.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.657+0000 7f5dd9f7c640 1 -- 172.21.15.148:0/2945573001 shutdown_connections 2024-03-29T04:11:08.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.657+0000 7f5dd9f7c640 1 -- 172.21.15.148:0/2945573001 wait complete. 2024-03-29T04:11:08.647 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:11:08.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.773+0000 7f74eee4e640 1 Processor -- start 2024-03-29T04:11:08.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.773+0000 7f74eee4e640 1 -- start start 2024-03-29T04:11:08.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.774+0000 7f74eee4e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f74e80b2f00 0x7f74e8162680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:08.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.774+0000 7f74eee4e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74e80b3430 0x7f74e806b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:08.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.774+0000 7f74eee4e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74e80cf330 con 0x7f74e815e3c0 2024-03-29T04:11:08.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.774+0000 7f74eee4e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f74e80cf960 con 0x7f74e80b2f00 2024-03-29T04:11:08.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.774+0000 7f74eee4e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f74e8162cf0 con 0x7f74e80b3430 2024-03-29T04:11:08.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.774+0000 7f74e7fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f74e80b2f00 0x7f74e8162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:08.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.774+0000 7f74ecbea640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f74e815e3c0 0x7f74e8161f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35714/0 (socket says 172.21.15.148:35714) 2024-03-29T04:11:08.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.774+0000 7f74e7fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f74e80b2f00 0x7f74e8162680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43204/0 (socket says 172.21.15.148:43204) 2024-03-29T04:11:08.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.774+0000 7f74ecbea640 1 -- 172.21.15.148:0/387497590 learned_addr learned my addr 172.21.15.148:0/387497590 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:08.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.774+0000 7f74ed3eb640 1 --2- 172.21.15.148:0/387497590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74e80b3430 0x7f74e806b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:08.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.774+0000 7f74e7fff640 1 -- 172.21.15.148:0/387497590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74e80b3430 msgr2=0x7f74e806b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:08.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.774+0000 7f74e7fff640 1 --2- 172.21.15.148:0/387497590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74e80b3430 0x7f74e806b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.774+0000 7f74e7fff640 1 -- 172.21.15.148:0/387497590 >> v1:172.21.15.148:6789/0 conn(0x7f74e815e3c0 legacy=0x7f74e8161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:08.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.774+0000 7f74e77fe640 1 -- 172.21.15.148:0/387497590 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1483147102 0 0) 0x7f74e80cf330 con 0x7f74e815e3c0 2024-03-29T04:11:08.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.774+0000 7f74e7fff640 1 -- 172.21.15.148:0/387497590 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74e80cf5f0 con 0x7f74e80b2f00 2024-03-29T04:11:08.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.775+0000 7f74e7fff640 1 --2- 172.21.15.148:0/387497590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74e80b2f00 0x7f74e8162680 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f74d40097a0 tx=0x7f74d402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e4305cfab74ffa server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:08.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.775+0000 7f74e77fe640 1 -- 172.21.15.148:0/387497590 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d40315a0 con 0x7f74e80b2f00 2024-03-29T04:11:08.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.775+0000 7f74e77fe640 1 -- 172.21.15.148:0/387497590 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74d4037070 con 0x7f74e80b2f00 2024-03-29T04:11:08.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.775+0000 7f74e77fe640 1 -- 172.21.15.148:0/387497590 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d4031960 con 0x7f74e80b2f00 2024-03-29T04:11:08.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.776+0000 7f74eee4e640 1 -- 172.21.15.148:0/387497590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74e80b2f00 msgr2=0x7f74e8162680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:08.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.776+0000 7f74eee4e640 1 --2- 172.21.15.148:0/387497590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74e80b2f00 0x7f74e8162680 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f74d40097a0 tx=0x7f74d402b840 comp rx=0 tx=0).stop 2024-03-29T04:11:08.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.776+0000 7f74eee4e640 1 -- 172.21.15.148:0/387497590 shutdown_connections 2024-03-29T04:11:08.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.776+0000 7f74eee4e640 1 --2- 172.21.15.148:0/387497590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74e80b3430 0x7f74e806b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.776+0000 7f74eee4e640 1 --2- 172.21.15.148:0/387497590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74e80b2f00 0x7f74e8162680 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.776+0000 7f74eee4e640 1 -- 172.21.15.148:0/387497590 >> 172.21.15.148:0/387497590 conn(0x7f74e80b6430 msgr2=0x7f74e80d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:08.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.776+0000 7f74eee4e640 1 -- 172.21.15.148:0/387497590 shutdown_connections 2024-03-29T04:11:08.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.776+0000 7f74eee4e640 1 -- 172.21.15.148:0/387497590 wait complete. 2024-03-29T04:11:08.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.777+0000 7f74eee4e640 1 Processor -- start 2024-03-29T04:11:08.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.777+0000 7f74eee4e640 1 -- start start 2024-03-29T04:11:08.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.778+0000 7f74eee4e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74e80b3430 0x7f74e815f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:08.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.778+0000 7f74eee4e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74e815e3c0 0x7f74e8160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:08.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.778+0000 7f74eee4e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74e8153430 0x7f74e81538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:08.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.778+0000 7f74eee4e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f74e80b3b20 con 0x7f74e80b3430 2024-03-29T04:11:08.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.778+0000 7f74eee4e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f74e80b39a0 con 0x7f74e8153430 2024-03-29T04:11:08.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.778+0000 7f74eee4e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f74e806bd30 con 0x7f74e815e3c0 2024-03-29T04:11:08.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.778+0000 7f74ecbea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74e80b3430 0x7f74e815f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:08.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.778+0000 7f74ecbea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74e80b3430 0x7f74e815f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43216/0 (socket says 172.21.15.148:43216) 2024-03-29T04:11:08.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.778+0000 7f74ecbea640 1 -- 172.21.15.148:0/3406217371 learned_addr learned my addr 172.21.15.148:0/3406217371 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:08.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.778+0000 7f74e7fff640 1 --2- 172.21.15.148:0/3406217371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74e815e3c0 0x7f74e8160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:08.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.778+0000 7f74ed3eb640 1 --2- 172.21.15.148:0/3406217371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74e8153430 0x7f74e81538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:08.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.778+0000 7f74ecbea640 1 -- 172.21.15.148:0/3406217371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74e815e3c0 msgr2=0x7f74e8160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:08.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.778+0000 7f74ecbea640 1 --2- 172.21.15.148:0/3406217371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74e815e3c0 0x7f74e8160280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.778+0000 7f74ecbea640 1 -- 172.21.15.148:0/3406217371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74e8153430 msgr2=0x7f74e81538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:08.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.778+0000 7f74ecbea640 1 --2- 172.21.15.148:0/3406217371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74e8153430 0x7f74e81538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.778+0000 7f74ecbea640 1 -- 172.21.15.148:0/3406217371 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74e8155ff0 con 0x7f74e80b3430 2024-03-29T04:11:08.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.779+0000 7f74ecbea640 1 --2- 172.21.15.148:0/3406217371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74e80b3430 0x7f74e815f610 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f74dc00c430 tx=0x7f74dc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:08.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.779+0000 7f74e57fa640 1 -- 172.21.15.148:0/3406217371 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74dc0074e0 con 0x7f74e80b3430 2024-03-29T04:11:08.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.779+0000 7f74e57fa640 1 -- 172.21.15.148:0/3406217371 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74dc0047e0 con 0x7f74e80b3430 2024-03-29T04:11:08.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.779+0000 7f74eee4e640 1 -- 172.21.15.148:0/3406217371 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74e815eb30 con 0x7f74e80b3430 2024-03-29T04:11:08.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.779+0000 7f74e57fa640 1 -- 172.21.15.148:0/3406217371 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74dc004bd0 con 0x7f74e80b3430 2024-03-29T04:11:08.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.779+0000 7f74eee4e640 1 -- 172.21.15.148:0/3406217371 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74e815f0f0 con 0x7f74e80b3430 2024-03-29T04:11:08.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.780+0000 7f74eee4e640 1 -- 172.21.15.148:0/3406217371 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74e8068850 con 0x7f74e80b3430 2024-03-29T04:11:08.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.780+0000 7f74e57fa640 1 -- 172.21.15.148:0/3406217371 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f74dc011070 con 0x7f74e80b3430 2024-03-29T04:11:08.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.781+0000 7f74e57fa640 1 --2- 172.21.15.148:0/3406217371 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f74cc06c530 0x7f74cc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:08.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.781+0000 7f74e57fa640 1 -- 172.21.15.148:0/3406217371 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f74dc08b5c0 con 0x7f74e80b3430 2024-03-29T04:11:08.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.781+0000 7f74e7fff640 1 --2- 172.21.15.148:0/3406217371 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f74cc06c530 0x7f74cc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:08.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.781+0000 7f74e7fff640 1 --2- 172.21.15.148:0/3406217371 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f74cc06c530 0x7f74cc06e9f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f74d4036000 tx=0x7f74d4008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:08.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.785+0000 7f74e57fa640 1 -- 172.21.15.148:0/3406217371 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f74dc059190 con 0x7f74e80b3430 2024-03-29T04:11:08.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.965+0000 7f74eee4e640 1 -- 172.21.15.148:0/3406217371 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f74e80cf110 con 0x7f74e80b3430 2024-03-29T04:11:08.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.966+0000 7f74e57fa640 1 -- 172.21.15.148:0/3406217371 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f74dc05d490 con 0x7f74e80b3430 2024-03-29T04:11:08.945 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:11:08.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.968+0000 7f74eee4e640 1 -- 172.21.15.148:0/3406217371 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f74cc06c530 msgr2=0x7f74cc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:08.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.968+0000 7f74eee4e640 1 --2- 172.21.15.148:0/3406217371 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f74cc06c530 0x7f74cc06e9f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f74d4036000 tx=0x7f74d4008030 comp rx=0 tx=0).stop 2024-03-29T04:11:08.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.968+0000 7f74eee4e640 1 -- 172.21.15.148:0/3406217371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74e80b3430 msgr2=0x7f74e815f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:08.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.968+0000 7f74eee4e640 1 --2- 172.21.15.148:0/3406217371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74e80b3430 0x7f74e815f610 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f74dc00c430 tx=0x7f74dc00c8f0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.968+0000 7f74eee4e640 1 -- 172.21.15.148:0/3406217371 shutdown_connections 2024-03-29T04:11:08.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.968+0000 7f74eee4e640 1 --2- 172.21.15.148:0/3406217371 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f74cc06c530 0x7f74cc06e9f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.968+0000 7f74eee4e640 1 --2- 172.21.15.148:0/3406217371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74e8153430 0x7f74e81538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.968+0000 7f74eee4e640 1 --2- 172.21.15.148:0/3406217371 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74e815e3c0 0x7f74e8160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.968+0000 7f74eee4e640 1 --2- 172.21.15.148:0/3406217371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74e80b3430 0x7f74e815f610 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:08.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.968+0000 7f74eee4e640 1 -- 172.21.15.148:0/3406217371 >> 172.21.15.148:0/3406217371 conn(0x7f74e80b6430 msgr2=0x7f74e80b51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:08.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.968+0000 7f74eee4e640 1 -- 172.21.15.148:0/3406217371 shutdown_connections 2024-03-29T04:11:08.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:08.968+0000 7f74eee4e640 1 -- 172.21.15.148:0/3406217371 wait complete. 2024-03-29T04:11:08.961 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:11:09.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.084+0000 7fe4992a5640 1 Processor -- start 2024-03-29T04:11:09.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.084+0000 7fe4992a5640 1 -- start start 2024-03-29T04:11:09.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.085+0000 7fe4992a5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe49415e590 0x7fe4940b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:09.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.085+0000 7fe4992a5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe49415fa90 con 0x7fe4940afc80 2024-03-29T04:11:09.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.085+0000 7fe4992a5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4940b4470 con 0x7fe49406c6c0 2024-03-29T04:11:09.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.085+0000 7fe4992a5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe49406f670 con 0x7fe49415e590 2024-03-29T04:11:09.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.085+0000 7fe4927fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe4940afc80 0x7fe49415ec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35728/0 (socket says 172.21.15.148:35728) 2024-03-29T04:11:09.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.085+0000 7fe4927fc640 1 -- 172.21.15.148:0/797287378 learned_addr learned my addr 172.21.15.148:0/797287378 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:09.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.085+0000 7fe491ffb640 1 -- 172.21.15.148:0/797287378 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3729426588 0 0) 0x7fe49415fa90 con 0x7fe4940afc80 2024-03-29T04:11:09.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.085+0000 7fe491ffb640 1 -- 172.21.15.148:0/797287378 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe47c003520 con 0x7fe4940afc80 2024-03-29T04:11:09.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.085+0000 7fe492ffd640 1 --2- 172.21.15.148:0/797287378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe49415e590 0x7fe4940b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:09.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.085+0000 7fe491ffb640 1 -- 172.21.15.148:0/797287378 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1221749271 0 0) 0x7fe47c003520 con 0x7fe4940afc80 2024-03-29T04:11:09.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.085+0000 7fe491ffb640 1 -- 172.21.15.148:0/797287378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe49415e590 msgr2=0x7fe4940b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:09.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.085+0000 7fe491ffb640 1 --2- 172.21.15.148:0/797287378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe49415e590 0x7fe4940b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.085+0000 7fe491ffb640 1 -- 172.21.15.148:0/797287378 >> v1:172.21.15.157:6789/0 conn(0x7fe49406c6c0 legacy=0x7fe49415f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:09.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.085+0000 7fe491ffb640 1 -- 172.21.15.148:0/797287378 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49406db80 con 0x7fe4940afc80 2024-03-29T04:11:09.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.085+0000 7fe491ffb640 1 -- 172.21.15.148:0/797287378 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7fe488001670 con 0x7fe4940afc80 2024-03-29T04:11:09.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.085+0000 7fe491ffb640 1 -- 172.21.15.148:0/797287378 >> v1:172.21.15.148:6789/0 conn(0x7fe4940afc80 legacy=0x7fe49415ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:09.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.085+0000 7fe491ffb640 1 --2- 172.21.15.148:0/797287378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe47c0039e0 0x7fe47c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:09.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.085+0000 7fe491ffb640 1 --2- 172.21.15.148:0/797287378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe47c004800 0x7fe47c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:09.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.085+0000 7fe491ffb640 1 --2- 172.21.15.148:0/797287378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe49415e590 0x7fe47c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:09.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.085+0000 7fe491ffb640 1 -- 172.21.15.148:0/797287378 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe47c00ba70 con 0x7fe47c0039e0 2024-03-29T04:11:09.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.085+0000 7fe491ffb640 1 -- 172.21.15.148:0/797287378 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe47c00bbf0 con 0x7fe47c004800 2024-03-29T04:11:09.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.085+0000 7fe491ffb640 1 -- 172.21.15.148:0/797287378 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe47c00e000 con 0x7fe49415e590 2024-03-29T04:11:09.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.086+0000 7fe492ffd640 1 --2- 172.21.15.148:0/797287378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe47c0039e0 0x7fe47c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:09.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.086+0000 7fe4927fc640 1 --2- 172.21.15.148:0/797287378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe47c004800 0x7fe47c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:09.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.086+0000 7fe4937fe640 1 --2- 172.21.15.148:0/797287378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe49415e590 0x7fe47c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:09.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.086+0000 7fe492ffd640 1 -- 172.21.15.148:0/797287378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe49415e590 msgr2=0x7fe47c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:09.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.086+0000 7fe492ffd640 1 --2- 172.21.15.148:0/797287378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe49415e590 0x7fe47c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.086+0000 7fe492ffd640 1 -- 172.21.15.148:0/797287378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe47c004800 msgr2=0x7fe47c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:09.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.086+0000 7fe492ffd640 1 --2- 172.21.15.148:0/797287378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe47c004800 0x7fe47c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.086+0000 7fe492ffd640 1 -- 172.21.15.148:0/797287378 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe47c003520 con 0x7fe47c0039e0 2024-03-29T04:11:09.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.087+0000 7fe492ffd640 1 --2- 172.21.15.148:0/797287378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe47c0039e0 0x7fe47c003e00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe484009a90 tx=0x7fe47c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23048a7bf22de16a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:09.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.087+0000 7fe491ffb640 1 -- 172.21.15.148:0/797287378 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484039070 con 0x7fe47c0039e0 2024-03-29T04:11:09.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.087+0000 7fe491ffb640 1 -- 172.21.15.148:0/797287378 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe48402c680 con 0x7fe47c0039e0 2024-03-29T04:11:09.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.087+0000 7fe491ffb640 1 -- 172.21.15.148:0/797287378 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484009690 con 0x7fe47c0039e0 2024-03-29T04:11:09.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.087+0000 7fe4992a5640 1 -- 172.21.15.148:0/797287378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe47c0039e0 msgr2=0x7fe47c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:09.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.087+0000 7fe4992a5640 1 --2- 172.21.15.148:0/797287378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe47c0039e0 0x7fe47c003e00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe484009a90 tx=0x7fe47c005830 comp rx=0 tx=0).stop 2024-03-29T04:11:09.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.087+0000 7fe4937fe640 1 -- 172.21.15.148:0/797287378 reap_dead start 2024-03-29T04:11:09.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.087+0000 7fe4992a5640 1 -- 172.21.15.148:0/797287378 shutdown_connections 2024-03-29T04:11:09.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.087+0000 7fe4992a5640 1 -- 172.21.15.148:0/797287378 >> 172.21.15.148:0/797287378 conn(0x7fe4940d0700 msgr2=0x7fe4940d0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:09.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.087+0000 7fe4992a5640 1 -- 172.21.15.148:0/797287378 shutdown_connections 2024-03-29T04:11:09.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.087+0000 7fe4992a5640 1 -- 172.21.15.148:0/797287378 wait complete. 2024-03-29T04:11:09.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.089+0000 7fe4992a5640 1 Processor -- start 2024-03-29T04:11:09.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.089+0000 7fe4992a5640 1 -- start start 2024-03-29T04:11:09.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.089+0000 7fe4992a5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe47c0039e0 0x7fe49415a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:09.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.089+0000 7fe4992a5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe494157d40 0x7fe49414d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:09.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.089+0000 7fe4992a5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4940b0b10 0x7fe4940b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:09.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.089+0000 7fe4992a5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe49415e970 con 0x7fe47c0039e0 2024-03-29T04:11:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.089+0000 7fe4992a5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe49406e140 con 0x7fe494157d40 2024-03-29T04:11:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.089+0000 7fe4992a5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe4940b4120 con 0x7fe4940b0b10 2024-03-29T04:11:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.089+0000 7fe492ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe47c0039e0 0x7fe49415a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.089+0000 7fe492ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe47c0039e0 0x7fe49415a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43234/0 (socket says 172.21.15.148:43234) 2024-03-29T04:11:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.089+0000 7fe492ffd640 1 -- 172.21.15.148:0/816297037 learned_addr learned my addr 172.21.15.148:0/816297037 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.089+0000 7fe4927fc640 1 --2- 172.21.15.148:0/816297037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe494157d40 0x7fe49414d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.090+0000 7fe4937fe640 1 --2- 172.21.15.148:0/816297037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4940b0b10 0x7fe4940b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.090+0000 7fe492ffd640 1 -- 172.21.15.148:0/816297037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4940b0b10 msgr2=0x7fe4940b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.090+0000 7fe492ffd640 1 --2- 172.21.15.148:0/816297037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4940b0b10 0x7fe4940b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.090+0000 7fe492ffd640 1 -- 172.21.15.148:0/816297037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe494157d40 msgr2=0x7fe49414d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.090+0000 7fe492ffd640 1 --2- 172.21.15.148:0/816297037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe494157d40 0x7fe49414d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.090+0000 7fe492ffd640 1 -- 172.21.15.148:0/816297037 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe494158490 con 0x7fe47c0039e0 2024-03-29T04:11:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.090+0000 7fe4937fe640 1 --2- 172.21.15.148:0/816297037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4940b0b10 0x7fe4940b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.090+0000 7fe4927fc640 1 --2- 172.21.15.148:0/816297037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe494157d40 0x7fe49414d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:09.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.090+0000 7fe492ffd640 1 --2- 172.21.15.148:0/816297037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe47c0039e0 0x7fe49415a020 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe484000c00 tx=0x7fe484004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:09.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.090+0000 7fe47b7fe640 1 -- 172.21.15.148:0/816297037 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484042020 con 0x7fe47c0039e0 2024-03-29T04:11:09.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.090+0000 7fe47b7fe640 1 -- 172.21.15.148:0/816297037 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe48403a040 con 0x7fe47c0039e0 2024-03-29T04:11:09.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.090+0000 7fe47b7fe640 1 -- 172.21.15.148:0/816297037 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484039070 con 0x7fe47c0039e0 2024-03-29T04:11:09.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.090+0000 7fe4992a5640 1 -- 172.21.15.148:0/816297037 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe49415c4c0 con 0x7fe47c0039e0 2024-03-29T04:11:09.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.090+0000 7fe4992a5640 1 -- 172.21.15.148:0/816297037 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe49415ca80 con 0x7fe47c0039e0 2024-03-29T04:11:09.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.091+0000 7fe4797fa640 1 -- 172.21.15.148:0/816297037 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe46c005290 con 0x7fe47c0039e0 2024-03-29T04:11:09.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.092+0000 7fe47b7fe640 1 -- 172.21.15.148:0/816297037 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe484045020 con 0x7fe47c0039e0 2024-03-29T04:11:09.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.092+0000 7fe47b7fe640 1 --2- 172.21.15.148:0/816297037 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe48006c530 0x7fe48006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:09.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.092+0000 7fe4927fc640 1 --2- 172.21.15.148:0/816297037 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe48006c530 0x7fe48006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:09.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.092+0000 7fe47b7fe640 1 -- 172.21.15.148:0/816297037 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe4840acec0 con 0x7fe47c0039e0 2024-03-29T04:11:09.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.093+0000 7fe4927fc640 1 --2- 172.21.15.148:0/816297037 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe48006c530 0x7fe48006e9f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe49414e960 tx=0x7fe488009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:09.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.096+0000 7fe47b7fe640 1 -- 172.21.15.148:0/816297037 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe48404c070 con 0x7fe47c0039e0 2024-03-29T04:11:09.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.288+0000 7fe4797fa640 1 -- 172.21.15.148:0/816297037 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe46c005850 con 0x7fe47c0039e0 2024-03-29T04:11:09.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.289+0000 7fe47b7fe640 1 -- 172.21.15.148:0/816297037 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fe48407b0e0 con 0x7fe47c0039e0 2024-03-29T04:11:09.268 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-03-29T04:11:09.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.291+0000 7fe4797fa640 1 -- 172.21.15.148:0/816297037 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe48006c530 msgr2=0x7fe48006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:09.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.291+0000 7fe4797fa640 1 --2- 172.21.15.148:0/816297037 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe48006c530 0x7fe48006e9f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe49414e960 tx=0x7fe488009040 comp rx=0 tx=0).stop 2024-03-29T04:11:09.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.291+0000 7fe4797fa640 1 -- 172.21.15.148:0/816297037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe47c0039e0 msgr2=0x7fe49415a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:09.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.291+0000 7fe4797fa640 1 --2- 172.21.15.148:0/816297037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe47c0039e0 0x7fe49415a020 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe484000c00 tx=0x7fe484004730 comp rx=0 tx=0).stop 2024-03-29T04:11:09.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.291+0000 7fe4797fa640 1 -- 172.21.15.148:0/816297037 shutdown_connections 2024-03-29T04:11:09.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.291+0000 7fe4797fa640 1 --2- 172.21.15.148:0/816297037 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe48006c530 0x7fe48006e9f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.291+0000 7fe4797fa640 1 --2- 172.21.15.148:0/816297037 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4940b0b10 0x7fe4940b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.291+0000 7fe4797fa640 1 --2- 172.21.15.148:0/816297037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe494157d40 0x7fe49414d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.291+0000 7fe4797fa640 1 --2- 172.21.15.148:0/816297037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe47c0039e0 0x7fe49415a020 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.291+0000 7fe4797fa640 1 -- 172.21.15.148:0/816297037 >> 172.21.15.148:0/816297037 conn(0x7fe4940d0700 msgr2=0x7fe49415d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:09.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.291+0000 7fe4797fa640 1 -- 172.21.15.148:0/816297037 shutdown_connections 2024-03-29T04:11:09.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.291+0000 7fe4797fa640 1 -- 172.21.15.148:0/816297037 wait complete. 2024-03-29T04:11:09.280 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:11:09.281 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:11:09.299 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:11:09.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.425+0000 7f89e45c8640 1 Processor -- start 2024-03-29T04:11:09.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.425+0000 7f89e45c8640 1 -- start start 2024-03-29T04:11:09.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.426+0000 7f89e45c8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f89dc0b5990 0x7f89dc06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:09.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.426+0000 7f89e45c8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89dc0cd9e0 con 0x7f89dc0cd600 2024-03-29T04:11:09.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.426+0000 7f89e45c8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89dc06cbf0 con 0x7f89dc06c6c0 2024-03-29T04:11:09.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.426+0000 7f89e45c8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f89dc06d800 con 0x7f89dc0b5990 2024-03-29T04:11:09.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.427+0000 7f89e2b65640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f89dc0b5990 0x7f89dc06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:09.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.427+0000 7f89e2364640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f89dc0cd600 0x7f89dc15c140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35742/0 (socket says 172.21.15.148:35742) 2024-03-29T04:11:09.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.427+0000 7f89e2364640 1 -- 172.21.15.148:0/79647240 learned_addr learned my addr 172.21.15.148:0/79647240 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:09.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.427+0000 7f89e2b65640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f89dc0b5990 0x7f89dc06dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43246/0 (socket says 172.21.15.148:43246) 2024-03-29T04:11:09.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.427+0000 7f89e2b65640 1 -- 172.21.15.148:0/79647240 >> v1:172.21.15.157:6789/0 conn(0x7f89dc06c6c0 legacy=0x7f89dc0b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:09.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.427+0000 7f89e2b65640 1 -- 172.21.15.148:0/79647240 >> v1:172.21.15.148:6789/0 conn(0x7f89dc0cd600 legacy=0x7f89dc15c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:09.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.427+0000 7f89e1362640 1 -- 172.21.15.148:0/79647240 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 973336991 0 0) 0x7f89dc0cd9e0 con 0x7f89dc0cd600 2024-03-29T04:11:09.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.427+0000 7f89e2b65640 1 -- 172.21.15.148:0/79647240 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89dc06ced0 con 0x7f89dc0b5990 2024-03-29T04:11:09.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.428+0000 7f89e2b65640 1 --2- 172.21.15.148:0/79647240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89dc0b5990 0x7f89dc06dc90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f89d400bfa0 tx=0x7f89d402e0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55ad3c1c6a0f7d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:09.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.428+0000 7f89e1362640 1 -- 172.21.15.148:0/79647240 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d402ea50 con 0x7f89dc0b5990 2024-03-29T04:11:09.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.428+0000 7f89e1362640 1 -- 172.21.15.148:0/79647240 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89d402ebf0 con 0x7f89dc0b5990 2024-03-29T04:11:09.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.428+0000 7f89e1362640 1 -- 172.21.15.148:0/79647240 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d4009540 con 0x7f89dc0b5990 2024-03-29T04:11:09.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.428+0000 7f89e45c8640 1 -- 172.21.15.148:0/79647240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89dc0b5990 msgr2=0x7f89dc06dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:09.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.428+0000 7f89e45c8640 1 --2- 172.21.15.148:0/79647240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89dc0b5990 0x7f89dc06dc90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f89d400bfa0 tx=0x7f89d402e0b0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.428+0000 7f89e45c8640 1 -- 172.21.15.148:0/79647240 shutdown_connections 2024-03-29T04:11:09.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.428+0000 7f89e45c8640 1 --2- 172.21.15.148:0/79647240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89dc0b5990 0x7f89dc06dc90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.428+0000 7f89e45c8640 1 -- 172.21.15.148:0/79647240 >> 172.21.15.148:0/79647240 conn(0x7f89dc0b2c60 msgr2=0x7f89dc0b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:09.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.428+0000 7f89e45c8640 1 -- 172.21.15.148:0/79647240 shutdown_connections 2024-03-29T04:11:09.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.428+0000 7f89e45c8640 1 -- 172.21.15.148:0/79647240 wait complete. 2024-03-29T04:11:09.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.430+0000 7f89e45c8640 1 Processor -- start 2024-03-29T04:11:09.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.430+0000 7f89e45c8640 1 -- start start 2024-03-29T04:11:09.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.430+0000 7f89e45c8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89dc06c6c0 0x7f89dc140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:09.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.430+0000 7f89e45c8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89dc0b5990 0x7f89dc1f0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:09.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.430+0000 7f89e45c8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89dc0cd600 0x7f89dc072370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:09.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.430+0000 7f89e45c8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f89dc15c880 con 0x7f89dc06c6c0 2024-03-29T04:11:09.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.430+0000 7f89e45c8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f89dc06d490 con 0x7f89dc0b5990 2024-03-29T04:11:09.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.430+0000 7f89e45c8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f89dc15ca00 con 0x7f89dc0cd600 2024-03-29T04:11:09.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.430+0000 7f89e2364640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89dc06c6c0 0x7f89dc140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:09.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.430+0000 7f89e2364640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89dc06c6c0 0x7f89dc140ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43250/0 (socket says 172.21.15.148:43250) 2024-03-29T04:11:09.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.430+0000 7f89e2364640 1 -- 172.21.15.148:0/1491710117 learned_addr learned my addr 172.21.15.148:0/1491710117 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:09.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.430+0000 7f89e1b63640 1 --2- 172.21.15.148:0/1491710117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89dc0b5990 0x7f89dc1f0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:09.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.431+0000 7f89e2b65640 1 --2- 172.21.15.148:0/1491710117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89dc0cd600 0x7f89dc072370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:09.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.431+0000 7f89e2364640 1 -- 172.21.15.148:0/1491710117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89dc0cd600 msgr2=0x7f89dc072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:09.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.431+0000 7f89e2364640 1 --2- 172.21.15.148:0/1491710117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89dc0cd600 0x7f89dc072370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.431+0000 7f89e2364640 1 -- 172.21.15.148:0/1491710117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89dc0b5990 msgr2=0x7f89dc1f0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:09.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.431+0000 7f89e2364640 1 --2- 172.21.15.148:0/1491710117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89dc0b5990 0x7f89dc1f0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.431+0000 7f89e2364640 1 -- 172.21.15.148:0/1491710117 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89dc141010 con 0x7f89dc06c6c0 2024-03-29T04:11:09.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.431+0000 7f89e1b63640 1 --2- 172.21.15.148:0/1491710117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89dc0b5990 0x7f89dc1f0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:09.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.431+0000 7f89e2364640 1 --2- 172.21.15.148:0/1491710117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89dc06c6c0 0x7f89dc140ad0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f89d800c430 tx=0x7f89d800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:09.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.431+0000 7f89d2ffd640 1 -- 172.21.15.148:0/1491710117 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d80074e0 con 0x7f89dc06c6c0 2024-03-29T04:11:09.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.431+0000 7f89d2ffd640 1 -- 172.21.15.148:0/1491710117 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89d80047e0 con 0x7f89dc06c6c0 2024-03-29T04:11:09.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.431+0000 7f89d2ffd640 1 -- 172.21.15.148:0/1491710117 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d8004bd0 con 0x7f89dc06c6c0 2024-03-29T04:11:09.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.431+0000 7f89e45c8640 1 -- 172.21.15.148:0/1491710117 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89dc153a10 con 0x7f89dc06c6c0 2024-03-29T04:11:09.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.431+0000 7f89e45c8640 1 -- 172.21.15.148:0/1491710117 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89dc153fd0 con 0x7f89dc06c6c0 2024-03-29T04:11:09.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.432+0000 7f89e45c8640 1 -- 172.21.15.148:0/1491710117 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89dc0687e0 con 0x7f89dc06c6c0 2024-03-29T04:11:09.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.433+0000 7f89d2ffd640 1 -- 172.21.15.148:0/1491710117 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f89d8011070 con 0x7f89dc06c6c0 2024-03-29T04:11:09.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.433+0000 7f89d2ffd640 1 --2- 172.21.15.148:0/1491710117 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f89c006c530 0x7f89c006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:09.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.434+0000 7f89d2ffd640 1 -- 172.21.15.148:0/1491710117 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f89d808b750 con 0x7f89dc06c6c0 2024-03-29T04:11:09.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.434+0000 7f89e1b63640 1 --2- 172.21.15.148:0/1491710117 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f89c006c530 0x7f89c006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:09.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.434+0000 7f89e1b63640 1 --2- 172.21.15.148:0/1491710117 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f89c006c530 0x7f89c006e9f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f89cc007fd0 tx=0x7f89cc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:09.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.442+0000 7f89d2ffd640 1 -- 172.21.15.148:0/1491710117 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f89d8059970 con 0x7f89dc06c6c0 2024-03-29T04:11:09.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.625+0000 7f89e45c8640 1 -- 172.21.15.148:0/1491710117 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f89dc0ce9f0 con 0x7f89dc06c6c0 2024-03-29T04:11:09.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.626+0000 7f89d2ffd640 1 -- 172.21.15.148:0/1491710117 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18858 (secure 0 0 0) 0x7f89d8059270 con 0x7f89dc06c6c0 2024-03-29T04:11:09.605 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:11:09.606 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:41.662934+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:11:09.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.628+0000 7f89e45c8640 1 -- 172.21.15.148:0/1491710117 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f89c006c530 msgr2=0x7f89c006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:09.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.628+0000 7f89e45c8640 1 --2- 172.21.15.148:0/1491710117 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f89c006c530 0x7f89c006e9f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f89cc007fd0 tx=0x7f89cc009040 comp rx=0 tx=0).stop 2024-03-29T04:11:09.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.628+0000 7f89e45c8640 1 -- 172.21.15.148:0/1491710117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89dc06c6c0 msgr2=0x7f89dc140ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:09.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.628+0000 7f89e45c8640 1 --2- 172.21.15.148:0/1491710117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89dc06c6c0 0x7f89dc140ad0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f89d800c430 tx=0x7f89d800c8f0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.628+0000 7f89e45c8640 1 -- 172.21.15.148:0/1491710117 shutdown_connections 2024-03-29T04:11:09.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.628+0000 7f89e45c8640 1 --2- 172.21.15.148:0/1491710117 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f89c006c530 0x7f89c006e9f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.628+0000 7f89e45c8640 1 --2- 172.21.15.148:0/1491710117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89dc0cd600 0x7f89dc072370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.628+0000 7f89e45c8640 1 --2- 172.21.15.148:0/1491710117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89dc0b5990 0x7f89dc1f0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.628+0000 7f89e45c8640 1 --2- 172.21.15.148:0/1491710117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89dc06c6c0 0x7f89dc140ad0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.628+0000 7f89e45c8640 1 -- 172.21.15.148:0/1491710117 >> 172.21.15.148:0/1491710117 conn(0x7f89dc0b2c60 msgr2=0x7f89dc1e3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:09.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.629+0000 7f89e45c8640 1 -- 172.21.15.148:0/1491710117 shutdown_connections 2024-03-29T04:11:09.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.629+0000 7f89e45c8640 1 -- 172.21.15.148:0/1491710117 wait complete. 2024-03-29T04:11:09.622 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:47.898351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:57.922313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:58.829405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:10:02.551763+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:11:09.622 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:11:09.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.747+0000 7f72d2270640 1 Processor -- start 2024-03-29T04:11:09.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.747+0000 7f72d2270640 1 -- start start 2024-03-29T04:11:09.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.748+0000 7f72d2270640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f72cc15f200 0x7f72cc15f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:09.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.748+0000 7f72d2270640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72cc0b4fb0 0x7f72cc0cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:09.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.748+0000 7f72d2270640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f72cc0b5430 0x7f72cc0cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:09.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.748+0000 7f72d2270640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f72cc0d09e0 con 0x7f72cc0b5430 2024-03-29T04:11:09.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.748+0000 7f72d2270640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f72cc0cfdd0 con 0x7f72cc15f200 2024-03-29T04:11:09.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.748+0000 7f72d2270640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f72cc0d00b0 con 0x7f72cc0b4fb0 2024-03-29T04:11:09.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.748+0000 7f72d080d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f72cc0b5430 0x7f72cc0cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:09.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.748+0000 7f72d080d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f72cc0b5430 0x7f72cc0cccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43260/0 (socket says 172.21.15.148:43260) 2024-03-29T04:11:09.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.748+0000 7f72d080d640 1 -- 172.21.15.148:0/3439280580 learned_addr learned my addr 172.21.15.148:0/3439280580 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:09.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.748+0000 7f72cb7fe640 1 --2- 172.21.15.148:0/3439280580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72cc0b4fb0 0x7f72cc0cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:09.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.748+0000 7f72cbfff640 1 --2- 172.21.15.148:0/3439280580 >> v2:172.21.15.157:3300/0 conn(0x7f72cc15f200 0x7f72cc15f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:09.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.748+0000 7f72d080d640 1 -- 172.21.15.148:0/3439280580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72cc0b4fb0 msgr2=0x7f72cc0cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:09.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.748+0000 7f72d080d640 1 --2- 172.21.15.148:0/3439280580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72cc0b4fb0 0x7f72cc0cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.748+0000 7f72d080d640 1 -- 172.21.15.148:0/3439280580 >> v2:172.21.15.157:3300/0 conn(0x7f72cc15f200 msgr2=0x7f72cc15f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:09.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.748+0000 7f72d080d640 1 --2- 172.21.15.148:0/3439280580 >> v2:172.21.15.157:3300/0 conn(0x7f72cc15f200 0x7f72cc15f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.748+0000 7f72d080d640 1 -- 172.21.15.148:0/3439280580 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72cc0d0390 con 0x7f72cc0b5430 2024-03-29T04:11:09.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.749+0000 7f72d080d640 1 --2- 172.21.15.148:0/3439280580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72cc0b5430 0x7f72cc0cccf0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f72cc0d3230 tx=0x7f72c402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63ea3d0bbf31c23 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:09.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.749+0000 7f72caffd640 1 -- 172.21.15.148:0/3439280580 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c402ca70 con 0x7f72cc0b5430 2024-03-29T04:11:09.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.749+0000 7f72caffd640 1 -- 172.21.15.148:0/3439280580 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72c402cc10 con 0x7f72cc0b5430 2024-03-29T04:11:09.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.749+0000 7f72caffd640 1 -- 172.21.15.148:0/3439280580 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c40343f0 con 0x7f72cc0b5430 2024-03-29T04:11:09.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.749+0000 7f72d2270640 1 -- 172.21.15.148:0/3439280580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72cc0b5430 msgr2=0x7f72cc0cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:09.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.749+0000 7f72d2270640 1 --2- 172.21.15.148:0/3439280580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72cc0b5430 0x7f72cc0cccf0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f72cc0d3230 tx=0x7f72c402bed0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.750+0000 7f72d2270640 1 -- 172.21.15.148:0/3439280580 shutdown_connections 2024-03-29T04:11:09.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.750+0000 7f72d2270640 1 --2- 172.21.15.148:0/3439280580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72cc0b5430 0x7f72cc0cccf0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.750+0000 7f72d2270640 1 --2- 172.21.15.148:0/3439280580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72cc0b4fb0 0x7f72cc0cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.750+0000 7f72d2270640 1 --2- 172.21.15.148:0/3439280580 >> v2:172.21.15.157:3300/0 conn(0x7f72cc15f200 0x7f72cc15f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.750+0000 7f72d2270640 1 -- 172.21.15.148:0/3439280580 >> 172.21.15.148:0/3439280580 conn(0x7f72cc0b3c70 msgr2=0x7f72cc0b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:09.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.750+0000 7f72d2270640 1 -- 172.21.15.148:0/3439280580 shutdown_connections 2024-03-29T04:11:09.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.750+0000 7f72d2270640 1 -- 172.21.15.148:0/3439280580 wait complete. 2024-03-29T04:11:09.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.751+0000 7f72d2270640 1 Processor -- start 2024-03-29T04:11:09.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.751+0000 7f72d2270640 1 -- start start 2024-03-29T04:11:09.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.752+0000 7f72d2270640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72cc0b4fb0 0x7f72cc06eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:09.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.752+0000 7f72d2270640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72cc0b5430 0x7f72cc06c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:09.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.752+0000 7f72d2270640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72cc15f200 0x7f72cc0cba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:09.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.752+0000 7f72d2270640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc1621a0 con 0x7f72cc15f200 2024-03-29T04:11:09.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.752+0000 7f72d2270640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc162020 con 0x7f72cc0b4fb0 2024-03-29T04:11:09.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.752+0000 7f72d2270640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f72cc162320 con 0x7f72cc0b5430 2024-03-29T04:11:09.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.752+0000 7f72d080d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72cc15f200 0x7f72cc0cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:09.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.752+0000 7f72d080d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72cc15f200 0x7f72cc0cba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43264/0 (socket says 172.21.15.148:43264) 2024-03-29T04:11:09.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.752+0000 7f72d080d640 1 -- 172.21.15.148:0/1605457622 learned_addr learned my addr 172.21.15.148:0/1605457622 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:09.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.752+0000 7f72cbfff640 1 --2- 172.21.15.148:0/1605457622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72cc0b4fb0 0x7f72cc06eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:09.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.752+0000 7f72cb7fe640 1 --2- 172.21.15.148:0/1605457622 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72cc0b5430 0x7f72cc06c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:09.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.752+0000 7f72d080d640 1 -- 172.21.15.148:0/1605457622 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72cc0b5430 msgr2=0x7f72cc06c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:09.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.752+0000 7f72d080d640 1 --2- 172.21.15.148:0/1605457622 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72cc0b5430 0x7f72cc06c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.752+0000 7f72d080d640 1 -- 172.21.15.148:0/1605457622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72cc0b4fb0 msgr2=0x7f72cc06eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:09.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.752+0000 7f72d080d640 1 --2- 172.21.15.148:0/1605457622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72cc0b4fb0 0x7f72cc06eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.752+0000 7f72d080d640 1 -- 172.21.15.148:0/1605457622 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72cc070220 con 0x7f72cc15f200 2024-03-29T04:11:09.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.752+0000 7f72cbfff640 1 --2- 172.21.15.148:0/1605457622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72cc0b4fb0 0x7f72cc06eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:09.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.752+0000 7f72d080d640 1 --2- 172.21.15.148:0/1605457622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72cc15f200 0x7f72cc0cba00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f72c4036040 tx=0x7f72c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:09.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.752+0000 7f72c8ff9640 1 -- 172.21.15.148:0/1605457622 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c4043020 con 0x7f72cc15f200 2024-03-29T04:11:09.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.752+0000 7f72c8ff9640 1 -- 172.21.15.148:0/1605457622 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72c403e450 con 0x7f72cc15f200 2024-03-29T04:11:09.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.752+0000 7f72d2270640 1 -- 172.21.15.148:0/1605457622 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72cc06f180 con 0x7f72cc15f200 2024-03-29T04:11:09.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.752+0000 7f72c8ff9640 1 -- 172.21.15.148:0/1605457622 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c4009510 con 0x7f72cc15f200 2024-03-29T04:11:09.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.752+0000 7f72d2270640 1 -- 172.21.15.148:0/1605457622 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72cc06f740 con 0x7f72cc15f200 2024-03-29T04:11:09.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.753+0000 7f72d2270640 1 -- 172.21.15.148:0/1605457622 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72cc068850 con 0x7f72cc15f200 2024-03-29T04:11:09.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.754+0000 7f72c8ff9640 1 -- 172.21.15.148:0/1605457622 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f72c4046020 con 0x7f72cc15f200 2024-03-29T04:11:09.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.754+0000 7f72c8ff9640 1 --2- 172.21.15.148:0/1605457622 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f72b806c530 0x7f72b806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:09.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.754+0000 7f72cbfff640 1 --2- 172.21.15.148:0/1605457622 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f72b806c530 0x7f72b806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:09.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.754+0000 7f72c8ff9640 1 -- 172.21.15.148:0/1605457622 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f72c40ae1c0 con 0x7f72cc15f200 2024-03-29T04:11:09.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.755+0000 7f72cbfff640 1 --2- 172.21.15.148:0/1605457622 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f72b806c530 0x7f72b806e9f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009af0 tx=0x7f72c0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:09.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.759+0000 7f72c8ff9640 1 -- 172.21.15.148:0/1605457622 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f72c404d030 con 0x7f72cc15f200 2024-03-29T04:11:09.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.936+0000 7f72d2270640 1 -- 172.21.15.148:0/1605457622 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f72cc0d1630 con 0x7f72cc15f200 2024-03-29T04:11:09.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.937+0000 7f72c8ff9640 1 -- 172.21.15.148:0/1605457622 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f72c407c3e0 con 0x7f72cc15f200 2024-03-29T04:11:09.916 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-29T04:11:09.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.939+0000 7f72d2270640 1 -- 172.21.15.148:0/1605457622 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f72b806c530 msgr2=0x7f72b806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:09.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.939+0000 7f72d2270640 1 --2- 172.21.15.148:0/1605457622 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f72b806c530 0x7f72b806e9f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009af0 tx=0x7f72c0008040 comp rx=0 tx=0).stop 2024-03-29T04:11:09.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.939+0000 7f72d2270640 1 -- 172.21.15.148:0/1605457622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72cc15f200 msgr2=0x7f72cc0cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:09.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.939+0000 7f72d2270640 1 --2- 172.21.15.148:0/1605457622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72cc15f200 0x7f72cc0cba00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f72c4036040 tx=0x7f72c4004a40 comp rx=0 tx=0).stop 2024-03-29T04:11:09.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.939+0000 7f72d2270640 1 -- 172.21.15.148:0/1605457622 shutdown_connections 2024-03-29T04:11:09.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.939+0000 7f72d2270640 1 --2- 172.21.15.148:0/1605457622 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f72b806c530 0x7f72b806e9f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.939+0000 7f72d2270640 1 --2- 172.21.15.148:0/1605457622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72cc15f200 0x7f72cc0cba00 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.939+0000 7f72d2270640 1 --2- 172.21.15.148:0/1605457622 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72cc0b5430 0x7f72cc06c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.939+0000 7f72d2270640 1 --2- 172.21.15.148:0/1605457622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72cc0b4fb0 0x7f72cc06eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:09.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.939+0000 7f72d2270640 1 -- 172.21.15.148:0/1605457622 >> 172.21.15.148:0/1605457622 conn(0x7f72cc0b3c70 msgr2=0x7f72cc160800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:09.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.939+0000 7f72d2270640 1 -- 172.21.15.148:0/1605457622 shutdown_connections 2024-03-29T04:11:09.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:09.939+0000 7f72d2270640 1 -- 172.21.15.148:0/1605457622 wait complete. 2024-03-29T04:11:09.929 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:11:10.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.056+0000 7fa93f7cd640 1 Processor -- start 2024-03-29T04:11:10.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.057+0000 7fa93f7cd640 1 -- start start 2024-03-29T04:11:10.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.057+0000 7fa93f7cd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa9380b3430 0x7fa938162680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:10.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.057+0000 7fa93f7cd640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9380cf330 con 0x7fa9380b2f00 2024-03-29T04:11:10.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.057+0000 7fa93f7cd640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa938162cf0 con 0x7fa93815e3c0 2024-03-29T04:11:10.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.057+0000 7fa93f7cd640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa9380cf960 con 0x7fa9380b3430 2024-03-29T04:11:10.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.058+0000 7fa93cd68640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa9380b2f00 0x7fa93806b7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35748/0 (socket says 172.21.15.148:35748) 2024-03-29T04:11:10.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.058+0000 7fa93cd68640 1 -- 172.21.15.148:0/1552418782 learned_addr learned my addr 172.21.15.148:0/1552418782 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:10.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.058+0000 7fa93dd6a640 1 --2- 172.21.15.148:0/1552418782 >> v2:172.21.15.157:3300/0 conn(0x7fa9380b3430 0x7fa938162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:10.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.058+0000 7fa937fff640 1 -- 172.21.15.148:0/1552418782 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 295776817 0 0) 0x7fa9380cf330 con 0x7fa9380b2f00 2024-03-29T04:11:10.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.058+0000 7fa937fff640 1 -- 172.21.15.148:0/1552418782 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa920003520 con 0x7fa9380b2f00 2024-03-29T04:11:10.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.058+0000 7fa937fff640 1 -- 172.21.15.148:0/1552418782 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2787552062 0 0) 0x7fa920003520 con 0x7fa9380b2f00 2024-03-29T04:11:10.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.058+0000 7fa937fff640 1 -- 172.21.15.148:0/1552418782 >> v2:172.21.15.157:3300/0 conn(0x7fa9380b3430 msgr2=0x7fa938162680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:10.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.058+0000 7fa937fff640 1 --2- 172.21.15.148:0/1552418782 >> v2:172.21.15.157:3300/0 conn(0x7fa9380b3430 0x7fa938162680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:10.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.058+0000 7fa937fff640 1 -- 172.21.15.148:0/1552418782 >> v1:172.21.15.157:6789/0 conn(0x7fa93815e3c0 legacy=0x7fa938161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:10.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.058+0000 7fa937fff640 1 -- 172.21.15.148:0/1552418782 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9380cf5f0 con 0x7fa9380b2f00 2024-03-29T04:11:10.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.058+0000 7fa937fff640 1 -- 172.21.15.148:0/1552418782 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7fa92c001670 con 0x7fa9380b2f00 2024-03-29T04:11:10.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.058+0000 7fa937fff640 1 -- 172.21.15.148:0/1552418782 >> v1:172.21.15.148:6789/0 conn(0x7fa9380b2f00 legacy=0x7fa93806b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:10.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.058+0000 7fa937fff640 1 --2- 172.21.15.148:0/1552418782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9200039b0 0x7fa920003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:10.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.058+0000 7fa937fff640 1 --2- 172.21.15.148:0/1552418782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa920004800 0x7fa920004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:10.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.058+0000 7fa937fff640 1 --2- 172.21.15.148:0/1552418782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9200051c0 0x7fa92000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:10.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.058+0000 7fa937fff640 1 -- 172.21.15.148:0/1552418782 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa9380cf960 con 0x7fa9200039b0 2024-03-29T04:11:10.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.058+0000 7fa937fff640 1 -- 172.21.15.148:0/1552418782 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa92000c160 con 0x7fa9200051c0 2024-03-29T04:11:10.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.058+0000 7fa937fff640 1 -- 172.21.15.148:0/1552418782 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa92000c360 con 0x7fa920004800 2024-03-29T04:11:10.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.058+0000 7fa93d569640 1 --2- 172.21.15.148:0/1552418782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9200039b0 0x7fa920003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:10.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.058+0000 7fa93dd6a640 1 --2- 172.21.15.148:0/1552418782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9200051c0 0x7fa92000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:10.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.058+0000 7fa93cd68640 1 --2- 172.21.15.148:0/1552418782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa920004800 0x7fa920004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:10.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.059+0000 7fa93d569640 1 -- 172.21.15.148:0/1552418782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa920004800 msgr2=0x7fa920004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:10.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.059+0000 7fa93d569640 1 --2- 172.21.15.148:0/1552418782 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa920004800 0x7fa920004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:10.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.059+0000 7fa93d569640 1 -- 172.21.15.148:0/1552418782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9200051c0 msgr2=0x7fa92000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:10.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.059+0000 7fa93d569640 1 --2- 172.21.15.148:0/1552418782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9200051c0 0x7fa92000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:10.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.059+0000 7fa93dd6a640 1 -- 172.21.15.148:0/1552418782 reap_dead start 2024-03-29T04:11:10.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.059+0000 7fa93d569640 1 -- 172.21.15.148:0/1552418782 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa920003520 con 0x7fa9200039b0 2024-03-29T04:11:10.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.060+0000 7fa93d569640 1 --2- 172.21.15.148:0/1552418782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9200039b0 0x7fa920003dd0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa928001750 tx=0x7fa920005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=760f620bffda74 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:10.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.060+0000 7fa937fff640 1 -- 172.21.15.148:0/1552418782 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa928001370 con 0x7fa9200039b0 2024-03-29T04:11:10.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.060+0000 7fa937fff640 1 -- 172.21.15.148:0/1552418782 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9280052d0 con 0x7fa9200039b0 2024-03-29T04:11:10.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.060+0000 7fa937fff640 1 -- 172.21.15.148:0/1552418782 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa928003830 con 0x7fa9200039b0 2024-03-29T04:11:10.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.061+0000 7fa93f7cd640 1 -- 172.21.15.148:0/1552418782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9200039b0 msgr2=0x7fa920003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:10.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.061+0000 7fa93f7cd640 1 --2- 172.21.15.148:0/1552418782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9200039b0 0x7fa920003dd0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa928001750 tx=0x7fa920005830 comp rx=0 tx=0).stop 2024-03-29T04:11:10.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.061+0000 7fa93f7cd640 1 -- 172.21.15.148:0/1552418782 shutdown_connections 2024-03-29T04:11:10.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.061+0000 7fa93f7cd640 1 --2- 172.21.15.148:0/1552418782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9200039b0 0x7fa920003dd0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:10.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.061+0000 7fa93f7cd640 1 -- 172.21.15.148:0/1552418782 >> 172.21.15.148:0/1552418782 conn(0x7fa9380b6430 msgr2=0x7fa9380d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:10.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.061+0000 7fa93f7cd640 1 -- 172.21.15.148:0/1552418782 shutdown_connections 2024-03-29T04:11:10.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.061+0000 7fa93f7cd640 1 -- 172.21.15.148:0/1552418782 wait complete. 2024-03-29T04:11:10.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.062+0000 7fa93f7cd640 1 Processor -- start 2024-03-29T04:11:10.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.062+0000 7fa93f7cd640 1 -- start start 2024-03-29T04:11:10.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.063+0000 7fa93f7cd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9200039b0 0x7fa938157450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:10.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.063+0000 7fa93f7cd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa93815fbf0 0x7fa938160050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:10.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.063+0000 7fa93f7cd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa93815e9b0 0x7fa93815ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:10.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.063+0000 7fa93f7cd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa9380b3b20 con 0x7fa93815e9b0 2024-03-29T04:11:10.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.063+0000 7fa93f7cd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa9380b39a0 con 0x7fa9200039b0 2024-03-29T04:11:10.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.063+0000 7fa93f7cd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa938161f70 con 0x7fa93815fbf0 2024-03-29T04:11:10.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.063+0000 7fa93dd6a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa93815e9b0 0x7fa93815ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:10.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.063+0000 7fa93dd6a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa93815e9b0 0x7fa93815ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43294/0 (socket says 172.21.15.148:43294) 2024-03-29T04:11:10.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.063+0000 7fa93dd6a640 1 -- 172.21.15.148:0/1429137053 learned_addr learned my addr 172.21.15.148:0/1429137053 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:10.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.063+0000 7fa93cd68640 1 --2- 172.21.15.148:0/1429137053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa93815fbf0 0x7fa938160050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:10.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.063+0000 7fa93d569640 1 --2- 172.21.15.148:0/1429137053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9200039b0 0x7fa938157450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:10.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.063+0000 7fa93dd6a640 1 -- 172.21.15.148:0/1429137053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa93815fbf0 msgr2=0x7fa938160050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:10.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.063+0000 7fa93dd6a640 1 --2- 172.21.15.148:0/1429137053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa93815fbf0 0x7fa938160050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:10.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.063+0000 7fa93dd6a640 1 -- 172.21.15.148:0/1429137053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9200039b0 msgr2=0x7fa938157450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:10.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.063+0000 7fa93dd6a640 1 --2- 172.21.15.148:0/1429137053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9200039b0 0x7fa938157450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:10.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.063+0000 7fa93dd6a640 1 -- 172.21.15.148:0/1429137053 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa93815f470 con 0x7fa93815e9b0 2024-03-29T04:11:10.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.063+0000 7fa93dd6a640 1 --2- 172.21.15.148:0/1429137053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa93815e9b0 0x7fa93815ee30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa9380cb300 tx=0x7fa93000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:10.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.063+0000 7fa935ffb640 1 -- 172.21.15.148:0/1429137053 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93000fd80 con 0x7fa93815e9b0 2024-03-29T04:11:10.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.063+0000 7fa935ffb640 1 -- 172.21.15.148:0/1429137053 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa93001d070 con 0x7fa93815e9b0 2024-03-29T04:11:10.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.063+0000 7fa93f7cd640 1 -- 172.21.15.148:0/1429137053 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa93815f750 con 0x7fa93815e9b0 2024-03-29T04:11:10.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.063+0000 7fa935ffb640 1 -- 172.21.15.148:0/1429137053 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa930015670 con 0x7fa93815e9b0 2024-03-29T04:11:10.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.064+0000 7fa93f7cd640 1 -- 172.21.15.148:0/1429137053 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9381e82f0 con 0x7fa93815e9b0 2024-03-29T04:11:10.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.065+0000 7fa93f7cd640 1 -- 172.21.15.148:0/1429137053 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9380d1a00 con 0x7fa93815e9b0 2024-03-29T04:11:10.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.065+0000 7fa935ffb640 1 -- 172.21.15.148:0/1429137053 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa930015810 con 0x7fa93815e9b0 2024-03-29T04:11:10.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.065+0000 7fa935ffb640 1 --2- 172.21.15.148:0/1429137053 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa92406c560 0x7fa92406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:10.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.066+0000 7fa935ffb640 1 -- 172.21.15.148:0/1429137053 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa93008c5c0 con 0x7fa93815e9b0 2024-03-29T04:11:10.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.066+0000 7fa93d569640 1 --2- 172.21.15.148:0/1429137053 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa92406c560 0x7fa92406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:10.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.066+0000 7fa93d569640 1 --2- 172.21.15.148:0/1429137053 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa92406c560 0x7fa92406ea20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa928033000 tx=0x7fa928009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:10.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.070+0000 7fa935ffb640 1 -- 172.21.15.148:0/1429137053 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa9380d1a00 con 0x7fa93815e9b0 2024-03-29T04:11:10.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.287+0000 7fa93f7cd640 1 -- 172.21.15.148:0/1429137053 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa938068850 con 0x7fa93815e9b0 2024-03-29T04:11:10.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.287+0000 7fa935ffb640 1 -- 172.21.15.148:0/1429137053 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fa930027020 con 0x7fa93815e9b0 2024-03-29T04:11:10.267 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:11:10.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.290+0000 7fa93f7cd640 1 -- 172.21.15.148:0/1429137053 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa92406c560 msgr2=0x7fa92406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:10.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.290+0000 7fa93f7cd640 1 --2- 172.21.15.148:0/1429137053 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa92406c560 0x7fa92406ea20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa928033000 tx=0x7fa928009030 comp rx=0 tx=0).stop 2024-03-29T04:11:10.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.290+0000 7fa93f7cd640 1 -- 172.21.15.148:0/1429137053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa93815e9b0 msgr2=0x7fa93815ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:10.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.290+0000 7fa93f7cd640 1 --2- 172.21.15.148:0/1429137053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa93815e9b0 0x7fa93815ee30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa9380cb300 tx=0x7fa93000f580 comp rx=0 tx=0).stop 2024-03-29T04:11:10.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.290+0000 7fa93f7cd640 1 -- 172.21.15.148:0/1429137053 shutdown_connections 2024-03-29T04:11:10.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.290+0000 7fa93f7cd640 1 --2- 172.21.15.148:0/1429137053 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa92406c560 0x7fa92406ea20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:10.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.290+0000 7fa93f7cd640 1 --2- 172.21.15.148:0/1429137053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa93815e9b0 0x7fa93815ee30 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:10.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.290+0000 7fa93f7cd640 1 --2- 172.21.15.148:0/1429137053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa93815fbf0 0x7fa938160050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:10.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.290+0000 7fa93f7cd640 1 --2- 172.21.15.148:0/1429137053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9200039b0 0x7fa938157450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:10.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.290+0000 7fa93f7cd640 1 -- 172.21.15.148:0/1429137053 >> 172.21.15.148:0/1429137053 conn(0x7fa9380b6430 msgr2=0x7fa9380d2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:10.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.290+0000 7fa93f7cd640 1 -- 172.21.15.148:0/1429137053 shutdown_connections 2024-03-29T04:11:10.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.290+0000 7fa93f7cd640 1 -- 172.21.15.148:0/1429137053 wait complete. 2024-03-29T04:11:10.283 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:11:10.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.410+0000 7f90c5d72640 1 Processor -- start 2024-03-29T04:11:10.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.410+0000 7f90c5d72640 1 -- start start 2024-03-29T04:11:10.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.410+0000 7f90c5d72640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f90c00b2f00 0x7f90c0162680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:10.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.410+0000 7f90c5d72640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f90c00b3430 0x7f90c006b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:10.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.410+0000 7f90c5d72640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90c00cf330 con 0x7f90c015e3c0 2024-03-29T04:11:10.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.410+0000 7f90c5d72640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f90c00cf960 con 0x7f90c00b2f00 2024-03-29T04:11:10.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.410+0000 7f90c5d72640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f90c0162cf0 con 0x7f90c00b3430 2024-03-29T04:11:10.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.411+0000 7f90beffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f90c00b2f00 0x7f90c0162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:10.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.411+0000 7f90beffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f90c00b2f00 0x7f90c0162680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43304/0 (socket says 172.21.15.148:43304) 2024-03-29T04:11:10.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.411+0000 7f90bf7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f90c015e3c0 0x7f90c0161f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35760/0 (socket says 172.21.15.148:35760) 2024-03-29T04:11:10.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.411+0000 7f90beffd640 1 -- 172.21.15.148:0/701279295 learned_addr learned my addr 172.21.15.148:0/701279295 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:10.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.411+0000 7f90bffff640 1 --2- 172.21.15.148:0/701279295 >> v2:172.21.15.157:3300/0 conn(0x7f90c00b3430 0x7f90c006b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:10.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.411+0000 7f90beffd640 1 -- 172.21.15.148:0/701279295 >> v2:172.21.15.157:3300/0 conn(0x7f90c00b3430 msgr2=0x7f90c006b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:10.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.411+0000 7f90beffd640 1 --2- 172.21.15.148:0/701279295 >> v2:172.21.15.157:3300/0 conn(0x7f90c00b3430 0x7f90c006b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:10.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.411+0000 7f90beffd640 1 -- 172.21.15.148:0/701279295 >> v1:172.21.15.148:6789/0 conn(0x7f90c015e3c0 legacy=0x7f90c0161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:10.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.411+0000 7f90be7fc640 1 -- 172.21.15.148:0/701279295 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3845875195 0 0) 0x7f90c00cf330 con 0x7f90c015e3c0 2024-03-29T04:11:10.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.411+0000 7f90beffd640 1 -- 172.21.15.148:0/701279295 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c00cf5f0 con 0x7f90c00b2f00 2024-03-29T04:11:10.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.412+0000 7f90beffd640 1 --2- 172.21.15.148:0/701279295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90c00b2f00 0x7f90c0162680 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f90c00cf8d0 tx=0x7f90a802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e87d9f817d9ad7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:10.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.412+0000 7f90be7fc640 1 -- 172.21.15.148:0/701279295 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8035020 con 0x7f90c00b2f00 2024-03-29T04:11:10.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.412+0000 7f90be7fc640 1 -- 172.21.15.148:0/701279295 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a8031870 con 0x7f90c00b2f00 2024-03-29T04:11:10.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.412+0000 7f90be7fc640 1 -- 172.21.15.148:0/701279295 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8031c40 con 0x7f90c00b2f00 2024-03-29T04:11:10.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.412+0000 7f90c5d72640 1 -- 172.21.15.148:0/701279295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90c00b2f00 msgr2=0x7f90c0162680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:10.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.413+0000 7f90c5d72640 1 --2- 172.21.15.148:0/701279295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90c00b2f00 0x7f90c0162680 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f90c00cf8d0 tx=0x7f90a802b910 comp rx=0 tx=0).stop 2024-03-29T04:11:10.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.413+0000 7f90c5d72640 1 -- 172.21.15.148:0/701279295 shutdown_connections 2024-03-29T04:11:10.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.413+0000 7f90c5d72640 1 --2- 172.21.15.148:0/701279295 >> v2:172.21.15.157:3300/0 conn(0x7f90c00b3430 0x7f90c006b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:10.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.413+0000 7f90c5d72640 1 --2- 172.21.15.148:0/701279295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90c00b2f00 0x7f90c0162680 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:10.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.413+0000 7f90c5d72640 1 -- 172.21.15.148:0/701279295 >> 172.21.15.148:0/701279295 conn(0x7f90c00b6430 msgr2=0x7f90c00d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:10.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.413+0000 7f90c5d72640 1 -- 172.21.15.148:0/701279295 shutdown_connections 2024-03-29T04:11:10.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.413+0000 7f90c5d72640 1 -- 172.21.15.148:0/701279295 wait complete. 2024-03-29T04:11:10.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.414+0000 7f90c5d72640 1 Processor -- start 2024-03-29T04:11:10.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.414+0000 7f90c5d72640 1 -- start start 2024-03-29T04:11:10.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.415+0000 7f90c5d72640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90c00b3430 0x7f90c00ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:10.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.415+0000 7f90c5d72640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90c015e3c0 0x7f90c00c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:10.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.415+0000 7f90c5d72640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90c0161650 0x7f90c00cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:10.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.415+0000 7f90c5d72640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f90c00b3b20 con 0x7f90c0161650 2024-03-29T04:11:10.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.415+0000 7f90c5d72640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f90c00b39a0 con 0x7f90c015e3c0 2024-03-29T04:11:10.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.415+0000 7f90c5d72640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f90c006bd30 con 0x7f90c00b3430 2024-03-29T04:11:10.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.415+0000 7f90bffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90c0161650 0x7f90c00cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:10.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.415+0000 7f90bffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90c0161650 0x7f90c00cb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43310/0 (socket says 172.21.15.148:43310) 2024-03-29T04:11:10.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.415+0000 7f90bffff640 1 -- 172.21.15.148:0/114250313 learned_addr learned my addr 172.21.15.148:0/114250313 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:10.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.415+0000 7f90bf7fe640 1 --2- 172.21.15.148:0/114250313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90c00b3430 0x7f90c00ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:10.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.415+0000 7f90bffff640 1 -- 172.21.15.148:0/114250313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90c00b3430 msgr2=0x7f90c00ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:10.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.415+0000 7f90beffd640 1 --2- 172.21.15.148:0/114250313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90c015e3c0 0x7f90c00c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:10.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.415+0000 7f90bffff640 1 --2- 172.21.15.148:0/114250313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90c00b3430 0x7f90c00ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:10.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.415+0000 7f90bffff640 1 -- 172.21.15.148:0/114250313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90c015e3c0 msgr2=0x7f90c00c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:10.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.415+0000 7f90bffff640 1 --2- 172.21.15.148:0/114250313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90c015e3c0 0x7f90c00c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:10.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.415+0000 7f90bffff640 1 -- 172.21.15.148:0/114250313 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c00cbd10 con 0x7f90c0161650 2024-03-29T04:11:10.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.415+0000 7f90bf7fe640 1 --2- 172.21.15.148:0/114250313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90c00b3430 0x7f90c00ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:10.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.415+0000 7f90beffd640 1 --2- 172.21.15.148:0/114250313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90c015e3c0 0x7f90c00c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:10.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.415+0000 7f90bffff640 1 --2- 172.21.15.148:0/114250313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90c0161650 0x7f90c00cb3a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f90b0007e40 tx=0x7f90b000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:10.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.415+0000 7f90b7fff640 1 -- 172.21.15.148:0/114250313 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0014620 con 0x7f90c0161650 2024-03-29T04:11:10.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.415+0000 7f90b7fff640 1 -- 172.21.15.148:0/114250313 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90b0019070 con 0x7f90c0161650 2024-03-29T04:11:10.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.416+0000 7f90c5d72640 1 -- 172.21.15.148:0/114250313 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90c00cbff0 con 0x7f90c0161650 2024-03-29T04:11:10.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.416+0000 7f90b7fff640 1 -- 172.21.15.148:0/114250313 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0014a20 con 0x7f90c0161650 2024-03-29T04:11:10.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.416+0000 7f90c5d72640 1 -- 172.21.15.148:0/114250313 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c00cc5b0 con 0x7f90c0161650 2024-03-29T04:11:10.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.417+0000 7f90c5d72640 1 -- 172.21.15.148:0/114250313 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90c0068850 con 0x7f90c0161650 2024-03-29T04:11:10.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.417+0000 7f90b7fff640 1 -- 172.21.15.148:0/114250313 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f90b0014bc0 con 0x7f90c0161650 2024-03-29T04:11:10.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.417+0000 7f90b7fff640 1 --2- 172.21.15.148:0/114250313 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f90a006c530 0x7f90a006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:10.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.417+0000 7f90b7fff640 1 -- 172.21.15.148:0/114250313 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f90b008c350 con 0x7f90c0161650 2024-03-29T04:11:10.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.418+0000 7f90bf7fe640 1 --2- 172.21.15.148:0/114250313 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f90a006c530 0x7f90a006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:10.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.418+0000 7f90bf7fe640 1 --2- 172.21.15.148:0/114250313 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f90a006c530 0x7f90a006e9f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f90b8000f60 tx=0x7f90b8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:10.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.422+0000 7f90b7fff640 1 -- 172.21.15.148:0/114250313 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f90b005a600 con 0x7f90c0161650 2024-03-29T04:11:10.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.701+0000 7f90c5d72640 1 -- 172.21.15.148:0/114250313 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f90c00cb0c0 con 0x7f90c0161650 2024-03-29T04:11:10.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.701+0000 7f90b7fff640 1 -- 172.21.15.148:0/114250313 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f90b0059f00 con 0x7f90c0161650 2024-03-29T04:11:10.681 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-29T04:11:10.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.703+0000 7f90c5d72640 1 -- 172.21.15.148:0/114250313 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f90a006c530 msgr2=0x7f90a006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:10.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.703+0000 7f90c5d72640 1 --2- 172.21.15.148:0/114250313 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f90a006c530 0x7f90a006e9f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f90b8000f60 tx=0x7f90b8009040 comp rx=0 tx=0).stop 2024-03-29T04:11:10.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.703+0000 7f90c5d72640 1 -- 172.21.15.148:0/114250313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90c0161650 msgr2=0x7f90c00cb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:10.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.703+0000 7f90c5d72640 1 --2- 172.21.15.148:0/114250313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90c0161650 0x7f90c00cb3a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f90b0007e40 tx=0x7f90b000e4b0 comp rx=0 tx=0).stop 2024-03-29T04:11:10.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.703+0000 7f90c5d72640 1 -- 172.21.15.148:0/114250313 shutdown_connections 2024-03-29T04:11:10.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.703+0000 7f90c5d72640 1 --2- 172.21.15.148:0/114250313 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f90a006c530 0x7f90a006e9f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:10.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.703+0000 7f90c5d72640 1 --2- 172.21.15.148:0/114250313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90c0161650 0x7f90c00cb3a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:10.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.703+0000 7f90c5d72640 1 --2- 172.21.15.148:0/114250313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90c015e3c0 0x7f90c00c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:10.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.703+0000 7f90c5d72640 1 --2- 172.21.15.148:0/114250313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90c00b3430 0x7f90c00ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:10.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.704+0000 7f90c5d72640 1 -- 172.21.15.148:0/114250313 >> 172.21.15.148:0/114250313 conn(0x7f90c00b6430 msgr2=0x7f90c00b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:10.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.704+0000 7f90c5d72640 1 -- 172.21.15.148:0/114250313 shutdown_connections 2024-03-29T04:11:10.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.704+0000 7f90c5d72640 1 -- 172.21.15.148:0/114250313 wait complete. 2024-03-29T04:11:10.694 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:11:10.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.820+0000 7fd32447b640 1 Processor -- start 2024-03-29T04:11:10.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.820+0000 7fd32447b640 1 -- start start 2024-03-29T04:11:10.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.822+0000 7fd32447b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd31c0cc6a0 0x7fd31c0cff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:10.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.822+0000 7fd32447b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd31c0b5030 con 0x7fd31c15f560 2024-03-29T04:11:10.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.822+0000 7fd32447b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd31c0b5310 con 0x7fd31c0ce110 2024-03-29T04:11:10.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.822+0000 7fd32447b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd31c0ce7d0 con 0x7fd31c0cc6a0 2024-03-29T04:11:10.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.822+0000 7fd322a18640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd31c15f560 0x7fd31c0b7220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35766/0 (socket says 172.21.15.148:35766) 2024-03-29T04:11:10.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.822+0000 7fd322a18640 1 -- 172.21.15.148:0/977942177 learned_addr learned my addr 172.21.15.148:0/977942177 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:10.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.822+0000 7fd322217640 1 --2- 172.21.15.148:0/977942177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd31c0cc6a0 0x7fd31c0cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:10.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.822+0000 7fd321215640 1 -- 172.21.15.148:0/977942177 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2388388233 0 0) 0x7fd31c0b5030 con 0x7fd31c15f560 2024-03-29T04:11:10.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.822+0000 7fd321215640 1 -- 172.21.15.148:0/977942177 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd308003520 con 0x7fd31c15f560 2024-03-29T04:11:10.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.822+0000 7fd321215640 1 -- 172.21.15.148:0/977942177 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1712254594 0 0) 0x7fd31c0b5310 con 0x7fd31c0ce110 2024-03-29T04:11:10.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.822+0000 7fd321215640 1 -- 172.21.15.148:0/977942177 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd31c0b5030 con 0x7fd31c0ce110 2024-03-29T04:11:10.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.823+0000 7fd322217640 1 -- 172.21.15.148:0/977942177 >> v1:172.21.15.157:6789/0 conn(0x7fd31c0ce110 legacy=0x7fd31c0cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:10.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.823+0000 7fd321215640 1 -- 172.21.15.148:0/977942177 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 174324707 0 0) 0x7fd308003520 con 0x7fd31c15f560 2024-03-29T04:11:10.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.823+0000 7fd322217640 1 -- 172.21.15.148:0/977942177 >> v1:172.21.15.148:6789/0 conn(0x7fd31c15f560 legacy=0x7fd31c0b7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:10.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.823+0000 7fd322217640 1 -- 172.21.15.148:0/977942177 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31c0b55f0 con 0x7fd31c0cc6a0 2024-03-29T04:11:10.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.823+0000 7fd322217640 1 --2- 172.21.15.148:0/977942177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd31c0cc6a0 0x7fd31c0cff50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd3180097d0 tx=0x7fd31802b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=154ee62388930845 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:10.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.824+0000 7fd321215640 1 -- 172.21.15.148:0/977942177 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318037070 con 0x7fd31c0cc6a0 2024-03-29T04:11:10.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.824+0000 7fd321215640 1 -- 172.21.15.148:0/977942177 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd318030510 con 0x7fd31c0cc6a0 2024-03-29T04:11:10.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.824+0000 7fd321215640 1 -- 172.21.15.148:0/977942177 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3140019f0 con 0x7fd31c0cc6a0 2024-03-29T04:11:10.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.824+0000 7fd32447b640 1 -- 172.21.15.148:0/977942177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd31c0cc6a0 msgr2=0x7fd31c0cff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:10.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.824+0000 7fd32447b640 1 --2- 172.21.15.148:0/977942177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd31c0cc6a0 0x7fd31c0cff50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd3180097d0 tx=0x7fd31802b770 comp rx=0 tx=0).stop 2024-03-29T04:11:10.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.824+0000 7fd32447b640 1 -- 172.21.15.148:0/977942177 shutdown_connections 2024-03-29T04:11:10.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.824+0000 7fd32447b640 1 --2- 172.21.15.148:0/977942177 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd31c0cc6a0 0x7fd31c0cff50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:10.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.824+0000 7fd32447b640 1 -- 172.21.15.148:0/977942177 >> 172.21.15.148:0/977942177 conn(0x7fd31c0cead0 msgr2=0x7fd31c0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:10.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.824+0000 7fd32447b640 1 -- 172.21.15.148:0/977942177 shutdown_connections 2024-03-29T04:11:10.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.824+0000 7fd32447b640 1 -- 172.21.15.148:0/977942177 wait complete. 2024-03-29T04:11:10.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.825+0000 7fd32447b640 1 Processor -- start 2024-03-29T04:11:10.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.825+0000 7fd32447b640 1 -- start start 2024-03-29T04:11:10.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.826+0000 7fd32447b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd31c0cc6a0 0x7fd31c161fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:10.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.826+0000 7fd32447b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd31c0ce110 0x7fd31c1562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:10.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.826+0000 7fd32447b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd31c15f560 0x7fd31c1567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:10.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.826+0000 7fd32447b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c0ce570 con 0x7fd31c15f560 2024-03-29T04:11:10.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.826+0000 7fd32447b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c0b6d90 con 0x7fd31c0ce110 2024-03-29T04:11:10.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.826+0000 7fd32447b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd31c158ee0 con 0x7fd31c0cc6a0 2024-03-29T04:11:10.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.826+0000 7fd322a18640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd31c15f560 0x7fd31c1567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:10.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.826+0000 7fd322a18640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd31c15f560 0x7fd31c1567f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43320/0 (socket says 172.21.15.148:43320) 2024-03-29T04:11:10.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.826+0000 7fd322a18640 1 -- 172.21.15.148:0/395446211 learned_addr learned my addr 172.21.15.148:0/395446211 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:10.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.826+0000 7fd322217640 1 --2- 172.21.15.148:0/395446211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd31c0cc6a0 0x7fd31c161fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:10.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.826+0000 7fd321a16640 1 --2- 172.21.15.148:0/395446211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd31c0ce110 0x7fd31c1562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:10.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.827+0000 7fd322a18640 1 -- 172.21.15.148:0/395446211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd31c0cc6a0 msgr2=0x7fd31c161fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:10.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.827+0000 7fd322a18640 1 --2- 172.21.15.148:0/395446211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd31c0cc6a0 0x7fd31c161fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:10.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.827+0000 7fd322a18640 1 -- 172.21.15.148:0/395446211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd31c0ce110 msgr2=0x7fd31c1562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:10.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.827+0000 7fd322a18640 1 --2- 172.21.15.148:0/395446211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd31c0ce110 0x7fd31c1562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:10.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.827+0000 7fd322a18640 1 -- 172.21.15.148:0/395446211 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31c1f85f0 con 0x7fd31c15f560 2024-03-29T04:11:10.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.827+0000 7fd322a18640 1 --2- 172.21.15.148:0/395446211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd31c15f560 0x7fd31c1567f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd31400cc30 tx=0x7fd31400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:10.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.827+0000 7fd312ffd640 1 -- 172.21.15.148:0/395446211 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314017070 con 0x7fd31c15f560 2024-03-29T04:11:10.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.827+0000 7fd312ffd640 1 -- 172.21.15.148:0/395446211 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd31400ce40 con 0x7fd31c15f560 2024-03-29T04:11:10.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.827+0000 7fd32447b640 1 -- 172.21.15.148:0/395446211 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd31c1f88d0 con 0x7fd31c15f560 2024-03-29T04:11:10.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.827+0000 7fd312ffd640 1 -- 172.21.15.148:0/395446211 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314012620 con 0x7fd31c15f560 2024-03-29T04:11:10.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.827+0000 7fd32447b640 1 -- 172.21.15.148:0/395446211 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd31c1624e0 con 0x7fd31c15f560 2024-03-29T04:11:10.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.828+0000 7fd32447b640 1 -- 172.21.15.148:0/395446211 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd31c1f8ec0 con 0x7fd31c15f560 2024-03-29T04:11:10.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.828+0000 7fd312ffd640 1 -- 172.21.15.148:0/395446211 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd31401a070 con 0x7fd31c15f560 2024-03-29T04:11:10.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.829+0000 7fd312ffd640 1 --2- 172.21.15.148:0/395446211 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd30006c530 0x7fd30006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:10.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.829+0000 7fd322217640 1 --2- 172.21.15.148:0/395446211 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd30006c530 0x7fd30006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:10.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.829+0000 7fd312ffd640 1 -- 172.21.15.148:0/395446211 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd31408b990 con 0x7fd31c15f560 2024-03-29T04:11:10.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.829+0000 7fd322217640 1 --2- 172.21.15.148:0/395446211 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd30006c530 0x7fd30006e9f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd318034000 tx=0x7fd318008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:10.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:10.833+0000 7fd312ffd640 1 -- 172.21.15.148:0/395446211 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd314059560 con 0x7fd31c15f560 2024-03-29T04:11:10.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.014+0000 7fd32447b640 1 -- 172.21.15.148:0/395446211 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd31c068850 con 0x7fd31c15f560 2024-03-29T04:11:10.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.015+0000 7fd312ffd640 1 -- 172.21.15.148:0/395446211 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fd31405d860 con 0x7fd31c15f560 2024-03-29T04:11:10.994 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:11:10.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.017+0000 7fd32447b640 1 -- 172.21.15.148:0/395446211 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd30006c530 msgr2=0x7fd30006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:10.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.017+0000 7fd32447b640 1 --2- 172.21.15.148:0/395446211 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd30006c530 0x7fd30006e9f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd318034000 tx=0x7fd318008030 comp rx=0 tx=0).stop 2024-03-29T04:11:10.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.017+0000 7fd32447b640 1 -- 172.21.15.148:0/395446211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd31c15f560 msgr2=0x7fd31c1567f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:10.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.017+0000 7fd32447b640 1 --2- 172.21.15.148:0/395446211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd31c15f560 0x7fd31c1567f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd31400cc30 tx=0x7fd31400a430 comp rx=0 tx=0).stop 2024-03-29T04:11:10.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.017+0000 7fd32447b640 1 -- 172.21.15.148:0/395446211 shutdown_connections 2024-03-29T04:11:10.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.017+0000 7fd32447b640 1 --2- 172.21.15.148:0/395446211 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd30006c530 0x7fd30006e9f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:10.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.017+0000 7fd32447b640 1 --2- 172.21.15.148:0/395446211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd31c15f560 0x7fd31c1567f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:10.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.017+0000 7fd32447b640 1 --2- 172.21.15.148:0/395446211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd31c0ce110 0x7fd31c1562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:10.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.017+0000 7fd32447b640 1 --2- 172.21.15.148:0/395446211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd31c0cc6a0 0x7fd31c161fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:10.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.017+0000 7fd32447b640 1 -- 172.21.15.148:0/395446211 >> 172.21.15.148:0/395446211 conn(0x7fd31c0cead0 msgr2=0x7fd31c0d0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:10.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.017+0000 7fd32447b640 1 -- 172.21.15.148:0/395446211 shutdown_connections 2024-03-29T04:11:10.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.017+0000 7fd32447b640 1 -- 172.21.15.148:0/395446211 wait complete. 2024-03-29T04:11:11.007 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:11:11.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.130+0000 7f03fb33d640 1 Processor -- start 2024-03-29T04:11:11.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.131+0000 7f03fb33d640 1 -- start start 2024-03-29T04:11:11.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.132+0000 7f03fb33d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f03f415e3e0 0x7f03f40d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:11.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.132+0000 7f03fb33d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f03f40ba030 0x7f03f40d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:11.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.132+0000 7f03fb33d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03f415e7c0 con 0x7f03f40d10d0 2024-03-29T04:11:11.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.132+0000 7f03fb33d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f03f40d3840 con 0x7f03f415e3e0 2024-03-29T04:11:11.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.132+0000 7f03fb33d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f03f40d2c30 con 0x7f03f40ba030 2024-03-29T04:11:11.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.132+0000 7f03f90d9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f03f415e3e0 0x7f03f40d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:11.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.132+0000 7f03f98da640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f03f40d10d0 0x7f03f40b72d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35782/0 (socket says 172.21.15.148:35782) 2024-03-29T04:11:11.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.132+0000 7f03f98da640 1 -- 172.21.15.148:0/1790792614 learned_addr learned my addr 172.21.15.148:0/1790792614 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:11.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.132+0000 7f03f88d8640 1 --2- 172.21.15.148:0/1790792614 >> v2:172.21.15.157:3300/0 conn(0x7f03f40ba030 0x7f03f40d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:11.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.132+0000 7f03f90d9640 1 -- 172.21.15.148:0/1790792614 >> v2:172.21.15.157:3300/0 conn(0x7f03f40ba030 msgr2=0x7f03f40d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:11.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.132+0000 7f03f90d9640 1 --2- 172.21.15.148:0/1790792614 >> v2:172.21.15.157:3300/0 conn(0x7f03f40ba030 0x7f03f40d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:11.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.132+0000 7f03f90d9640 1 -- 172.21.15.148:0/1790792614 >> v1:172.21.15.148:6789/0 conn(0x7f03f40d10d0 legacy=0x7f03f40b72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:11.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.133+0000 7f03f90d9640 1 -- 172.21.15.148:0/1790792614 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f40d2f10 con 0x7f03f415e3e0 2024-03-29T04:11:11.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.132+0000 7f03ebfff640 1 -- 172.21.15.148:0/1790792614 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1887422788 0 0) 0x7f03f415e7c0 con 0x7f03f40d10d0 2024-03-29T04:11:11.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.133+0000 7f03f90d9640 1 --2- 172.21.15.148:0/1790792614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03f415e3e0 0x7f03f40d3cd0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f03f40d05d0 tx=0x7f03f002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d84e50809d409c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:11.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.133+0000 7f03ebfff640 1 -- 172.21.15.148:0/1790792614 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0034070 con 0x7f03f415e3e0 2024-03-29T04:11:11.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.133+0000 7f03ebfff640 1 -- 172.21.15.148:0/1790792614 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f0030610 con 0x7f03f415e3e0 2024-03-29T04:11:11.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.133+0000 7f03ebfff640 1 -- 172.21.15.148:0/1790792614 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f00309e0 con 0x7f03f415e3e0 2024-03-29T04:11:11.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.134+0000 7f03fb33d640 1 -- 172.21.15.148:0/1790792614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03f415e3e0 msgr2=0x7f03f40d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:11.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.134+0000 7f03fb33d640 1 --2- 172.21.15.148:0/1790792614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03f415e3e0 0x7f03f40d3cd0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f03f40d05d0 tx=0x7f03f002b960 comp rx=0 tx=0).stop 2024-03-29T04:11:11.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.134+0000 7f03fb33d640 1 -- 172.21.15.148:0/1790792614 shutdown_connections 2024-03-29T04:11:11.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.134+0000 7f03fb33d640 1 --2- 172.21.15.148:0/1790792614 >> v2:172.21.15.157:3300/0 conn(0x7f03f40ba030 0x7f03f40d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:11.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.134+0000 7f03fb33d640 1 --2- 172.21.15.148:0/1790792614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03f415e3e0 0x7f03f40d3cd0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:11.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.134+0000 7f03fb33d640 1 -- 172.21.15.148:0/1790792614 >> 172.21.15.148:0/1790792614 conn(0x7f03f40b6210 msgr2=0x7f03f40b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:11.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.134+0000 7f03fb33d640 1 -- 172.21.15.148:0/1790792614 shutdown_connections 2024-03-29T04:11:11.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.134+0000 7f03fb33d640 1 -- 172.21.15.148:0/1790792614 wait complete. 2024-03-29T04:11:11.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.135+0000 7f03fb33d640 1 Processor -- start 2024-03-29T04:11:11.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.135+0000 7f03fb33d640 1 -- start start 2024-03-29T04:11:11.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.136+0000 7f03fb33d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03f40ba030 0x7f03f41f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:11.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.136+0000 7f03fb33d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03f40d10d0 0x7f03f4159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:11.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.136+0000 7f03fb33d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03f415e3e0 0x7f03f41f5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:11.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.136+0000 7f03fb33d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f03f40d2870 con 0x7f03f40ba030 2024-03-29T04:11:11.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.136+0000 7f03fb33d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f03f40d34d0 con 0x7f03f40d10d0 2024-03-29T04:11:11.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.136+0000 7f03fb33d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f03f40d29f0 con 0x7f03f415e3e0 2024-03-29T04:11:11.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.136+0000 7f03f90d9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03f40ba030 0x7f03f41f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:11.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.136+0000 7f03f90d9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03f40ba030 0x7f03f41f5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43334/0 (socket says 172.21.15.148:43334) 2024-03-29T04:11:11.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.136+0000 7f03f90d9640 1 -- 172.21.15.148:0/3285440720 learned_addr learned my addr 172.21.15.148:0/3285440720 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:11.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.136+0000 7f03f90d9640 1 -- 172.21.15.148:0/3285440720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03f415e3e0 msgr2=0x7f03f41f5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:11.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.136+0000 7f03f98da640 1 --2- 172.21.15.148:0/3285440720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03f415e3e0 0x7f03f41f5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:11.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.136+0000 7f03f88d8640 1 --2- 172.21.15.148:0/3285440720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03f40d10d0 0x7f03f4159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:11.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.136+0000 7f03f90d9640 1 --2- 172.21.15.148:0/3285440720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03f415e3e0 0x7f03f41f5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:11.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.136+0000 7f03f90d9640 1 -- 172.21.15.148:0/3285440720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03f40d10d0 msgr2=0x7f03f4159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:11.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.136+0000 7f03f90d9640 1 --2- 172.21.15.148:0/3285440720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03f40d10d0 0x7f03f4159180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:11.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.136+0000 7f03f90d9640 1 -- 172.21.15.148:0/3285440720 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f40cb480 con 0x7f03f40ba030 2024-03-29T04:11:11.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.136+0000 7f03f98da640 1 --2- 172.21.15.148:0/3285440720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03f415e3e0 0x7f03f41f5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:11.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.136+0000 7f03f88d8640 1 --2- 172.21.15.148:0/3285440720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03f40d10d0 0x7f03f4159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:11.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.137+0000 7f03f90d9640 1 --2- 172.21.15.148:0/3285440720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03f40ba030 0x7f03f41f5cb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009890 tx=0x7f03f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:11.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.137+0000 7f03e9ffb640 1 -- 172.21.15.148:0/3285440720 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0034040 con 0x7f03f40ba030 2024-03-29T04:11:11.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.137+0000 7f03e9ffb640 1 -- 172.21.15.148:0/3285440720 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f003acc0 con 0x7f03f40ba030 2024-03-29T04:11:11.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.137+0000 7f03fb33d640 1 -- 172.21.15.148:0/3285440720 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f40cb720 con 0x7f03f40ba030 2024-03-29T04:11:11.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.137+0000 7f03e9ffb640 1 -- 172.21.15.148:0/3285440720 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0044640 con 0x7f03f40ba030 2024-03-29T04:11:11.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.137+0000 7f03fb33d640 1 -- 172.21.15.148:0/3285440720 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f41f5950 con 0x7f03f40ba030 2024-03-29T04:11:11.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.138+0000 7f03fb33d640 1 -- 172.21.15.148:0/3285440720 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03f40687e0 con 0x7f03f40ba030 2024-03-29T04:11:11.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.138+0000 7f03e9ffb640 1 -- 172.21.15.148:0/3285440720 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f03f0037070 con 0x7f03f40ba030 2024-03-29T04:11:11.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.139+0000 7f03e9ffb640 1 --2- 172.21.15.148:0/3285440720 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f03d806c560 0x7f03d806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:11.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.139+0000 7f03f88d8640 1 --2- 172.21.15.148:0/3285440720 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f03d806c560 0x7f03d806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:11.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.139+0000 7f03e9ffb640 1 -- 172.21.15.148:0/3285440720 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f03f00ad8c0 con 0x7f03f40ba030 2024-03-29T04:11:11.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.139+0000 7f03f88d8640 1 --2- 172.21.15.148:0/3285440720 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f03d806c560 0x7f03d806ea20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f03f40c9a60 tx=0x7f03e4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:11.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.143+0000 7f03e9ffb640 1 -- 172.21.15.148:0/3285440720 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f03f004a060 con 0x7f03f40ba030 2024-03-29T04:11:11.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.331+0000 7f03fb33d640 1 -- 172.21.15.148:0/3285440720 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f03f40b93c0 con 0x7f03f40ba030 2024-03-29T04:11:11.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.331+0000 7f03e9ffb640 1 -- 172.21.15.148:0/3285440720 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f03f003f070 con 0x7f03f40ba030 2024-03-29T04:11:11.310 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-03-29T04:11:11.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.333+0000 7f03fb33d640 1 -- 172.21.15.148:0/3285440720 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f03d806c560 msgr2=0x7f03d806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:11.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.333+0000 7f03fb33d640 1 --2- 172.21.15.148:0/3285440720 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f03d806c560 0x7f03d806ea20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f03f40c9a60 tx=0x7f03e4008040 comp rx=0 tx=0).stop 2024-03-29T04:11:11.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.333+0000 7f03fb33d640 1 -- 172.21.15.148:0/3285440720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03f40ba030 msgr2=0x7f03f41f5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:11.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.333+0000 7f03fb33d640 1 --2- 172.21.15.148:0/3285440720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03f40ba030 0x7f03f41f5cb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009890 tx=0x7f03f00046c0 comp rx=0 tx=0).stop 2024-03-29T04:11:11.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.333+0000 7f03fb33d640 1 -- 172.21.15.148:0/3285440720 shutdown_connections 2024-03-29T04:11:11.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.333+0000 7f03fb33d640 1 --2- 172.21.15.148:0/3285440720 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f03d806c560 0x7f03d806ea20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:11.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.333+0000 7f03fb33d640 1 --2- 172.21.15.148:0/3285440720 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03f415e3e0 0x7f03f41f5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:11.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.333+0000 7f03fb33d640 1 --2- 172.21.15.148:0/3285440720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03f40d10d0 0x7f03f4159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:11.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.333+0000 7f03fb33d640 1 --2- 172.21.15.148:0/3285440720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03f40ba030 0x7f03f41f5cb0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:11.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.333+0000 7f03fb33d640 1 -- 172.21.15.148:0/3285440720 >> 172.21.15.148:0/3285440720 conn(0x7f03f40b6210 msgr2=0x7f03f40cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:11.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.333+0000 7f03fb33d640 1 -- 172.21.15.148:0/3285440720 shutdown_connections 2024-03-29T04:11:11.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.333+0000 7f03fb33d640 1 -- 172.21.15.148:0/3285440720 wait complete. 2024-03-29T04:11:11.323 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2024-03-29T04:11:11.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:11.323 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-03-29T04:11:11.356 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:11:11.356 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-03-29T04:11:11.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:11.356 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-29T04:11:11.414 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-29T04:11:11.414 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:11:11.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.536+0000 7f0c9e6d3640 1 Processor -- start 2024-03-29T04:11:11.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.536+0000 7f0c9e6d3640 1 -- start start 2024-03-29T04:11:11.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.537+0000 7f0c9e6d3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0c9815e3e0 0x7f0c980d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:11.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.537+0000 7f0c9e6d3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0c980d10d0 0x7f0c980d1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:11.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.537+0000 7f0c9e6d3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c9815e7c0 con 0x7f0c980ba030 2024-03-29T04:11:11.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.537+0000 7f0c9e6d3640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f0c980d1a70 con 0x7f0c9815e3e0 2024-03-29T04:11:11.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.537+0000 7f0c9e6d3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f0c980d3840 con 0x7f0c980d10d0 2024-03-29T04:11:11.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.537+0000 7f0c977fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0c980ba030 0x7f0c980d2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35794/0 (socket says 172.21.15.148:35794) 2024-03-29T04:11:11.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.537+0000 7f0c97fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0c9815e3e0 0x7f0c980d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:11.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.537+0000 7f0c977fe640 1 -- 172.21.15.148:0/2209783252 learned_addr learned my addr 172.21.15.148:0/2209783252 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:11.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.537+0000 7f0c9cc70640 1 --2- 172.21.15.148:0/2209783252 >> v2:172.21.15.157:3300/0 conn(0x7f0c980d10d0 0x7f0c980d1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:11.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.537+0000 7f0c97fff640 1 -- 172.21.15.148:0/2209783252 >> v2:172.21.15.157:3300/0 conn(0x7f0c980d10d0 msgr2=0x7f0c980d1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:11.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.537+0000 7f0c97fff640 1 --2- 172.21.15.148:0/2209783252 >> v2:172.21.15.157:3300/0 conn(0x7f0c980d10d0 0x7f0c980d1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:11.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.537+0000 7f0c97fff640 1 -- 172.21.15.148:0/2209783252 >> v1:172.21.15.148:6789/0 conn(0x7f0c980ba030 legacy=0x7f0c980d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:11.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.537+0000 7f0c97fff640 1 -- 172.21.15.148:0/2209783252 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c980d2c30 con 0x7f0c9815e3e0 2024-03-29T04:11:11.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.537+0000 7f0c9cc70640 1 --2- 172.21.15.148:0/2209783252 >> v2:172.21.15.157:3300/0 conn(0x7f0c980d10d0 0x7f0c980d1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:11.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.537+0000 7f0c96ffd640 1 -- 172.21.15.148:0/2209783252 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1521811933 0 0) 0x7f0c9815e7c0 con 0x7f0c980ba030 2024-03-29T04:11:11.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.538+0000 7f0c97fff640 1 --2- 172.21.15.148:0/2209783252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c9815e3e0 0x7f0c980d3cd0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0c980c7c30 tx=0x7f0c8002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c7c389a6d14af1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:11.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.538+0000 7f0c96ffd640 1 -- 172.21.15.148:0/2209783252 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80034070 con 0x7f0c9815e3e0 2024-03-29T04:11:11.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.539+0000 7f0c96ffd640 1 -- 172.21.15.148:0/2209783252 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c80030610 con 0x7f0c9815e3e0 2024-03-29T04:11:11.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.539+0000 7f0c96ffd640 1 -- 172.21.15.148:0/2209783252 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c800309e0 con 0x7f0c9815e3e0 2024-03-29T04:11:11.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.539+0000 7f0c9e6d3640 1 -- 172.21.15.148:0/2209783252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c9815e3e0 msgr2=0x7f0c980d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:11.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.539+0000 7f0c9e6d3640 1 --2- 172.21.15.148:0/2209783252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c9815e3e0 0x7f0c980d3cd0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0c980c7c30 tx=0x7f0c8002b960 comp rx=0 tx=0).stop 2024-03-29T04:11:11.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.539+0000 7f0c9e6d3640 1 -- 172.21.15.148:0/2209783252 shutdown_connections 2024-03-29T04:11:11.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.539+0000 7f0c9e6d3640 1 --2- 172.21.15.148:0/2209783252 >> v2:172.21.15.157:3300/0 conn(0x7f0c980d10d0 0x7f0c980d1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:11.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.539+0000 7f0c9e6d3640 1 --2- 172.21.15.148:0/2209783252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c9815e3e0 0x7f0c980d3cd0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:11.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.539+0000 7f0c9e6d3640 1 -- 172.21.15.148:0/2209783252 >> 172.21.15.148:0/2209783252 conn(0x7f0c980b6210 msgr2=0x7f0c980b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:11.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.539+0000 7f0c9e6d3640 1 -- 172.21.15.148:0/2209783252 shutdown_connections 2024-03-29T04:11:11.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.539+0000 7f0c9e6d3640 1 -- 172.21.15.148:0/2209783252 wait complete. 2024-03-29T04:11:11.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.540+0000 7f0c9e6d3640 1 Processor -- start 2024-03-29T04:11:11.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.541+0000 7f0c9e6d3640 1 -- start start 2024-03-29T04:11:11.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.542+0000 7f0c9e6d3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c980ba030 0x7f0c9814dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:11.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.542+0000 7f0c9e6d3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c980d10d0 0x7f0c9814cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:11.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.542+0000 7f0c9e6d3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c9815e3e0 0x7f0c98160130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:11.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.542+0000 7f0c9e6d3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0c980d34d0 con 0x7f0c980ba030 2024-03-29T04:11:11.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.542+0000 7f0c9e6d3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0c980d31f0 con 0x7f0c9815e3e0 2024-03-29T04:11:11.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.542+0000 7f0c9e6d3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0c98167f50 con 0x7f0c980d10d0 2024-03-29T04:11:11.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.542+0000 7f0c97fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c980ba030 0x7f0c9814dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:11.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.542+0000 7f0c97fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c980ba030 0x7f0c9814dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43348/0 (socket says 172.21.15.148:43348) 2024-03-29T04:11:11.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.542+0000 7f0c97fff640 1 -- 172.21.15.148:0/3430471256 learned_addr learned my addr 172.21.15.148:0/3430471256 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:11.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.542+0000 7f0c9cc70640 1 --2- 172.21.15.148:0/3430471256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c9815e3e0 0x7f0c98160130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:11.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.542+0000 7f0c977fe640 1 --2- 172.21.15.148:0/3430471256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c980d10d0 0x7f0c9814cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:11.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.542+0000 7f0c97fff640 1 -- 172.21.15.148:0/3430471256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c980d10d0 msgr2=0x7f0c9814cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:11.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.542+0000 7f0c97fff640 1 --2- 172.21.15.148:0/3430471256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c980d10d0 0x7f0c9814cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:11.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.542+0000 7f0c97fff640 1 -- 172.21.15.148:0/3430471256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c9815e3e0 msgr2=0x7f0c98160130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:11.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.542+0000 7f0c97fff640 1 --2- 172.21.15.148:0/3430471256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c9815e3e0 0x7f0c98160130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:11.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.542+0000 7f0c97fff640 1 -- 172.21.15.148:0/3430471256 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c9814e630 con 0x7f0c980ba030 2024-03-29T04:11:11.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.542+0000 7f0c97fff640 1 --2- 172.21.15.148:0/3430471256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c980ba030 0x7f0c9814dde0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0c80009890 tx=0x7f0c800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:11.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.543+0000 7f0c94ff9640 1 -- 172.21.15.148:0/3430471256 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80034040 con 0x7f0c980ba030 2024-03-29T04:11:11.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.543+0000 7f0c94ff9640 1 -- 172.21.15.148:0/3430471256 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c8003acc0 con 0x7f0c980ba030 2024-03-29T04:11:11.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.543+0000 7f0c9e6d3640 1 -- 172.21.15.148:0/3430471256 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c980c7e40 con 0x7f0c980ba030 2024-03-29T04:11:11.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.543+0000 7f0c94ff9640 1 -- 172.21.15.148:0/3430471256 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80044640 con 0x7f0c980ba030 2024-03-29T04:11:11.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.543+0000 7f0c9e6d3640 1 -- 172.21.15.148:0/3430471256 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c980c8400 con 0x7f0c980ba030 2024-03-29T04:11:11.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.544+0000 7f0c9e6d3640 1 -- 172.21.15.148:0/3430471256 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c980687e0 con 0x7f0c980ba030 2024-03-29T04:11:11.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.544+0000 7f0c94ff9640 1 -- 172.21.15.148:0/3430471256 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0c80037070 con 0x7f0c980ba030 2024-03-29T04:11:11.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.545+0000 7f0c94ff9640 1 --2- 172.21.15.148:0/3430471256 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f0c7806c530 0x7f0c7806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:11.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.545+0000 7f0c94ff9640 1 -- 172.21.15.148:0/3430471256 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f0c800ad780 con 0x7f0c980ba030 2024-03-29T04:11:11.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.545+0000 7f0c977fe640 1 --2- 172.21.15.148:0/3430471256 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f0c7806c530 0x7f0c7806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:11.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.545+0000 7f0c977fe640 1 --2- 172.21.15.148:0/3430471256 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f0c7806c530 0x7f0c7806e9f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0c88007f70 tx=0x7f0c88007f00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:11.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.549+0000 7f0c94ff9640 1 -- 172.21.15.148:0/3430471256 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0c8004a060 con 0x7f0c980ba030 2024-03-29T04:11:11.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.768+0000 7f0c9e6d3640 1 -- 172.21.15.148:0/3430471256 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c980d1820 con 0x7f0c980ba030 2024-03-29T04:11:11.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.769+0000 7f0c94ff9640 1 -- 172.21.15.148:0/3430471256 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f0c8003f070 con 0x7f0c980ba030 2024-03-29T04:11:11.748 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:11:11.748 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/36660496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":36660496},{"type":"v1","addr":"172.21.15.157:6834","nonce":36660496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2674481055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2674481055},{"type":"v1","addr":"172.21.15.157:6835","nonce":2674481055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3356449643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3356449643},{"type":"v1","addr":"172.21.15.148:6836","nonce":3356449643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:09:59.886367+0000","modified":"2024-03-29T04:10:22.957623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/2453181817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2453181817},{"type":"v1","addr":"172.21.15.148:6837","nonce":2453181817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-03-29T04:11:11.748 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 9 2024-03-29T04:11:11.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.771+0000 7f0c9e6d3640 1 -- 172.21.15.148:0/3430471256 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f0c7806c530 msgr2=0x7f0c7806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:11.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.771+0000 7f0c9e6d3640 1 --2- 172.21.15.148:0/3430471256 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f0c7806c530 0x7f0c7806e9f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0c88007f70 tx=0x7f0c88007f00 comp rx=0 tx=0).stop 2024-03-29T04:11:11.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.771+0000 7f0c9e6d3640 1 -- 172.21.15.148:0/3430471256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c980ba030 msgr2=0x7f0c9814dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:11.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.771+0000 7f0c9e6d3640 1 --2- 172.21.15.148:0/3430471256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c980ba030 0x7f0c9814dde0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0c80009890 tx=0x7f0c800046c0 comp rx=0 tx=0).stop 2024-03-29T04:11:11.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.771+0000 7f0c9e6d3640 1 -- 172.21.15.148:0/3430471256 shutdown_connections 2024-03-29T04:11:11.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.771+0000 7f0c9e6d3640 1 --2- 172.21.15.148:0/3430471256 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f0c7806c530 0x7f0c7806e9f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:11.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.771+0000 7f0c9e6d3640 1 --2- 172.21.15.148:0/3430471256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c9815e3e0 0x7f0c98160130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:11.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.771+0000 7f0c9e6d3640 1 --2- 172.21.15.148:0/3430471256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c980d10d0 0x7f0c9814cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:11.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.771+0000 7f0c9e6d3640 1 --2- 172.21.15.148:0/3430471256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c980ba030 0x7f0c9814dde0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:11.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.771+0000 7f0c9e6d3640 1 -- 172.21.15.148:0/3430471256 >> 172.21.15.148:0/3430471256 conn(0x7f0c980b6210 msgr2=0x7f0c98156cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:11.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.771+0000 7f0c9e6d3640 1 -- 172.21.15.148:0/3430471256 shutdown_connections 2024-03-29T04:11:11.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.771+0000 7f0c9e6d3640 1 -- 172.21.15.148:0/3430471256 wait complete. 2024-03-29T04:11:11.761 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:11:11.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.883+0000 7f7340022640 1 Processor -- start 2024-03-29T04:11:11.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.883+0000 7f7340022640 1 -- start start 2024-03-29T04:11:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.884+0000 7f7340022640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f733815e770 0x7f73380ce710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.884+0000 7f7340022640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f733806b5b0 0x7f7338072520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.884+0000 7f7340022640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f733806dc80 con 0x7f7338072a60 2024-03-29T04:11:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.884+0000 7f7340022640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f73380b4090 con 0x7f733806b5b0 2024-03-29T04:11:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.884+0000 7f7340022640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f73380b5ec0 con 0x7f733815e770 2024-03-29T04:11:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.884+0000 7f733d5bd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f733806b5b0 0x7f7338072520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.884+0000 7f733ddbe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f733815e770 0x7f73380ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.884+0000 7f733e5bf640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7338072a60 0x7f73380cf860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.148:32806/0 (socket says 172.21.15.148:32806) 2024-03-29T04:11:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.884+0000 7f733e5bf640 1 -- 172.21.15.148:0/1775086309 learned_addr learned my addr 172.21.15.148:0/1775086309 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.884+0000 7f733d5bd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f733806b5b0 0x7f7338072520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:46358/0 (socket says 172.21.15.148:46358) 2024-03-29T04:11:11.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.884+0000 7f733ddbe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f733815e770 0x7f73380ce710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.148:40402/0 (socket says 172.21.15.148:40402) 2024-03-29T04:11:11.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.885+0000 7f733d5bd640 1 -- 172.21.15.148:0/1775086309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f733815e770 msgr2=0x7f73380ce710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:11.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.885+0000 7f733d5bd640 1 --2- 172.21.15.148:0/1775086309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f733815e770 0x7f73380ce710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:11.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.885+0000 7f733d5bd640 1 -- 172.21.15.148:0/1775086309 >> v1:172.21.15.157:6789/0 conn(0x7f7338072a60 legacy=0x7f73380cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:11.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.885+0000 7f733d5bd640 1 -- 172.21.15.148:0/1775086309 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73380b61a0 con 0x7f733806b5b0 2024-03-29T04:11:11.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.885+0000 7f733ddbe640 1 --2- 172.21.15.148:0/1775086309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f733815e770 0x7f73380ce710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:11:11.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.885+0000 7f733d5bd640 1 --2- 172.21.15.148:0/1775086309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f733806b5b0 0x7f7338072520 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f73340097a0 tx=0x7f733402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc59bfe99f29aa28 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:11.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.886+0000 7f733cdbc640 1 -- 172.21.15.148:0/1775086309 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7334035020 con 0x7f733806b5b0 2024-03-29T04:11:11.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.886+0000 7f733cdbc640 1 -- 172.21.15.148:0/1775086309 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7334031830 con 0x7f733806b5b0 2024-03-29T04:11:11.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.886+0000 7f733cdbc640 1 -- 172.21.15.148:0/1775086309 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7334031c00 con 0x7f733806b5b0 2024-03-29T04:11:11.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.886+0000 7f7340022640 1 -- 172.21.15.148:0/1775086309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f733806b5b0 msgr2=0x7f7338072520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:11.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.886+0000 7f7340022640 1 --2- 172.21.15.148:0/1775086309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f733806b5b0 0x7f7338072520 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f73340097a0 tx=0x7f733402b8d0 comp rx=0 tx=0).stop 2024-03-29T04:11:11.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.886+0000 7f7340022640 1 -- 172.21.15.148:0/1775086309 shutdown_connections 2024-03-29T04:11:11.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.886+0000 7f7340022640 1 --2- 172.21.15.148:0/1775086309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f733806b5b0 0x7f7338072520 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:11.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.886+0000 7f7340022640 1 --2- 172.21.15.148:0/1775086309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f733815e770 0x7f73380ce710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:11.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.886+0000 7f7340022640 1 -- 172.21.15.148:0/1775086309 >> 172.21.15.148:0/1775086309 conn(0x7f73380d0f50 msgr2=0x7f73380d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:11.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.887+0000 7f7340022640 1 -- 172.21.15.148:0/1775086309 shutdown_connections 2024-03-29T04:11:11.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.887+0000 7f7340022640 1 -- 172.21.15.148:0/1775086309 wait complete. 2024-03-29T04:11:11.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.888+0000 7f7340022640 1 Processor -- start 2024-03-29T04:11:11.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.888+0000 7f7340022640 1 -- start start 2024-03-29T04:11:11.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.890+0000 7f7340022640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f733806b5b0 0x7f733815f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:11.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.890+0000 7f7340022640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7338072a60 0x7f73381e7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:11.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.890+0000 7f7340022640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f733815e770 0x7f73381e8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:11.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.890+0000 7f7340022640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f733806da20 con 0x7f733806b5b0 2024-03-29T04:11:11.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.890+0000 7f7340022640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f733806d8a0 con 0x7f7338072a60 2024-03-29T04:11:11.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.890+0000 7f7340022640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f73380b5ae0 con 0x7f733815e770 2024-03-29T04:11:11.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.890+0000 7f733ddbe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f733806b5b0 0x7f733815f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:11.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.890+0000 7f733ddbe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f733806b5b0 0x7f733815f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43364/0 (socket says 172.21.15.148:43364) 2024-03-29T04:11:11.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.890+0000 7f733ddbe640 1 -- 172.21.15.148:0/603909981 learned_addr learned my addr 172.21.15.148:0/603909981 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:11.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.890+0000 7f733d5bd640 1 --2- 172.21.15.148:0/603909981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7338072a60 0x7f73381e7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:11.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.890+0000 7f733e5bf640 1 --2- 172.21.15.148:0/603909981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f733815e770 0x7f73381e8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:11.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.890+0000 7f733ddbe640 1 -- 172.21.15.148:0/603909981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f733815e770 msgr2=0x7f73381e8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:11.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.890+0000 7f733ddbe640 1 --2- 172.21.15.148:0/603909981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f733815e770 0x7f73381e8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:11.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.890+0000 7f733ddbe640 1 -- 172.21.15.148:0/603909981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7338072a60 msgr2=0x7f73381e7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:11.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.890+0000 7f733ddbe640 1 --2- 172.21.15.148:0/603909981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7338072a60 0x7f73381e7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:11.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.890+0000 7f733ddbe640 1 -- 172.21.15.148:0/603909981 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7338155940 con 0x7f733806b5b0 2024-03-29T04:11:11.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.890+0000 7f733d5bd640 1 --2- 172.21.15.148:0/603909981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7338072a60 0x7f73381e7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:11.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.890+0000 7f733ddbe640 1 --2- 172.21.15.148:0/603909981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f733806b5b0 0x7f733815f240 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f732800ebe0 tx=0x7f732800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:11.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.890+0000 7f732e7fc640 1 -- 172.21.15.148:0/603909981 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7328017070 con 0x7f733806b5b0 2024-03-29T04:11:11.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.890+0000 7f732e7fc640 1 -- 172.21.15.148:0/603909981 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f732800cc30 con 0x7f733806b5b0 2024-03-29T04:11:11.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.890+0000 7f7340022640 1 -- 172.21.15.148:0/603909981 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7338155c20 con 0x7f733806b5b0 2024-03-29T04:11:11.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.890+0000 7f732e7fc640 1 -- 172.21.15.148:0/603909981 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7328013470 con 0x7f733806b5b0 2024-03-29T04:11:11.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.890+0000 7f7340022640 1 -- 172.21.15.148:0/603909981 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7338154aa0 con 0x7f733806b5b0 2024-03-29T04:11:11.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.892+0000 7f7340022640 1 -- 172.21.15.148:0/603909981 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f733813f650 con 0x7f733806b5b0 2024-03-29T04:11:11.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.892+0000 7f732e7fc640 1 -- 172.21.15.148:0/603909981 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f732801a070 con 0x7f733806b5b0 2024-03-29T04:11:11.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.892+0000 7f732e7fc640 1 --2- 172.21.15.148:0/603909981 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f731c06c530 0x7f731c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:11.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.892+0000 7f732e7fc640 1 -- 172.21.15.148:0/603909981 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f732808be10 con 0x7f733806b5b0 2024-03-29T04:11:11.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.892+0000 7f733d5bd640 1 --2- 172.21.15.148:0/603909981 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f731c06c530 0x7f731c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:11.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.893+0000 7f733d5bd640 1 --2- 172.21.15.148:0/603909981 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f731c06c530 0x7f731c06e9f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7334036000 tx=0x7f7334008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:11.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:11.897+0000 7f732e7fc640 1 -- 172.21.15.148:0/603909981 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f733813f650 con 0x7f733806b5b0 2024-03-29T04:11:12.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.082+0000 7f7340022640 1 -- 172.21.15.148:0/603909981 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73380687e0 con 0x7f733806b5b0 2024-03-29T04:11:12.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.083+0000 7f732e7fc640 1 -- 172.21.15.148:0/603909981 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18858 (secure 0 0 0) 0x7f732805a030 con 0x7f733806b5b0 2024-03-29T04:11:12.062 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:11:12.062 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:41.662934+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:11:12.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.085+0000 7f7340022640 1 -- 172.21.15.148:0/603909981 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f731c06c530 msgr2=0x7f731c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:12.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.085+0000 7f7340022640 1 --2- 172.21.15.148:0/603909981 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f731c06c530 0x7f731c06e9f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7334036000 tx=0x7f7334008030 comp rx=0 tx=0).stop 2024-03-29T04:11:12.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.085+0000 7f7340022640 1 -- 172.21.15.148:0/603909981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f733806b5b0 msgr2=0x7f733815f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:12.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.085+0000 7f7340022640 1 --2- 172.21.15.148:0/603909981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f733806b5b0 0x7f733815f240 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f732800ebe0 tx=0x7f732800c520 comp rx=0 tx=0).stop 2024-03-29T04:11:12.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.085+0000 7f7340022640 1 -- 172.21.15.148:0/603909981 shutdown_connections 2024-03-29T04:11:12.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.085+0000 7f7340022640 1 --2- 172.21.15.148:0/603909981 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f731c06c530 0x7f731c06e9f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:12.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.085+0000 7f7340022640 1 --2- 172.21.15.148:0/603909981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f733815e770 0x7f73381e8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:12.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.085+0000 7f7340022640 1 --2- 172.21.15.148:0/603909981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7338072a60 0x7f73381e7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:12.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.085+0000 7f7340022640 1 --2- 172.21.15.148:0/603909981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f733806b5b0 0x7f733815f240 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:12.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.085+0000 7f7340022640 1 -- 172.21.15.148:0/603909981 >> 172.21.15.148:0/603909981 conn(0x7f73380d0f50 msgr2=0x7f73380b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:12.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.085+0000 7f7340022640 1 -- 172.21.15.148:0/603909981 shutdown_connections 2024-03-29T04:11:12.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.085+0000 7f7340022640 1 -- 172.21.15.148:0/603909981 wait complete. 2024-03-29T04:11:12.075 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:47.898351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:57.922313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:58.829405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:10:02.551763+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:11:12.075 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:11:12.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.201+0000 7f0acd5cd640 1 Processor -- start 2024-03-29T04:11:12.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.201+0000 7f0acd5cd640 1 -- start start 2024-03-29T04:11:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.202+0000 7f0acd5cd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0ac80ccec0 0x7f0ac81693b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.202+0000 7f0acd5cd640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ac80b56c0 con 0x7f0ac815f200 2024-03-29T04:11:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.202+0000 7f0acd5cd640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ac80cfdd0 con 0x7f0ac815f5e0 2024-03-29T04:11:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.202+0000 7f0acd5cd640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f0ac80d09e0 con 0x7f0ac80ccec0 2024-03-29T04:11:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.202+0000 7f0ac77fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0ac80ccec0 0x7f0ac81693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.202+0000 7f0ac6ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0ac815f200 0x7f0ac80b4fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35808/0 (socket says 172.21.15.148:35808) 2024-03-29T04:11:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.202+0000 7f0ac6ffd640 1 -- 172.21.15.148:0/2925760501 learned_addr learned my addr 172.21.15.148:0/2925760501 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.202+0000 7f0ac77fe640 1 -- 172.21.15.148:0/2925760501 >> v1:172.21.15.157:6789/0 conn(0x7f0ac815f5e0 legacy=0x7f0ac80cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.202+0000 7f0ac77fe640 1 -- 172.21.15.148:0/2925760501 >> v1:172.21.15.148:6789/0 conn(0x7f0ac815f200 legacy=0x7f0ac80b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.202+0000 7f0ac5ffb640 1 -- 172.21.15.148:0/2925760501 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 236793887 0 0) 0x7f0ac80b56c0 con 0x7f0ac815f200 2024-03-29T04:11:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.202+0000 7f0ac77fe640 1 -- 172.21.15.148:0/2925760501 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac80d00b0 con 0x7f0ac80ccec0 2024-03-29T04:11:12.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.203+0000 7f0ac77fe640 1 --2- 172.21.15.148:0/2925760501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ac80ccec0 0x7f0ac81693b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0009d90 tx=0x7f0ac002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8eec7a4e53293b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:12.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.204+0000 7f0ac5ffb640 1 -- 172.21.15.148:0/2925760501 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0039070 con 0x7f0ac80ccec0 2024-03-29T04:11:12.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.204+0000 7f0ac5ffb640 1 -- 172.21.15.148:0/2925760501 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ac002cc60 con 0x7f0ac80ccec0 2024-03-29T04:11:12.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.204+0000 7f0ac5ffb640 1 -- 172.21.15.148:0/2925760501 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0034660 con 0x7f0ac80ccec0 2024-03-29T04:11:12.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.204+0000 7f0acd5cd640 1 -- 172.21.15.148:0/2925760501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ac80ccec0 msgr2=0x7f0ac81693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:12.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.204+0000 7f0acd5cd640 1 --2- 172.21.15.148:0/2925760501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ac80ccec0 0x7f0ac81693b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0009d90 tx=0x7f0ac002beb0 comp rx=0 tx=0).stop 2024-03-29T04:11:12.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.204+0000 7f0acd5cd640 1 -- 172.21.15.148:0/2925760501 shutdown_connections 2024-03-29T04:11:12.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.204+0000 7f0acd5cd640 1 --2- 172.21.15.148:0/2925760501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ac80ccec0 0x7f0ac81693b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:12.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.204+0000 7f0acd5cd640 1 -- 172.21.15.148:0/2925760501 >> 172.21.15.148:0/2925760501 conn(0x7f0ac80b3c70 msgr2=0x7f0ac80b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:12.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.204+0000 7f0acd5cd640 1 -- 172.21.15.148:0/2925760501 shutdown_connections 2024-03-29T04:11:12.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.204+0000 7f0acd5cd640 1 -- 172.21.15.148:0/2925760501 wait complete. 2024-03-29T04:11:12.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.206+0000 7f0acd5cd640 1 Processor -- start 2024-03-29T04:11:12.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.206+0000 7f0acd5cd640 1 -- start start 2024-03-29T04:11:12.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.206+0000 7f0acd5cd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ac80ccec0 0x7f0ac814d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:12.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.206+0000 7f0acd5cd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ac815f200 0x7f0ac80c9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:12.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.206+0000 7f0acd5cd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ac815f5e0 0x7f0ac8156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:12.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.206+0000 7f0acd5cd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac80b7b40 con 0x7f0ac80ccec0 2024-03-29T04:11:12.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.206+0000 7f0acd5cd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac80d0670 con 0x7f0ac815f5e0 2024-03-29T04:11:12.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.206+0000 7f0acd5cd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0ac80b7cc0 con 0x7f0ac815f200 2024-03-29T04:11:12.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.206+0000 7f0ac6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ac80ccec0 0x7f0ac814d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:12.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.207+0000 7f0ac6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ac80ccec0 0x7f0ac814d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43380/0 (socket says 172.21.15.148:43380) 2024-03-29T04:11:12.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.207+0000 7f0ac6ffd640 1 -- 172.21.15.148:0/3126533481 learned_addr learned my addr 172.21.15.148:0/3126533481 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:12.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.207+0000 7f0ac67fc640 1 --2- 172.21.15.148:0/3126533481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ac815f200 0x7f0ac80c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:12.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.207+0000 7f0ac77fe640 1 --2- 172.21.15.148:0/3126533481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ac815f5e0 0x7f0ac8156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:12.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.207+0000 7f0ac6ffd640 1 -- 172.21.15.148:0/3126533481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ac815f200 msgr2=0x7f0ac80c9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:12.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.207+0000 7f0ac6ffd640 1 --2- 172.21.15.148:0/3126533481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ac815f200 0x7f0ac80c9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:12.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.207+0000 7f0ac6ffd640 1 -- 172.21.15.148:0/3126533481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ac815f5e0 msgr2=0x7f0ac8156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:12.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.207+0000 7f0ac6ffd640 1 --2- 172.21.15.148:0/3126533481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ac815f5e0 0x7f0ac8156290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:12.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.207+0000 7f0ac6ffd640 1 -- 172.21.15.148:0/3126533481 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac80c6c20 con 0x7f0ac80ccec0 2024-03-29T04:11:12.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.207+0000 7f0ac77fe640 1 --2- 172.21.15.148:0/3126533481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ac815f5e0 0x7f0ac8156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:12.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.207+0000 7f0ac6ffd640 1 --2- 172.21.15.148:0/3126533481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ac80ccec0 0x7f0ac814d7d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0abc00c4d0 tx=0x7f0abc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:12.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.207+0000 7f0aaf7fe640 1 -- 172.21.15.148:0/3126533481 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc018070 con 0x7f0ac80ccec0 2024-03-29T04:11:12.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.207+0000 7f0aaf7fe640 1 -- 172.21.15.148:0/3126533481 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0abc007500 con 0x7f0ac80ccec0 2024-03-29T04:11:12.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.207+0000 7f0acd5cd640 1 -- 172.21.15.148:0/3126533481 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac80c6f00 con 0x7f0ac80ccec0 2024-03-29T04:11:12.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.207+0000 7f0aaf7fe640 1 -- 172.21.15.148:0/3126533481 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc004a20 con 0x7f0ac80ccec0 2024-03-29T04:11:12.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.207+0000 7f0acd5cd640 1 -- 172.21.15.148:0/3126533481 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac81545d0 con 0x7f0ac80ccec0 2024-03-29T04:11:12.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.208+0000 7f0acd5cd640 1 -- 172.21.15.148:0/3126533481 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ac80758d0 con 0x7f0ac80ccec0 2024-03-29T04:11:12.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.209+0000 7f0aaf7fe640 1 -- 172.21.15.148:0/3126533481 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0abc01b070 con 0x7f0ac80ccec0 2024-03-29T04:11:12.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.209+0000 7f0aaf7fe640 1 --2- 172.21.15.148:0/3126533481 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f0ab406c530 0x7f0ab406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:12.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.209+0000 7f0aaf7fe640 1 -- 172.21.15.148:0/3126533481 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f0abc08acf0 con 0x7f0ac80ccec0 2024-03-29T04:11:12.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.209+0000 7f0ac67fc640 1 --2- 172.21.15.148:0/3126533481 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f0ab406c530 0x7f0ab406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:12.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.210+0000 7f0ac67fc640 1 --2- 172.21.15.148:0/3126533481 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f0ab406c530 0x7f0ab406e9f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8156e10 tx=0x7f0ab8009060 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:12.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.213+0000 7f0aaf7fe640 1 -- 172.21.15.148:0/3126533481 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0abc08f070 con 0x7f0ac80ccec0 2024-03-29T04:11:12.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.395+0000 7f0acd5cd640 1 -- 172.21.15.148:0/3126533481 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0ac80d1630 con 0x7f0ac80ccec0 2024-03-29T04:11:12.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.396+0000 7f0aaf7fe640 1 -- 172.21.15.148:0/3126533481 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f0abc058950 con 0x7f0ac80ccec0 2024-03-29T04:11:12.375 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-29T04:11:12.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.398+0000 7f0acd5cd640 1 -- 172.21.15.148:0/3126533481 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f0ab406c530 msgr2=0x7f0ab406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:12.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.398+0000 7f0acd5cd640 1 --2- 172.21.15.148:0/3126533481 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f0ab406c530 0x7f0ab406e9f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8156e10 tx=0x7f0ab8009060 comp rx=0 tx=0).stop 2024-03-29T04:11:12.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.398+0000 7f0acd5cd640 1 -- 172.21.15.148:0/3126533481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ac80ccec0 msgr2=0x7f0ac814d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:12.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.398+0000 7f0acd5cd640 1 --2- 172.21.15.148:0/3126533481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ac80ccec0 0x7f0ac814d7d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0abc00c4d0 tx=0x7f0abc00c990 comp rx=0 tx=0).stop 2024-03-29T04:11:12.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.398+0000 7f0acd5cd640 1 -- 172.21.15.148:0/3126533481 shutdown_connections 2024-03-29T04:11:12.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.398+0000 7f0acd5cd640 1 --2- 172.21.15.148:0/3126533481 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f0ab406c530 0x7f0ab406e9f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:12.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.398+0000 7f0acd5cd640 1 --2- 172.21.15.148:0/3126533481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ac815f5e0 0x7f0ac8156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:12.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.398+0000 7f0acd5cd640 1 --2- 172.21.15.148:0/3126533481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ac815f200 0x7f0ac80c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:12.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.398+0000 7f0acd5cd640 1 --2- 172.21.15.148:0/3126533481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ac80ccec0 0x7f0ac814d7d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:12.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.398+0000 7f0acd5cd640 1 -- 172.21.15.148:0/3126533481 >> 172.21.15.148:0/3126533481 conn(0x7f0ac80b3c70 msgr2=0x7f0ac81577a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:12.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.398+0000 7f0acd5cd640 1 -- 172.21.15.148:0/3126533481 shutdown_connections 2024-03-29T04:11:12.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.398+0000 7f0acd5cd640 1 -- 172.21.15.148:0/3126533481 wait complete. 2024-03-29T04:11:12.391 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:11:12.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.517+0000 7fbba3f90640 1 Processor -- start 2024-03-29T04:11:12.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.517+0000 7fbba3f90640 1 -- start start 2024-03-29T04:11:12.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.518+0000 7fbba3f90640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb9c15f5e0 0x7fbb9c0cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:12.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.518+0000 7fbba3f90640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb9c0b56c0 con 0x7fbb9c0cccf0 2024-03-29T04:11:12.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.518+0000 7fbba3f90640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb9c0cfdd0 con 0x7fbb9c15f200 2024-03-29T04:11:12.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.518+0000 7fbba3f90640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbb9c0d09e0 con 0x7fbb9c15f5e0 2024-03-29T04:11:12.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.518+0000 7fbba252d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbb9c0cccf0 0x7fbb9c0b6b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35824/0 (socket says 172.21.15.148:35824) 2024-03-29T04:11:12.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.518+0000 7fbba252d640 1 -- 172.21.15.148:0/781839449 learned_addr learned my addr 172.21.15.148:0/781839449 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:12.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.518+0000 7fbba0d2a640 1 -- 172.21.15.148:0/781839449 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3722111963 0 0) 0x7fbb9c0b56c0 con 0x7fbb9c0cccf0 2024-03-29T04:11:12.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.518+0000 7fbba152b640 1 --2- 172.21.15.148:0/781839449 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb9c15f5e0 0x7fbb9c0cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:12.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.518+0000 7fbba0d2a640 1 -- 172.21.15.148:0/781839449 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb88003520 con 0x7fbb9c0cccf0 2024-03-29T04:11:12.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.519+0000 7fbba0d2a640 1 -- 172.21.15.148:0/781839449 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1066985590 0 0) 0x7fbb88003520 con 0x7fbb9c0cccf0 2024-03-29T04:11:12.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.519+0000 7fbba0d2a640 1 -- 172.21.15.148:0/781839449 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb9c15f5e0 msgr2=0x7fbb9c0cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:12.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.519+0000 7fbba0d2a640 1 --2- 172.21.15.148:0/781839449 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb9c15f5e0 0x7fbb9c0cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:12.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.519+0000 7fbba0d2a640 1 -- 172.21.15.148:0/781839449 >> v1:172.21.15.157:6789/0 conn(0x7fbb9c15f200 legacy=0x7fbb9c0b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:12.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.519+0000 7fbba0d2a640 1 -- 172.21.15.148:0/781839449 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9c0d00b0 con 0x7fbb9c0cccf0 2024-03-29T04:11:12.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.519+0000 7fbba0d2a640 1 -- 172.21.15.148:0/781839449 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7fbb940019f0 con 0x7fbb9c0cccf0 2024-03-29T04:11:12.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.519+0000 7fbba0d2a640 1 -- 172.21.15.148:0/781839449 >> v1:172.21.15.148:6789/0 conn(0x7fbb9c0cccf0 legacy=0x7fbb9c0b6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:12.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.519+0000 7fbba0d2a640 1 --2- 172.21.15.148:0/781839449 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb9c15f5e0 0x7fbb880039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:12.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.519+0000 7fbba0d2a640 1 --2- 172.21.15.148:0/781839449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb88003f60 0x7fbb88004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:12.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.519+0000 7fbba0d2a640 1 --2- 172.21.15.148:0/781839449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb88004d40 0x7fbb880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:12.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.519+0000 7fbba0d2a640 1 -- 172.21.15.148:0/781839449 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c0d09e0 con 0x7fbb88003f60 2024-03-29T04:11:12.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.519+0000 7fbba0d2a640 1 -- 172.21.15.148:0/781839449 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8800baa0 con 0x7fbb88004d40 2024-03-29T04:11:12.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.519+0000 7fbba0d2a640 1 -- 172.21.15.148:0/781839449 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbb8800bca0 con 0x7fbb9c15f5e0 2024-03-29T04:11:12.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.519+0000 7fbba152b640 1 --2- 172.21.15.148:0/781839449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb88003f60 0x7fbb88004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:12.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.519+0000 7fbba1d2c640 1 --2- 172.21.15.148:0/781839449 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb9c15f5e0 0x7fbb880039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:12.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.519+0000 7fbba252d640 1 --2- 172.21.15.148:0/781839449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb88004d40 0x7fbb880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:12.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.519+0000 7fbba152b640 1 -- 172.21.15.148:0/781839449 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb9c15f5e0 msgr2=0x7fbb880039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:12.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.519+0000 7fbba152b640 1 --2- 172.21.15.148:0/781839449 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb9c15f5e0 0x7fbb880039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:12.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.519+0000 7fbba152b640 1 -- 172.21.15.148:0/781839449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb88004d40 msgr2=0x7fbb880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:12.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.519+0000 7fbba152b640 1 --2- 172.21.15.148:0/781839449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb88004d40 0x7fbb880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:12.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.519+0000 7fbba152b640 1 -- 172.21.15.148:0/781839449 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb88003520 con 0x7fbb88003f60 2024-03-29T04:11:12.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.519+0000 7fbba252d640 1 --2- 172.21.15.148:0/781839449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb88004d40 0x7fbb880051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:12.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.520+0000 7fbba152b640 1 --2- 172.21.15.148:0/781839449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb88003f60 0x7fbb88004800 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009ac0 tx=0x7fbb88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad85d6622566dd82 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:12.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.520+0000 7fbba0d2a640 1 -- 172.21.15.148:0/781839449 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98039070 con 0x7fbb88003f60 2024-03-29T04:11:12.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.520+0000 7fbba0d2a640 1 -- 172.21.15.148:0/781839449 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb9802c6b0 con 0x7fbb88003f60 2024-03-29T04:11:12.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.520+0000 7fbba0d2a640 1 -- 172.21.15.148:0/781839449 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98009690 con 0x7fbb88003f60 2024-03-29T04:11:12.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.521+0000 7fbba3f90640 1 -- 172.21.15.148:0/781839449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb88003f60 msgr2=0x7fbb88004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:12.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.521+0000 7fbba3f90640 1 --2- 172.21.15.148:0/781839449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb88003f60 0x7fbb88004800 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009ac0 tx=0x7fbb88005830 comp rx=0 tx=0).stop 2024-03-29T04:11:12.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.521+0000 7fbba252d640 1 -- 172.21.15.148:0/781839449 reap_dead start 2024-03-29T04:11:12.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.521+0000 7fbba3f90640 1 -- 172.21.15.148:0/781839449 shutdown_connections 2024-03-29T04:11:12.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.521+0000 7fbba3f90640 1 -- 172.21.15.148:0/781839449 >> 172.21.15.148:0/781839449 conn(0x7fbb9c0b3c70 msgr2=0x7fbb9c0b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:12.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.521+0000 7fbba3f90640 1 -- 172.21.15.148:0/781839449 shutdown_connections 2024-03-29T04:11:12.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.521+0000 7fbba3f90640 1 -- 172.21.15.148:0/781839449 wait complete. 2024-03-29T04:11:12.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.522+0000 7fbba3f90640 1 Processor -- start 2024-03-29T04:11:12.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.522+0000 7fbba3f90640 1 -- start start 2024-03-29T04:11:12.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.523+0000 7fbba3f90640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb88003f60 0x7fbb9c150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:12.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.523+0000 7fbba3f90640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb9c14d4d0 0x7fbb9c14d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:12.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.523+0000 7fbba3f90640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb9c0c9e40 0x7fbb9c0ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:12.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.523+0000 7fbba3f90640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c0cc7b0 con 0x7fbb9c14d4d0 2024-03-29T04:11:12.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.523+0000 7fbba3f90640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c0d0670 con 0x7fbb88003f60 2024-03-29T04:11:12.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.523+0000 7fbba3f90640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbb9c0cc930 con 0x7fbb9c0c9e40 2024-03-29T04:11:12.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.523+0000 7fbba152b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb9c14d4d0 0x7fbb9c14d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:12.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.523+0000 7fbba152b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb9c14d4d0 0x7fbb9c14d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43398/0 (socket says 172.21.15.148:43398) 2024-03-29T04:11:12.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.523+0000 7fbba152b640 1 -- 172.21.15.148:0/2170734305 learned_addr learned my addr 172.21.15.148:0/2170734305 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:12.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.523+0000 7fbba252d640 1 --2- 172.21.15.148:0/2170734305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb9c0c9e40 0x7fbb9c0ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:12.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.523+0000 7fbba1d2c640 1 --2- 172.21.15.148:0/2170734305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb88003f60 0x7fbb9c150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:12.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.523+0000 7fbba152b640 1 -- 172.21.15.148:0/2170734305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb9c0c9e40 msgr2=0x7fbb9c0ca2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:12.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.523+0000 7fbba152b640 1 --2- 172.21.15.148:0/2170734305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb9c0c9e40 0x7fbb9c0ca2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:12.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.523+0000 7fbba152b640 1 -- 172.21.15.148:0/2170734305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb88003f60 msgr2=0x7fbb9c150f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:12.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.523+0000 7fbba152b640 1 --2- 172.21.15.148:0/2170734305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb88003f60 0x7fbb9c150f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:12.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.523+0000 7fbba152b640 1 -- 172.21.15.148:0/2170734305 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9c0cabe0 con 0x7fbb9c14d4d0 2024-03-29T04:11:12.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.523+0000 7fbba1d2c640 1 --2- 172.21.15.148:0/2170734305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb88003f60 0x7fbb9c150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:12.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.523+0000 7fbba152b640 1 --2- 172.21.15.148:0/2170734305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb9c14d4d0 0x7fbb9c14d950 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbb98000c00 tx=0x7fbb98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:12.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.523+0000 7fbb927fc640 1 -- 172.21.15.148:0/2170734305 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98042020 con 0x7fbb9c14d4d0 2024-03-29T04:11:12.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.523+0000 7fbb927fc640 1 -- 172.21.15.148:0/2170734305 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb9803a040 con 0x7fbb9c14d4d0 2024-03-29T04:11:12.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.523+0000 7fbba3f90640 1 -- 172.21.15.148:0/2170734305 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb9c0c5ec0 con 0x7fbb9c14d4d0 2024-03-29T04:11:12.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.523+0000 7fbb927fc640 1 -- 172.21.15.148:0/2170734305 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98039070 con 0x7fbb9c14d4d0 2024-03-29T04:11:12.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.523+0000 7fbba3f90640 1 -- 172.21.15.148:0/2170734305 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb9c14de90 con 0x7fbb9c14d4d0 2024-03-29T04:11:12.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.524+0000 7fbba3f90640 1 -- 172.21.15.148:0/2170734305 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb9c0c65e0 con 0x7fbb9c14d4d0 2024-03-29T04:11:12.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.525+0000 7fbb927fc640 1 -- 172.21.15.148:0/2170734305 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbb98045020 con 0x7fbb9c14d4d0 2024-03-29T04:11:12.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.525+0000 7fbb927fc640 1 --2- 172.21.15.148:0/2170734305 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbb8006c530 0x7fbb8006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:12.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.525+0000 7fbb927fc640 1 -- 172.21.15.148:0/2170734305 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fbb980ace20 con 0x7fbb9c14d4d0 2024-03-29T04:11:12.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.525+0000 7fbba1d2c640 1 --2- 172.21.15.148:0/2170734305 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbb8006c530 0x7fbb8006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:12.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.526+0000 7fbba1d2c640 1 --2- 172.21.15.148:0/2170734305 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbb8006c530 0x7fbb8006e9f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0036b0 tx=0x7fbb8c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:12.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.530+0000 7fbb927fc640 1 -- 172.21.15.148:0/2170734305 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbb9804c070 con 0x7fbb9c14d4d0 2024-03-29T04:11:12.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.734+0000 7fbba3f90640 1 -- 172.21.15.148:0/2170734305 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbb9c0d1630 con 0x7fbb9c14d4d0 2024-03-29T04:11:12.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.734+0000 7fbb927fc640 1 -- 172.21.15.148:0/2170734305 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fbb9807b0d0 con 0x7fbb9c14d4d0 2024-03-29T04:11:12.713 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:11:12.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.736+0000 7fbba3f90640 1 -- 172.21.15.148:0/2170734305 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbb8006c530 msgr2=0x7fbb8006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:12.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.736+0000 7fbba3f90640 1 --2- 172.21.15.148:0/2170734305 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbb8006c530 0x7fbb8006e9f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0036b0 tx=0x7fbb8c009040 comp rx=0 tx=0).stop 2024-03-29T04:11:12.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.736+0000 7fbba3f90640 1 -- 172.21.15.148:0/2170734305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb9c14d4d0 msgr2=0x7fbb9c14d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:12.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.736+0000 7fbba3f90640 1 --2- 172.21.15.148:0/2170734305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb9c14d4d0 0x7fbb9c14d950 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbb98000c00 tx=0x7fbb98004730 comp rx=0 tx=0).stop 2024-03-29T04:11:12.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.736+0000 7fbba3f90640 1 -- 172.21.15.148:0/2170734305 shutdown_connections 2024-03-29T04:11:12.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.736+0000 7fbba3f90640 1 --2- 172.21.15.148:0/2170734305 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbb8006c530 0x7fbb8006e9f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:12.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.736+0000 7fbba3f90640 1 --2- 172.21.15.148:0/2170734305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb9c0c9e40 0x7fbb9c0ca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:12.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.736+0000 7fbba3f90640 1 --2- 172.21.15.148:0/2170734305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb9c14d4d0 0x7fbb9c14d950 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:12.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.736+0000 7fbba3f90640 1 --2- 172.21.15.148:0/2170734305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb88003f60 0x7fbb9c150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:12.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.736+0000 7fbba3f90640 1 -- 172.21.15.148:0/2170734305 >> 172.21.15.148:0/2170734305 conn(0x7fbb9c0b3c70 msgr2=0x7fbb9c0ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:12.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.737+0000 7fbba3f90640 1 -- 172.21.15.148:0/2170734305 shutdown_connections 2024-03-29T04:11:12.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.737+0000 7fbba3f90640 1 -- 172.21.15.148:0/2170734305 wait complete. 2024-03-29T04:11:12.727 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:11:12.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.853+0000 7f8776cf2640 1 Processor -- start 2024-03-29T04:11:12.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.853+0000 7f8776cf2640 1 -- start start 2024-03-29T04:11:12.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.854+0000 7f8776cf2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f87700b3430 0x7f8770162680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:12.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.854+0000 7f8776cf2640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87700cf330 con 0x7f877015e3c0 2024-03-29T04:11:12.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.854+0000 7f8776cf2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8770162cf0 con 0x7f87700b2f00 2024-03-29T04:11:12.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.854+0000 7f8776cf2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f87700cf960 con 0x7f87700b3430 2024-03-29T04:11:12.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.854+0000 7f8774a8e640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f877015e3c0 0x7f8770161f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35836/0 (socket says 172.21.15.148:35836) 2024-03-29T04:11:12.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.854+0000 7f8774a8e640 1 -- 172.21.15.148:0/1683073413 learned_addr learned my addr 172.21.15.148:0/1683073413 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:12.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.854+0000 7f877528f640 1 --2- 172.21.15.148:0/1683073413 >> v2:172.21.15.157:3300/0 conn(0x7f87700b3430 0x7f8770162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:12.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.854+0000 7f876f7fe640 1 -- 172.21.15.148:0/1683073413 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1870349720 0 0) 0x7f87700cf330 con 0x7f877015e3c0 2024-03-29T04:11:12.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.854+0000 7f876f7fe640 1 -- 172.21.15.148:0/1683073413 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8758003520 con 0x7f877015e3c0 2024-03-29T04:11:12.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.855+0000 7f876f7fe640 1 -- 172.21.15.148:0/1683073413 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 308159538 0 0) 0x7f8770162cf0 con 0x7f87700b2f00 2024-03-29T04:11:12.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.855+0000 7f876f7fe640 1 -- 172.21.15.148:0/1683073413 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87700cf330 con 0x7f87700b2f00 2024-03-29T04:11:12.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.855+0000 7f876f7fe640 1 -- 172.21.15.148:0/1683073413 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3020690805 0 0) 0x7f8758003520 con 0x7f877015e3c0 2024-03-29T04:11:12.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.855+0000 7f877528f640 1 -- 172.21.15.148:0/1683073413 >> v1:172.21.15.157:6789/0 conn(0x7f87700b2f00 legacy=0x7f877006b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:12.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.855+0000 7f877528f640 1 -- 172.21.15.148:0/1683073413 >> v1:172.21.15.148:6789/0 conn(0x7f877015e3c0 legacy=0x7f8770161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:12.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.855+0000 7f877528f640 1 -- 172.21.15.148:0/1683073413 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87700cf5f0 con 0x7f87700b3430 2024-03-29T04:11:12.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.856+0000 7f877528f640 1 --2- 172.21.15.148:0/1683073413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87700b3430 0x7f8770162680 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8768009d90 tx=0x7f876802bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c4ddbac85eb240c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:12.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.856+0000 7f876f7fe640 1 -- 172.21.15.148:0/1683073413 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8768039070 con 0x7f87700b3430 2024-03-29T04:11:12.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.856+0000 7f876f7fe640 1 -- 172.21.15.148:0/1683073413 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f876802c8b0 con 0x7f87700b3430 2024-03-29T04:11:12.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.856+0000 7f876f7fe640 1 -- 172.21.15.148:0/1683073413 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8760001670 con 0x7f87700b3430 2024-03-29T04:11:12.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.856+0000 7f8776cf2640 1 -- 172.21.15.148:0/1683073413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87700b3430 msgr2=0x7f8770162680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:12.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.856+0000 7f8776cf2640 1 --2- 172.21.15.148:0/1683073413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87700b3430 0x7f8770162680 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8768009d90 tx=0x7f876802bd80 comp rx=0 tx=0).stop 2024-03-29T04:11:12.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.856+0000 7f8776cf2640 1 -- 172.21.15.148:0/1683073413 shutdown_connections 2024-03-29T04:11:12.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.856+0000 7f8776cf2640 1 --2- 172.21.15.148:0/1683073413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87700b3430 0x7f8770162680 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:12.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.856+0000 7f8776cf2640 1 -- 172.21.15.148:0/1683073413 >> 172.21.15.148:0/1683073413 conn(0x7f87700b6430 msgr2=0x7f87700d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:12.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.856+0000 7f8776cf2640 1 -- 172.21.15.148:0/1683073413 shutdown_connections 2024-03-29T04:11:12.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.856+0000 7f8776cf2640 1 -- 172.21.15.148:0/1683073413 wait complete. 2024-03-29T04:11:12.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.857+0000 7f8776cf2640 1 Processor -- start 2024-03-29T04:11:12.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.857+0000 7f8776cf2640 1 -- start start 2024-03-29T04:11:12.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.858+0000 7f8776cf2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87700b3430 0x7f87700cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:12.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.858+0000 7f8776cf2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f877015e3c0 0x7f8770160950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:12.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.858+0000 7f8776cf2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87700cb3a0 0x7f8770152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:12.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.858+0000 7f8776cf2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f87700b3b20 con 0x7f87700cb3a0 2024-03-29T04:11:12.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.858+0000 7f8776cf2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f87700b39a0 con 0x7f87700b3430 2024-03-29T04:11:12.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.858+0000 7f8776cf2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f877006bf00 con 0x7f877015e3c0 2024-03-29T04:11:12.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.858+0000 7f877528f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87700cb3a0 0x7f8770152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:12.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.858+0000 7f877528f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87700cb3a0 0x7f8770152520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43410/0 (socket says 172.21.15.148:43410) 2024-03-29T04:11:12.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.858+0000 7f877528f640 1 -- 172.21.15.148:0/4143107391 learned_addr learned my addr 172.21.15.148:0/4143107391 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:12.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.858+0000 7f8774a8e640 1 --2- 172.21.15.148:0/4143107391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87700b3430 0x7f87700cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:12.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.859+0000 7f876ffff640 1 --2- 172.21.15.148:0/4143107391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f877015e3c0 0x7f8770160950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:12.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.859+0000 7f877528f640 1 -- 172.21.15.148:0/4143107391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f877015e3c0 msgr2=0x7f8770160950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:12.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.859+0000 7f877528f640 1 --2- 172.21.15.148:0/4143107391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f877015e3c0 0x7f8770160950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:12.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.859+0000 7f877528f640 1 -- 172.21.15.148:0/4143107391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87700b3430 msgr2=0x7f87700cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:12.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.859+0000 7f877528f640 1 --2- 172.21.15.148:0/4143107391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87700b3430 0x7f87700cc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:12.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.859+0000 7f877528f640 1 -- 172.21.15.148:0/4143107391 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87700cb940 con 0x7f87700cb3a0 2024-03-29T04:11:12.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.859+0000 7f877528f640 1 --2- 172.21.15.148:0/4143107391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87700cb3a0 0x7f8770152520 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8768009eb0 tx=0x7f8768004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:12.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.859+0000 7f876d7fa640 1 -- 172.21.15.148:0/4143107391 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8768039050 con 0x7f87700cb3a0 2024-03-29T04:11:12.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.859+0000 7f876d7fa640 1 -- 172.21.15.148:0/4143107391 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8768045070 con 0x7f87700cb3a0 2024-03-29T04:11:12.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.859+0000 7f876d7fa640 1 -- 172.21.15.148:0/4143107391 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8768009450 con 0x7f87700cb3a0 2024-03-29T04:11:12.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.859+0000 7f8776cf2640 1 -- 172.21.15.148:0/4143107391 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87700cbc20 con 0x7f87700cb3a0 2024-03-29T04:11:12.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.859+0000 7f8776cf2640 1 -- 172.21.15.148:0/4143107391 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87700cc1e0 con 0x7f87700cb3a0 2024-03-29T04:11:12.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.860+0000 7f8776cf2640 1 -- 172.21.15.148:0/4143107391 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8770068850 con 0x7f87700cb3a0 2024-03-29T04:11:12.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.861+0000 7f876d7fa640 1 -- 172.21.15.148:0/4143107391 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8768040690 con 0x7f87700cb3a0 2024-03-29T04:11:12.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.862+0000 7f876d7fa640 1 --2- 172.21.15.148:0/4143107391 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f875c06c560 0x7f875c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:12.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.862+0000 7f876d7fa640 1 -- 172.21.15.148:0/4143107391 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f87680ae190 con 0x7f87700cb3a0 2024-03-29T04:11:12.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.862+0000 7f8774a8e640 1 --2- 172.21.15.148:0/4143107391 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f875c06c560 0x7f875c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:12.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.862+0000 7f8774a8e640 1 --2- 172.21.15.148:0/4143107391 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f875c06c560 0x7f875c06ea20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8760000fd0 tx=0x7f8760001150 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:12.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:12.866+0000 7f876d7fa640 1 -- 172.21.15.148:0/4143107391 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f876807be20 con 0x7f87700cb3a0 2024-03-29T04:11:13.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.047+0000 7f8776cf2640 1 -- 172.21.15.148:0/4143107391 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f87700b9750 con 0x7f87700cb3a0 2024-03-29T04:11:13.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.048+0000 7f876d7fa640 1 -- 172.21.15.148:0/4143107391 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f8768080120 con 0x7f87700cb3a0 2024-03-29T04:11:13.027 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-29T04:11:13.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.049+0000 7f8776cf2640 1 -- 172.21.15.148:0/4143107391 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f875c06c560 msgr2=0x7f875c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:13.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.049+0000 7f8776cf2640 1 --2- 172.21.15.148:0/4143107391 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f875c06c560 0x7f875c06ea20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8760000fd0 tx=0x7f8760001150 comp rx=0 tx=0).stop 2024-03-29T04:11:13.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.049+0000 7f8776cf2640 1 -- 172.21.15.148:0/4143107391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87700cb3a0 msgr2=0x7f8770152520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:13.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.049+0000 7f8776cf2640 1 --2- 172.21.15.148:0/4143107391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87700cb3a0 0x7f8770152520 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8768009eb0 tx=0x7f8768004a40 comp rx=0 tx=0).stop 2024-03-29T04:11:13.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.050+0000 7f8776cf2640 1 -- 172.21.15.148:0/4143107391 shutdown_connections 2024-03-29T04:11:13.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.050+0000 7f8776cf2640 1 --2- 172.21.15.148:0/4143107391 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f875c06c560 0x7f875c06ea20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.050+0000 7f8776cf2640 1 --2- 172.21.15.148:0/4143107391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87700cb3a0 0x7f8770152520 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.050+0000 7f8776cf2640 1 --2- 172.21.15.148:0/4143107391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f877015e3c0 0x7f8770160950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.050+0000 7f8776cf2640 1 --2- 172.21.15.148:0/4143107391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87700b3430 0x7f87700cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.050+0000 7f8776cf2640 1 -- 172.21.15.148:0/4143107391 >> 172.21.15.148:0/4143107391 conn(0x7f87700b6430 msgr2=0x7f8770161c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:13.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.050+0000 7f8776cf2640 1 -- 172.21.15.148:0/4143107391 shutdown_connections 2024-03-29T04:11:13.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.050+0000 7f8776cf2640 1 -- 172.21.15.148:0/4143107391 wait complete. 2024-03-29T04:11:13.040 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:11:13.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.164+0000 7f5f402c8640 1 Processor -- start 2024-03-29T04:11:13.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.164+0000 7f5f402c8640 1 -- start start 2024-03-29T04:11:13.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.165+0000 7f5f402c8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5f380b2f00 0x7f5f38162680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:13.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.165+0000 7f5f402c8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5f380b3430 0x7f5f3806b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:13.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.165+0000 7f5f402c8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f380cf330 con 0x7f5f3815e3c0 2024-03-29T04:11:13.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.165+0000 7f5f402c8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f5f380cf960 con 0x7f5f380b2f00 2024-03-29T04:11:13.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.165+0000 7f5f402c8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5f38162cf0 con 0x7f5f380b3430 2024-03-29T04:11:13.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.165+0000 7f5f3d863640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5f380b2f00 0x7f5f38162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:13.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.165+0000 7f5f3d863640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5f380b2f00 0x7f5f38162680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43426/0 (socket says 172.21.15.148:43426) 2024-03-29T04:11:13.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.165+0000 7f5f3d863640 1 -- 172.21.15.148:0/3483825739 learned_addr learned my addr 172.21.15.148:0/3483825739 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:13.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.166+0000 7f5f3e865640 1 --2- 172.21.15.148:0/3483825739 >> v2:172.21.15.157:3300/0 conn(0x7f5f380b3430 0x7f5f3806b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:13.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.166+0000 7f5f3d863640 1 -- 172.21.15.148:0/3483825739 >> v2:172.21.15.157:3300/0 conn(0x7f5f380b3430 msgr2=0x7f5f3806b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:13.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.166+0000 7f5f3d863640 1 --2- 172.21.15.148:0/3483825739 >> v2:172.21.15.157:3300/0 conn(0x7f5f380b3430 0x7f5f3806b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.166+0000 7f5f3d863640 1 -- 172.21.15.148:0/3483825739 >> v1:172.21.15.157:6789/0 conn(0x7f5f3815e3c0 legacy=0x7f5f38161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:13.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.166+0000 7f5f3d863640 1 -- 172.21.15.148:0/3483825739 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f380cf5f0 con 0x7f5f380b2f00 2024-03-29T04:11:13.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.166+0000 7f5f3d863640 1 --2- 172.21.15.148:0/3483825739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f380b2f00 0x7f5f38162680 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5f380cf8d0 tx=0x7f5f2c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2db3c618348b10c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:13.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.167+0000 7f5f3d062640 1 -- 172.21.15.148:0/3483825739 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c02e070 con 0x7f5f380b2f00 2024-03-29T04:11:13.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.167+0000 7f5f3d062640 1 -- 172.21.15.148:0/3483825739 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f2c037070 con 0x7f5f380b2f00 2024-03-29T04:11:13.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.167+0000 7f5f3d062640 1 -- 172.21.15.148:0/3483825739 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c031a00 con 0x7f5f380b2f00 2024-03-29T04:11:13.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.167+0000 7f5f402c8640 1 -- 172.21.15.148:0/3483825739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f380b2f00 msgr2=0x7f5f38162680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:13.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.167+0000 7f5f402c8640 1 --2- 172.21.15.148:0/3483825739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f380b2f00 0x7f5f38162680 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5f380cf8d0 tx=0x7f5f2c02b880 comp rx=0 tx=0).stop 2024-03-29T04:11:13.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.167+0000 7f5f402c8640 1 -- 172.21.15.148:0/3483825739 shutdown_connections 2024-03-29T04:11:13.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.167+0000 7f5f402c8640 1 --2- 172.21.15.148:0/3483825739 >> v2:172.21.15.157:3300/0 conn(0x7f5f380b3430 0x7f5f3806b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.167+0000 7f5f402c8640 1 --2- 172.21.15.148:0/3483825739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f380b2f00 0x7f5f38162680 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.167+0000 7f5f402c8640 1 -- 172.21.15.148:0/3483825739 >> 172.21.15.148:0/3483825739 conn(0x7f5f380b6430 msgr2=0x7f5f380d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:13.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.167+0000 7f5f402c8640 1 -- 172.21.15.148:0/3483825739 shutdown_connections 2024-03-29T04:11:13.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.167+0000 7f5f402c8640 1 -- 172.21.15.148:0/3483825739 wait complete. 2024-03-29T04:11:13.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.168+0000 7f5f402c8640 1 Processor -- start 2024-03-29T04:11:13.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.169+0000 7f5f402c8640 1 -- start start 2024-03-29T04:11:13.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.169+0000 7f5f402c8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f380b3430 0x7f5f380c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:13.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.169+0000 7f5f402c8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f3815e3c0 0x7f5f380cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:13.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.169+0000 7f5f402c8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f380cdfc0 0x7f5f380cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:13.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.169+0000 7f5f402c8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5f380b3b20 con 0x7f5f380cdfc0 2024-03-29T04:11:13.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.169+0000 7f5f402c8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5f380b39a0 con 0x7f5f3815e3c0 2024-03-29T04:11:13.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.169+0000 7f5f402c8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5f3806bd30 con 0x7f5f380b3430 2024-03-29T04:11:13.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.169+0000 7f5f3e865640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f380cdfc0 0x7f5f380cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:13.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.169+0000 7f5f3e865640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f380cdfc0 0x7f5f380cb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43428/0 (socket says 172.21.15.148:43428) 2024-03-29T04:11:13.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.169+0000 7f5f3e865640 1 -- 172.21.15.148:0/2575331184 learned_addr learned my addr 172.21.15.148:0/2575331184 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:13.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.169+0000 7f5f3d863640 1 --2- 172.21.15.148:0/2575331184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f3815e3c0 0x7f5f380cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:13.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.169+0000 7f5f3e064640 1 --2- 172.21.15.148:0/2575331184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f380b3430 0x7f5f380c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:13.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.170+0000 7f5f3e865640 1 -- 172.21.15.148:0/2575331184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f380b3430 msgr2=0x7f5f380c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:13.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.170+0000 7f5f3e865640 1 --2- 172.21.15.148:0/2575331184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f380b3430 0x7f5f380c7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.170+0000 7f5f3e865640 1 -- 172.21.15.148:0/2575331184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f3815e3c0 msgr2=0x7f5f380cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:13.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.170+0000 7f5f3e865640 1 --2- 172.21.15.148:0/2575331184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f3815e3c0 0x7f5f380cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.170+0000 7f5f3e865640 1 -- 172.21.15.148:0/2575331184 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f380c8700 con 0x7f5f380cdfc0 2024-03-29T04:11:13.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.170+0000 7f5f3e865640 1 --2- 172.21.15.148:0/2575331184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f380cdfc0 0x7f5f380cb8e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000cd70 tx=0x7f5f3000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:13.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.170+0000 7f5f36ffd640 1 -- 172.21.15.148:0/2575331184 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30003b90 con 0x7f5f380cdfc0 2024-03-29T04:11:13.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.170+0000 7f5f36ffd640 1 -- 172.21.15.148:0/2575331184 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f30003d30 con 0x7f5f380cdfc0 2024-03-29T04:11:13.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.170+0000 7f5f402c8640 1 -- 172.21.15.148:0/2575331184 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f380c89e0 con 0x7f5f380cdfc0 2024-03-29T04:11:13.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.170+0000 7f5f36ffd640 1 -- 172.21.15.148:0/2575331184 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30004240 con 0x7f5f380cdfc0 2024-03-29T04:11:13.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.170+0000 7f5f402c8640 1 -- 172.21.15.148:0/2575331184 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f380c8fa0 con 0x7f5f380cdfc0 2024-03-29T04:11:13.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.171+0000 7f5f402c8640 1 -- 172.21.15.148:0/2575331184 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f38068850 con 0x7f5f380cdfc0 2024-03-29T04:11:13.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.172+0000 7f5f36ffd640 1 -- 172.21.15.148:0/2575331184 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5f300043e0 con 0x7f5f380cdfc0 2024-03-29T04:11:13.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.172+0000 7f5f36ffd640 1 --2- 172.21.15.148:0/2575331184 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5f2406c530 0x7f5f2406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:13.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.172+0000 7f5f36ffd640 1 -- 172.21.15.148:0/2575331184 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f5f3008c430 con 0x7f5f380cdfc0 2024-03-29T04:11:13.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.172+0000 7f5f3e064640 1 --2- 172.21.15.148:0/2575331184 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5f2406c530 0x7f5f2406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:13.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.172+0000 7f5f3e064640 1 --2- 172.21.15.148:0/2575331184 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5f2406c530 0x7f5f2406e9f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5f380d1830 tx=0x7f5f28009060 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:13.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.176+0000 7f5f36ffd640 1 -- 172.21.15.148:0/2575331184 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5f3005a090 con 0x7f5f380cdfc0 2024-03-29T04:11:13.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.350+0000 7f5f402c8640 1 -- 172.21.15.148:0/2575331184 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5f380cf110 con 0x7f5f380cdfc0 2024-03-29T04:11:13.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.351+0000 7f5f36ffd640 1 -- 172.21.15.148:0/2575331184 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f5f3005e390 con 0x7f5f380cdfc0 2024-03-29T04:11:13.330 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:11:13.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.353+0000 7f5f402c8640 1 -- 172.21.15.148:0/2575331184 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5f2406c530 msgr2=0x7f5f2406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:13.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.353+0000 7f5f402c8640 1 --2- 172.21.15.148:0/2575331184 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5f2406c530 0x7f5f2406e9f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5f380d1830 tx=0x7f5f28009060 comp rx=0 tx=0).stop 2024-03-29T04:11:13.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.353+0000 7f5f402c8640 1 -- 172.21.15.148:0/2575331184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f380cdfc0 msgr2=0x7f5f380cb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:13.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.353+0000 7f5f402c8640 1 --2- 172.21.15.148:0/2575331184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f380cdfc0 0x7f5f380cb8e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000cd70 tx=0x7f5f3000a660 comp rx=0 tx=0).stop 2024-03-29T04:11:13.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.353+0000 7f5f402c8640 1 -- 172.21.15.148:0/2575331184 shutdown_connections 2024-03-29T04:11:13.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.353+0000 7f5f402c8640 1 --2- 172.21.15.148:0/2575331184 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5f2406c530 0x7f5f2406e9f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.353+0000 7f5f402c8640 1 --2- 172.21.15.148:0/2575331184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f380cdfc0 0x7f5f380cb8e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.353+0000 7f5f402c8640 1 --2- 172.21.15.148:0/2575331184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f3815e3c0 0x7f5f380cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.353+0000 7f5f402c8640 1 --2- 172.21.15.148:0/2575331184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f380b3430 0x7f5f380c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.353+0000 7f5f402c8640 1 -- 172.21.15.148:0/2575331184 >> 172.21.15.148:0/2575331184 conn(0x7f5f380b6430 msgr2=0x7f5f380d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:13.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.353+0000 7f5f402c8640 1 -- 172.21.15.148:0/2575331184 shutdown_connections 2024-03-29T04:11:13.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.353+0000 7f5f402c8640 1 -- 172.21.15.148:0/2575331184 wait complete. 2024-03-29T04:11:13.343 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:11:13.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.465+0000 7f01a4b72640 1 Processor -- start 2024-03-29T04:11:13.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.466+0000 7f01a4b72640 1 -- start start 2024-03-29T04:11:13.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.467+0000 7f01a4b72640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f01a015e3e0 0x7f01a00d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:13.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.467+0000 7f01a4b72640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f01a00ba030 0x7f01a00d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:13.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.467+0000 7f01a4b72640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01a00d10d0 0x7f01a00d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:13.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.467+0000 7f01a4b72640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f01a00d3840 con 0x7f01a00ba030 2024-03-29T04:11:13.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.467+0000 7f01a4b72640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f01a015e7c0 con 0x7f01a015e3e0 2024-03-29T04:11:13.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.467+0000 7f01a4b72640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f01a00d2c30 con 0x7f01a00d10d0 2024-03-29T04:11:13.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.467+0000 7f0197fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f01a00ba030 0x7f01a00d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:13.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.467+0000 7f0197fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f01a00ba030 0x7f01a00d2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43438/0 (socket says 172.21.15.148:43438) 2024-03-29T04:11:13.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.467+0000 7f0197fff640 1 -- 172.21.15.148:0/784409750 learned_addr learned my addr 172.21.15.148:0/784409750 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:13.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.467+0000 7f019ed9d640 1 --2- 172.21.15.148:0/784409750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01a00d10d0 0x7f01a00d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:13.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.467+0000 7f019e59c640 1 --2- 172.21.15.148:0/784409750 >> v2:172.21.15.157:3300/0 conn(0x7f01a015e3e0 0x7f01a00d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:13.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.467+0000 7f0197fff640 1 -- 172.21.15.148:0/784409750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01a00d10d0 msgr2=0x7f01a00d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:13.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.467+0000 7f0197fff640 1 --2- 172.21.15.148:0/784409750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01a00d10d0 0x7f01a00d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.467+0000 7f0197fff640 1 -- 172.21.15.148:0/784409750 >> v2:172.21.15.157:3300/0 conn(0x7f01a015e3e0 msgr2=0x7f01a00d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:13.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.467+0000 7f0197fff640 1 --2- 172.21.15.148:0/784409750 >> v2:172.21.15.157:3300/0 conn(0x7f01a015e3e0 0x7f01a00d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.467+0000 7f0197fff640 1 -- 172.21.15.148:0/784409750 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a00d2f10 con 0x7f01a00ba030 2024-03-29T04:11:13.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.467+0000 7f019e59c640 1 --2- 172.21.15.148:0/784409750 >> v2:172.21.15.157:3300/0 conn(0x7f01a015e3e0 0x7f01a00d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:13.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.468+0000 7f0197fff640 1 --2- 172.21.15.148:0/784409750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01a00ba030 0x7f01a00d2330 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f01a00b4eb0 tx=0x7f018802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0fc138ea6e2556c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:13.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.468+0000 7f019dd9b640 1 -- 172.21.15.148:0/784409750 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0188034070 con 0x7f01a00ba030 2024-03-29T04:11:13.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.468+0000 7f019dd9b640 1 -- 172.21.15.148:0/784409750 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0188030860 con 0x7f01a00ba030 2024-03-29T04:11:13.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.468+0000 7f019dd9b640 1 -- 172.21.15.148:0/784409750 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0188030c30 con 0x7f01a00ba030 2024-03-29T04:11:13.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.469+0000 7f01a4b72640 1 -- 172.21.15.148:0/784409750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01a00ba030 msgr2=0x7f01a00d2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:13.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.469+0000 7f01a4b72640 1 --2- 172.21.15.148:0/784409750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01a00ba030 0x7f01a00d2330 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f01a00b4eb0 tx=0x7f018802b9a0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.469+0000 7f01a4b72640 1 -- 172.21.15.148:0/784409750 shutdown_connections 2024-03-29T04:11:13.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.469+0000 7f01a4b72640 1 --2- 172.21.15.148:0/784409750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01a00ba030 0x7f01a00d2330 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.469+0000 7f01a4b72640 1 --2- 172.21.15.148:0/784409750 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01a00d10d0 0x7f01a00d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.469+0000 7f01a4b72640 1 --2- 172.21.15.148:0/784409750 >> v2:172.21.15.157:3300/0 conn(0x7f01a015e3e0 0x7f01a00d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.469+0000 7f01a4b72640 1 -- 172.21.15.148:0/784409750 >> 172.21.15.148:0/784409750 conn(0x7f01a00b6210 msgr2=0x7f01a00b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:13.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.469+0000 7f01a4b72640 1 -- 172.21.15.148:0/784409750 shutdown_connections 2024-03-29T04:11:13.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.469+0000 7f01a4b72640 1 -- 172.21.15.148:0/784409750 wait complete. 2024-03-29T04:11:13.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.470+0000 7f01a4b72640 1 Processor -- start 2024-03-29T04:11:13.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.471+0000 7f01a4b72640 1 -- start start 2024-03-29T04:11:13.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.472+0000 7f01a4b72640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01a00ba030 0x7f01a0154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:13.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.472+0000 7f01a4b72640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01a00d10d0 0x7f01a00c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:13.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.472+0000 7f01a4b72640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a015e3e0 0x7f01a01587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:13.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.472+0000 7f01a4b72640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f01a00d2870 con 0x7f01a00d10d0 2024-03-29T04:11:13.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.472+0000 7f01a4b72640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f01a00d34d0 con 0x7f01a015e3e0 2024-03-29T04:11:13.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.472+0000 7f01a4b72640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f01a00d29f0 con 0x7f01a00ba030 2024-03-29T04:11:13.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.472+0000 7f0197fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01a00d10d0 0x7f01a00c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:13.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.472+0000 7f0197fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01a00d10d0 0x7f01a00c9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43446/0 (socket says 172.21.15.148:43446) 2024-03-29T04:11:13.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.472+0000 7f0197fff640 1 -- 172.21.15.148:0/2846752104 learned_addr learned my addr 172.21.15.148:0/2846752104 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:13.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.472+0000 7f019e59c640 1 --2- 172.21.15.148:0/2846752104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01a00ba030 0x7f01a0154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:13.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.472+0000 7f019ed9d640 1 --2- 172.21.15.148:0/2846752104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a015e3e0 0x7f01a01587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:13.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.472+0000 7f0197fff640 1 -- 172.21.15.148:0/2846752104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01a00ba030 msgr2=0x7f01a0154210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:13.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.472+0000 7f0197fff640 1 --2- 172.21.15.148:0/2846752104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01a00ba030 0x7f01a0154210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.472+0000 7f0197fff640 1 -- 172.21.15.148:0/2846752104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a015e3e0 msgr2=0x7f01a01587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:13.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.472+0000 7f0197fff640 1 --2- 172.21.15.148:0/2846752104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a015e3e0 0x7f01a01587e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.472+0000 7f0197fff640 1 -- 172.21.15.148:0/2846752104 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a01547f0 con 0x7f01a00d10d0 2024-03-29T04:11:13.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.472+0000 7f0197fff640 1 --2- 172.21.15.148:0/2846752104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01a00d10d0 0x7f01a00c9f90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0188000c00 tx=0x7f0188004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:13.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.472+0000 7f0196ffd640 1 -- 172.21.15.148:0/2846752104 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0188048070 con 0x7f01a00d10d0 2024-03-29T04:11:13.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.472+0000 7f0196ffd640 1 -- 172.21.15.148:0/2846752104 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0188030660 con 0x7f01a00d10d0 2024-03-29T04:11:13.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.472+0000 7f01a4b72640 1 -- 172.21.15.148:0/2846752104 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01a0154ad0 con 0x7f01a00d10d0 2024-03-29T04:11:13.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.473+0000 7f0196ffd640 1 -- 172.21.15.148:0/2846752104 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0188034040 con 0x7f01a00d10d0 2024-03-29T04:11:13.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.473+0000 7f01a4b72640 1 -- 172.21.15.148:0/2846752104 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01a0155090 con 0x7f01a00d10d0 2024-03-29T04:11:13.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.474+0000 7f01a4b72640 1 -- 172.21.15.148:0/2846752104 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01a015a6d0 con 0x7f01a00d10d0 2024-03-29T04:11:13.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.474+0000 7f0196ffd640 1 -- 172.21.15.148:0/2846752104 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f018804b070 con 0x7f01a00d10d0 2024-03-29T04:11:13.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.475+0000 7f0196ffd640 1 --2- 172.21.15.148:0/2846752104 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f018006c530 0x7f018006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:13.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.475+0000 7f019e59c640 1 --2- 172.21.15.148:0/2846752104 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f018006c530 0x7f018006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:13.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.475+0000 7f0196ffd640 1 -- 172.21.15.148:0/2846752104 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f01880adb20 con 0x7f01a00d10d0 2024-03-29T04:11:13.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.475+0000 7f019e59c640 1 --2- 172.21.15.148:0/2846752104 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f018006c530 0x7f018006e9f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f01a00cdd50 tx=0x7f0190008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:13.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.479+0000 7f0196ffd640 1 -- 172.21.15.148:0/2846752104 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f018807bdd0 con 0x7f01a00d10d0 2024-03-29T04:11:13.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.755+0000 7f01a4b72640 1 -- 172.21.15.148:0/2846752104 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01a00ce870 con 0x7f01a00d10d0 2024-03-29T04:11:13.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.755+0000 7f0196ffd640 1 -- 172.21.15.148:0/2846752104 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f018807b6d0 con 0x7f01a00d10d0 2024-03-29T04:11:13.734 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:11:13.735 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/36660496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":36660496},{"type":"v1","addr":"172.21.15.157:6834","nonce":36660496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2674481055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2674481055},{"type":"v1","addr":"172.21.15.157:6835","nonce":2674481055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3356449643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3356449643},{"type":"v1","addr":"172.21.15.148:6836","nonce":3356449643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:09:59.886367+0000","modified":"2024-03-29T04:10:22.957623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/2453181817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2453181817},{"type":"v1","addr":"172.21.15.148:6837","nonce":2453181817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-03-29T04:11:13.735 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 9 2024-03-29T04:11:13.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.757+0000 7f01a4b72640 1 -- 172.21.15.148:0/2846752104 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f018006c530 msgr2=0x7f018006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:13.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.757+0000 7f01a4b72640 1 --2- 172.21.15.148:0/2846752104 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f018006c530 0x7f018006e9f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f01a00cdd50 tx=0x7f0190008040 comp rx=0 tx=0).stop 2024-03-29T04:11:13.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.757+0000 7f01a4b72640 1 -- 172.21.15.148:0/2846752104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01a00d10d0 msgr2=0x7f01a00c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:13.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.757+0000 7f01a4b72640 1 --2- 172.21.15.148:0/2846752104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01a00d10d0 0x7f01a00c9f90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0188000c00 tx=0x7f0188004730 comp rx=0 tx=0).stop 2024-03-29T04:11:13.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.757+0000 7f01a4b72640 1 -- 172.21.15.148:0/2846752104 shutdown_connections 2024-03-29T04:11:13.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.757+0000 7f01a4b72640 1 --2- 172.21.15.148:0/2846752104 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f018006c530 0x7f018006e9f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.757+0000 7f01a4b72640 1 --2- 172.21.15.148:0/2846752104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a015e3e0 0x7f01a01587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.757+0000 7f01a4b72640 1 --2- 172.21.15.148:0/2846752104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01a00d10d0 0x7f01a00c9f90 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.757+0000 7f01a4b72640 1 --2- 172.21.15.148:0/2846752104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01a00ba030 0x7f01a0154210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.757+0000 7f01a4b72640 1 -- 172.21.15.148:0/2846752104 >> 172.21.15.148:0/2846752104 conn(0x7f01a00b6210 msgr2=0x7f01a00b6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:13.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.757+0000 7f01a4b72640 1 -- 172.21.15.148:0/2846752104 shutdown_connections 2024-03-29T04:11:13.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.757+0000 7f01a4b72640 1 -- 172.21.15.148:0/2846752104 wait complete. 2024-03-29T04:11:13.748 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:11:13.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.897+0000 7f1b99ac5640 1 Processor -- start 2024-03-29T04:11:13.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.897+0000 7f1b99ac5640 1 -- start start 2024-03-29T04:11:13.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.897+0000 7f1b99ac5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1b9415e3e0 0x7f1b940d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:13.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.897+0000 7f1b99ac5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b940d10d0 0x7f1b940d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:13.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.897+0000 7f1b99ac5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b9415e7c0 con 0x7f1b940ba030 2024-03-29T04:11:13.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.897+0000 7f1b99ac5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f1b940d3840 con 0x7f1b9415e3e0 2024-03-29T04:11:13.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.897+0000 7f1b99ac5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1b940d2c30 con 0x7f1b940d10d0 2024-03-29T04:11:13.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.898+0000 7f1b92ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f1b940ba030 0x7f1b940d2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35838/0 (socket says 172.21.15.148:35838) 2024-03-29T04:11:13.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.898+0000 7f1b92ffd640 1 -- 172.21.15.148:0/2656124308 learned_addr learned my addr 172.21.15.148:0/2656124308 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:13.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.898+0000 7f1b93fff640 1 --2- 172.21.15.148:0/2656124308 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b940d10d0 0x7f1b940d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:13.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.898+0000 7f1b937fe640 1 --2- 172.21.15.148:0/2656124308 >> v2:172.21.15.157:3300/0 conn(0x7f1b9415e3e0 0x7f1b940d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:13.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.898+0000 7f1b927fc640 1 -- 172.21.15.148:0/2656124308 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1005195378 0 0) 0x7f1b9415e7c0 con 0x7f1b940ba030 2024-03-29T04:11:13.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.898+0000 7f1b927fc640 1 -- 172.21.15.148:0/2656124308 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b7c003520 con 0x7f1b940ba030 2024-03-29T04:11:13.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.898+0000 7f1b927fc640 1 -- 172.21.15.148:0/2656124308 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3880969425 0 0) 0x7f1b7c003520 con 0x7f1b940ba030 2024-03-29T04:11:13.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.898+0000 7f1b927fc640 1 -- 172.21.15.148:0/2656124308 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b940d10d0 msgr2=0x7f1b940d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:13.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.898+0000 7f1b927fc640 1 --2- 172.21.15.148:0/2656124308 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b940d10d0 0x7f1b940d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.898+0000 7f1b927fc640 1 -- 172.21.15.148:0/2656124308 >> v2:172.21.15.157:3300/0 conn(0x7f1b9415e3e0 msgr2=0x7f1b940d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:13.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.898+0000 7f1b927fc640 1 --2- 172.21.15.148:0/2656124308 >> v2:172.21.15.157:3300/0 conn(0x7f1b9415e3e0 0x7f1b940d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.898+0000 7f1b927fc640 1 -- 172.21.15.148:0/2656124308 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b940d2f10 con 0x7f1b940ba030 2024-03-29T04:11:13.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.898+0000 7f1b927fc640 1 -- 172.21.15.148:0/2656124308 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f1b88001690 con 0x7f1b940ba030 2024-03-29T04:11:13.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.898+0000 7f1b927fc640 1 -- 172.21.15.148:0/2656124308 >> v1:172.21.15.148:6789/0 conn(0x7f1b940ba030 legacy=0x7f1b940d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:13.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.898+0000 7f1b927fc640 1 --2- 172.21.15.148:0/2656124308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b7c0039b0 0x7f1b7c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:13.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.898+0000 7f1b927fc640 1 --2- 172.21.15.148:0/2656124308 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b940d10d0 0x7f1b7c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:13.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.898+0000 7f1b927fc640 1 --2- 172.21.15.148:0/2656124308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b7c004d40 0x7f1b7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:13.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.898+0000 7f1b927fc640 1 -- 172.21.15.148:0/2656124308 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1b940d3840 con 0x7f1b7c0039b0 2024-03-29T04:11:13.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.898+0000 7f1b927fc640 1 -- 172.21.15.148:0/2656124308 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1b940d2c30 con 0x7f1b7c004d40 2024-03-29T04:11:13.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.898+0000 7f1b927fc640 1 -- 172.21.15.148:0/2656124308 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1b7c00b9c0 con 0x7f1b940d10d0 2024-03-29T04:11:13.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.898+0000 7f1b937fe640 1 --2- 172.21.15.148:0/2656124308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b7c0039b0 0x7f1b7c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:13.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.898+0000 7f1b93fff640 1 --2- 172.21.15.148:0/2656124308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b7c004d40 0x7f1b7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:13.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.898+0000 7f1b92ffd640 1 --2- 172.21.15.148:0/2656124308 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b940d10d0 0x7f1b7c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:13.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.899+0000 7f1b937fe640 1 -- 172.21.15.148:0/2656124308 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b940d10d0 msgr2=0x7f1b7c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:13.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.899+0000 7f1b937fe640 1 --2- 172.21.15.148:0/2656124308 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b940d10d0 0x7f1b7c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.899+0000 7f1b937fe640 1 -- 172.21.15.148:0/2656124308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b7c004d40 msgr2=0x7f1b7c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:13.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.899+0000 7f1b937fe640 1 --2- 172.21.15.148:0/2656124308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b7c004d40 0x7f1b7c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.899+0000 7f1b937fe640 1 -- 172.21.15.148:0/2656124308 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7c003520 con 0x7f1b7c0039b0 2024-03-29T04:11:13.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.899+0000 7f1b93fff640 1 --2- 172.21.15.148:0/2656124308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b7c004d40 0x7f1b7c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:13.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.899+0000 7f1b937fe640 1 --2- 172.21.15.148:0/2656124308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b7c0039b0 0x7f1b7c003dd0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1b84009ac0 tx=0x7f1b7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f2d712e640f3d3a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:13.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.899+0000 7f1b927fc640 1 -- 172.21.15.148:0/2656124308 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b84039070 con 0x7f1b7c0039b0 2024-03-29T04:11:13.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.899+0000 7f1b927fc640 1 -- 172.21.15.148:0/2656124308 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b8402c710 con 0x7f1b7c0039b0 2024-03-29T04:11:13.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.899+0000 7f1b927fc640 1 -- 172.21.15.148:0/2656124308 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b84009690 con 0x7f1b7c0039b0 2024-03-29T04:11:13.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.900+0000 7f1b99ac5640 1 -- 172.21.15.148:0/2656124308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b7c0039b0 msgr2=0x7f1b7c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:13.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.900+0000 7f1b99ac5640 1 --2- 172.21.15.148:0/2656124308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b7c0039b0 0x7f1b7c003dd0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1b84009ac0 tx=0x7f1b7c005830 comp rx=0 tx=0).stop 2024-03-29T04:11:13.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.900+0000 7f1b93fff640 1 -- 172.21.15.148:0/2656124308 reap_dead start 2024-03-29T04:11:13.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.900+0000 7f1b99ac5640 1 -- 172.21.15.148:0/2656124308 shutdown_connections 2024-03-29T04:11:13.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.900+0000 7f1b99ac5640 1 -- 172.21.15.148:0/2656124308 >> 172.21.15.148:0/2656124308 conn(0x7f1b940b6210 msgr2=0x7f1b940b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:13.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.900+0000 7f1b99ac5640 1 -- 172.21.15.148:0/2656124308 shutdown_connections 2024-03-29T04:11:13.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.900+0000 7f1b99ac5640 1 -- 172.21.15.148:0/2656124308 wait complete. 2024-03-29T04:11:13.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.902+0000 7f1b99ac5640 1 Processor -- start 2024-03-29T04:11:13.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.902+0000 7f1b99ac5640 1 -- start start 2024-03-29T04:11:13.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.903+0000 7f1b99ac5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b7c0039b0 0x7f1b9414a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:13.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.903+0000 7f1b99ac5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b940c9a00 0x7f1b940c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:13.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.903+0000 7f1b99ac5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b940c7ec0 0x7f1b940c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:13.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.903+0000 7f1b99ac5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1b940d1580 con 0x7f1b940c7ec0 2024-03-29T04:11:13.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.903+0000 7f1b99ac5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1b940d34d0 con 0x7f1b7c0039b0 2024-03-29T04:11:13.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.903+0000 7f1b99ac5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1b940d1700 con 0x7f1b940c9a00 2024-03-29T04:11:13.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.903+0000 7f1b93fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b940c7ec0 0x7f1b940c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:13.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.904+0000 7f1b93fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b940c7ec0 0x7f1b940c8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43458/0 (socket says 172.21.15.148:43458) 2024-03-29T04:11:13.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.904+0000 7f1b93fff640 1 -- 172.21.15.148:0/3576420842 learned_addr learned my addr 172.21.15.148:0/3576420842 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:13.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.904+0000 7f1b937fe640 1 --2- 172.21.15.148:0/3576420842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b7c0039b0 0x7f1b9414a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:13.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.904+0000 7f1b92ffd640 1 --2- 172.21.15.148:0/3576420842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b940c9a00 0x7f1b940c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:13.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.904+0000 7f1b93fff640 1 -- 172.21.15.148:0/3576420842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b940c9a00 msgr2=0x7f1b940c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:13.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.904+0000 7f1b93fff640 1 --2- 172.21.15.148:0/3576420842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b940c9a00 0x7f1b940c9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.904+0000 7f1b93fff640 1 -- 172.21.15.148:0/3576420842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b7c0039b0 msgr2=0x7f1b9414a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:13.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.904+0000 7f1b93fff640 1 --2- 172.21.15.148:0/3576420842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b7c0039b0 0x7f1b9414a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:13.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.904+0000 7f1b93fff640 1 -- 172.21.15.148:0/3576420842 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9414ace0 con 0x7f1b940c7ec0 2024-03-29T04:11:13.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.904+0000 7f1b93fff640 1 --2- 172.21.15.148:0/3576420842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b940c7ec0 0x7f1b940c8340 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1b940d1c80 tx=0x7f1b8c010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:13.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.904+0000 7f1b7bfff640 1 -- 172.21.15.148:0/3576420842 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c01a030 con 0x7f1b940c7ec0 2024-03-29T04:11:13.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.904+0000 7f1b7bfff640 1 -- 172.21.15.148:0/3576420842 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b8c010d80 con 0x7f1b940c7ec0 2024-03-29T04:11:13.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.904+0000 7f1b7bfff640 1 -- 172.21.15.148:0/3576420842 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c003bd0 con 0x7f1b940c7ec0 2024-03-29T04:11:13.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.904+0000 7f1b99ac5640 1 -- 172.21.15.148:0/3576420842 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b940c8bf0 con 0x7f1b940c7ec0 2024-03-29T04:11:13.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.904+0000 7f1b99ac5640 1 -- 172.21.15.148:0/3576420842 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b940cb480 con 0x7f1b940c7ec0 2024-03-29T04:11:13.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.905+0000 7f1b99ac5640 1 -- 172.21.15.148:0/3576420842 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b940687e0 con 0x7f1b940c7ec0 2024-03-29T04:11:13.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.906+0000 7f1b7bfff640 1 -- 172.21.15.148:0/3576420842 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1b8c01d070 con 0x7f1b940c7ec0 2024-03-29T04:11:13.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.906+0000 7f1b7bfff640 1 --2- 172.21.15.148:0/3576420842 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1b8006c560 0x7f1b8006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:13.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.906+0000 7f1b937fe640 1 --2- 172.21.15.148:0/3576420842 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1b8006c560 0x7f1b8006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:13.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.906+0000 7f1b7bfff640 1 -- 172.21.15.148:0/3576420842 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f1b8c08c3f0 con 0x7f1b940c7ec0 2024-03-29T04:11:13.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.907+0000 7f1b937fe640 1 --2- 172.21.15.148:0/3576420842 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1b8006c560 0x7f1b8006ea20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1b84036000 tx=0x7f1b84030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:13.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:13.911+0000 7f1b7bfff640 1 -- 172.21.15.148:0/3576420842 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1b9414ace0 con 0x7f1b940c7ec0 2024-03-29T04:11:14.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.093+0000 7f1b99ac5640 1 -- 172.21.15.148:0/3576420842 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b940b8ae0 con 0x7f1b940c7ec0 2024-03-29T04:11:14.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.094+0000 7f1b7bfff640 1 -- 172.21.15.148:0/3576420842 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18858 (secure 0 0 0) 0x7f1b8c05a050 con 0x7f1b940c7ec0 2024-03-29T04:11:14.073 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:11:14.073 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:41.662934+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:11:14.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.096+0000 7f1b99ac5640 1 -- 172.21.15.148:0/3576420842 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1b8006c560 msgr2=0x7f1b8006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:14.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.096+0000 7f1b99ac5640 1 --2- 172.21.15.148:0/3576420842 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1b8006c560 0x7f1b8006ea20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1b84036000 tx=0x7f1b84030000 comp rx=0 tx=0).stop 2024-03-29T04:11:14.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.096+0000 7f1b99ac5640 1 -- 172.21.15.148:0/3576420842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b940c7ec0 msgr2=0x7f1b940c8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:14.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.096+0000 7f1b99ac5640 1 --2- 172.21.15.148:0/3576420842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b940c7ec0 0x7f1b940c8340 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1b940d1c80 tx=0x7f1b8c010590 comp rx=0 tx=0).stop 2024-03-29T04:11:14.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.096+0000 7f1b99ac5640 1 -- 172.21.15.148:0/3576420842 shutdown_connections 2024-03-29T04:11:14.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.096+0000 7f1b99ac5640 1 --2- 172.21.15.148:0/3576420842 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1b8006c560 0x7f1b8006ea20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:14.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.096+0000 7f1b99ac5640 1 --2- 172.21.15.148:0/3576420842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b940c7ec0 0x7f1b940c8340 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:14.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.096+0000 7f1b99ac5640 1 --2- 172.21.15.148:0/3576420842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b940c9a00 0x7f1b940c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:14.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.096+0000 7f1b99ac5640 1 --2- 172.21.15.148:0/3576420842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b7c0039b0 0x7f1b9414a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:14.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.096+0000 7f1b99ac5640 1 -- 172.21.15.148:0/3576420842 >> 172.21.15.148:0/3576420842 conn(0x7f1b940b6210 msgr2=0x7f1b940b9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:14.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.096+0000 7f1b99ac5640 1 -- 172.21.15.148:0/3576420842 shutdown_connections 2024-03-29T04:11:14.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.096+0000 7f1b99ac5640 1 -- 172.21.15.148:0/3576420842 wait complete. 2024-03-29T04:11:14.087 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-29T04:11:14.087 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:11:14.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.232+0000 7fdced5e2640 1 Processor -- start 2024-03-29T04:11:14.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.232+0000 7fdced5e2640 1 -- start start 2024-03-29T04:11:14.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.234+0000 7fdced5e2640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdce80ceb50 0x7fdce80cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:14.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.234+0000 7fdced5e2640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdce80d0250 con 0x7fdce81608b0 2024-03-29T04:11:14.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.234+0000 7fdced5e2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdce80d19d0 con 0x7fdce815c5f0 2024-03-29T04:11:14.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.234+0000 7fdced5e2640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fdce80ccde0 con 0x7fdce80ceb50 2024-03-29T04:11:14.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.234+0000 7fdce6ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdce80ceb50 0x7fdce80cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:14.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.234+0000 7fdce77fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fdce81608b0 0x7fdce815cc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35850/0 (socket says 172.21.15.148:35850) 2024-03-29T04:11:14.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.234+0000 7fdce77fe640 1 -- 172.21.15.148:0/204276464 learned_addr learned my addr 172.21.15.148:0/204276464 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:14.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.234+0000 7fdce6ffd640 1 -- 172.21.15.148:0/204276464 >> v1:172.21.15.157:6789/0 conn(0x7fdce815c5f0 legacy=0x7fdce80cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:14.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.234+0000 7fdce6ffd640 1 -- 172.21.15.148:0/204276464 >> v1:172.21.15.148:6789/0 conn(0x7fdce81608b0 legacy=0x7fdce815cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:14.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.234+0000 7fdce6ffd640 1 -- 172.21.15.148:0/204276464 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce80ccb00 con 0x7fdce80ceb50 2024-03-29T04:11:14.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.235+0000 7fdce6ffd640 1 --2- 172.21.15.148:0/204276464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce80ceb50 0x7fdce80cfd10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdce80d1ba0 tx=0x7fdce002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6baa41433fd0d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:14.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.235+0000 7fdce5ffb640 1 -- 172.21.15.148:0/204276464 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce0035020 con 0x7fdce80ceb50 2024-03-29T04:11:14.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.235+0000 7fdce5ffb640 1 -- 172.21.15.148:0/204276464 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdce0031830 con 0x7fdce80ceb50 2024-03-29T04:11:14.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.235+0000 7fdce5ffb640 1 -- 172.21.15.148:0/204276464 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce0031c00 con 0x7fdce80ceb50 2024-03-29T04:11:14.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.236+0000 7fdced5e2640 1 -- 172.21.15.148:0/204276464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce80ceb50 msgr2=0x7fdce80cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:14.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.236+0000 7fdced5e2640 1 --2- 172.21.15.148:0/204276464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce80ceb50 0x7fdce80cfd10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdce80d1ba0 tx=0x7fdce002b8d0 comp rx=0 tx=0).stop 2024-03-29T04:11:14.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.236+0000 7fdced5e2640 1 -- 172.21.15.148:0/204276464 shutdown_connections 2024-03-29T04:11:14.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.236+0000 7fdced5e2640 1 --2- 172.21.15.148:0/204276464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce80ceb50 0x7fdce80cfd10 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:14.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.236+0000 7fdced5e2640 1 -- 172.21.15.148:0/204276464 >> 172.21.15.148:0/204276464 conn(0x7fdce80b3c70 msgr2=0x7fdce80b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:14.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.236+0000 7fdced5e2640 1 -- 172.21.15.148:0/204276464 shutdown_connections 2024-03-29T04:11:14.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.236+0000 7fdced5e2640 1 -- 172.21.15.148:0/204276464 wait complete. 2024-03-29T04:11:14.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.237+0000 7fdced5e2640 1 Processor -- start 2024-03-29T04:11:14.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.237+0000 7fdced5e2640 1 -- start start 2024-03-29T04:11:14.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.238+0000 7fdced5e2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce80ceb50 0x7fdce81e70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:14.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.238+0000 7fdced5e2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdce81608b0 0x7fdce80cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:14.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.238+0000 7fdced5e2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdce80cb2b0 0x7fdce815ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:14.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.238+0000 7fdced5e2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdce80ce540 con 0x7fdce80ceb50 2024-03-29T04:11:14.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.238+0000 7fdced5e2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdce80cc790 con 0x7fdce81608b0 2024-03-29T04:11:14.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.238+0000 7fdced5e2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdce80ce6c0 con 0x7fdce80cb2b0 2024-03-29T04:11:14.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.238+0000 7fdce6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce80ceb50 0x7fdce81e70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:14.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.238+0000 7fdce6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce80ceb50 0x7fdce81e70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43482/0 (socket says 172.21.15.148:43482) 2024-03-29T04:11:14.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.238+0000 7fdce6ffd640 1 -- 172.21.15.148:0/423060757 learned_addr learned my addr 172.21.15.148:0/423060757 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:14.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.238+0000 7fdce67fc640 1 --2- 172.21.15.148:0/423060757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdce81608b0 0x7fdce80cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:14.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.238+0000 7fdce6ffd640 1 -- 172.21.15.148:0/423060757 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdce80cb2b0 msgr2=0x7fdce815ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:14.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.238+0000 7fdce77fe640 1 --2- 172.21.15.148:0/423060757 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdce80cb2b0 0x7fdce815ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:14.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.238+0000 7fdce6ffd640 1 --2- 172.21.15.148:0/423060757 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdce80cb2b0 0x7fdce815ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:14.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.238+0000 7fdce6ffd640 1 -- 172.21.15.148:0/423060757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdce81608b0 msgr2=0x7fdce80cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:14.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.238+0000 7fdce6ffd640 1 --2- 172.21.15.148:0/423060757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdce81608b0 0x7fdce80cad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:14.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.238+0000 7fdce6ffd640 1 -- 172.21.15.148:0/423060757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce815da40 con 0x7fdce80ceb50 2024-03-29T04:11:14.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.238+0000 7fdce6ffd640 1 --2- 172.21.15.148:0/423060757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce80ceb50 0x7fdce81e70b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009890 tx=0x7fdce00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:14.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.239+0000 7fdccf7fe640 1 -- 172.21.15.148:0/423060757 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce003b9c0 con 0x7fdce80ceb50 2024-03-29T04:11:14.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.239+0000 7fdccf7fe640 1 -- 172.21.15.148:0/423060757 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdce003bb60 con 0x7fdce80ceb50 2024-03-29T04:11:14.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.239+0000 7fdccf7fe640 1 -- 172.21.15.148:0/423060757 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce00453f0 con 0x7fdce80ceb50 2024-03-29T04:11:14.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.239+0000 7fdced5e2640 1 -- 172.21.15.148:0/423060757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdce8141d60 con 0x7fdce80ceb50 2024-03-29T04:11:14.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.239+0000 7fdced5e2640 1 -- 172.21.15.148:0/423060757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdce8142320 con 0x7fdce80ceb50 2024-03-29T04:11:14.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.240+0000 7fdced5e2640 1 -- 172.21.15.148:0/423060757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdce80b4050 con 0x7fdce80ceb50 2024-03-29T04:11:14.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.241+0000 7fdccf7fe640 1 -- 172.21.15.148:0/423060757 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdce0038020 con 0x7fdce80ceb50 2024-03-29T04:11:14.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.241+0000 7fdccf7fe640 1 --2- 172.21.15.148:0/423060757 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fdcd406c560 0x7fdcd406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:14.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.241+0000 7fdce67fc640 1 --2- 172.21.15.148:0/423060757 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fdcd406c560 0x7fdcd406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:14.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.241+0000 7fdccf7fe640 1 -- 172.21.15.148:0/423060757 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fdce00ad860 con 0x7fdce80ceb50 2024-03-29T04:11:14.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.242+0000 7fdce67fc640 1 --2- 172.21.15.148:0/423060757 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fdcd406c560 0x7fdcd406ea20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc007fd0 tx=0x7fdcdc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:14.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.245+0000 7fdccf7fe640 1 -- 172.21.15.148:0/423060757 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fdce80b4050 con 0x7fdce80ceb50 2024-03-29T04:11:14.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.469+0000 7fdced5e2640 1 -- 172.21.15.148:0/423060757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdce80c5a40 con 0x7fdce80ceb50 2024-03-29T04:11:14.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.470+0000 7fdccf7fe640 1 -- 172.21.15.148:0/423060757 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7fdce80c5a40 con 0x7fdce80ceb50 2024-03-29T04:11:14.449 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:11:14.449 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/36660496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":36660496},{"type":"v1","addr":"172.21.15.157:6834","nonce":36660496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2674481055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2674481055},{"type":"v1","addr":"172.21.15.157:6835","nonce":2674481055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3356449643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3356449643},{"type":"v1","addr":"172.21.15.148:6836","nonce":3356449643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:09:59.886367+0000","modified":"2024-03-29T04:10:22.957623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/2453181817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2453181817},{"type":"v1","addr":"172.21.15.148:6837","nonce":2453181817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-03-29T04:11:14.449 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 9 2024-03-29T04:11:14.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.471+0000 7fdced5e2640 1 -- 172.21.15.148:0/423060757 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fdcd406c560 msgr2=0x7fdcd406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:14.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.471+0000 7fdced5e2640 1 --2- 172.21.15.148:0/423060757 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fdcd406c560 0x7fdcd406ea20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc007fd0 tx=0x7fdcdc009040 comp rx=0 tx=0).stop 2024-03-29T04:11:14.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.472+0000 7fdced5e2640 1 -- 172.21.15.148:0/423060757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce80ceb50 msgr2=0x7fdce81e70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:14.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.472+0000 7fdced5e2640 1 --2- 172.21.15.148:0/423060757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce80ceb50 0x7fdce81e70b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009890 tx=0x7fdce00046c0 comp rx=0 tx=0).stop 2024-03-29T04:11:14.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.472+0000 7fdced5e2640 1 -- 172.21.15.148:0/423060757 shutdown_connections 2024-03-29T04:11:14.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.472+0000 7fdced5e2640 1 --2- 172.21.15.148:0/423060757 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fdcd406c560 0x7fdcd406ea20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:14.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.472+0000 7fdced5e2640 1 --2- 172.21.15.148:0/423060757 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdce80cb2b0 0x7fdce815ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:14.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.472+0000 7fdced5e2640 1 --2- 172.21.15.148:0/423060757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdce81608b0 0x7fdce80cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:14.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.472+0000 7fdced5e2640 1 --2- 172.21.15.148:0/423060757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce80ceb50 0x7fdce81e70b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:14.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.472+0000 7fdced5e2640 1 -- 172.21.15.148:0/423060757 >> 172.21.15.148:0/423060757 conn(0x7fdce80b3c70 msgr2=0x7fdce80b4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:14.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.472+0000 7fdced5e2640 1 -- 172.21.15.148:0/423060757 shutdown_connections 2024-03-29T04:11:14.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.472+0000 7fdced5e2640 1 -- 172.21.15.148:0/423060757 wait complete. 2024-03-29T04:11:14.462 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:11:14.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.584+0000 7fb5a1c3c640 1 Processor -- start 2024-03-29T04:11:14.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.584+0000 7fb5a1c3c640 1 -- start start 2024-03-29T04:11:14.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.584+0000 7fb5a1c3c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb59c15c5f0 0x7fb59c0cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:14.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.584+0000 7fb5a1c3c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb59c1608b0 0x7fb59c15cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:14.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.584+0000 7fb5a1c3c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb59c0d19d0 con 0x7fb59c0ceb50 2024-03-29T04:11:14.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.584+0000 7fb5a1c3c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb59c0cf470 con 0x7fb59c1608b0 2024-03-29T04:11:14.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.584+0000 7fb5a1c3c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb59c0ccde0 con 0x7fb59c15c5f0 2024-03-29T04:11:14.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.585+0000 7fb59b7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb59c0ceb50 0x7fb59c0cfd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35864/0 (socket says 172.21.15.148:35864) 2024-03-29T04:11:14.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.585+0000 7fb59b7fe640 1 -- 172.21.15.148:0/1864964647 learned_addr learned my addr 172.21.15.148:0/1864964647 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:14.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.585+0000 7fb593fff640 1 --2- 172.21.15.148:0/1864964647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb59c15c5f0 0x7fb59c0cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:14.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.585+0000 7fb59affd640 1 -- 172.21.15.148:0/1864964647 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2569111255 0 0) 0x7fb59c0d19d0 con 0x7fb59c0ceb50 2024-03-29T04:11:14.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.585+0000 7fb59bfff640 1 --2- 172.21.15.148:0/1864964647 >> v2:172.21.15.157:3300/0 conn(0x7fb59c1608b0 0x7fb59c15cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:14.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.585+0000 7fb59affd640 1 -- 172.21.15.148:0/1864964647 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb588003520 con 0x7fb59c0ceb50 2024-03-29T04:11:14.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.585+0000 7fb59affd640 1 -- 172.21.15.148:0/1864964647 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2211551074 0 0) 0x7fb588003520 con 0x7fb59c0ceb50 2024-03-29T04:11:14.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.585+0000 7fb59affd640 1 -- 172.21.15.148:0/1864964647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb59c15c5f0 msgr2=0x7fb59c0cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:14.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.585+0000 7fb59affd640 1 --2- 172.21.15.148:0/1864964647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb59c15c5f0 0x7fb59c0cef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:14.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.585+0000 7fb59affd640 1 -- 172.21.15.148:0/1864964647 >> v2:172.21.15.157:3300/0 conn(0x7fb59c1608b0 msgr2=0x7fb59c15cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:14.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.585+0000 7fb59affd640 1 --2- 172.21.15.148:0/1864964647 >> v2:172.21.15.157:3300/0 conn(0x7fb59c1608b0 0x7fb59c15cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:14.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.585+0000 7fb59affd640 1 -- 172.21.15.148:0/1864964647 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb59c0ccb00 con 0x7fb59c0ceb50 2024-03-29T04:11:14.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.585+0000 7fb59affd640 1 -- 172.21.15.148:0/1864964647 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7fb58c001690 con 0x7fb59c0ceb50 2024-03-29T04:11:14.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.585+0000 7fb59affd640 1 -- 172.21.15.148:0/1864964647 >> v1:172.21.15.148:6789/0 conn(0x7fb59c0ceb50 legacy=0x7fb59c0cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:14.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.585+0000 7fb59affd640 1 --2- 172.21.15.148:0/1864964647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5880039b0 0x7fb588003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:14.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.585+0000 7fb59affd640 1 --2- 172.21.15.148:0/1864964647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5880043a0 0x7fb588004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:14.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.585+0000 7fb59affd640 1 --2- 172.21.15.148:0/1864964647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb588004ef0 0x7fb58800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:14.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.585+0000 7fb59affd640 1 -- 172.21.15.148:0/1864964647 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c0cf470 con 0x7fb588004ef0 2024-03-29T04:11:14.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.585+0000 7fb59affd640 1 -- 172.21.15.148:0/1864964647 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c0ccde0 con 0x7fb5880043a0 2024-03-29T04:11:14.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.585+0000 7fb59affd640 1 -- 172.21.15.148:0/1864964647 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb588010310 con 0x7fb5880039b0 2024-03-29T04:11:14.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.585+0000 7fb593fff640 1 --2- 172.21.15.148:0/1864964647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb59c15c5f0 0x7fb59c0cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:11:14.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.585+0000 7fb59bfff640 1 --2- 172.21.15.148:0/1864964647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb588004ef0 0x7fb58800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:14.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.585+0000 7fb59b7fe640 1 --2- 172.21.15.148:0/1864964647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5880039b0 0x7fb588003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:14.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.586+0000 7fb593fff640 1 --2- 172.21.15.148:0/1864964647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5880043a0 0x7fb588004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:14.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.586+0000 7fb59bfff640 1 -- 172.21.15.148:0/1864964647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5880039b0 msgr2=0x7fb588003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:14.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.586+0000 7fb59bfff640 1 --2- 172.21.15.148:0/1864964647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5880039b0 0x7fb588003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:14.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.586+0000 7fb59bfff640 1 -- 172.21.15.148:0/1864964647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5880043a0 msgr2=0x7fb588004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:14.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.586+0000 7fb59bfff640 1 --2- 172.21.15.148:0/1864964647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5880043a0 0x7fb588004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:14.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.586+0000 7fb59bfff640 1 -- 172.21.15.148:0/1864964647 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb588003520 con 0x7fb588004ef0 2024-03-29T04:11:14.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.586+0000 7fb59b7fe640 1 --2- 172.21.15.148:0/1864964647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5880039b0 0x7fb588003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:14.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.586+0000 7fb59bfff640 1 --2- 172.21.15.148:0/1864964647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb588004ef0 0x7fb58800b9c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb594014a00 tx=0x7fb588005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6dcb20beaeaf7f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:14.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.586+0000 7fb59affd640 1 -- 172.21.15.148:0/1864964647 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb594041070 con 0x7fb588004ef0 2024-03-29T04:11:14.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.587+0000 7fb59affd640 1 -- 172.21.15.148:0/1864964647 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb59403c640 con 0x7fb588004ef0 2024-03-29T04:11:14.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.587+0000 7fb59affd640 1 -- 172.21.15.148:0/1864964647 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb594009d00 con 0x7fb588004ef0 2024-03-29T04:11:14.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.587+0000 7fb5a1c3c640 1 -- 172.21.15.148:0/1864964647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb588004ef0 msgr2=0x7fb58800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:14.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.587+0000 7fb5a1c3c640 1 --2- 172.21.15.148:0/1864964647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb588004ef0 0x7fb58800b9c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb594014a00 tx=0x7fb588005830 comp rx=0 tx=0).stop 2024-03-29T04:11:14.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.587+0000 7fb59bfff640 1 -- 172.21.15.148:0/1864964647 reap_dead start 2024-03-29T04:11:14.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.587+0000 7fb5a1c3c640 1 -- 172.21.15.148:0/1864964647 shutdown_connections 2024-03-29T04:11:14.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.587+0000 7fb5a1c3c640 1 -- 172.21.15.148:0/1864964647 >> 172.21.15.148:0/1864964647 conn(0x7fb59c0b3c70 msgr2=0x7fb59c0b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:14.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.587+0000 7fb5a1c3c640 1 -- 172.21.15.148:0/1864964647 shutdown_connections 2024-03-29T04:11:14.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.587+0000 7fb5a1c3c640 1 -- 172.21.15.148:0/1864964647 wait complete. 2024-03-29T04:11:14.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.588+0000 7fb5a1c3c640 1 Processor -- start 2024-03-29T04:11:14.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.588+0000 7fb5a1c3c640 1 -- start start 2024-03-29T04:11:14.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.589+0000 7fb5a1c3c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb588004ef0 0x7fb59c14a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:14.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.589+0000 7fb5a1c3c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb59c0c5c70 0x7fb59c0c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:14.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.589+0000 7fb5a1c3c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb59c0c71c0 0x7fb59c0c7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:14.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.589+0000 7fb5a1c3c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c0ce540 con 0x7fb588004ef0 2024-03-29T04:11:14.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.589+0000 7fb5a1c3c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c0cc790 con 0x7fb59c0c5c70 2024-03-29T04:11:14.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.589+0000 7fb5a1c3c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb59c0ce6c0 con 0x7fb59c0c71c0 2024-03-29T04:11:14.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.589+0000 7fb59b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb588004ef0 0x7fb59c14a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:14.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.589+0000 7fb59b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb588004ef0 0x7fb59c14a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43498/0 (socket says 172.21.15.148:43498) 2024-03-29T04:11:14.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.589+0000 7fb59b7fe640 1 -- 172.21.15.148:0/3580119090 learned_addr learned my addr 172.21.15.148:0/3580119090 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:14.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.589+0000 7fb59bfff640 1 --2- 172.21.15.148:0/3580119090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb59c0c71c0 0x7fb59c0c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:14.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.589+0000 7fb593fff640 1 --2- 172.21.15.148:0/3580119090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb59c0c5c70 0x7fb59c0c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:14.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.589+0000 7fb59b7fe640 1 -- 172.21.15.148:0/3580119090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb59c0c71c0 msgr2=0x7fb59c0c7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:14.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.589+0000 7fb59b7fe640 1 --2- 172.21.15.148:0/3580119090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb59c0c71c0 0x7fb59c0c7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:14.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.589+0000 7fb59b7fe640 1 -- 172.21.15.148:0/3580119090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb59c0c5c70 msgr2=0x7fb59c0c60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:14.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.589+0000 7fb59b7fe640 1 --2- 172.21.15.148:0/3580119090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb59c0c5c70 0x7fb59c0c60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:14.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.589+0000 7fb59b7fe640 1 -- 172.21.15.148:0/3580119090 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb59c0c66f0 con 0x7fb588004ef0 2024-03-29T04:11:14.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.589+0000 7fb59bfff640 1 --2- 172.21.15.148:0/3580119090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb59c0c71c0 0x7fb59c0c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:14.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.590+0000 7fb59b7fe640 1 --2- 172.21.15.148:0/3580119090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb588004ef0 0x7fb59c14a4b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00e870 tx=0x7fb58c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:14.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.590+0000 7fb598ff9640 1 -- 172.21.15.148:0/3580119090 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c00c8d0 con 0x7fb588004ef0 2024-03-29T04:11:14.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.590+0000 7fb598ff9640 1 -- 172.21.15.148:0/3580119090 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb58c018070 con 0x7fb588004ef0 2024-03-29T04:11:14.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.590+0000 7fb5a1c3c640 1 -- 172.21.15.148:0/3580119090 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb59c0c69d0 con 0x7fb588004ef0 2024-03-29T04:11:14.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.590+0000 7fb598ff9640 1 -- 172.21.15.148:0/3580119090 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c00cc70 con 0x7fb588004ef0 2024-03-29T04:11:14.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.590+0000 7fb5a1c3c640 1 -- 172.21.15.148:0/3580119090 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb59c0b4ee0 con 0x7fb588004ef0 2024-03-29T04:11:14.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.591+0000 7fb5a1c3c640 1 -- 172.21.15.148:0/3580119090 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb59c0687e0 con 0x7fb588004ef0 2024-03-29T04:11:14.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.591+0000 7fb598ff9640 1 -- 172.21.15.148:0/3580119090 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb58c01d460 con 0x7fb588004ef0 2024-03-29T04:11:14.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.592+0000 7fb598ff9640 1 --2- 172.21.15.148:0/3580119090 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb57c06c530 0x7fb57c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:14.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.592+0000 7fb598ff9640 1 -- 172.21.15.148:0/3580119090 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb58c08baf0 con 0x7fb588004ef0 2024-03-29T04:11:14.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.592+0000 7fb593fff640 1 --2- 172.21.15.148:0/3580119090 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb57c06c530 0x7fb57c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:14.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.592+0000 7fb593fff640 1 --2- 172.21.15.148:0/3580119090 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb57c06c530 0x7fb57c06e9f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb588004780 tx=0x7fb584008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:14.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.596+0000 7fb598ff9640 1 -- 172.21.15.148:0/3580119090 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb58c059d10 con 0x7fb588004ef0 2024-03-29T04:11:14.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.809+0000 7fb5a1c3c640 1 -- 172.21.15.148:0/3580119090 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb59c0701d0 con 0x7fb588004ef0 2024-03-29T04:11:14.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.809+0000 7fb598ff9640 1 -- 172.21.15.148:0/3580119090 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7fb58c059610 con 0x7fb588004ef0 2024-03-29T04:11:14.789 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:11:14.789 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/36660496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":36660496},{"type":"v1","addr":"172.21.15.157:6834","nonce":36660496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2674481055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2674481055},{"type":"v1","addr":"172.21.15.157:6835","nonce":2674481055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3356449643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3356449643},{"type":"v1","addr":"172.21.15.148:6836","nonce":3356449643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:09:59.886367+0000","modified":"2024-03-29T04:10:22.957623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4290":{"gid":4290,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/2453181817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2453181817},{"type":"v1","addr":"172.21.15.148:6837","nonce":2453181817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4290,"qdb_cluster":[4290]},"id":1}]} 2024-03-29T04:11:14.790 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 9 2024-03-29T04:11:14.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.811+0000 7fb5a1c3c640 1 -- 172.21.15.148:0/3580119090 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb57c06c530 msgr2=0x7fb57c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:14.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.811+0000 7fb5a1c3c640 1 --2- 172.21.15.148:0/3580119090 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb57c06c530 0x7fb57c06e9f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb588004780 tx=0x7fb584008040 comp rx=0 tx=0).stop 2024-03-29T04:11:14.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.811+0000 7fb5a1c3c640 1 -- 172.21.15.148:0/3580119090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb588004ef0 msgr2=0x7fb59c14a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:14.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.811+0000 7fb5a1c3c640 1 --2- 172.21.15.148:0/3580119090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb588004ef0 0x7fb59c14a4b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00e870 tx=0x7fb58c00ed30 comp rx=0 tx=0).stop 2024-03-29T04:11:14.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.811+0000 7fb5a1c3c640 1 -- 172.21.15.148:0/3580119090 shutdown_connections 2024-03-29T04:11:14.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.811+0000 7fb5a1c3c640 1 --2- 172.21.15.148:0/3580119090 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb57c06c530 0x7fb57c06e9f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:14.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.811+0000 7fb5a1c3c640 1 --2- 172.21.15.148:0/3580119090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb59c0c71c0 0x7fb59c0c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:14.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.811+0000 7fb5a1c3c640 1 --2- 172.21.15.148:0/3580119090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb59c0c5c70 0x7fb59c0c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:14.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.811+0000 7fb5a1c3c640 1 --2- 172.21.15.148:0/3580119090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb588004ef0 0x7fb59c14a4b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:14.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.811+0000 7fb5a1c3c640 1 -- 172.21.15.148:0/3580119090 >> 172.21.15.148:0/3580119090 conn(0x7fb59c0b3c70 msgr2=0x7fb59c0d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:14.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.811+0000 7fb5a1c3c640 1 -- 172.21.15.148:0/3580119090 shutdown_connections 2024-03-29T04:11:14.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.811+0000 7fb5a1c3c640 1 -- 172.21.15.148:0/3580119090 wait complete. 2024-03-29T04:11:14.802 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:11:14.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.923+0000 7f0333bf4640 1 Processor -- start 2024-03-29T04:11:14.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.923+0000 7f0333bf4640 1 -- start start 2024-03-29T04:11:14.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.924+0000 7f0333bf4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f032c15e3e0 0x7f032c0d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:14.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.924+0000 7f0333bf4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f032c0d10d0 0x7f032c0d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:14.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.924+0000 7f0333bf4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f032c15e7c0 con 0x7f032c0ba030 2024-03-29T04:11:14.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.924+0000 7f0333bf4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f032c0d3840 con 0x7f032c0d10d0 2024-03-29T04:11:14.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.924+0000 7f0333bf4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f032c0d2c30 con 0x7f032c15e3e0 2024-03-29T04:11:14.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.925+0000 7f0332191640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f032c0d10d0 0x7f032c0d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:14.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.925+0000 7f0332191640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f032c0d10d0 0x7f032c0d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43500/0 (socket says 172.21.15.148:43500) 2024-03-29T04:11:14.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.925+0000 7f0332191640 1 -- 172.21.15.148:0/1078582684 learned_addr learned my addr 172.21.15.148:0/1078582684 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:14.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.925+0000 7f0331990640 1 --2- 172.21.15.148:0/1078582684 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f032c15e3e0 0x7f032c0d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:14.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.925+0000 7f0332191640 1 -- 172.21.15.148:0/1078582684 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f032c15e3e0 msgr2=0x7f032c0d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:14.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.925+0000 7f0332191640 1 --2- 172.21.15.148:0/1078582684 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f032c15e3e0 0x7f032c0d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:14.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.925+0000 7f0332191640 1 -- 172.21.15.148:0/1078582684 >> v1:172.21.15.157:6789/0 conn(0x7f032c0ba030 legacy=0x7f032c0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:14.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.925+0000 7f0332191640 1 -- 172.21.15.148:0/1078582684 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032c0d2f10 con 0x7f032c0d10d0 2024-03-29T04:11:14.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.925+0000 7f0331990640 1 --2- 172.21.15.148:0/1078582684 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f032c15e3e0 0x7f032c0d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:14.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.926+0000 7f0332191640 1 --2- 172.21.15.148:0/1078582684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f032c0d10d0 0x7f032c0d1580 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0324009e50 tx=0x7f032402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db27ea56c67ac74 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:14.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.926+0000 7f033098e640 1 -- 172.21.15.148:0/1078582684 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032402cb00 con 0x7f032c0d10d0 2024-03-29T04:11:14.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.926+0000 7f033098e640 1 -- 172.21.15.148:0/1078582684 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f032402cca0 con 0x7f032c0d10d0 2024-03-29T04:11:14.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.926+0000 7f033098e640 1 -- 172.21.15.148:0/1078582684 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324034610 con 0x7f032c0d10d0 2024-03-29T04:11:14.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.926+0000 7f0333bf4640 1 -- 172.21.15.148:0/1078582684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f032c0d10d0 msgr2=0x7f032c0d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:14.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.926+0000 7f0333bf4640 1 --2- 172.21.15.148:0/1078582684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f032c0d10d0 0x7f032c0d1580 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0324009e50 tx=0x7f032402bf60 comp rx=0 tx=0).stop 2024-03-29T04:11:14.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.926+0000 7f0333bf4640 1 -- 172.21.15.148:0/1078582684 shutdown_connections 2024-03-29T04:11:14.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.926+0000 7f0333bf4640 1 --2- 172.21.15.148:0/1078582684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f032c0d10d0 0x7f032c0d1580 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:14.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.926+0000 7f0333bf4640 1 --2- 172.21.15.148:0/1078582684 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f032c15e3e0 0x7f032c0d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:14.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.926+0000 7f0333bf4640 1 -- 172.21.15.148:0/1078582684 >> 172.21.15.148:0/1078582684 conn(0x7f032c0b6210 msgr2=0x7f032c0b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:14.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.926+0000 7f0333bf4640 1 -- 172.21.15.148:0/1078582684 shutdown_connections 2024-03-29T04:11:14.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.926+0000 7f0333bf4640 1 -- 172.21.15.148:0/1078582684 wait complete. 2024-03-29T04:11:14.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.927+0000 7f0333bf4640 1 Processor -- start 2024-03-29T04:11:14.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.928+0000 7f0333bf4640 1 -- start start 2024-03-29T04:11:14.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.928+0000 7f0333bf4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f032c0ba030 0x7f032c150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:14.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.928+0000 7f0333bf4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c0d10d0 0x7f032c0cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:14.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.928+0000 7f0333bf4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f032c15e3e0 0x7f032c154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:14.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.928+0000 7f0333bf4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f032c167f50 con 0x7f032c15e3e0 2024-03-29T04:11:14.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.928+0000 7f0333bf4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f032c0d34d0 con 0x7f032c0d10d0 2024-03-29T04:11:14.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.928+0000 7f0333bf4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f032c1680d0 con 0x7f032c0ba030 2024-03-29T04:11:14.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.928+0000 7f0332191640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f032c15e3e0 0x7f032c154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:14.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.928+0000 7f0332191640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f032c15e3e0 0x7f032c154350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43504/0 (socket says 172.21.15.148:43504) 2024-03-29T04:11:14.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.928+0000 7f0332191640 1 -- 172.21.15.148:0/2557626712 learned_addr learned my addr 172.21.15.148:0/2557626712 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:14.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.928+0000 7f0331990640 1 --2- 172.21.15.148:0/2557626712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f032c0ba030 0x7f032c150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:14.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.928+0000 7f033118f640 1 --2- 172.21.15.148:0/2557626712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c0d10d0 0x7f032c0cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:14.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.929+0000 7f0332191640 1 -- 172.21.15.148:0/2557626712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f032c0ba030 msgr2=0x7f032c150750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:14.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.929+0000 7f0332191640 1 --2- 172.21.15.148:0/2557626712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f032c0ba030 0x7f032c150750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:14.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.929+0000 7f0332191640 1 -- 172.21.15.148:0/2557626712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c0d10d0 msgr2=0x7f032c0cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:14.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.929+0000 7f0332191640 1 --2- 172.21.15.148:0/2557626712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c0d10d0 0x7f032c0cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:14.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.929+0000 7f0332191640 1 -- 172.21.15.148:0/2557626712 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032c155b50 con 0x7f032c15e3e0 2024-03-29T04:11:14.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.929+0000 7f033118f640 1 --2- 172.21.15.148:0/2557626712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c0d10d0 0x7f032c0cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:14.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.929+0000 7f0332191640 1 --2- 172.21.15.148:0/2557626712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f032c15e3e0 0x7f032c154350 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0324036040 tx=0x7f0324004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:14.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.929+0000 7f03227fc640 1 -- 172.21.15.148:0/2557626712 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324043020 con 0x7f032c15e3e0 2024-03-29T04:11:14.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.929+0000 7f03227fc640 1 -- 172.21.15.148:0/2557626712 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f032402ce50 con 0x7f032c15e3e0 2024-03-29T04:11:14.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.929+0000 7f0333bf4640 1 -- 172.21.15.148:0/2557626712 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f032c1602f0 con 0x7f032c15e3e0 2024-03-29T04:11:14.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.929+0000 7f03227fc640 1 -- 172.21.15.148:0/2557626712 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032403e650 con 0x7f032c15e3e0 2024-03-29T04:11:14.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.929+0000 7f0333bf4640 1 -- 172.21.15.148:0/2557626712 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f032c1608b0 con 0x7f032c15e3e0 2024-03-29T04:11:14.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.930+0000 7f0333bf4640 1 -- 172.21.15.148:0/2557626712 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f032c0d0720 con 0x7f032c15e3e0 2024-03-29T04:11:14.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.931+0000 7f03227fc640 1 -- 172.21.15.148:0/2557626712 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0324046020 con 0x7f032c15e3e0 2024-03-29T04:11:14.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.931+0000 7f03227fc640 1 --2- 172.21.15.148:0/2557626712 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f031006c560 0x7f031006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:14.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.931+0000 7f03227fc640 1 -- 172.21.15.148:0/2557626712 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f03240adec0 con 0x7f032c15e3e0 2024-03-29T04:11:14.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.931+0000 7f0331990640 1 --2- 172.21.15.148:0/2557626712 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f031006c560 0x7f031006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:14.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.932+0000 7f0331990640 1 --2- 172.21.15.148:0/2557626712 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f031006c560 0x7f031006ea20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f031c00aac0 tx=0x7f031c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:14.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:14.936+0000 7f03227fc640 1 -- 172.21.15.148:0/2557626712 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f032c0d0720 con 0x7f032c15e3e0 2024-03-29T04:11:15.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.113+0000 7f0333bf4640 1 -- 172.21.15.148:0/2557626712 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f032c0687e0 con 0x7f032c15e3e0 2024-03-29T04:11:15.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.115+0000 7f03227fc640 1 -- 172.21.15.148:0/2557626712 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18858 (secure 0 0 0) 0x7f032404d070 con 0x7f032c15e3e0 2024-03-29T04:11:15.095 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:11:15.095 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:10:41.662934+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:57.922313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:58.829405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:10:02.551763+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:11:15.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.117+0000 7f0333bf4640 1 -- 172.21.15.148:0/2557626712 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f031006c560 msgr2=0x7f031006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:15.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.117+0000 7f0333bf4640 1 --2- 172.21.15.148:0/2557626712 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f031006c560 0x7f031006ea20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f031c00aac0 tx=0x7f031c008040 comp rx=0 tx=0).stop 2024-03-29T04:11:15.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.117+0000 7f0333bf4640 1 -- 172.21.15.148:0/2557626712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f032c15e3e0 msgr2=0x7f032c154350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:15.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.117+0000 7f0333bf4640 1 --2- 172.21.15.148:0/2557626712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f032c15e3e0 0x7f032c154350 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0324036040 tx=0x7f0324004a40 comp rx=0 tx=0).stop 2024-03-29T04:11:15.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.117+0000 7f0333bf4640 1 -- 172.21.15.148:0/2557626712 shutdown_connections 2024-03-29T04:11:15.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.117+0000 7f0333bf4640 1 --2- 172.21.15.148:0/2557626712 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f031006c560 0x7f031006ea20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:15.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.117+0000 7f0333bf4640 1 --2- 172.21.15.148:0/2557626712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f032c15e3e0 0x7f032c154350 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:15.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.117+0000 7f0333bf4640 1 --2- 172.21.15.148:0/2557626712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f032c0d10d0 0x7f032c0cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:15.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.117+0000 7f0333bf4640 1 --2- 172.21.15.148:0/2557626712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f032c0ba030 0x7f032c150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:15.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.117+0000 7f0333bf4640 1 -- 172.21.15.148:0/2557626712 >> 172.21.15.148:0/2557626712 conn(0x7f032c0b6210 msgr2=0x7f032c0b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:15.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.117+0000 7f0333bf4640 1 -- 172.21.15.148:0/2557626712 shutdown_connections 2024-03-29T04:11:15.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.117+0000 7f0333bf4640 1 -- 172.21.15.148:0/2557626712 wait complete. 2024-03-29T04:11:15.109 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-29T04:11:15.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.231+0000 7f64e1693640 1 Processor -- start 2024-03-29T04:11:15.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.231+0000 7f64e1693640 1 -- start start 2024-03-29T04:11:15.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.232+0000 7f64e1693640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f64dc0cd600 0x7f64dc06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:15.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.232+0000 7f64e1693640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f64dc06c6c0 0x7f64dc15c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:15.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.232+0000 7f64e1693640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64dc0cd9e0 con 0x7f64dc0b5280 2024-03-29T04:11:15.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.232+0000 7f64e1693640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f64dc06d800 con 0x7f64dc06c6c0 2024-03-29T04:11:15.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.232+0000 7f64e1693640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f64dc15c680 con 0x7f64dc0cd600 2024-03-29T04:11:15.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.232+0000 7f64da7fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f64dc06c6c0 0x7f64dc15c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:15.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.232+0000 7f64da7fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f64dc06c6c0 0x7f64dc15c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43506/0 (socket says 172.21.15.148:43506) 2024-03-29T04:11:15.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.232+0000 7f64da7fc640 1 -- 172.21.15.148:0/254419604 learned_addr learned my addr 172.21.15.148:0/254419604 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:15.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.233+0000 7f64daffd640 1 --2- 172.21.15.148:0/254419604 >> v2:172.21.15.157:3300/0 conn(0x7f64dc0cd600 0x7f64dc06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:15.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.233+0000 7f64da7fc640 1 -- 172.21.15.148:0/254419604 >> v2:172.21.15.157:3300/0 conn(0x7f64dc0cd600 msgr2=0x7f64dc06dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:15.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.233+0000 7f64da7fc640 1 --2- 172.21.15.148:0/254419604 >> v2:172.21.15.157:3300/0 conn(0x7f64dc0cd600 0x7f64dc06dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:15.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.233+0000 7f64da7fc640 1 -- 172.21.15.148:0/254419604 >> v1:172.21.15.157:6789/0 conn(0x7f64dc0b5280 legacy=0x7f64dc0b5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:15.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.233+0000 7f64da7fc640 1 -- 172.21.15.148:0/254419604 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64dc06cbf0 con 0x7f64dc06c6c0 2024-03-29T04:11:15.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.233+0000 7f64da7fc640 1 --2- 172.21.15.148:0/254419604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64dc06c6c0 0x7f64dc15c140 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0739d0 tx=0x7f64d002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12eb00bbb94ebf67 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:15.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.233+0000 7f64d9ffb640 1 -- 172.21.15.148:0/254419604 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d0035020 con 0x7f64dc06c6c0 2024-03-29T04:11:15.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.234+0000 7f64d9ffb640 1 -- 172.21.15.148:0/254419604 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64d0031870 con 0x7f64dc06c6c0 2024-03-29T04:11:15.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.234+0000 7f64d9ffb640 1 -- 172.21.15.148:0/254419604 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d0031c40 con 0x7f64dc06c6c0 2024-03-29T04:11:15.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.234+0000 7f64e1693640 1 -- 172.21.15.148:0/254419604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64dc06c6c0 msgr2=0x7f64dc15c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:15.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.234+0000 7f64e1693640 1 --2- 172.21.15.148:0/254419604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64dc06c6c0 0x7f64dc15c140 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0739d0 tx=0x7f64d002b910 comp rx=0 tx=0).stop 2024-03-29T04:11:15.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.234+0000 7f64e1693640 1 -- 172.21.15.148:0/254419604 shutdown_connections 2024-03-29T04:11:15.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.234+0000 7f64e1693640 1 --2- 172.21.15.148:0/254419604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64dc06c6c0 0x7f64dc15c140 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:15.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.234+0000 7f64e1693640 1 --2- 172.21.15.148:0/254419604 >> v2:172.21.15.157:3300/0 conn(0x7f64dc0cd600 0x7f64dc06dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:15.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.234+0000 7f64e1693640 1 -- 172.21.15.148:0/254419604 >> 172.21.15.148:0/254419604 conn(0x7f64dc0b2c60 msgr2=0x7f64dc0b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:15.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.234+0000 7f64e1693640 1 -- 172.21.15.148:0/254419604 shutdown_connections 2024-03-29T04:11:15.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.234+0000 7f64e1693640 1 -- 172.21.15.148:0/254419604 wait complete. 2024-03-29T04:11:15.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.235+0000 7f64e1693640 1 Processor -- start 2024-03-29T04:11:15.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.235+0000 7f64e1693640 1 -- start start 2024-03-29T04:11:15.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.236+0000 7f64e1693640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64dc06c6c0 0x7f64dc154980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:15.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.236+0000 7f64e1693640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64dc0b5280 0x7f64dc1448e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:15.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.236+0000 7f64e1693640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64dc0cd600 0x7f64dc0d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:15.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.236+0000 7f64e1693640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc06d490 con 0x7f64dc0cd600 2024-03-29T04:11:15.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.236+0000 7f64e1693640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc06d1b0 con 0x7f64dc0b5280 2024-03-29T04:11:15.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.236+0000 7f64e1693640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f64dc15c880 con 0x7f64dc06c6c0 2024-03-29T04:11:15.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.236+0000 7f64db7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64dc0cd600 0x7f64dc0d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:15.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.236+0000 7f64db7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64dc0cd600 0x7f64dc0d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43518/0 (socket says 172.21.15.148:43518) 2024-03-29T04:11:15.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.236+0000 7f64db7fe640 1 -- 172.21.15.148:0/2785500070 learned_addr learned my addr 172.21.15.148:0/2785500070 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:15.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.236+0000 7f64da7fc640 1 --2- 172.21.15.148:0/2785500070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64dc0b5280 0x7f64dc1448e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:15.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.236+0000 7f64daffd640 1 --2- 172.21.15.148:0/2785500070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64dc06c6c0 0x7f64dc154980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:15.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.237+0000 7f64db7fe640 1 -- 172.21.15.148:0/2785500070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64dc06c6c0 msgr2=0x7f64dc154980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:15.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.237+0000 7f64db7fe640 1 --2- 172.21.15.148:0/2785500070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64dc06c6c0 0x7f64dc154980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:15.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.237+0000 7f64db7fe640 1 -- 172.21.15.148:0/2785500070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64dc0b5280 msgr2=0x7f64dc1448e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:15.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.237+0000 7f64db7fe640 1 --2- 172.21.15.148:0/2785500070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64dc0b5280 0x7f64dc1448e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:15.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.237+0000 7f64db7fe640 1 -- 172.21.15.148:0/2785500070 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64dc1581e0 con 0x7f64dc0cd600 2024-03-29T04:11:15.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.237+0000 7f64da7fc640 1 --2- 172.21.15.148:0/2785500070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64dc0b5280 0x7f64dc1448e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:15.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.237+0000 7f64db7fe640 1 --2- 172.21.15.148:0/2785500070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64dc0cd600 0x7f64dc0d00d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f64d400c7b0 tx=0x7f64d400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:15.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.237+0000 7f64c37fe640 1 -- 172.21.15.148:0/2785500070 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d4017070 con 0x7f64dc0cd600 2024-03-29T04:11:15.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.237+0000 7f64c37fe640 1 -- 172.21.15.148:0/2785500070 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64d4003da0 con 0x7f64dc0cd600 2024-03-29T04:11:15.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.237+0000 7f64e1693640 1 -- 172.21.15.148:0/2785500070 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64dc1584c0 con 0x7f64dc0cd600 2024-03-29T04:11:15.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.237+0000 7f64c37fe640 1 -- 172.21.15.148:0/2785500070 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d40052b0 con 0x7f64dc0cd600 2024-03-29T04:11:15.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.237+0000 7f64e1693640 1 -- 172.21.15.148:0/2785500070 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64dc158a80 con 0x7f64dc0cd600 2024-03-29T04:11:15.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.238+0000 7f64e1693640 1 -- 172.21.15.148:0/2785500070 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64dc0687e0 con 0x7f64dc0cd600 2024-03-29T04:11:15.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.238+0000 7f64c37fe640 1 -- 172.21.15.148:0/2785500070 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f64d401a070 con 0x7f64dc0cd600 2024-03-29T04:11:15.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.239+0000 7f64c37fe640 1 --2- 172.21.15.148:0/2785500070 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f64c806c530 0x7f64c806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:15.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.239+0000 7f64c37fe640 1 -- 172.21.15.148:0/2785500070 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f64d408b090 con 0x7f64dc0cd600 2024-03-29T04:11:15.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.239+0000 7f64daffd640 1 --2- 172.21.15.148:0/2785500070 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f64c806c530 0x7f64c806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:15.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.239+0000 7f64daffd640 1 --2- 172.21.15.148:0/2785500070 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f64c806c530 0x7f64c806e9f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f64cc00b7c0 tx=0x7f64cc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:15.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.243+0000 7f64c37fe640 1 -- 172.21.15.148:0/2785500070 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f64d4059340 con 0x7f64dc0cd600 2024-03-29T04:11:15.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.515+0000 7f64e1693640 1 -- 172.21.15.148:0/2785500070 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f64dc06ddc0 con 0x7f64dc0cd600 2024-03-29T04:11:15.495 INFO:tasks.ceph.mon.a.smithi148.stderr:2024-03-29T04:11:15.515+0000 7fa221fb2640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T04:11:15.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.517+0000 7f64c37fe640 1 -- 172.21.15.148:0/2785500070 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7f64d4058c40 con 0x7f64dc0cd600 2024-03-29T04:11:15.497 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-29T04:11:15.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.519+0000 7f64e1693640 1 -- 172.21.15.148:0/2785500070 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f64c806c530 msgr2=0x7f64c806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:15.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.519+0000 7f64e1693640 1 --2- 172.21.15.148:0/2785500070 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f64c806c530 0x7f64c806e9f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f64cc00b7c0 tx=0x7f64cc008040 comp rx=0 tx=0).stop 2024-03-29T04:11:15.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.519+0000 7f64e1693640 1 -- 172.21.15.148:0/2785500070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64dc0cd600 msgr2=0x7f64dc0d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:15.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.519+0000 7f64e1693640 1 --2- 172.21.15.148:0/2785500070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64dc0cd600 0x7f64dc0d00d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f64d400c7b0 tx=0x7f64d400cc70 comp rx=0 tx=0).stop 2024-03-29T04:11:15.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.519+0000 7f64e1693640 1 -- 172.21.15.148:0/2785500070 shutdown_connections 2024-03-29T04:11:15.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.519+0000 7f64e1693640 1 --2- 172.21.15.148:0/2785500070 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f64c806c530 0x7f64c806e9f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:15.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.519+0000 7f64e1693640 1 --2- 172.21.15.148:0/2785500070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64dc0cd600 0x7f64dc0d00d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:15.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.519+0000 7f64e1693640 1 --2- 172.21.15.148:0/2785500070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64dc0b5280 0x7f64dc1448e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:15.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.519+0000 7f64e1693640 1 --2- 172.21.15.148:0/2785500070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64dc06c6c0 0x7f64dc154980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:15.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.519+0000 7f64e1693640 1 -- 172.21.15.148:0/2785500070 >> 172.21.15.148:0/2785500070 conn(0x7f64dc0b2c60 msgr2=0x7f64dc0b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:15.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.519+0000 7f64e1693640 1 -- 172.21.15.148:0/2785500070 shutdown_connections 2024-03-29T04:11:15.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.519+0000 7f64e1693640 1 -- 172.21.15.148:0/2785500070 wait complete. 2024-03-29T04:11:15.501 INFO:tasks.ceph.mds.a.smithi148.stderr: -6144> 2024-03-29T04:09:53.483+0000 7fded860d640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-03-29T04:11:15.502 INFO:tasks.ceph.mds.a.smithi148.stderr: -4929> 2024-03-29T04:10:04.495+0000 7fded2601640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T04:11:15.515 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-29T04:11:15.534 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2024-03-29T04:11:15.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.640+0000 7ff89b6b2640 1 Processor -- start 2024-03-29T04:11:15.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.640+0000 7ff89b6b2640 1 -- start start 2024-03-29T04:11:15.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.641+0000 7ff89b6b2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff89415e3c0 0x7ff894161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:15.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.641+0000 7ff89b6b2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8940b3430 0x7ff89406b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:15.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.641+0000 7ff89b6b2640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8940cf330 con 0x7ff8940b2f00 2024-03-29T04:11:15.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.641+0000 7ff89b6b2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff8940cf960 con 0x7ff89415e3c0 2024-03-29T04:11:15.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.641+0000 7ff89b6b2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff894162cf0 con 0x7ff8940b3430 2024-03-29T04:11:15.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.641+0000 7ff898c4d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff8940b2f00 0x7ff8941624b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49314/0 (socket says 172.21.15.148:49314) 2024-03-29T04:11:15.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.641+0000 7ff898c4d640 1 -- 172.21.15.148:0/245624730 learned_addr learned my addr 172.21.15.148:0/245624730 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:15.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.641+0000 7ff88bfff640 1 -- 172.21.15.148:0/245624730 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1708866745 0 0) 0x7ff8940cf330 con 0x7ff8940b2f00 2024-03-29T04:11:15.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.641+0000 7ff899c4f640 1 --2- 172.21.15.148:0/245624730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8940b3430 0x7ff89406b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:15.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.641+0000 7ff89944e640 1 --2- 172.21.15.148:0/245624730 >> v2:172.21.15.157:3300/0 conn(0x7ff89415e3c0 0x7ff894161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:15.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.641+0000 7ff88bfff640 1 -- 172.21.15.148:0/245624730 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff880003520 con 0x7ff8940b2f00 2024-03-29T04:11:15.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.641+0000 7ff88bfff640 1 -- 172.21.15.148:0/245624730 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1427791371 0 0) 0x7ff880003520 con 0x7ff8940b2f00 2024-03-29T04:11:15.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.641+0000 7ff88bfff640 1 -- 172.21.15.148:0/245624730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8940b3430 msgr2=0x7ff89406b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:15.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.641+0000 7ff88bfff640 1 --2- 172.21.15.148:0/245624730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8940b3430 0x7ff89406b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:15.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.641+0000 7ff88bfff640 1 -- 172.21.15.148:0/245624730 >> v2:172.21.15.157:3300/0 conn(0x7ff89415e3c0 msgr2=0x7ff894161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:15.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.641+0000 7ff88bfff640 1 --2- 172.21.15.148:0/245624730 >> v2:172.21.15.157:3300/0 conn(0x7ff89415e3c0 0x7ff894161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:15.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.641+0000 7ff88bfff640 1 -- 172.21.15.148:0/245624730 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8940cf5f0 con 0x7ff8940b2f00 2024-03-29T04:11:15.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.641+0000 7ff88bfff640 1 -- 172.21.15.148:0/245624730 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7ff884001670 con 0x7ff8940b2f00 2024-03-29T04:11:15.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.641+0000 7ff88bfff640 1 -- 172.21.15.148:0/245624730 >> v1:172.21.15.148:6789/0 conn(0x7ff8940b2f00 legacy=0x7ff8941624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:15.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.641+0000 7ff88bfff640 1 --2- 172.21.15.148:0/245624730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8940b3430 0x7ff8800039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:15.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.641+0000 7ff88bfff640 1 --2- 172.21.15.148:0/245624730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff880003f30 0x7ff880004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:15.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.641+0000 7ff88bfff640 1 --2- 172.21.15.148:0/245624730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff880004d40 0x7ff8800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:15.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.641+0000 7ff88bfff640 1 -- 172.21.15.148:0/245624730 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff8940cf960 con 0x7ff880004d40 2024-03-29T04:11:15.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.641+0000 7ff88bfff640 1 -- 172.21.15.148:0/245624730 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff894162cf0 con 0x7ff880003f30 2024-03-29T04:11:15.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.641+0000 7ff88bfff640 1 -- 172.21.15.148:0/245624730 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff88000ba10 con 0x7ff8940b3430 2024-03-29T04:11:15.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.642+0000 7ff899c4f640 1 --2- 172.21.15.148:0/245624730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff880004d40 0x7ff8800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:15.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.642+0000 7ff89944e640 1 --2- 172.21.15.148:0/245624730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8940b3430 0x7ff8800039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:15.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.642+0000 7ff898c4d640 1 --2- 172.21.15.148:0/245624730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff880003f30 0x7ff880004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:15.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.642+0000 7ff899c4f640 1 -- 172.21.15.148:0/245624730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8940b3430 msgr2=0x7ff8800039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:15.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.642+0000 7ff899c4f640 1 --2- 172.21.15.148:0/245624730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8940b3430 0x7ff8800039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:15.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.642+0000 7ff899c4f640 1 -- 172.21.15.148:0/245624730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff880003f30 msgr2=0x7ff880004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:15.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.642+0000 7ff899c4f640 1 --2- 172.21.15.148:0/245624730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff880003f30 0x7ff880004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:15.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.642+0000 7ff899c4f640 1 -- 172.21.15.148:0/245624730 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff880003520 con 0x7ff880004d40 2024-03-29T04:11:15.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.643+0000 7ff899c4f640 1 --2- 172.21.15.148:0/245624730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff880004d40 0x7ff8800051c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff88c00e170 tx=0x7ff880005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ad25437c1f0d72 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:15.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.643+0000 7ff88bfff640 1 -- 172.21.15.148:0/245624730 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c039070 con 0x7ff880004d40 2024-03-29T04:11:15.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.643+0000 7ff88bfff640 1 -- 172.21.15.148:0/245624730 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff88c02cbf0 con 0x7ff880004d40 2024-03-29T04:11:15.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.643+0000 7ff88bfff640 1 -- 172.21.15.148:0/245624730 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c009c10 con 0x7ff880004d40 2024-03-29T04:11:15.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.643+0000 7ff89b6b2640 1 -- 172.21.15.148:0/245624730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff880004d40 msgr2=0x7ff8800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:15.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.643+0000 7ff89b6b2640 1 --2- 172.21.15.148:0/245624730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff880004d40 0x7ff8800051c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff88c00e170 tx=0x7ff880005830 comp rx=0 tx=0).stop 2024-03-29T04:11:15.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.643+0000 7ff899c4f640 1 -- 172.21.15.148:0/245624730 reap_dead start 2024-03-29T04:11:15.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.643+0000 7ff89b6b2640 1 -- 172.21.15.148:0/245624730 shutdown_connections 2024-03-29T04:11:15.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.643+0000 7ff89b6b2640 1 -- 172.21.15.148:0/245624730 >> 172.21.15.148:0/245624730 conn(0x7ff8940b6430 msgr2=0x7ff8940d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:15.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.643+0000 7ff89b6b2640 1 -- 172.21.15.148:0/245624730 shutdown_connections 2024-03-29T04:11:15.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.643+0000 7ff89b6b2640 1 -- 172.21.15.148:0/245624730 wait complete. 2024-03-29T04:11:15.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.644+0000 7ff89b6b2640 1 Processor -- start 2024-03-29T04:11:15.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.645+0000 7ff89b6b2640 1 -- start start 2024-03-29T04:11:15.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.645+0000 7ff89b6b2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff880004d40 0x7ff8940cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:15.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.645+0000 7ff89b6b2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8940cbd20 0x7ff8941529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:15.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.645+0000 7ff89b6b2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8941602d0 0x7ff8940b88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:15.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.645+0000 7ff89b6b2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff8940b3b20 con 0x7ff880004d40 2024-03-29T04:11:15.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.645+0000 7ff89b6b2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff8940b39a0 con 0x7ff8941602d0 2024-03-29T04:11:15.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.645+0000 7ff89b6b2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff89406b7f0 con 0x7ff8940cbd20 2024-03-29T04:11:15.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.645+0000 7ff89944e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff880004d40 0x7ff8940cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:15.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.645+0000 7ff89944e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff880004d40 0x7ff8940cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45494/0 (socket says 172.21.15.148:45494) 2024-03-29T04:11:15.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.645+0000 7ff89944e640 1 -- 172.21.15.148:0/1443631996 learned_addr learned my addr 172.21.15.148:0/1443631996 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:15.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.645+0000 7ff898c4d640 1 --2- 172.21.15.148:0/1443631996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8940cbd20 0x7ff8941529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:15.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.645+0000 7ff899c4f640 1 --2- 172.21.15.148:0/1443631996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8941602d0 0x7ff8940b88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:15.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.646+0000 7ff89944e640 1 -- 172.21.15.148:0/1443631996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8940cbd20 msgr2=0x7ff8941529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:15.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.646+0000 7ff89944e640 1 --2- 172.21.15.148:0/1443631996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8940cbd20 0x7ff8941529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:15.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.646+0000 7ff89944e640 1 -- 172.21.15.148:0/1443631996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8941602d0 msgr2=0x7ff8940b88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:15.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.646+0000 7ff89944e640 1 --2- 172.21.15.148:0/1443631996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8941602d0 0x7ff8940b88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:15.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.646+0000 7ff89944e640 1 -- 172.21.15.148:0/1443631996 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8940cc9f0 con 0x7ff880004d40 2024-03-29T04:11:15.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.646+0000 7ff898c4d640 1 --2- 172.21.15.148:0/1443631996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8940cbd20 0x7ff8941529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:15.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.646+0000 7ff89944e640 1 --2- 172.21.15.148:0/1443631996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff880004d40 0x7ff8940cc410 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff890011aa0 tx=0x7ff890011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:15.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.646+0000 7ff889ffb640 1 -- 172.21.15.148:0/1443631996 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89000fb80 con 0x7ff880004d40 2024-03-29T04:11:15.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.646+0000 7ff889ffb640 1 -- 172.21.15.148:0/1443631996 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff89000fd20 con 0x7ff880004d40 2024-03-29T04:11:15.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.646+0000 7ff89b6b2640 1 -- 172.21.15.148:0/1443631996 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8940cccd0 con 0x7ff880004d40 2024-03-29T04:11:15.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.646+0000 7ff889ffb640 1 -- 172.21.15.148:0/1443631996 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff890014660 con 0x7ff880004d40 2024-03-29T04:11:15.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.646+0000 7ff89b6b2640 1 -- 172.21.15.148:0/1443631996 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff894160750 con 0x7ff880004d40 2024-03-29T04:11:15.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.647+0000 7ff89b6b2640 1 -- 172.21.15.148:0/1443631996 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff894068850 con 0x7ff880004d40 2024-03-29T04:11:15.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.648+0000 7ff889ffb640 1 -- 172.21.15.148:0/1443631996 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff890012040 con 0x7ff880004d40 2024-03-29T04:11:15.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.648+0000 7ff889ffb640 1 --2- 172.21.15.148:0/1443631996 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff87806c530 0x7ff87806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:15.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.648+0000 7ff889ffb640 1 -- 172.21.15.148:0/1443631996 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7ff89008c130 con 0x7ff880004d40 2024-03-29T04:11:15.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.648+0000 7ff898c4d640 1 --2- 172.21.15.148:0/1443631996 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff87806c530 0x7ff87806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:15.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.648+0000 7ff898c4d640 1 --2- 172.21.15.148:0/1443631996 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff87806c530 0x7ff87806e9f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff8940cabc0 tx=0x7ff884009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:15.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.653+0000 7ff889ffb640 1 -- 172.21.15.148:0/1443631996 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff890019030 con 0x7ff880004d40 2024-03-29T04:11:15.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:15.866+0000 7ff89b6b2640 1 -- 172.21.15.148:0/1443631996 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff8940ccea0 con 0x7ff880004d40 2024-03-29T04:11:16.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.519+0000 7ff889ffb640 1 -- 172.21.15.148:0/1443631996 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7ff89005a380 con 0x7ff880004d40 2024-03-29T04:11:16.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.522+0000 7ff89b6b2640 1 -- 172.21.15.148:0/1443631996 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff87806c530 msgr2=0x7ff87806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:16.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.522+0000 7ff89b6b2640 1 --2- 172.21.15.148:0/1443631996 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff87806c530 0x7ff87806e9f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff8940cabc0 tx=0x7ff884009040 comp rx=0 tx=0).stop 2024-03-29T04:11:16.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.522+0000 7ff89b6b2640 1 -- 172.21.15.148:0/1443631996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff880004d40 msgr2=0x7ff8940cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:16.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.522+0000 7ff89b6b2640 1 --2- 172.21.15.148:0/1443631996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff880004d40 0x7ff8940cc410 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff890011aa0 tx=0x7ff890011f60 comp rx=0 tx=0).stop 2024-03-29T04:11:16.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.522+0000 7ff89b6b2640 1 -- 172.21.15.148:0/1443631996 shutdown_connections 2024-03-29T04:11:16.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.522+0000 7ff89b6b2640 1 --2- 172.21.15.148:0/1443631996 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff87806c530 0x7ff87806e9f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:16.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.522+0000 7ff89b6b2640 1 --2- 172.21.15.148:0/1443631996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8941602d0 0x7ff8940b88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:16.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.522+0000 7ff89b6b2640 1 --2- 172.21.15.148:0/1443631996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8940cbd20 0x7ff8941529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:16.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.522+0000 7ff89b6b2640 1 --2- 172.21.15.148:0/1443631996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff880004d40 0x7ff8940cc410 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:16.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.522+0000 7ff89b6b2640 1 -- 172.21.15.148:0/1443631996 >> 172.21.15.148:0/1443631996 conn(0x7ff8940b6430 msgr2=0x7ff89415c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:16.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.522+0000 7ff89b6b2640 1 -- 172.21.15.148:0/1443631996 shutdown_connections 2024-03-29T04:11:16.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.522+0000 7ff89b6b2640 1 -- 172.21.15.148:0/1443631996 wait complete. 2024-03-29T04:11:16.513 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-29T04:11:16.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.680+0000 7efda90b2640 1 Processor -- start 2024-03-29T04:11:16.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.680+0000 7efda90b2640 1 -- start start 2024-03-29T04:11:16.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.681+0000 7efda90b2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efda40d3840 0x7efda40d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:16.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.681+0000 7efda90b2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efda40b36e0 0x7efda41615d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:16.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.681+0000 7efda90b2640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efda40d27b0 con 0x7efda40d3c20 2024-03-29T04:11:16.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.681+0000 7efda90b2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7efda40d2a50 con 0x7efda40b36e0 2024-03-29T04:11:16.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.681+0000 7efda90b2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efda40d2d30 con 0x7efda40d3840 2024-03-29T04:11:16.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.681+0000 7efda359e640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7efda40d3c20 0x7efda41621b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49324/0 (socket says 172.21.15.148:49324) 2024-03-29T04:11:16.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.681+0000 7efda359e640 1 -- 172.21.15.148:0/628416277 learned_addr learned my addr 172.21.15.148:0/628416277 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:16.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.682+0000 7efda2d9d640 1 --2- 172.21.15.148:0/628416277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efda40d3840 0x7efda40d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:16.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.682+0000 7efda259c640 1 --2- 172.21.15.148:0/628416277 >> v2:172.21.15.157:3300/0 conn(0x7efda40b36e0 0x7efda41615d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:16.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.682+0000 7efda1d9b640 1 -- 172.21.15.148:0/628416277 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3460791631 0 0) 0x7efda40d27b0 con 0x7efda40d3c20 2024-03-29T04:11:16.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.682+0000 7efda1d9b640 1 -- 172.21.15.148:0/628416277 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd90003520 con 0x7efda40d3c20 2024-03-29T04:11:16.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.682+0000 7efda1d9b640 1 -- 172.21.15.148:0/628416277 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3749634177 0 0) 0x7efd90003520 con 0x7efda40d3c20 2024-03-29T04:11:16.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.682+0000 7efda1d9b640 1 -- 172.21.15.148:0/628416277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efda40d3840 msgr2=0x7efda40d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:16.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.682+0000 7efda1d9b640 1 --2- 172.21.15.148:0/628416277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efda40d3840 0x7efda40d2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:16.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.682+0000 7efda1d9b640 1 -- 172.21.15.148:0/628416277 >> v2:172.21.15.157:3300/0 conn(0x7efda40b36e0 msgr2=0x7efda41615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:16.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.682+0000 7efda1d9b640 1 --2- 172.21.15.148:0/628416277 >> v2:172.21.15.157:3300/0 conn(0x7efda40b36e0 0x7efda41615d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:16.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.682+0000 7efda1d9b640 1 -- 172.21.15.148:0/628416277 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda40d3010 con 0x7efda40d3c20 2024-03-29T04:11:16.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.682+0000 7efda1d9b640 1 -- 172.21.15.148:0/628416277 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7efd9c001a10 con 0x7efda40d3c20 2024-03-29T04:11:16.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.682+0000 7efda1d9b640 1 -- 172.21.15.148:0/628416277 >> v1:172.21.15.148:6789/0 conn(0x7efda40d3c20 legacy=0x7efda41621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:16.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.682+0000 7efda1d9b640 1 --2- 172.21.15.148:0/628416277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efda40d3840 0x7efd900039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:16.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.682+0000 7efda1d9b640 1 --2- 172.21.15.148:0/628416277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd90003f30 0x7efd90004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:16.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.682+0000 7efda1d9b640 1 --2- 172.21.15.148:0/628416277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd90004d40 0x7efd900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:16.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.682+0000 7efda1d9b640 1 -- 172.21.15.148:0/628416277 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efda40d2a50 con 0x7efd90003f30 2024-03-29T04:11:16.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.682+0000 7efda1d9b640 1 -- 172.21.15.148:0/628416277 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efda40d2d30 con 0x7efd90004d40 2024-03-29T04:11:16.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.682+0000 7efda1d9b640 1 -- 172.21.15.148:0/628416277 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efd9000b9c0 con 0x7efda40d3840 2024-03-29T04:11:16.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.682+0000 7efda259c640 1 --2- 172.21.15.148:0/628416277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd90003f30 0x7efd90004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:16.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.682+0000 7efda2d9d640 1 --2- 172.21.15.148:0/628416277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efda40d3840 0x7efd900039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:16.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.682+0000 7efda359e640 1 --2- 172.21.15.148:0/628416277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd90004d40 0x7efd900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:16.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.682+0000 7efda259c640 1 -- 172.21.15.148:0/628416277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efda40d3840 msgr2=0x7efd900039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:16.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.682+0000 7efda259c640 1 --2- 172.21.15.148:0/628416277 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efda40d3840 0x7efd900039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:16.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.682+0000 7efda259c640 1 -- 172.21.15.148:0/628416277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd90004d40 msgr2=0x7efd900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:16.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.682+0000 7efda259c640 1 --2- 172.21.15.148:0/628416277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd90004d40 0x7efd900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:16.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.682+0000 7efda259c640 1 -- 172.21.15.148:0/628416277 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd90003520 con 0x7efd90003f30 2024-03-29T04:11:16.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.683+0000 7efda259c640 1 --2- 172.21.15.148:0/628416277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd90003f30 0x7efd90004800 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009ac0 tx=0x7efd90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91b32a54cc9cd58 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:16.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.683+0000 7efda1d9b640 1 -- 172.21.15.148:0/628416277 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c039070 con 0x7efd90003f30 2024-03-29T04:11:16.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.683+0000 7efda1d9b640 1 -- 172.21.15.148:0/628416277 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd8c02c6d0 con 0x7efd90003f30 2024-03-29T04:11:16.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.683+0000 7efda1d9b640 1 -- 172.21.15.148:0/628416277 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c009690 con 0x7efd90003f30 2024-03-29T04:11:16.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.684+0000 7efda90b2640 1 -- 172.21.15.148:0/628416277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd90003f30 msgr2=0x7efd90004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:16.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.684+0000 7efda90b2640 1 --2- 172.21.15.148:0/628416277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd90003f30 0x7efd90004800 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009ac0 tx=0x7efd90005830 comp rx=0 tx=0).stop 2024-03-29T04:11:16.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.684+0000 7efda359e640 1 -- 172.21.15.148:0/628416277 reap_dead start 2024-03-29T04:11:16.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.684+0000 7efda90b2640 1 -- 172.21.15.148:0/628416277 shutdown_connections 2024-03-29T04:11:16.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.684+0000 7efda90b2640 1 -- 172.21.15.148:0/628416277 >> 172.21.15.148:0/628416277 conn(0x7efda40d42d0 msgr2=0x7efda40d46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:16.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.684+0000 7efda90b2640 1 -- 172.21.15.148:0/628416277 shutdown_connections 2024-03-29T04:11:16.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.684+0000 7efda90b2640 1 -- 172.21.15.148:0/628416277 wait complete. 2024-03-29T04:11:16.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.686+0000 7efda90b2640 1 Processor -- start 2024-03-29T04:11:16.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.687+0000 7efda90b2640 1 -- start start 2024-03-29T04:11:16.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.688+0000 7efda90b2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd90003f30 0x7efda415ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:16.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.688+0000 7efda90b2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efda4158520 0x7efda41505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:16.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.688+0000 7efda90b2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efda415c4e0 0x7efda40b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:16.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.688+0000 7efda90b2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efda406ce70 con 0x7efda415c4e0 2024-03-29T04:11:16.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.688+0000 7efda90b2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efda406ccf0 con 0x7efda4158520 2024-03-29T04:11:16.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.688+0000 7efda90b2640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efda406cff0 con 0x7efd90003f30 2024-03-29T04:11:16.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.688+0000 7efda359e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efda415c4e0 0x7efda40b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:16.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.688+0000 7efda359e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efda415c4e0 0x7efda40b5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45502/0 (socket says 172.21.15.148:45502) 2024-03-29T04:11:16.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.688+0000 7efda359e640 1 -- 172.21.15.148:0/2552010056 learned_addr learned my addr 172.21.15.148:0/2552010056 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:16.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.688+0000 7efda259c640 1 --2- 172.21.15.148:0/2552010056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efda4158520 0x7efda41505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:16.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.688+0000 7efda2d9d640 1 --2- 172.21.15.148:0/2552010056 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd90003f30 0x7efda415ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:16.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.688+0000 7efda359e640 1 -- 172.21.15.148:0/2552010056 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd90003f30 msgr2=0x7efda415ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:16.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.688+0000 7efda359e640 1 --2- 172.21.15.148:0/2552010056 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd90003f30 0x7efda415ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:16.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.688+0000 7efda359e640 1 -- 172.21.15.148:0/2552010056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efda4158520 msgr2=0x7efda41505e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:16.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.688+0000 7efda359e640 1 --2- 172.21.15.148:0/2552010056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efda4158520 0x7efda41505e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:16.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.688+0000 7efda359e640 1 -- 172.21.15.148:0/2552010056 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda415cd10 con 0x7efda415c4e0 2024-03-29T04:11:16.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.688+0000 7efda2d9d640 1 --2- 172.21.15.148:0/2552010056 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd90003f30 0x7efda415ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:16.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.688+0000 7efda359e640 1 --2- 172.21.15.148:0/2552010056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efda415c4e0 0x7efda40b5090 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7efda40d2980 tx=0x7efd9c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:16.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.689+0000 7efd9b7fe640 1 -- 172.21.15.148:0/2552010056 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c00eb60 con 0x7efda415c4e0 2024-03-29T04:11:16.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.689+0000 7efd9b7fe640 1 -- 172.21.15.148:0/2552010056 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd9c00ed20 con 0x7efda415c4e0 2024-03-29T04:11:16.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.689+0000 7efd9b7fe640 1 -- 172.21.15.148:0/2552010056 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c014670 con 0x7efda415c4e0 2024-03-29T04:11:16.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.689+0000 7efda90b2640 1 -- 172.21.15.148:0/2552010056 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda415cff0 con 0x7efda415c4e0 2024-03-29T04:11:16.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.689+0000 7efda90b2640 1 -- 172.21.15.148:0/2552010056 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda415d5b0 con 0x7efda415c4e0 2024-03-29T04:11:16.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.690+0000 7efda90b2640 1 -- 172.21.15.148:0/2552010056 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efda4068740 con 0x7efda415c4e0 2024-03-29T04:11:16.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.690+0000 7efd9b7fe640 1 -- 172.21.15.148:0/2552010056 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efd9c014810 con 0x7efda415c4e0 2024-03-29T04:11:16.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.691+0000 7efd9b7fe640 1 --2- 172.21.15.148:0/2552010056 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efd8406c560 0x7efd8406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:16.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.691+0000 7efd9b7fe640 1 -- 172.21.15.148:0/2552010056 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7efd9c08b860 con 0x7efda415c4e0 2024-03-29T04:11:16.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.691+0000 7efda2d9d640 1 --2- 172.21.15.148:0/2552010056 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efd8406c560 0x7efd8406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:16.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.691+0000 7efda2d9d640 1 --2- 172.21.15.148:0/2552010056 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efd8406c560 0x7efd8406ea20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7efd940099f0 tx=0x7efd94008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:16.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.695+0000 7efd9b7fe640 1 -- 172.21.15.148:0/2552010056 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efd9c059340 con 0x7efda415c4e0 2024-03-29T04:11:16.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:16.874+0000 7efda90b2640 1 -- 172.21.15.148:0/2552010056 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7efda40d2050 con 0x7efda415c4e0 2024-03-29T04:11:17.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.798+0000 7efd9b7fe640 1 -- 172.21.15.148:0/2552010056 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) ==== 181+0+0 (secure 0 0 0) 0x7efd9c05d640 con 0x7efda415c4e0 2024-03-29T04:11:17.779 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' removed 2024-03-29T04:11:17.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.801+0000 7efda90b2640 1 -- 172.21.15.148:0/2552010056 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efd8406c560 msgr2=0x7efd8406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:17.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.801+0000 7efda90b2640 1 --2- 172.21.15.148:0/2552010056 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efd8406c560 0x7efd8406ea20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7efd940099f0 tx=0x7efd94008040 comp rx=0 tx=0).stop 2024-03-29T04:11:17.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.801+0000 7efda90b2640 1 -- 172.21.15.148:0/2552010056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efda415c4e0 msgr2=0x7efda40b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:17.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.801+0000 7efda90b2640 1 --2- 172.21.15.148:0/2552010056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efda415c4e0 0x7efda40b5090 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7efda40d2980 tx=0x7efd9c00e530 comp rx=0 tx=0).stop 2024-03-29T04:11:17.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.801+0000 7efda90b2640 1 -- 172.21.15.148:0/2552010056 shutdown_connections 2024-03-29T04:11:17.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.801+0000 7efda90b2640 1 --2- 172.21.15.148:0/2552010056 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efd8406c560 0x7efd8406ea20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:17.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.801+0000 7efda90b2640 1 --2- 172.21.15.148:0/2552010056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efda415c4e0 0x7efda40b5090 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:17.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.801+0000 7efda90b2640 1 --2- 172.21.15.148:0/2552010056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efda4158520 0x7efda41505e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:17.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.801+0000 7efda90b2640 1 --2- 172.21.15.148:0/2552010056 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd90003f30 0x7efda415ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:17.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.801+0000 7efda90b2640 1 -- 172.21.15.148:0/2552010056 >> 172.21.15.148:0/2552010056 conn(0x7efda40d42d0 msgr2=0x7efda40d46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:17.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.801+0000 7efda90b2640 1 -- 172.21.15.148:0/2552010056 shutdown_connections 2024-03-29T04:11:17.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.802+0000 7efda90b2640 1 -- 172.21.15.148:0/2552010056 wait complete. 2024-03-29T04:11:17.801 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-29T04:11:17.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.933+0000 7f6029396640 1 Processor -- start 2024-03-29T04:11:17.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.933+0000 7f6029396640 1 -- start start 2024-03-29T04:11:17.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.934+0000 7f6029396640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60240cc140 0x7f60240b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:17.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.934+0000 7f6029396640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f60240b58b0 0x7f6024072c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:17.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.934+0000 7f6029396640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60240b4f30 con 0x7f6024073180 2024-03-29T04:11:17.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.934+0000 7f6029396640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f60240ca8d0 con 0x7f60240b58b0 2024-03-29T04:11:17.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.934+0000 7f6029396640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f60240735e0 con 0x7f60240cc140 2024-03-29T04:11:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.934+0000 7f60237fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6024073180 0x7f602406b7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49332/0 (socket says 172.21.15.148:49332) 2024-03-29T04:11:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.934+0000 7f60237fe640 1 -- 172.21.15.148:0/2605066605 learned_addr learned my addr 172.21.15.148:0/2605066605 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.934+0000 7f6022ffd640 1 --2- 172.21.15.148:0/2605066605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60240cc140 0x7f60240b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.934+0000 7f60227fc640 1 --2- 172.21.15.148:0/2605066605 >> v2:172.21.15.157:3300/0 conn(0x7f60240b58b0 0x7f6024072c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.934+0000 7f6021ffb640 1 -- 172.21.15.148:0/2605066605 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 643798744 0 0) 0x7f60240b4f30 con 0x7f6024073180 2024-03-29T04:11:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.934+0000 7f6021ffb640 1 -- 172.21.15.148:0/2605066605 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f600c003520 con 0x7f6024073180 2024-03-29T04:11:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.935+0000 7f6021ffb640 1 -- 172.21.15.148:0/2605066605 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 13172223 0 0) 0x7f600c003520 con 0x7f6024073180 2024-03-29T04:11:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.935+0000 7f6022ffd640 1 -- 172.21.15.148:0/2605066605 >> v2:172.21.15.157:3300/0 conn(0x7f60240b58b0 msgr2=0x7f6024072c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.935+0000 7f6022ffd640 1 --2- 172.21.15.148:0/2605066605 >> v2:172.21.15.157:3300/0 conn(0x7f60240b58b0 0x7f6024072c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.935+0000 7f6022ffd640 1 -- 172.21.15.148:0/2605066605 >> v1:172.21.15.148:6789/0 conn(0x7f6024073180 legacy=0x7f602406b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.935+0000 7f6022ffd640 1 -- 172.21.15.148:0/2605066605 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60240cd760 con 0x7f60240cc140 2024-03-29T04:11:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.935+0000 7f6022ffd640 1 --2- 172.21.15.148:0/2605066605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60240cc140 0x7f60240b5d90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6014009d90 tx=0x7f601402bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52ece91f33c79b9e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.936+0000 7f6021ffb640 1 -- 172.21.15.148:0/2605066605 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6014039070 con 0x7f60240cc140 2024-03-29T04:11:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.936+0000 7f6021ffb640 1 -- 172.21.15.148:0/2605066605 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f601402c8a0 con 0x7f60240cc140 2024-03-29T04:11:17.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.936+0000 7f6021ffb640 1 -- 172.21.15.148:0/2605066605 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c001670 con 0x7f60240cc140 2024-03-29T04:11:17.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.936+0000 7f6029396640 1 -- 172.21.15.148:0/2605066605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60240cc140 msgr2=0x7f60240b5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:17.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.936+0000 7f6029396640 1 --2- 172.21.15.148:0/2605066605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60240cc140 0x7f60240b5d90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6014009d90 tx=0x7f601402bd70 comp rx=0 tx=0).stop 2024-03-29T04:11:17.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.936+0000 7f6029396640 1 -- 172.21.15.148:0/2605066605 shutdown_connections 2024-03-29T04:11:17.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.936+0000 7f6029396640 1 --2- 172.21.15.148:0/2605066605 >> v2:172.21.15.157:3300/0 conn(0x7f60240b58b0 0x7f6024072c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:17.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.936+0000 7f6029396640 1 --2- 172.21.15.148:0/2605066605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60240cc140 0x7f60240b5d90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:17.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.936+0000 7f6029396640 1 -- 172.21.15.148:0/2605066605 >> 172.21.15.148:0/2605066605 conn(0x7f60240d0280 msgr2=0x7f60240d0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:17.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.936+0000 7f6029396640 1 -- 172.21.15.148:0/2605066605 shutdown_connections 2024-03-29T04:11:17.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.936+0000 7f6029396640 1 -- 172.21.15.148:0/2605066605 wait complete. 2024-03-29T04:11:17.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.938+0000 7f6029396640 1 Processor -- start 2024-03-29T04:11:17.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.938+0000 7f6029396640 1 -- start start 2024-03-29T04:11:17.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.938+0000 7f6029396640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6024073180 0x7f602415f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:17.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.938+0000 7f6029396640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60240b58b0 0x7f602415da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:17.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.938+0000 7f6029396640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60240cc140 0x7f6024099940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:17.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.938+0000 7f6029396640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f60240b5540 con 0x7f6024073180 2024-03-29T04:11:17.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.938+0000 7f6029396640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f60240cd510 con 0x7f60240b58b0 2024-03-29T04:11:17.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.938+0000 7f6029396640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f60240b51d0 con 0x7f60240cc140 2024-03-29T04:11:17.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.938+0000 7f6022ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6024073180 0x7f602415f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:17.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.938+0000 7f6022ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6024073180 0x7f602415f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45514/0 (socket says 172.21.15.148:45514) 2024-03-29T04:11:17.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.938+0000 7f6022ffd640 1 -- 172.21.15.148:0/689044190 learned_addr learned my addr 172.21.15.148:0/689044190 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:17.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.938+0000 7f60237fe640 1 --2- 172.21.15.148:0/689044190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60240cc140 0x7f6024099940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:17.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.938+0000 7f60227fc640 1 --2- 172.21.15.148:0/689044190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60240b58b0 0x7f602415da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:17.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.939+0000 7f6022ffd640 1 -- 172.21.15.148:0/689044190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60240cc140 msgr2=0x7f6024099940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:17.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.939+0000 7f6022ffd640 1 --2- 172.21.15.148:0/689044190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60240cc140 0x7f6024099940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:17.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.939+0000 7f6022ffd640 1 -- 172.21.15.148:0/689044190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60240b58b0 msgr2=0x7f602415da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:17.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.939+0000 7f6022ffd640 1 --2- 172.21.15.148:0/689044190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60240b58b0 0x7f602415da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:17.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.939+0000 7f6022ffd640 1 -- 172.21.15.148:0/689044190 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f602415e110 con 0x7f6024073180 2024-03-29T04:11:17.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.939+0000 7f6022ffd640 1 --2- 172.21.15.148:0/689044190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6024073180 0x7f602415f5f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6014037040 tx=0x7f6014004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:17.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.939+0000 7f600b7fe640 1 -- 172.21.15.148:0/689044190 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6014044020 con 0x7f6024073180 2024-03-29T04:11:17.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.939+0000 7f600b7fe640 1 -- 172.21.15.148:0/689044190 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6014034ce0 con 0x7f6024073180 2024-03-29T04:11:17.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.939+0000 7f6029396640 1 -- 172.21.15.148:0/689044190 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f602415e3f0 con 0x7f6024073180 2024-03-29T04:11:17.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.939+0000 7f600b7fe640 1 -- 172.21.15.148:0/689044190 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6014039040 con 0x7f6024073180 2024-03-29T04:11:17.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.939+0000 7f6029396640 1 -- 172.21.15.148:0/689044190 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f602415e9b0 con 0x7f6024073180 2024-03-29T04:11:17.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.940+0000 7f6029396640 1 -- 172.21.15.148:0/689044190 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6024068740 con 0x7f6024073180 2024-03-29T04:11:17.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.941+0000 7f600b7fe640 1 -- 172.21.15.148:0/689044190 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6014047020 con 0x7f6024073180 2024-03-29T04:11:17.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.941+0000 7f600b7fe640 1 --2- 172.21.15.148:0/689044190 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f601006c530 0x7f601006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:17.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.941+0000 7f600b7fe640 1 -- 172.21.15.148:0/689044190 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6383+0+0 (secure 0 0 0) 0x7f60140adbf0 con 0x7f6024073180 2024-03-29T04:11:17.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.941+0000 7f60227fc640 1 --2- 172.21.15.148:0/689044190 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f601006c530 0x7f601006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:17.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.941+0000 7f60227fc640 1 --2- 172.21.15.148:0/689044190 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f601006c530 0x7f601006e9f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6024157d80 tx=0x7f6018009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:17.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:17.945+0000 7f600b7fe640 1 -- 172.21.15.148:0/689044190 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f601404e070 con 0x7f6024073180 2024-03-29T04:11:18.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.121+0000 7f6029396640 1 -- 172.21.15.148:0/689044190 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f60240cf410 con 0x7f6024073180 2024-03-29T04:11:18.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.805+0000 7f600b7fe640 1 -- 172.21.15.148:0/689044190 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) ==== 169+0+0 (secure 0 0 0) 0x7f601407b920 con 0x7f6024073180 2024-03-29T04:11:18.786 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' removed 2024-03-29T04:11:18.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.808+0000 7f6029396640 1 -- 172.21.15.148:0/689044190 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f601006c530 msgr2=0x7f601006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:18.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.808+0000 7f6029396640 1 --2- 172.21.15.148:0/689044190 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f601006c530 0x7f601006e9f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6024157d80 tx=0x7f6018009190 comp rx=0 tx=0).stop 2024-03-29T04:11:18.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.808+0000 7f6029396640 1 -- 172.21.15.148:0/689044190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6024073180 msgr2=0x7f602415f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:18.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.808+0000 7f6029396640 1 --2- 172.21.15.148:0/689044190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6024073180 0x7f602415f5f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6014037040 tx=0x7f6014004a40 comp rx=0 tx=0).stop 2024-03-29T04:11:18.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.808+0000 7f6029396640 1 -- 172.21.15.148:0/689044190 shutdown_connections 2024-03-29T04:11:18.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.808+0000 7f6029396640 1 --2- 172.21.15.148:0/689044190 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f601006c530 0x7f601006e9f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:18.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.808+0000 7f6029396640 1 --2- 172.21.15.148:0/689044190 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60240cc140 0x7f6024099940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:18.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.808+0000 7f6029396640 1 --2- 172.21.15.148:0/689044190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60240b58b0 0x7f602415da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:18.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.808+0000 7f6029396640 1 --2- 172.21.15.148:0/689044190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6024073180 0x7f602415f5f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:18.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.808+0000 7f6029396640 1 -- 172.21.15.148:0/689044190 >> 172.21.15.148:0/689044190 conn(0x7f60240d0280 msgr2=0x7f60240afec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:18.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.808+0000 7f6029396640 1 -- 172.21.15.148:0/689044190 shutdown_connections 2024-03-29T04:11:18.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.808+0000 7f6029396640 1 -- 172.21.15.148:0/689044190 wait complete. 2024-03-29T04:11:18.803 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-29T04:11:18.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.932+0000 7f53a54a5640 1 Processor -- start 2024-03-29T04:11:18.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.932+0000 7f53a54a5640 1 -- start start 2024-03-29T04:11:18.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.933+0000 7f53a54a5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f53a0147a40 0x7f53a00ae290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:18.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.933+0000 7f53a54a5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53a00aecc0 con 0x7f53a00afca0 2024-03-29T04:11:18.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.933+0000 7f53a54a5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53a015fbf0 con 0x7f53a015eb00 2024-03-29T04:11:18.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.933+0000 7f53a54a5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f53a015fed0 con 0x7f53a0147a40 2024-03-29T04:11:18.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.933+0000 7f539f7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f53a00afca0 0x7f53a01609b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49334/0 (socket says 172.21.15.148:49334) 2024-03-29T04:11:18.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.933+0000 7f539f7fe640 1 -- 172.21.15.148:0/3936876995 learned_addr learned my addr 172.21.15.148:0/3936876995 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:18.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.934+0000 7f539dffb640 1 -- 172.21.15.148:0/3936876995 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2538298120 0 0) 0x7f53a00aecc0 con 0x7f53a00afca0 2024-03-29T04:11:18.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.934+0000 7f539effd640 1 --2- 172.21.15.148:0/3936876995 >> v2:172.21.15.157:3300/0 conn(0x7f53a0147a40 0x7f53a00ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:18.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.934+0000 7f539dffb640 1 -- 172.21.15.148:0/3936876995 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5388003520 con 0x7f53a00afca0 2024-03-29T04:11:18.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.934+0000 7f539dffb640 1 -- 172.21.15.148:0/3936876995 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 393749383 0 0) 0x7f5388003520 con 0x7f53a00afca0 2024-03-29T04:11:18.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.934+0000 7f539dffb640 1 -- 172.21.15.148:0/3936876995 >> v2:172.21.15.157:3300/0 conn(0x7f53a0147a40 msgr2=0x7f53a00ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:18.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.934+0000 7f539dffb640 1 --2- 172.21.15.148:0/3936876995 >> v2:172.21.15.157:3300/0 conn(0x7f53a0147a40 0x7f53a00ae290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:18.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.934+0000 7f539dffb640 1 -- 172.21.15.148:0/3936876995 >> v1:172.21.15.157:6789/0 conn(0x7f53a015eb00 legacy=0x7f53a00af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:18.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.934+0000 7f539dffb640 1 -- 172.21.15.148:0/3936876995 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a01601b0 con 0x7f53a00afca0 2024-03-29T04:11:18.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.934+0000 7f539dffb640 1 -- 172.21.15.148:0/3936876995 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f5398001670 con 0x7f53a00afca0 2024-03-29T04:11:18.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.934+0000 7f539dffb640 1 -- 172.21.15.148:0/3936876995 >> v1:172.21.15.148:6789/0 conn(0x7f53a00afca0 legacy=0x7f53a01609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:18.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.934+0000 7f539dffb640 1 --2- 172.21.15.148:0/3936876995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53880039b0 0x7f5388003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:18.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.934+0000 7f539dffb640 1 --2- 172.21.15.148:0/3936876995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5388004800 0x7f5388004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:18.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.934+0000 7f539dffb640 1 --2- 172.21.15.148:0/3936876995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53880051c0 0x7f538800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:18.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.934+0000 7f539dffb640 1 -- 172.21.15.148:0/3936876995 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f53a015fed0 con 0x7f5388004800 2024-03-29T04:11:18.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.934+0000 7f539dffb640 1 -- 172.21.15.148:0/3936876995 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f538800c160 con 0x7f53880039b0 2024-03-29T04:11:18.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.934+0000 7f539dffb640 1 -- 172.21.15.148:0/3936876995 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f538800c360 con 0x7f53880051c0 2024-03-29T04:11:18.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.934+0000 7f539e7fc640 1 --2- 172.21.15.148:0/3936876995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5388004800 0x7f5388004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:18.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.934+0000 7f539effd640 1 --2- 172.21.15.148:0/3936876995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53880039b0 0x7f5388003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:18.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.934+0000 7f539f7fe640 1 --2- 172.21.15.148:0/3936876995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53880051c0 0x7f538800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:18.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.934+0000 7f539e7fc640 1 -- 172.21.15.148:0/3936876995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53880051c0 msgr2=0x7f538800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:18.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.934+0000 7f539e7fc640 1 --2- 172.21.15.148:0/3936876995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53880051c0 0x7f538800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:18.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.934+0000 7f539e7fc640 1 -- 172.21.15.148:0/3936876995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53880039b0 msgr2=0x7f5388003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:18.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.934+0000 7f539e7fc640 1 --2- 172.21.15.148:0/3936876995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53880039b0 0x7f5388003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:18.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.934+0000 7f539e7fc640 1 -- 172.21.15.148:0/3936876995 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5388003520 con 0x7f5388004800 2024-03-29T04:11:18.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.934+0000 7f539f7fe640 1 -- 172.21.15.148:0/3936876995 reap_dead start 2024-03-29T04:11:18.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.934+0000 7f539effd640 1 --2- 172.21.15.148:0/3936876995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53880039b0 0x7f5388003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:18.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.935+0000 7f539e7fc640 1 --2- 172.21.15.148:0/3936876995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5388004800 0x7f5388004c80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f53a00709b0 tx=0x7f5388005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205800d261dc572d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:18.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.935+0000 7f539dffb640 1 -- 172.21.15.148:0/3936876995 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394001370 con 0x7f5388004800 2024-03-29T04:11:18.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.935+0000 7f539dffb640 1 -- 172.21.15.148:0/3936876995 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53940052d0 con 0x7f5388004800 2024-03-29T04:11:18.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.935+0000 7f539dffb640 1 -- 172.21.15.148:0/3936876995 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394003830 con 0x7f5388004800 2024-03-29T04:11:18.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.936+0000 7f53a54a5640 1 -- 172.21.15.148:0/3936876995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5388004800 msgr2=0x7f5388004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:18.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.936+0000 7f53a54a5640 1 --2- 172.21.15.148:0/3936876995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5388004800 0x7f5388004c80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f53a00709b0 tx=0x7f5388005830 comp rx=0 tx=0).stop 2024-03-29T04:11:18.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.936+0000 7f53a54a5640 1 -- 172.21.15.148:0/3936876995 shutdown_connections 2024-03-29T04:11:18.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.936+0000 7f53a54a5640 1 --2- 172.21.15.148:0/3936876995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5388004800 0x7f5388004c80 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:18.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.936+0000 7f53a54a5640 1 -- 172.21.15.148:0/3936876995 >> 172.21.15.148:0/3936876995 conn(0x7f53a0072490 msgr2=0x7f53a00728a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:18.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.936+0000 7f53a54a5640 1 -- 172.21.15.148:0/3936876995 shutdown_connections 2024-03-29T04:11:18.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.936+0000 7f53a54a5640 1 -- 172.21.15.148:0/3936876995 wait complete. 2024-03-29T04:11:18.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.937+0000 7f53a54a5640 1 Processor -- start 2024-03-29T04:11:18.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.937+0000 7f53a54a5640 1 -- start start 2024-03-29T04:11:18.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.938+0000 7f53a54a5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5388004800 0x7f53a01f15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:18.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.938+0000 7f53a54a5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53a013cec0 0x7f53a0160870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:18.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.938+0000 7f53a54a5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53a013d340 0x7f53a00cf200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:18.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.938+0000 7f53a54a5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f53a0147fa0 con 0x7f53a013d340 2024-03-29T04:11:18.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.938+0000 7f53a54a5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f53a0147e20 con 0x7f53a013cec0 2024-03-29T04:11:18.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.938+0000 7f53a54a5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f53a00af590 con 0x7f5388004800 2024-03-29T04:11:18.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.938+0000 7f539f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53a013d340 0x7f53a00cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:18.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.938+0000 7f539f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53a013d340 0x7f53a00cf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45526/0 (socket says 172.21.15.148:45526) 2024-03-29T04:11:18.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.938+0000 7f539f7fe640 1 -- 172.21.15.148:0/2433950536 learned_addr learned my addr 172.21.15.148:0/2433950536 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:18.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.938+0000 7f539effd640 1 --2- 172.21.15.148:0/2433950536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5388004800 0x7f53a01f15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:18.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.938+0000 7f539e7fc640 1 --2- 172.21.15.148:0/2433950536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53a013cec0 0x7f53a0160870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:18.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.938+0000 7f539f7fe640 1 -- 172.21.15.148:0/2433950536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5388004800 msgr2=0x7f53a01f15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:18.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.938+0000 7f539f7fe640 1 --2- 172.21.15.148:0/2433950536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5388004800 0x7f53a01f15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:18.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.938+0000 7f539f7fe640 1 -- 172.21.15.148:0/2433950536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53a013cec0 msgr2=0x7f53a0160870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:18.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.938+0000 7f539f7fe640 1 --2- 172.21.15.148:0/2433950536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53a013cec0 0x7f53a0160870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:18.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.938+0000 7f539f7fe640 1 -- 172.21.15.148:0/2433950536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a0161330 con 0x7f53a013d340 2024-03-29T04:11:18.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.938+0000 7f539f7fe640 1 --2- 172.21.15.148:0/2433950536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53a013d340 0x7f53a00cf200 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f53a0070a00 tx=0x7f539800e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:18.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.938+0000 7f53877fe640 1 -- 172.21.15.148:0/2433950536 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5398010510 con 0x7f53a013d340 2024-03-29T04:11:18.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.938+0000 7f53877fe640 1 -- 172.21.15.148:0/2433950536 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53980106d0 con 0x7f53a013d340 2024-03-29T04:11:18.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.938+0000 7f53a54a5640 1 -- 172.21.15.148:0/2433950536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a0161610 con 0x7f53a013d340 2024-03-29T04:11:18.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.938+0000 7f53877fe640 1 -- 172.21.15.148:0/2433950536 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5398010ac0 con 0x7f53a013d340 2024-03-29T04:11:18.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.938+0000 7f53a54a5640 1 -- 172.21.15.148:0/2433950536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a013c9a0 con 0x7f53a013d340 2024-03-29T04:11:18.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.939+0000 7f53a54a5640 1 -- 172.21.15.148:0/2433950536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53a0068740 con 0x7f53a013d340 2024-03-29T04:11:18.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.940+0000 7f53877fe640 1 -- 172.21.15.148:0/2433950536 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f539801b020 con 0x7f53a013d340 2024-03-29T04:11:18.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.940+0000 7f53877fe640 1 --2- 172.21.15.148:0/2433950536 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f538c06c560 0x7f538c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:18.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.940+0000 7f539effd640 1 --2- 172.21.15.148:0/2433950536 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f538c06c560 0x7f538c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:18.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.940+0000 7f53877fe640 1 -- 172.21.15.148:0/2433950536 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5853+0+0 (secure 0 0 0) 0x7f539808ae50 con 0x7f53a013d340 2024-03-29T04:11:18.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.941+0000 7f539effd640 1 --2- 172.21.15.148:0/2433950536 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f538c06c560 0x7f538c06ea20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f53a00b3bf0 tx=0x7f539000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:18.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:18.945+0000 7f53877fe640 1 -- 172.21.15.148:0/2433950536 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5398059470 con 0x7f53a013d340 2024-03-29T04:11:19.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:19.127+0000 7f53a54a5640 1 -- 172.21.15.148:0/2433950536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f53a013cb70 con 0x7f53a013d340 2024-03-29T04:11:19.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:19.807+0000 7f53877fe640 1 -- 172.21.15.148:0/2433950536 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) ==== 178+0+0 (secure 0 0 0) 0x7f5398058d70 con 0x7f53a013d340 2024-03-29T04:11:19.787 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data_ec' removed 2024-03-29T04:11:19.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:19.811+0000 7f53a54a5640 1 -- 172.21.15.148:0/2433950536 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f538c06c560 msgr2=0x7f538c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:19.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:19.811+0000 7f53a54a5640 1 --2- 172.21.15.148:0/2433950536 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f538c06c560 0x7f538c06ea20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f53a00b3bf0 tx=0x7f539000b040 comp rx=0 tx=0).stop 2024-03-29T04:11:19.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:19.811+0000 7f53a54a5640 1 -- 172.21.15.148:0/2433950536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53a013d340 msgr2=0x7f53a00cf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:19.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:19.811+0000 7f53a54a5640 1 --2- 172.21.15.148:0/2433950536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53a013d340 0x7f53a00cf200 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f53a0070a00 tx=0x7f539800e960 comp rx=0 tx=0).stop 2024-03-29T04:11:19.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:19.811+0000 7f53a54a5640 1 -- 172.21.15.148:0/2433950536 shutdown_connections 2024-03-29T04:11:19.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:19.811+0000 7f53a54a5640 1 --2- 172.21.15.148:0/2433950536 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f538c06c560 0x7f538c06ea20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:19.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:19.811+0000 7f53a54a5640 1 --2- 172.21.15.148:0/2433950536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53a013d340 0x7f53a00cf200 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:19.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:19.811+0000 7f53a54a5640 1 --2- 172.21.15.148:0/2433950536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53a013cec0 0x7f53a0160870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:19.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:19.811+0000 7f53a54a5640 1 --2- 172.21.15.148:0/2433950536 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5388004800 0x7f53a01f15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:19.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:19.811+0000 7f53a54a5640 1 -- 172.21.15.148:0/2433950536 >> 172.21.15.148:0/2433950536 conn(0x7f53a0072490 msgr2=0x7f53a0072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:19.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:19.811+0000 7f53a54a5640 1 -- 172.21.15.148:0/2433950536 shutdown_connections 2024-03-29T04:11:19.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:19.811+0000 7f53a54a5640 1 -- 172.21.15.148:0/2433950536 wait complete. 2024-03-29T04:11:19.804 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-29T04:11:19.805 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-29T04:11:19.805 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-29T04:11:19.805 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:11:19.805 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-29T04:11:19.805 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-29T04:11:19.805 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-29T04:11:19.805 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:11:19.806 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-29T04:11:19.806 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-29T04:11:19.806 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-29T04:11:19.806 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:11:19.806 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-29T04:11:19.806 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-29T04:11:19.807 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-29T04:11:19.807 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:11:19.816 INFO:tasks.ceph.mds.c:Stopped 2024-03-29T04:11:19.816 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-29T04:11:19.817 INFO:tasks.ceph.mds.b:Stopped 2024-03-29T04:11:19.817 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-29T04:11:19.818 INFO:tasks.ceph.mds.d:Stopped 2024-03-29T04:11:19.818 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-29T04:11:19.860 INFO:tasks.ceph.mds.b:Started 2024-03-29T04:11:19.861 INFO:tasks.ceph.mds.c:Started 2024-03-29T04:11:19.861 INFO:tasks.ceph.mds.d:Started 2024-03-29T04:11:19.861 DEBUG:teuthology.parallel:result is None 2024-03-29T04:11:19.861 DEBUG:teuthology.parallel:result is None 2024-03-29T04:11:19.862 DEBUG:teuthology.parallel:result is None 2024-03-29T04:11:19.935 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2024-03-29T04:11:19.941 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-03-29T04:11:19.941 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-03-29T04:11:20.051 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-03-29T04:11:25.817 INFO:tasks.ceph.mds.a:Stopped 2024-03-29T04:11:25.817 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-29T04:11:25.819 INFO:tasks.ceph.mds.a:Started 2024-03-29T04:11:25.820 DEBUG:teuthology.parallel:result is None 2024-03-29T04:11:25.820 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-29T04:11:25.908 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2024-03-29T04:11:25.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.955+0000 7f4b997bd640 1 Processor -- start 2024-03-29T04:11:25.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.955+0000 7f4b997bd640 1 -- start start 2024-03-29T04:11:25.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.956+0000 7f4b997bd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4b9415e3e0 0x7f4b940d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:25.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.956+0000 7f4b997bd640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b9415e7c0 con 0x7f4b940ba030 2024-03-29T04:11:25.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.956+0000 7f4b997bd640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b940d2c30 con 0x7f4b940d10d0 2024-03-29T04:11:25.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.956+0000 7f4b997bd640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f4b940d3840 con 0x7f4b9415e3e0 2024-03-29T04:11:25.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.956+0000 7f4b927fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4b940ba030 0x7f4b940d2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50690/0 (socket says 172.21.15.148:50690) 2024-03-29T04:11:25.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.956+0000 7f4b927fc640 1 -- 172.21.15.148:0/1779714544 learned_addr learned my addr 172.21.15.148:0/1779714544 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:25.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.956+0000 7f4b92ffd640 1 --2- 172.21.15.148:0/1779714544 >> v2:172.21.15.157:3300/0 conn(0x7f4b9415e3e0 0x7f4b940d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:25.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.956+0000 7f4b92ffd640 1 -- 172.21.15.148:0/1779714544 >> v1:172.21.15.157:6789/0 conn(0x7f4b940d10d0 legacy=0x7f4b94167f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:25.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.956+0000 7f4b92ffd640 1 -- 172.21.15.148:0/1779714544 >> v1:172.21.15.148:6789/0 conn(0x7f4b940ba030 legacy=0x7f4b940d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:25.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.956+0000 7f4b92ffd640 1 -- 172.21.15.148:0/1779714544 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b940d2f10 con 0x7f4b9415e3e0 2024-03-29T04:11:25.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.957+0000 7f4b92ffd640 1 --2- 172.21.15.148:0/1779714544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b9415e3e0 0x7f4b940d3cd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4b840097a0 tx=0x7f4b8402b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff34784d9ebe8876 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:25.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.958+0000 7f4b91ffb640 1 -- 172.21.15.148:0/1779714544 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8402f810 con 0x7f4b9415e3e0 2024-03-29T04:11:25.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.958+0000 7f4b91ffb640 1 -- 172.21.15.148:0/1779714544 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b8402f9b0 con 0x7f4b9415e3e0 2024-03-29T04:11:25.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.958+0000 7f4b91ffb640 1 -- 172.21.15.148:0/1779714544 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8402fd80 con 0x7f4b9415e3e0 2024-03-29T04:11:25.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.958+0000 7f4b997bd640 1 -- 172.21.15.148:0/1779714544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b9415e3e0 msgr2=0x7f4b940d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:25.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.958+0000 7f4b997bd640 1 --2- 172.21.15.148:0/1779714544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b9415e3e0 0x7f4b940d3cd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4b840097a0 tx=0x7f4b8402b9f0 comp rx=0 tx=0).stop 2024-03-29T04:11:25.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.958+0000 7f4b997bd640 1 -- 172.21.15.148:0/1779714544 shutdown_connections 2024-03-29T04:11:25.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.958+0000 7f4b997bd640 1 --2- 172.21.15.148:0/1779714544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b9415e3e0 0x7f4b940d3cd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:25.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.958+0000 7f4b997bd640 1 -- 172.21.15.148:0/1779714544 >> 172.21.15.148:0/1779714544 conn(0x7f4b940b6210 msgr2=0x7f4b940b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:25.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.958+0000 7f4b997bd640 1 -- 172.21.15.148:0/1779714544 shutdown_connections 2024-03-29T04:11:25.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.958+0000 7f4b997bd640 1 -- 172.21.15.148:0/1779714544 wait complete. 2024-03-29T04:11:25.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.960+0000 7f4b997bd640 1 Processor -- start 2024-03-29T04:11:25.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.960+0000 7f4b997bd640 1 -- start start 2024-03-29T04:11:25.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.961+0000 7f4b997bd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b940ba030 0x7f4b94155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:25.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.961+0000 7f4b997bd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b940d10d0 0x7f4b94153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:25.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.961+0000 7f4b997bd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b9415e3e0 0x7f4b940cc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:25.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.961+0000 7f4b997bd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4b940d1530 con 0x7f4b940ba030 2024-03-29T04:11:25.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.961+0000 7f4b997bd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4b940d34d0 con 0x7f4b940d10d0 2024-03-29T04:11:25.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.961+0000 7f4b997bd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4b940d16b0 con 0x7f4b9415e3e0 2024-03-29T04:11:25.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.961+0000 7f4b92ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b940ba030 0x7f4b94155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:25.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.961+0000 7f4b92ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b940ba030 0x7f4b94155590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39234/0 (socket says 172.21.15.148:39234) 2024-03-29T04:11:25.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.961+0000 7f4b92ffd640 1 -- 172.21.15.148:0/839315591 learned_addr learned my addr 172.21.15.148:0/839315591 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:25.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.961+0000 7f4b927fc640 1 --2- 172.21.15.148:0/839315591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b940d10d0 0x7f4b94153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:25.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.961+0000 7f4b937fe640 1 --2- 172.21.15.148:0/839315591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b9415e3e0 0x7f4b940cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:25.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.961+0000 7f4b92ffd640 1 -- 172.21.15.148:0/839315591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b9415e3e0 msgr2=0x7f4b940cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:25.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.961+0000 7f4b92ffd640 1 --2- 172.21.15.148:0/839315591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b9415e3e0 0x7f4b940cc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:25.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.961+0000 7f4b92ffd640 1 -- 172.21.15.148:0/839315591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b940d10d0 msgr2=0x7f4b94153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:25.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.961+0000 7f4b92ffd640 1 --2- 172.21.15.148:0/839315591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b940d10d0 0x7f4b94153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:25.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.961+0000 7f4b92ffd640 1 -- 172.21.15.148:0/839315591 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b940cca80 con 0x7f4b940ba030 2024-03-29T04:11:25.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.961+0000 7f4b927fc640 1 --2- 172.21.15.148:0/839315591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b940d10d0 0x7f4b94153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:25.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.962+0000 7f4b92ffd640 1 --2- 172.21.15.148:0/839315591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b940ba030 0x7f4b94155590 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4b840098c0 tx=0x7f4b84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:25.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.962+0000 7f4b7b7fe640 1 -- 172.21.15.148:0/839315591 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b84047070 con 0x7f4b940ba030 2024-03-29T04:11:25.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.962+0000 7f4b7b7fe640 1 -- 172.21.15.148:0/839315591 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b84037c60 con 0x7f4b940ba030 2024-03-29T04:11:25.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.962+0000 7f4b997bd640 1 -- 172.21.15.148:0/839315591 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b94155d00 con 0x7f4b940ba030 2024-03-29T04:11:25.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.962+0000 7f4b7b7fe640 1 -- 172.21.15.148:0/839315591 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8402fe60 con 0x7f4b940ba030 2024-03-29T04:11:25.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.962+0000 7f4b997bd640 1 -- 172.21.15.148:0/839315591 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b941562c0 con 0x7f4b940ba030 2024-03-29T04:11:25.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.963+0000 7f4b997bd640 1 -- 172.21.15.148:0/839315591 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b940687e0 con 0x7f4b940ba030 2024-03-29T04:11:25.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.963+0000 7f4b7b7fe640 1 -- 172.21.15.148:0/839315591 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4b8404a070 con 0x7f4b940ba030 2024-03-29T04:11:25.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.963+0000 7f4b7b7fe640 1 --2- 172.21.15.148:0/839315591 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4b8006c530 0x7f4b8006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:25.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.964+0000 7f4b7b7fe640 1 -- 172.21.15.148:0/839315591 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5335+0+0 (secure 0 0 0) 0x7f4b840ad1e0 con 0x7f4b940ba030 2024-03-29T04:11:25.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.964+0000 7f4b927fc640 1 --2- 172.21.15.148:0/839315591 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4b8006c530 0x7f4b8006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:25.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.964+0000 7f4b927fc640 1 --2- 172.21.15.148:0/839315591 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4b8006c530 0x7f4b8006e9f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4b941609b0 tx=0x7f4b88009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:25.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:25.969+0000 7f4b7b7fe640 1 -- 172.21.15.148:0/839315591 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4b84033030 con 0x7f4b940ba030 2024-03-29T04:11:26.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:26.152+0000 7f4b997bd640 1 -- 172.21.15.148:0/839315591 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f4b940b75c0 con 0x7f4b940ba030 2024-03-29T04:11:26.765 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-03-29T04:11:26.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:26.958+0000 7f4b7b7fe640 1 -- 172.21.15.148:0/839315591 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v35) ==== 97+0+0 (secure 0 0 0) 0x7f4b8407bb20 con 0x7f4b940ba030 2024-03-29T04:11:26.937 INFO:teuthology.orchestra.run.smithi148.stderr: removed all blocklist entries 2024-03-29T04:11:26.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:26.961+0000 7f4b997bd640 1 -- 172.21.15.148:0/839315591 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4b8006c530 msgr2=0x7f4b8006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:26.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:26.961+0000 7f4b997bd640 1 --2- 172.21.15.148:0/839315591 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4b8006c530 0x7f4b8006e9f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4b941609b0 tx=0x7f4b88009040 comp rx=0 tx=0).stop 2024-03-29T04:11:26.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:26.961+0000 7f4b997bd640 1 -- 172.21.15.148:0/839315591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b940ba030 msgr2=0x7f4b94155590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:26.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:26.961+0000 7f4b997bd640 1 --2- 172.21.15.148:0/839315591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b940ba030 0x7f4b94155590 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4b840098c0 tx=0x7f4b84004730 comp rx=0 tx=0).stop 2024-03-29T04:11:26.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:26.961+0000 7f4b997bd640 1 -- 172.21.15.148:0/839315591 shutdown_connections 2024-03-29T04:11:26.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:26.961+0000 7f4b997bd640 1 --2- 172.21.15.148:0/839315591 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f4b8006c530 0x7f4b8006e9f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:26.961+0000 7f4b997bd640 1 --2- 172.21.15.148:0/839315591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b9415e3e0 0x7f4b940cc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:26.961+0000 7f4b997bd640 1 --2- 172.21.15.148:0/839315591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b940d10d0 0x7f4b94153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:26.961+0000 7f4b997bd640 1 --2- 172.21.15.148:0/839315591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b940ba030 0x7f4b94155590 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:26.961+0000 7f4b997bd640 1 -- 172.21.15.148:0/839315591 >> 172.21.15.148:0/839315591 conn(0x7f4b940b6210 msgr2=0x7f4b940b7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:26.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:26.961+0000 7f4b997bd640 1 -- 172.21.15.148:0/839315591 shutdown_connections 2024-03-29T04:11:26.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:26.961+0000 7f4b997bd640 1 -- 172.21.15.148:0/839315591 wait complete. 2024-03-29T04:11:26.954 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-29T04:11:27.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.082+0000 7f0919560640 1 Processor -- start 2024-03-29T04:11:27.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.082+0000 7f0919560640 1 -- start start 2024-03-29T04:11:27.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.082+0000 7f0919560640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09140d1810 0x7f09140d2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.082+0000 7f0919560640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09140d2d50 con 0x7f09140d3df0 2024-03-29T04:11:27.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.082+0000 7f0919560640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09140d3030 con 0x7f091415e400 2024-03-29T04:11:27.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.082+0000 7f0919560640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f09140d3960 con 0x7f09140d1810 2024-03-29T04:11:27.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.083+0000 7f09137fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f09140d3df0 0x7f09140b7440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50696/0 (socket says 172.21.15.148:50696) 2024-03-29T04:11:27.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.083+0000 7f09137fe640 1 -- 172.21.15.148:0/2430142192 learned_addr learned my addr 172.21.15.148:0/2430142192 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:27.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.083+0000 7f0911ffb640 1 -- 172.21.15.148:0/2430142192 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1337350276 0 0) 0x7f09140d2d50 con 0x7f09140d3df0 2024-03-29T04:11:27.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.083+0000 7f0911ffb640 1 -- 172.21.15.148:0/2430142192 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08fc003520 con 0x7f09140d3df0 2024-03-29T04:11:27.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.083+0000 7f09127fc640 1 --2- 172.21.15.148:0/2430142192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09140d1810 0x7f09140d2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.083+0000 7f0911ffb640 1 -- 172.21.15.148:0/2430142192 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3691288859 0 0) 0x7f08fc003520 con 0x7f09140d3df0 2024-03-29T04:11:27.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.083+0000 7f0911ffb640 1 -- 172.21.15.148:0/2430142192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09140d1810 msgr2=0x7f09140d2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:27.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.083+0000 7f0911ffb640 1 --2- 172.21.15.148:0/2430142192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09140d1810 0x7f09140d2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.083+0000 7f0911ffb640 1 -- 172.21.15.148:0/2430142192 >> v1:172.21.15.157:6789/0 conn(0x7f091415e400 legacy=0x7f09140d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.083+0000 7f0911ffb640 1 -- 172.21.15.148:0/2430142192 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09140d3310 con 0x7f09140d3df0 2024-03-29T04:11:27.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.083+0000 7f0911ffb640 1 -- 172.21.15.148:0/2430142192 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f090c0019f0 con 0x7f09140d3df0 2024-03-29T04:11:27.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.083+0000 7f0911ffb640 1 -- 172.21.15.148:0/2430142192 >> v1:172.21.15.148:6789/0 conn(0x7f09140d3df0 legacy=0x7f09140b7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.083+0000 7f0911ffb640 1 --2- 172.21.15.148:0/2430142192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08fc0039e0 0x7f08fc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.083+0000 7f0911ffb640 1 --2- 172.21.15.148:0/2430142192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08fc004800 0x7f08fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.083+0000 7f0911ffb640 1 --2- 172.21.15.148:0/2430142192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09140d1810 0x7f08fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.083+0000 7f0911ffb640 1 -- 172.21.15.148:0/2430142192 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f09140d3960 con 0x7f08fc004800 2024-03-29T04:11:27.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.083+0000 7f0911ffb640 1 -- 172.21.15.148:0/2430142192 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc00baa0 con 0x7f08fc0039e0 2024-03-29T04:11:27.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.083+0000 7f0911ffb640 1 -- 172.21.15.148:0/2430142192 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f08fc00bca0 con 0x7f09140d1810 2024-03-29T04:11:27.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.083+0000 7f09127fc640 1 --2- 172.21.15.148:0/2430142192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08fc004800 0x7f08fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.083+0000 7f09137fe640 1 --2- 172.21.15.148:0/2430142192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09140d1810 0x7f08fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.083+0000 7f0912ffd640 1 --2- 172.21.15.148:0/2430142192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08fc0039e0 0x7f08fc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.084+0000 7f09127fc640 1 -- 172.21.15.148:0/2430142192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09140d1810 msgr2=0x7f08fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:27.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.084+0000 7f09127fc640 1 --2- 172.21.15.148:0/2430142192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09140d1810 0x7f08fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.084+0000 7f09127fc640 1 -- 172.21.15.148:0/2430142192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08fc0039e0 msgr2=0x7f08fc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:27.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.084+0000 7f09127fc640 1 --2- 172.21.15.148:0/2430142192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08fc0039e0 0x7f08fc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.084+0000 7f09127fc640 1 -- 172.21.15.148:0/2430142192 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08fc003520 con 0x7f08fc004800 2024-03-29T04:11:27.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.084+0000 7f09127fc640 1 --2- 172.21.15.148:0/2430142192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08fc004800 0x7f08fc004c80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f09140b4c70 tx=0x7f08fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483f660f538e7e0a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:27.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.085+0000 7f0911ffb640 1 -- 172.21.15.148:0/2430142192 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908039070 con 0x7f08fc004800 2024-03-29T04:11:27.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.085+0000 7f0911ffb640 1 -- 172.21.15.148:0/2430142192 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f090802c630 con 0x7f08fc004800 2024-03-29T04:11:27.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.085+0000 7f0911ffb640 1 -- 172.21.15.148:0/2430142192 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908009690 con 0x7f08fc004800 2024-03-29T04:11:27.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.085+0000 7f0919560640 1 -- 172.21.15.148:0/2430142192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08fc004800 msgr2=0x7f08fc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.085+0000 7f0919560640 1 --2- 172.21.15.148:0/2430142192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08fc004800 0x7f08fc004c80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f09140b4c70 tx=0x7f08fc005830 comp rx=0 tx=0).stop 2024-03-29T04:11:27.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.085+0000 7f09137fe640 1 -- 172.21.15.148:0/2430142192 reap_dead start 2024-03-29T04:11:27.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.085+0000 7f0919560640 1 -- 172.21.15.148:0/2430142192 shutdown_connections 2024-03-29T04:11:27.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.085+0000 7f0919560640 1 -- 172.21.15.148:0/2430142192 >> 172.21.15.148:0/2430142192 conn(0x7f09140b6210 msgr2=0x7f09140b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:27.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.085+0000 7f0919560640 1 -- 172.21.15.148:0/2430142192 shutdown_connections 2024-03-29T04:11:27.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.085+0000 7f0919560640 1 -- 172.21.15.148:0/2430142192 wait complete. 2024-03-29T04:11:27.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.086+0000 7f0919560640 1 Processor -- start 2024-03-29T04:11:27.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.087+0000 7f0919560640 1 -- start start 2024-03-29T04:11:27.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.087+0000 7f0919560640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08fc004800 0x7f0914158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.087+0000 7f0919560640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0914152230 0x7f0914160390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.087+0000 7f0919560640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09140cc430 0x7f09140cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.087+0000 7f0919560640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f091415e960 con 0x7f0914152230 2024-03-29T04:11:27.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.087+0000 7f0919560640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f091415e7e0 con 0x7f08fc004800 2024-03-29T04:11:27.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.087+0000 7f0919560640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f09140d2670 con 0x7f09140cc430 2024-03-29T04:11:27.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.087+0000 7f09127fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0914152230 0x7f0914160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.087+0000 7f09127fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0914152230 0x7f0914160390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39252/0 (socket says 172.21.15.148:39252) 2024-03-29T04:11:27.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.087+0000 7f09127fc640 1 -- 172.21.15.148:0/3552465716 learned_addr learned my addr 172.21.15.148:0/3552465716 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:27.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.087+0000 7f0912ffd640 1 --2- 172.21.15.148:0/3552465716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08fc004800 0x7f0914158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.087+0000 7f09137fe640 1 --2- 172.21.15.148:0/3552465716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09140cc430 0x7f09140cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.088+0000 7f09127fc640 1 -- 172.21.15.148:0/3552465716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09140cc430 msgr2=0x7f09140cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.088+0000 7f09127fc640 1 --2- 172.21.15.148:0/3552465716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09140cc430 0x7f09140cc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.088+0000 7f09127fc640 1 -- 172.21.15.148:0/3552465716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08fc004800 msgr2=0x7f0914158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.088+0000 7f09127fc640 1 --2- 172.21.15.148:0/3552465716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08fc004800 0x7f0914158f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.088+0000 7f09127fc640 1 -- 172.21.15.148:0/3552465716 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09141618f0 con 0x7f0914152230 2024-03-29T04:11:27.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.088+0000 7f09127fc640 1 --2- 172.21.15.148:0/3552465716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0914152230 0x7f0914160390 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0908009b80 tx=0x7f0908004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:27.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.088+0000 7f08fb7fe640 1 -- 172.21.15.148:0/3552465716 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908042020 con 0x7f0914152230 2024-03-29T04:11:27.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.088+0000 7f08fb7fe640 1 -- 172.21.15.148:0/3552465716 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f090803a040 con 0x7f0914152230 2024-03-29T04:11:27.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.088+0000 7f0919560640 1 -- 172.21.15.148:0/3552465716 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0914156080 con 0x7f0914152230 2024-03-29T04:11:27.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.088+0000 7f08fb7fe640 1 -- 172.21.15.148:0/3552465716 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908039070 con 0x7f0914152230 2024-03-29T04:11:27.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.088+0000 7f0919560640 1 -- 172.21.15.148:0/3552465716 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0914158760 con 0x7f0914152230 2024-03-29T04:11:27.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.089+0000 7f0919560640 1 -- 172.21.15.148:0/3552465716 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0914152030 con 0x7f0914152230 2024-03-29T04:11:27.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.089+0000 7f08fb7fe640 1 -- 172.21.15.148:0/3552465716 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0908045020 con 0x7f0914152230 2024-03-29T04:11:27.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.090+0000 7f08fb7fe640 1 --2- 172.21.15.148:0/3552465716 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f090006c530 0x7f090006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.090+0000 7f08fb7fe640 1 -- 172.21.15.148:0/3552465716 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f09080ac830 con 0x7f0914152230 2024-03-29T04:11:27.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.090+0000 7f0912ffd640 1 --2- 172.21.15.148:0/3552465716 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f090006c530 0x7f090006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.090+0000 7f0912ffd640 1 --2- 172.21.15.148:0/3552465716 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f090006c530 0x7f090006e9f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f09040017e0 tx=0x7f0904009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:27.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.094+0000 7f08fb7fe640 1 -- 172.21.15.148:0/3552465716 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0914152030 con 0x7f0914152230 2024-03-29T04:11:27.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.336+0000 7f0919560640 1 -- 172.21.15.148:0/3552465716 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f09140687e0 con 0x7f0914152230 2024-03-29T04:11:27.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.336+0000 7f08fb7fe640 1 -- 172.21.15.148:0/3552465716 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7f090804c070 con 0x7f0914152230 2024-03-29T04:11:27.315 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:11:27.315 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-03-29T04:11:27.315 INFO:teuthology.orchestra.run.smithi148.stdout: "auth_dump": [ 2024-03-29T04:11:27.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-29T04:11:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.a", 2024-03-29T04:11:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQDfPgZmTrdyMBAAvPgJ04SmD7mCUz2pcAdZ3g==", 2024-03-29T04:11:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-29T04:11:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-03-29T04:11:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-03-29T04:11:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-03-29T04:11:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-29T04:11:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-29T04:11:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.b", 2024-03-29T04:11:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQDgPgZmvrdIARAA4vZcLEbAmmcLdaOZbgls8A==", 2024-03-29T04:11:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-29T04:11:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-03-29T04:11:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-03-29T04:11:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-03-29T04:11:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-29T04:11:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-29T04:11:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.c", 2024-03-29T04:11:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQDfPgZmXvzaNxAAOuU3MIAJe5jS9MAoxcgP9Q==", 2024-03-29T04:11:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-29T04:11:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-03-29T04:11:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-03-29T04:11:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-03-29T04:11:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-29T04:11:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-29T04:11:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.d", 2024-03-29T04:11:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQDgPgZmK5OnCBAA/MqxtSGpybkRCfz71RftJw==", 2024-03-29T04:11:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-29T04:11:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-03-29T04:11:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-03-29T04:11:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-03-29T04:11:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-29T04:11:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-29T04:11:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.0", 2024-03-29T04:11:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQDjPgZmXItfKxAA2Jikd2Er5Kn/zTRZj3aeHg==", 2024-03-29T04:11:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-29T04:11:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-03-29T04:11:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-03-29T04:11:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-29T04:11:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-29T04:11:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.1", 2024-03-29T04:11:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQDmPgZmTiJPKRAAECwIVwapx3UnsEQG6IzVlg==", 2024-03-29T04:11:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-29T04:11:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-03-29T04:11:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-03-29T04:11:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-29T04:11:27.320 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:27.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:27.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-29T04:11:27.320 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.2", 2024-03-29T04:11:27.320 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQDpPgZmeNf2JxAANLSoLBF1pm2tRiYVQezq0A==", 2024-03-29T04:11:27.320 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-29T04:11:27.320 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-03-29T04:11:27.320 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-03-29T04:11:27.320 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-29T04:11:27.320 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:27.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:27.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-29T04:11:27.320 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.3", 2024-03-29T04:11:27.320 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQDsPgZmfKgbJhAAczXpuXVH2PniXy4WufH+lg==", 2024-03-29T04:11:27.321 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-29T04:11:27.321 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-03-29T04:11:27.321 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-03-29T04:11:27.321 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-29T04:11:27.321 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:27.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:27.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-29T04:11:27.321 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.4", 2024-03-29T04:11:27.321 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQDwPgZmwT4/MxAAuCDZmNYrb+bk5XwG3epZ+Q==", 2024-03-29T04:11:27.321 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-29T04:11:27.321 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-03-29T04:11:27.321 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-03-29T04:11:27.321 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-29T04:11:27.321 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:27.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:27.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-29T04:11:27.322 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.5", 2024-03-29T04:11:27.322 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQDzPgZmSoOiMRAAQefLkDGw3zw2jWszEcpFnw==", 2024-03-29T04:11:27.322 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-29T04:11:27.322 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-03-29T04:11:27.322 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-03-29T04:11:27.322 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-29T04:11:27.322 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:27.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:27.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-29T04:11:27.322 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.6", 2024-03-29T04:11:27.322 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQD2PgZm3uivLxAA1EfZ+gq3r0mgUbab0XcAAQ==", 2024-03-29T04:11:27.323 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-29T04:11:27.323 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-03-29T04:11:27.323 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-03-29T04:11:27.323 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-29T04:11:27.323 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:27.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:27.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-29T04:11:27.323 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.7", 2024-03-29T04:11:27.323 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQD5PgZmbFv0LRAAWX++H2hdVRnjF+vFhXS5mA==", 2024-03-29T04:11:27.323 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-29T04:11:27.323 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-03-29T04:11:27.323 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-03-29T04:11:27.323 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-29T04:11:27.323 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:27.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:27.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-29T04:11:27.324 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.0", 2024-03-29T04:11:27.324 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQDgPgZmzmjtDBAAZ0mAEI3GHeThXHriL59lXg==", 2024-03-29T04:11:27.324 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-29T04:11:27.324 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-03-29T04:11:27.324 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-03-29T04:11:27.324 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-03-29T04:11:27.324 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-03-29T04:11:27.324 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:27.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:27.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-29T04:11:27.324 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.1", 2024-03-29T04:11:27.324 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQDgPgZmOhImFhAAyeFmJdddMuNJNn529wAv7A==", 2024-03-29T04:11:27.324 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-29T04:11:27.324 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-03-29T04:11:27.325 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-03-29T04:11:27.325 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-03-29T04:11:27.325 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-03-29T04:11:27.325 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:27.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:27.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-29T04:11:27.325 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.2", 2024-03-29T04:11:27.325 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQDgPgZmeXm6EhAAcTw1856TQ/rI+Y/9YWGuMQ==", 2024-03-29T04:11:27.325 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-29T04:11:27.325 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-03-29T04:11:27.325 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-03-29T04:11:27.325 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-03-29T04:11:27.325 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-03-29T04:11:27.325 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:27.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:27.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-29T04:11:27.326 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.3", 2024-03-29T04:11:27.326 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQDgPgZmpOn5GxAA1qplaVeRlhA6pGPbpzdrrQ==", 2024-03-29T04:11:27.326 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-29T04:11:27.326 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-03-29T04:11:27.326 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-03-29T04:11:27.326 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-03-29T04:11:27.326 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-03-29T04:11:27.326 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:27.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:27.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-29T04:11:27.326 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.admin", 2024-03-29T04:11:27.326 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQDfPgZmCtCTExAAwyw/nMlI3rjtHohQAVeKBg==", 2024-03-29T04:11:27.326 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-29T04:11:27.327 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-03-29T04:11:27.327 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-03-29T04:11:27.327 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-03-29T04:11:27.327 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-29T04:11:27.327 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:27.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:27.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-29T04:11:27.327 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.bootstrap-mds", 2024-03-29T04:11:27.327 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQD/PgZmczwVJxAAWJ/lu0sA/QS1dgqEydVWRQ==", 2024-03-29T04:11:27.327 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-29T04:11:27.327 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile bootstrap-mds" 2024-03-29T04:11:27.327 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:27.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:27.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-29T04:11:27.327 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.bootstrap-mgr", 2024-03-29T04:11:27.328 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQD/PgZmI0UVJxAAcEKRrp05nuDPWL2EPR1wSA==", 2024-03-29T04:11:27.328 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-29T04:11:27.328 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-29T04:11:27.328 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:27.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:27.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-29T04:11:27.328 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.bootstrap-osd", 2024-03-29T04:11:27.328 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQD/PgZm3EsVJxAAw9ldRb5s635jGtSLeEMu/A==", 2024-03-29T04:11:27.328 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-29T04:11:27.328 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile bootstrap-osd" 2024-03-29T04:11:27.328 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:27.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:27.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-29T04:11:27.328 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.bootstrap-rbd", 2024-03-29T04:11:27.328 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQD/PgZmY1IVJxAAaUJDuBlj1qN+nyesZME1mQ==", 2024-03-29T04:11:27.329 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-29T04:11:27.329 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-29T04:11:27.329 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:27.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:27.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-29T04:11:27.329 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-29T04:11:27.329 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQD/PgZmQlkVJxAAcfKaCxS7uQmR90S5tm0fiw==", 2024-03-29T04:11:27.329 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-29T04:11:27.329 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-29T04:11:27.329 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:27.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:27.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-29T04:11:27.330 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.bootstrap-rgw", 2024-03-29T04:11:27.330 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQD/PgZmY2AVJxAAv0jazBvc21ACmoaXEudZGg==", 2024-03-29T04:11:27.330 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-29T04:11:27.330 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-29T04:11:27.330 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:27.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:27.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-29T04:11:27.330 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.x", 2024-03-29T04:11:27.330 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQDfPgZma5P5LBAA0aGdlks9ozVfEYCSPc/Kcg==", 2024-03-29T04:11:27.330 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-29T04:11:27.330 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-03-29T04:11:27.330 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-03-29T04:11:27.330 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-29T04:11:27.330 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:27.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:27.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-29T04:11:27.331 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.y", 2024-03-29T04:11:27.331 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQDfPgZmyAMBKRAA46y2Bz584MMFjMBsrsdVHg==", 2024-03-29T04:11:27.331 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-03-29T04:11:27.331 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-03-29T04:11:27.331 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-03-29T04:11:27.331 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-03-29T04:11:27.331 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:27.331 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:27.331 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-03-29T04:11:27.331 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-03-29T04:11:27.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.338+0000 7f0919560640 1 -- 172.21.15.148:0/3552465716 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f090006c530 msgr2=0x7f090006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.338+0000 7f0919560640 1 --2- 172.21.15.148:0/3552465716 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f090006c530 0x7f090006e9f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f09040017e0 tx=0x7f0904009040 comp rx=0 tx=0).stop 2024-03-29T04:11:27.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.338+0000 7f0919560640 1 -- 172.21.15.148:0/3552465716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0914152230 msgr2=0x7f0914160390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.338+0000 7f0919560640 1 --2- 172.21.15.148:0/3552465716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0914152230 0x7f0914160390 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0908009b80 tx=0x7f0908004730 comp rx=0 tx=0).stop 2024-03-29T04:11:27.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.338+0000 7f0919560640 1 -- 172.21.15.148:0/3552465716 shutdown_connections 2024-03-29T04:11:27.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.338+0000 7f0919560640 1 --2- 172.21.15.148:0/3552465716 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f090006c530 0x7f090006e9f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.338+0000 7f0919560640 1 --2- 172.21.15.148:0/3552465716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09140cc430 0x7f09140cc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.338+0000 7f0919560640 1 --2- 172.21.15.148:0/3552465716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0914152230 0x7f0914160390 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.338+0000 7f0919560640 1 --2- 172.21.15.148:0/3552465716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08fc004800 0x7f0914158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.338+0000 7f0919560640 1 -- 172.21.15.148:0/3552465716 >> 172.21.15.148:0/3552465716 conn(0x7f09140b6210 msgr2=0x7f09140b7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:27.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.338+0000 7f0919560640 1 -- 172.21.15.148:0/3552465716 shutdown_connections 2024-03-29T04:11:27.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.338+0000 7f0919560640 1 -- 172.21.15.148:0/3552465716 wait complete. 2024-03-29T04:11:27.334 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-29T04:11:27.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.456+0000 7fc3fe9ba640 1 Processor -- start 2024-03-29T04:11:27.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.456+0000 7fc3fe9ba640 1 -- start start 2024-03-29T04:11:27.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.457+0000 7fc3fe9ba640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc3f8072c30 0x7fc3f80ce710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.457+0000 7fc3fe9ba640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc3f806b5b0 0x7fc3f80cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.457+0000 7fc3fe9ba640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3f806dc80 con 0x7fc3f815e770 2024-03-29T04:11:27.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.457+0000 7fc3fe9ba640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fc3f80b4090 con 0x7fc3f806b5b0 2024-03-29T04:11:27.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.457+0000 7fc3fe9ba640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc3f80b5ec0 con 0x7fc3f8072c30 2024-03-29T04:11:27.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.457+0000 7fc3fcf57640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc3f806b5b0 0x7fc3f80cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.457+0000 7fc3f7fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc3f815e770 0x7fc3f8072520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50700/0 (socket says 172.21.15.148:50700) 2024-03-29T04:11:27.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.457+0000 7fc3f7fff640 1 -- 172.21.15.148:0/1725580301 learned_addr learned my addr 172.21.15.148:0/1725580301 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:27.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.457+0000 7fc3f77fe640 1 --2- 172.21.15.148:0/1725580301 >> v2:172.21.15.157:3300/0 conn(0x7fc3f8072c30 0x7fc3f80ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.457+0000 7fc3fcf57640 1 -- 172.21.15.148:0/1725580301 >> v2:172.21.15.157:3300/0 conn(0x7fc3f8072c30 msgr2=0x7fc3f80ce710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:27.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.457+0000 7fc3fcf57640 1 --2- 172.21.15.148:0/1725580301 >> v2:172.21.15.157:3300/0 conn(0x7fc3f8072c30 0x7fc3f80ce710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.457+0000 7fc3fcf57640 1 -- 172.21.15.148:0/1725580301 >> v1:172.21.15.148:6789/0 conn(0x7fc3f815e770 legacy=0x7fc3f8072520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.457+0000 7fc3f6ffd640 1 -- 172.21.15.148:0/1725580301 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2252396019 0 0) 0x7fc3f806dc80 con 0x7fc3f815e770 2024-03-29T04:11:27.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.457+0000 7fc3fcf57640 1 -- 172.21.15.148:0/1725580301 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f80b61a0 con 0x7fc3f806b5b0 2024-03-29T04:11:27.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.458+0000 7fc3fcf57640 1 --2- 172.21.15.148:0/1725580301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3f806b5b0 0x7fc3f80cf860 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc3f80b49f0 tx=0x7fc3f002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b36b63c5d140dbe server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:27.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.458+0000 7fc3f6ffd640 1 -- 172.21.15.148:0/1725580301 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0039070 con 0x7fc3f806b5b0 2024-03-29T04:11:27.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.458+0000 7fc3f6ffd640 1 -- 172.21.15.148:0/1725580301 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3f002cc80 con 0x7fc3f806b5b0 2024-03-29T04:11:27.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.458+0000 7fc3f6ffd640 1 -- 172.21.15.148:0/1725580301 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0034660 con 0x7fc3f806b5b0 2024-03-29T04:11:27.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.459+0000 7fc3fe9ba640 1 -- 172.21.15.148:0/1725580301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3f806b5b0 msgr2=0x7fc3f80cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.459+0000 7fc3fe9ba640 1 --2- 172.21.15.148:0/1725580301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3f806b5b0 0x7fc3f80cf860 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc3f80b49f0 tx=0x7fc3f002bed0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.459+0000 7fc3fe9ba640 1 -- 172.21.15.148:0/1725580301 shutdown_connections 2024-03-29T04:11:27.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.459+0000 7fc3fe9ba640 1 --2- 172.21.15.148:0/1725580301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3f806b5b0 0x7fc3f80cf860 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.459+0000 7fc3fe9ba640 1 --2- 172.21.15.148:0/1725580301 >> v2:172.21.15.157:3300/0 conn(0x7fc3f8072c30 0x7fc3f80ce710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.459+0000 7fc3fe9ba640 1 -- 172.21.15.148:0/1725580301 >> 172.21.15.148:0/1725580301 conn(0x7fc3f80d0f50 msgr2=0x7fc3f80d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:27.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.459+0000 7fc3fe9ba640 1 -- 172.21.15.148:0/1725580301 shutdown_connections 2024-03-29T04:11:27.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.459+0000 7fc3fe9ba640 1 -- 172.21.15.148:0/1725580301 wait complete. 2024-03-29T04:11:27.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.460+0000 7fc3fe9ba640 1 Processor -- start 2024-03-29T04:11:27.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.460+0000 7fc3fe9ba640 1 -- start start 2024-03-29T04:11:27.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.461+0000 7fc3fe9ba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3f806b5b0 0x7fc3f8153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.461+0000 7fc3fe9ba640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3f8072c30 0x7fc3f815d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.461+0000 7fc3fe9ba640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3f815e770 0x7fc3f81f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.461+0000 7fc3fe9ba640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f806da20 con 0x7fc3f8072c30 2024-03-29T04:11:27.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.461+0000 7fc3fe9ba640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f806d8a0 con 0x7fc3f806b5b0 2024-03-29T04:11:27.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.461+0000 7fc3fe9ba640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc3f80b5ae0 con 0x7fc3f815e770 2024-03-29T04:11:27.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.461+0000 7fc3f77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3f8072c30 0x7fc3f815d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.461+0000 7fc3f77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3f8072c30 0x7fc3f815d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39260/0 (socket says 172.21.15.148:39260) 2024-03-29T04:11:27.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.461+0000 7fc3f77fe640 1 -- 172.21.15.148:0/3650778050 learned_addr learned my addr 172.21.15.148:0/3650778050 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:27.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.461+0000 7fc3f7fff640 1 --2- 172.21.15.148:0/3650778050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3f806b5b0 0x7fc3f8153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.461+0000 7fc3fcf57640 1 --2- 172.21.15.148:0/3650778050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3f815e770 0x7fc3f81f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.461+0000 7fc3f77fe640 1 -- 172.21.15.148:0/3650778050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3f815e770 msgr2=0x7fc3f81f75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.461+0000 7fc3f77fe640 1 --2- 172.21.15.148:0/3650778050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3f815e770 0x7fc3f81f75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.461+0000 7fc3f77fe640 1 -- 172.21.15.148:0/3650778050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3f806b5b0 msgr2=0x7fc3f8153440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.461+0000 7fc3f77fe640 1 --2- 172.21.15.148:0/3650778050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3f806b5b0 0x7fc3f8153440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.461+0000 7fc3f77fe640 1 -- 172.21.15.148:0/3650778050 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f8153ad0 con 0x7fc3f8072c30 2024-03-29T04:11:27.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.462+0000 7fc3f77fe640 1 --2- 172.21.15.148:0/3650778050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3f8072c30 0x7fc3f815d960 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00c7f0 tx=0x7fc3ec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:27.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.462+0000 7fc3f4ff9640 1 -- 172.21.15.148:0/3650778050 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec015070 con 0x7fc3f8072c30 2024-03-29T04:11:27.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.462+0000 7fc3f4ff9640 1 -- 172.21.15.148:0/3650778050 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3ec004240 con 0x7fc3f8072c30 2024-03-29T04:11:27.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.462+0000 7fc3fe9ba640 1 -- 172.21.15.148:0/3650778050 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f8160800 con 0x7fc3f8072c30 2024-03-29T04:11:27.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.462+0000 7fc3f4ff9640 1 -- 172.21.15.148:0/3650778050 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec007620 con 0x7fc3f8072c30 2024-03-29T04:11:27.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.462+0000 7fc3fe9ba640 1 -- 172.21.15.148:0/3650778050 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f8160dc0 con 0x7fc3f8072c30 2024-03-29T04:11:27.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.463+0000 7fc3fe9ba640 1 -- 172.21.15.148:0/3650778050 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3f80687e0 con 0x7fc3f8072c30 2024-03-29T04:11:27.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.463+0000 7fc3f4ff9640 1 -- 172.21.15.148:0/3650778050 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc3ec0079d0 con 0x7fc3f8072c30 2024-03-29T04:11:27.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.464+0000 7fc3f4ff9640 1 --2- 172.21.15.148:0/3650778050 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc3e406c530 0x7fc3e406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.464+0000 7fc3f4ff9640 1 -- 172.21.15.148:0/3650778050 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fc3ec08b920 con 0x7fc3f8072c30 2024-03-29T04:11:27.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.464+0000 7fc3f7fff640 1 --2- 172.21.15.148:0/3650778050 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc3e406c530 0x7fc3e406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.464+0000 7fc3f7fff640 1 --2- 172.21.15.148:0/3650778050 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc3e406c530 0x7fc3e406e9f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8000f60 tx=0x7fc3e8007b50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:27.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.468+0000 7fc3f4ff9640 1 -- 172.21.15.148:0/3650778050 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc3ec05a190 con 0x7fc3f8072c30 2024-03-29T04:11:27.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.708+0000 7fc3fe9ba640 1 -- 172.21.15.148:0/3650778050 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fc3f8073870 con 0x7fc3f8072c30 2024-03-29T04:11:27.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.710+0000 7fc3f4ff9640 1 -- 172.21.15.148:0/3650778050 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7fc3ec059a90 con 0x7fc3f8072c30 2024-03-29T04:11:27.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.712+0000 7fc3fe9ba640 1 -- 172.21.15.148:0/3650778050 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc3e406c530 msgr2=0x7fc3e406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.712+0000 7fc3fe9ba640 1 --2- 172.21.15.148:0/3650778050 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc3e406c530 0x7fc3e406e9f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8000f60 tx=0x7fc3e8007b50 comp rx=0 tx=0).stop 2024-03-29T04:11:27.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.712+0000 7fc3fe9ba640 1 -- 172.21.15.148:0/3650778050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3f8072c30 msgr2=0x7fc3f815d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.712+0000 7fc3fe9ba640 1 --2- 172.21.15.148:0/3650778050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3f8072c30 0x7fc3f815d960 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00c7f0 tx=0x7fc3ec00ccb0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.712+0000 7fc3fe9ba640 1 -- 172.21.15.148:0/3650778050 shutdown_connections 2024-03-29T04:11:27.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.712+0000 7fc3fe9ba640 1 --2- 172.21.15.148:0/3650778050 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc3e406c530 0x7fc3e406e9f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.712+0000 7fc3fe9ba640 1 --2- 172.21.15.148:0/3650778050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3f815e770 0x7fc3f81f75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.712+0000 7fc3fe9ba640 1 --2- 172.21.15.148:0/3650778050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3f8072c30 0x7fc3f815d960 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.712+0000 7fc3fe9ba640 1 --2- 172.21.15.148:0/3650778050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3f806b5b0 0x7fc3f8153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.712+0000 7fc3fe9ba640 1 -- 172.21.15.148:0/3650778050 >> 172.21.15.148:0/3650778050 conn(0x7fc3f80d0f50 msgr2=0x7fc3f806cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:27.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.712+0000 7fc3fe9ba640 1 -- 172.21.15.148:0/3650778050 shutdown_connections 2024-03-29T04:11:27.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.712+0000 7fc3fe9ba640 1 -- 172.21.15.148:0/3650778050 wait complete. 2024-03-29T04:11:27.702 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-29T04:11:27.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.824+0000 7f1591fdb640 1 Processor -- start 2024-03-29T04:11:27.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.824+0000 7f1591fdb640 1 -- start start 2024-03-29T04:11:27.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.825+0000 7f1591fdb640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f158c15d3e0 0x7f158c0cd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.825+0000 7f1591fdb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f158c0cd650 0x7f158c06ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.825+0000 7f1591fdb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f158c0cfa20 con 0x7f158c0afce0 2024-03-29T04:11:27.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.825+0000 7f1591fdb640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f158c06b5b0 con 0x7f158c15d3e0 2024-03-29T04:11:27.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.825+0000 7f1591fdb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f158c0cf7c0 con 0x7f158c0cd650 2024-03-29T04:11:27.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.825+0000 7f158affd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f158c15d3e0 0x7f158c0cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.825+0000 7f158affd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f158c15d3e0 0x7f158c0cd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39270/0 (socket says 172.21.15.148:39270) 2024-03-29T04:11:27.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.825+0000 7f158affd640 1 -- 172.21.15.148:0/3497059098 learned_addr learned my addr 172.21.15.148:0/3497059098 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:27.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.825+0000 7f158bfff640 1 --2- 172.21.15.148:0/3497059098 >> v2:172.21.15.157:3300/0 conn(0x7f158c0cd650 0x7f158c06ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.825+0000 7f158affd640 1 -- 172.21.15.148:0/3497059098 >> v2:172.21.15.157:3300/0 conn(0x7f158c0cd650 msgr2=0x7f158c06ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:27.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.825+0000 7f158affd640 1 --2- 172.21.15.148:0/3497059098 >> v2:172.21.15.157:3300/0 conn(0x7f158c0cd650 0x7f158c06ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.825+0000 7f158affd640 1 -- 172.21.15.148:0/3497059098 >> v1:172.21.15.157:6789/0 conn(0x7f158c0afce0 legacy=0x7f158c0cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:27.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.825+0000 7f158affd640 1 -- 172.21.15.148:0/3497059098 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158c0cfd00 con 0x7f158c15d3e0 2024-03-29T04:11:27.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.825+0000 7f158bfff640 1 --2- 172.21.15.148:0/3497059098 >> v2:172.21.15.157:3300/0 conn(0x7f158c0cd650 0x7f158c06ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.826+0000 7f158affd640 1 --2- 172.21.15.148:0/3497059098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f158c15d3e0 0x7f158c0cd110 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f158c156d80 tx=0x7f158002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4719b4cc34f3f67f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.826+0000 7f158a7fc640 1 -- 172.21.15.148:0/3497059098 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1580035020 con 0x7f158c15d3e0 2024-03-29T04:11:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.826+0000 7f158a7fc640 1 -- 172.21.15.148:0/3497059098 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1580031870 con 0x7f158c15d3e0 2024-03-29T04:11:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.826+0000 7f158a7fc640 1 -- 172.21.15.148:0/3497059098 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1580031c40 con 0x7f158c15d3e0 2024-03-29T04:11:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.827+0000 7f1591fdb640 1 -- 172.21.15.148:0/3497059098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f158c15d3e0 msgr2=0x7f158c0cd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.827+0000 7f1591fdb640 1 --2- 172.21.15.148:0/3497059098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f158c15d3e0 0x7f158c0cd110 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f158c156d80 tx=0x7f158002b910 comp rx=0 tx=0).stop 2024-03-29T04:11:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.827+0000 7f1591fdb640 1 -- 172.21.15.148:0/3497059098 shutdown_connections 2024-03-29T04:11:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.827+0000 7f1591fdb640 1 --2- 172.21.15.148:0/3497059098 >> v2:172.21.15.157:3300/0 conn(0x7f158c0cd650 0x7f158c06ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.827+0000 7f1591fdb640 1 --2- 172.21.15.148:0/3497059098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f158c15d3e0 0x7f158c0cd110 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.827+0000 7f1591fdb640 1 -- 172.21.15.148:0/3497059098 >> 172.21.15.148:0/3497059098 conn(0x7f158c0d0700 msgr2=0x7f158c0d0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.827+0000 7f1591fdb640 1 -- 172.21.15.148:0/3497059098 shutdown_connections 2024-03-29T04:11:27.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.827+0000 7f1591fdb640 1 -- 172.21.15.148:0/3497059098 wait complete. 2024-03-29T04:11:27.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.828+0000 7f1591fdb640 1 Processor -- start 2024-03-29T04:11:27.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.828+0000 7f1591fdb640 1 -- start start 2024-03-29T04:11:27.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.829+0000 7f1591fdb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f158c0afce0 0x7f158c06ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.829+0000 7f1591fdb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f158c0cd650 0x7f158c1586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.829+0000 7f1591fdb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f158c15d3e0 0x7f158c156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.829+0000 7f1591fdb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f158c0d02c0 con 0x7f158c0cd650 2024-03-29T04:11:27.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.829+0000 7f1591fdb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f158c0cda80 con 0x7f158c0afce0 2024-03-29T04:11:27.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.829+0000 7f1591fdb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f158c0b00c0 con 0x7f158c15d3e0 2024-03-29T04:11:27.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.829+0000 7f158affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f158c0cd650 0x7f158c1586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.829+0000 7f158affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f158c0cd650 0x7f158c1586c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39276/0 (socket says 172.21.15.148:39276) 2024-03-29T04:11:27.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.829+0000 7f158affd640 1 -- 172.21.15.148:0/2507624687 learned_addr learned my addr 172.21.15.148:0/2507624687 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:27.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.829+0000 7f158affd640 1 -- 172.21.15.148:0/2507624687 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f158c15d3e0 msgr2=0x7f158c156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.829+0000 7f158affd640 1 --2- 172.21.15.148:0/2507624687 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f158c15d3e0 0x7f158c156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.829+0000 7f158affd640 1 -- 172.21.15.148:0/2507624687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f158c0afce0 msgr2=0x7f158c06ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.829+0000 7f158b7fe640 1 --2- 172.21.15.148:0/2507624687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f158c0afce0 0x7f158c06ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.829+0000 7f158affd640 1 --2- 172.21.15.148:0/2507624687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f158c0afce0 0x7f158c06ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.829+0000 7f158affd640 1 -- 172.21.15.148:0/2507624687 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158c1456c0 con 0x7f158c0cd650 2024-03-29T04:11:27.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.829+0000 7f158b7fe640 1 --2- 172.21.15.148:0/2507624687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f158c0afce0 0x7f158c06ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:27.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.829+0000 7f158affd640 1 --2- 172.21.15.148:0/2507624687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f158c0cd650 0x7f158c1586c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1580000c00 tx=0x7f15800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:27.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.830+0000 7f1573fff640 1 -- 172.21.15.148:0/2507624687 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1580040070 con 0x7f158c0cd650 2024-03-29T04:11:27.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.830+0000 7f1573fff640 1 -- 172.21.15.148:0/2507624687 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f158003a420 con 0x7f158c0cd650 2024-03-29T04:11:27.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.830+0000 7f1591fdb640 1 -- 172.21.15.148:0/2507624687 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f158c157620 con 0x7f158c0cd650 2024-03-29T04:11:27.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.830+0000 7f1573fff640 1 -- 172.21.15.148:0/2507624687 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1580035040 con 0x7f158c0cd650 2024-03-29T04:11:27.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.830+0000 7f1591fdb640 1 -- 172.21.15.148:0/2507624687 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f158c157be0 con 0x7f158c0cd650 2024-03-29T04:11:27.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.831+0000 7f1591fdb640 1 -- 172.21.15.148:0/2507624687 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f158c0687e0 con 0x7f158c0cd650 2024-03-29T04:11:27.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.831+0000 7f1573fff640 1 -- 172.21.15.148:0/2507624687 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f158004b060 con 0x7f158c0cd650 2024-03-29T04:11:27.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.831+0000 7f1573fff640 1 --2- 172.21.15.148:0/2507624687 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f157806c530 0x7f157806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.832+0000 7f1573fff640 1 -- 172.21.15.148:0/2507624687 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f15800ad460 con 0x7f158c0cd650 2024-03-29T04:11:27.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.832+0000 7f158b7fe640 1 --2- 172.21.15.148:0/2507624687 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f157806c530 0x7f157806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.832+0000 7f158b7fe640 1 --2- 172.21.15.148:0/2507624687 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f157806c530 0x7f157806e9f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f157c003eb0 tx=0x7f157c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:27.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:27.836+0000 7f1573fff640 1 -- 172.21.15.148:0/2507624687 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f158007bc40 con 0x7f158c0cd650 2024-03-29T04:11:28.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.083+0000 7f1591fdb640 1 -- 172.21.15.148:0/2507624687 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f158c15b600 con 0x7f158c0cd650 2024-03-29T04:11:28.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.085+0000 7f1573fff640 1 -- 172.21.15.148:0/2507624687 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f158007b540 con 0x7f158c0cd650 2024-03-29T04:11:28.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.087+0000 7f1591fdb640 1 -- 172.21.15.148:0/2507624687 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f157806c530 msgr2=0x7f157806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.087+0000 7f1591fdb640 1 --2- 172.21.15.148:0/2507624687 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f157806c530 0x7f157806e9f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f157c003eb0 tx=0x7f157c009040 comp rx=0 tx=0).stop 2024-03-29T04:11:28.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.087+0000 7f1591fdb640 1 -- 172.21.15.148:0/2507624687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f158c0cd650 msgr2=0x7f158c1586c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.087+0000 7f1591fdb640 1 --2- 172.21.15.148:0/2507624687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f158c0cd650 0x7f158c1586c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1580000c00 tx=0x7f15800046c0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.088+0000 7f1591fdb640 1 -- 172.21.15.148:0/2507624687 shutdown_connections 2024-03-29T04:11:28.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.088+0000 7f1591fdb640 1 --2- 172.21.15.148:0/2507624687 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f157806c530 0x7f157806e9f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.088+0000 7f1591fdb640 1 --2- 172.21.15.148:0/2507624687 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f158c15d3e0 0x7f158c156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.088+0000 7f1591fdb640 1 --2- 172.21.15.148:0/2507624687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f158c0cd650 0x7f158c1586c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.088+0000 7f1591fdb640 1 --2- 172.21.15.148:0/2507624687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f158c0afce0 0x7f158c06ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.088+0000 7f1591fdb640 1 -- 172.21.15.148:0/2507624687 >> 172.21.15.148:0/2507624687 conn(0x7f158c0d0700 msgr2=0x7f158c15ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:28.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.088+0000 7f1591fdb640 1 -- 172.21.15.148:0/2507624687 shutdown_connections 2024-03-29T04:11:28.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.088+0000 7f1591fdb640 1 -- 172.21.15.148:0/2507624687 wait complete. 2024-03-29T04:11:28.077 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-29T04:11:28.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.201+0000 7ff272fc8640 1 Processor -- start 2024-03-29T04:11:28.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.201+0000 7ff272fc8640 1 -- start start 2024-03-29T04:11:28.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.201+0000 7ff272fc8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff26c0d17e0 0x7ff26c0d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.201+0000 7ff272fc8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff26c15e7c0 con 0x7ff26c15e3e0 2024-03-29T04:11:28.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.201+0000 7ff272fc8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff26c0d2c30 con 0x7ff26c0ba030 2024-03-29T04:11:28.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.201+0000 7ff272fc8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff26c0d3840 con 0x7ff26c0d17e0 2024-03-29T04:11:28.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.202+0000 7ff270d64640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff26c15e3e0 0x7ff26c0d2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50708/0 (socket says 172.21.15.148:50708) 2024-03-29T04:11:28.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.202+0000 7ff270d64640 1 -- 172.21.15.148:0/1822790076 learned_addr learned my addr 172.21.15.148:0/1822790076 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:28.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.202+0000 7ff271565640 1 --2- 172.21.15.148:0/1822790076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff26c0d17e0 0x7ff26c0d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.202+0000 7ff26b7fe640 1 -- 172.21.15.148:0/1822790076 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3592863962 0 0) 0x7ff26c15e7c0 con 0x7ff26c15e3e0 2024-03-29T04:11:28.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.202+0000 7ff26b7fe640 1 -- 172.21.15.148:0/1822790076 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff25c003520 con 0x7ff26c15e3e0 2024-03-29T04:11:28.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.202+0000 7ff26b7fe640 1 -- 172.21.15.148:0/1822790076 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3253452149 0 0) 0x7ff26c0d2c30 con 0x7ff26c0ba030 2024-03-29T04:11:28.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.202+0000 7ff26b7fe640 1 -- 172.21.15.148:0/1822790076 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff26c15e7c0 con 0x7ff26c0ba030 2024-03-29T04:11:28.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.202+0000 7ff26b7fe640 1 -- 172.21.15.148:0/1822790076 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2240790556 0 0) 0x7ff25c003520 con 0x7ff26c15e3e0 2024-03-29T04:11:28.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.202+0000 7ff26b7fe640 1 -- 172.21.15.148:0/1822790076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff26c0d17e0 msgr2=0x7ff26c0d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:28.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.202+0000 7ff26b7fe640 1 --2- 172.21.15.148:0/1822790076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff26c0d17e0 0x7ff26c0d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.202+0000 7ff26b7fe640 1 -- 172.21.15.148:0/1822790076 >> v1:172.21.15.157:6789/0 conn(0x7ff26c0ba030 legacy=0x7ff26c0d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.202+0000 7ff26b7fe640 1 -- 172.21.15.148:0/1822790076 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26c0d2f10 con 0x7ff26c15e3e0 2024-03-29T04:11:28.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.202+0000 7ff26b7fe640 1 -- 172.21.15.148:0/1822790076 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7ff260001670 con 0x7ff26c15e3e0 2024-03-29T04:11:28.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.202+0000 7ff26b7fe640 1 -- 172.21.15.148:0/1822790076 >> v1:172.21.15.148:6789/0 conn(0x7ff26c15e3e0 legacy=0x7ff26c0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.202+0000 7ff26b7fe640 1 --2- 172.21.15.148:0/1822790076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff26c0d17e0 0x7ff25c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.202+0000 7ff26b7fe640 1 --2- 172.21.15.148:0/1822790076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff25c0040f0 0x7ff25c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.202+0000 7ff26b7fe640 1 --2- 172.21.15.148:0/1822790076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff25c004d40 0x7ff25c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.202+0000 7ff26b7fe640 1 -- 172.21.15.148:0/1822790076 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c0d3840 con 0x7ff25c0040f0 2024-03-29T04:11:28.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.202+0000 7ff26b7fe640 1 -- 172.21.15.148:0/1822790076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c00baa0 con 0x7ff25c004d40 2024-03-29T04:11:28.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.202+0000 7ff26b7fe640 1 -- 172.21.15.148:0/1822790076 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff25c00bca0 con 0x7ff26c0d17e0 2024-03-29T04:11:28.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.202+0000 7ff26bfff640 1 --2- 172.21.15.148:0/1822790076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff25c0040f0 0x7ff25c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.202+0000 7ff270d64640 1 --2- 172.21.15.148:0/1822790076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff26c0d17e0 0x7ff25c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.202+0000 7ff271565640 1 --2- 172.21.15.148:0/1822790076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff25c004d40 0x7ff25c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.203+0000 7ff26bfff640 1 -- 172.21.15.148:0/1822790076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff26c0d17e0 msgr2=0x7ff25c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:28.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.203+0000 7ff26bfff640 1 --2- 172.21.15.148:0/1822790076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff26c0d17e0 0x7ff25c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.203+0000 7ff26bfff640 1 -- 172.21.15.148:0/1822790076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff25c004d40 msgr2=0x7ff25c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:28.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.203+0000 7ff26bfff640 1 --2- 172.21.15.148:0/1822790076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff25c004d40 0x7ff25c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.203+0000 7ff26bfff640 1 -- 172.21.15.148:0/1822790076 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff25c003520 con 0x7ff25c0040f0 2024-03-29T04:11:28.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.203+0000 7ff26bfff640 1 --2- 172.21.15.148:0/1822790076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff25c0040f0 0x7ff25c004800 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff2580018d0 tx=0x7ff25c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8376cc934b08ade2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:28.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.204+0000 7ff26b7fe640 1 -- 172.21.15.148:0/1822790076 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff258037070 con 0x7ff25c0040f0 2024-03-29T04:11:28.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.204+0000 7ff26b7fe640 1 -- 172.21.15.148:0/1822790076 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2580052d0 con 0x7ff25c0040f0 2024-03-29T04:11:28.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.204+0000 7ff26b7fe640 1 -- 172.21.15.148:0/1822790076 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2580014f0 con 0x7ff25c0040f0 2024-03-29T04:11:28.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.204+0000 7ff272fc8640 1 -- 172.21.15.148:0/1822790076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff25c0040f0 msgr2=0x7ff25c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.204+0000 7ff272fc8640 1 --2- 172.21.15.148:0/1822790076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff25c0040f0 0x7ff25c004800 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff2580018d0 tx=0x7ff25c005830 comp rx=0 tx=0).stop 2024-03-29T04:11:28.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.204+0000 7ff271565640 1 -- 172.21.15.148:0/1822790076 reap_dead start 2024-03-29T04:11:28.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.204+0000 7ff272fc8640 1 -- 172.21.15.148:0/1822790076 shutdown_connections 2024-03-29T04:11:28.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.204+0000 7ff272fc8640 1 -- 172.21.15.148:0/1822790076 >> 172.21.15.148:0/1822790076 conn(0x7ff26c0b6210 msgr2=0x7ff26c0b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:28.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.204+0000 7ff272fc8640 1 -- 172.21.15.148:0/1822790076 shutdown_connections 2024-03-29T04:11:28.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.204+0000 7ff272fc8640 1 -- 172.21.15.148:0/1822790076 wait complete. 2024-03-29T04:11:28.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.205+0000 7ff272fc8640 1 Processor -- start 2024-03-29T04:11:28.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.205+0000 7ff272fc8640 1 -- start start 2024-03-29T04:11:28.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.206+0000 7ff272fc8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff25c0040f0 0x7ff26c0c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.206+0000 7ff272fc8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff26c0c8550 0x7ff26c15f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.206+0000 7ff272fc8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff26c15fca0 0x7ff26c160120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.206+0000 7ff272fc8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c0d3cd0 con 0x7ff25c0040f0 2024-03-29T04:11:28.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.206+0000 7ff272fc8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c0d34d0 con 0x7ff26c15fca0 2024-03-29T04:11:28.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.206+0000 7ff272fc8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff26c0d3e50 con 0x7ff26c0c8550 2024-03-29T04:11:28.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.206+0000 7ff270d64640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff25c0040f0 0x7ff26c0c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.206+0000 7ff270d64640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff25c0040f0 0x7ff26c0c8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39306/0 (socket says 172.21.15.148:39306) 2024-03-29T04:11:28.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.206+0000 7ff270d64640 1 -- 172.21.15.148:0/2391944343 learned_addr learned my addr 172.21.15.148:0/2391944343 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:28.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.206+0000 7ff271565640 1 --2- 172.21.15.148:0/2391944343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff26c15fca0 0x7ff26c160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.206+0000 7ff26bfff640 1 --2- 172.21.15.148:0/2391944343 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff26c0c8550 0x7ff26c15f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.207+0000 7ff270d64640 1 -- 172.21.15.148:0/2391944343 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff26c0c8550 msgr2=0x7ff26c15f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.207+0000 7ff270d64640 1 --2- 172.21.15.148:0/2391944343 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff26c0c8550 0x7ff26c15f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.207+0000 7ff270d64640 1 -- 172.21.15.148:0/2391944343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff26c15fca0 msgr2=0x7ff26c160120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.207+0000 7ff270d64640 1 --2- 172.21.15.148:0/2391944343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff26c15fca0 0x7ff26c160120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.207+0000 7ff270d64640 1 -- 172.21.15.148:0/2391944343 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26c160660 con 0x7ff25c0040f0 2024-03-29T04:11:28.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.207+0000 7ff271565640 1 --2- 172.21.15.148:0/2391944343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff26c15fca0 0x7ff26c160120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:28.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.207+0000 7ff270d64640 1 --2- 172.21.15.148:0/2391944343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff25c0040f0 0x7ff26c0c8010 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff260007340 tx=0x7ff260007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:28.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.207+0000 7ff2697fa640 1 -- 172.21.15.148:0/2391944343 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26000f860 con 0x7ff25c0040f0 2024-03-29T04:11:28.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.207+0000 7ff2697fa640 1 -- 172.21.15.148:0/2391944343 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff260018070 con 0x7ff25c0040f0 2024-03-29T04:11:28.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.207+0000 7ff272fc8640 1 -- 172.21.15.148:0/2391944343 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff26c153830 con 0x7ff25c0040f0 2024-03-29T04:11:28.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.207+0000 7ff2697fa640 1 -- 172.21.15.148:0/2391944343 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26000fc60 con 0x7ff25c0040f0 2024-03-29T04:11:28.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.207+0000 7ff272fc8640 1 -- 172.21.15.148:0/2391944343 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff26c0c90a0 con 0x7ff25c0040f0 2024-03-29T04:11:28.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.208+0000 7ff272fc8640 1 -- 172.21.15.148:0/2391944343 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff26c0687e0 con 0x7ff25c0040f0 2024-03-29T04:11:28.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.208+0000 7ff2697fa640 1 -- 172.21.15.148:0/2391944343 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff26001d460 con 0x7ff25c0040f0 2024-03-29T04:11:28.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.209+0000 7ff2697fa640 1 --2- 172.21.15.148:0/2391944343 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff25006c560 0x7ff25006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.209+0000 7ff2697fa640 1 -- 172.21.15.148:0/2391944343 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7ff26008b5d0 con 0x7ff25c0040f0 2024-03-29T04:11:28.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.209+0000 7ff26bfff640 1 --2- 172.21.15.148:0/2391944343 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff25006c560 0x7ff25006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.209+0000 7ff26bfff640 1 --2- 172.21.15.148:0/2391944343 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff25006c560 0x7ff25006ea20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff258034000 tx=0x7ff258031000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:28.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.213+0000 7ff2697fa640 1 -- 172.21.15.148:0/2391944343 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff260059db0 con 0x7ff25c0040f0 2024-03-29T04:11:28.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.455+0000 7ff272fc8640 1 -- 172.21.15.148:0/2391944343 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7ff26c0b65f0 con 0x7ff25c0040f0 2024-03-29T04:11:28.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.457+0000 7ff2697fa640 1 -- 172.21.15.148:0/2391944343 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7ff2600596b0 con 0x7ff25c0040f0 2024-03-29T04:11:28.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.459+0000 7ff272fc8640 1 -- 172.21.15.148:0/2391944343 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff25006c560 msgr2=0x7ff25006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.459+0000 7ff272fc8640 1 --2- 172.21.15.148:0/2391944343 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff25006c560 0x7ff25006ea20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff258034000 tx=0x7ff258031000 comp rx=0 tx=0).stop 2024-03-29T04:11:28.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.459+0000 7ff272fc8640 1 -- 172.21.15.148:0/2391944343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff25c0040f0 msgr2=0x7ff26c0c8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.459+0000 7ff272fc8640 1 --2- 172.21.15.148:0/2391944343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff25c0040f0 0x7ff26c0c8010 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff260007340 tx=0x7ff260007800 comp rx=0 tx=0).stop 2024-03-29T04:11:28.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.459+0000 7ff272fc8640 1 -- 172.21.15.148:0/2391944343 shutdown_connections 2024-03-29T04:11:28.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.459+0000 7ff272fc8640 1 --2- 172.21.15.148:0/2391944343 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff25006c560 0x7ff25006ea20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.459+0000 7ff272fc8640 1 --2- 172.21.15.148:0/2391944343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff26c15fca0 0x7ff26c160120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.459+0000 7ff272fc8640 1 --2- 172.21.15.148:0/2391944343 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff26c0c8550 0x7ff26c15f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.459+0000 7ff272fc8640 1 --2- 172.21.15.148:0/2391944343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff25c0040f0 0x7ff26c0c8010 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.459+0000 7ff272fc8640 1 -- 172.21.15.148:0/2391944343 >> 172.21.15.148:0/2391944343 conn(0x7ff26c0b6210 msgr2=0x7ff26c0b8520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:28.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.459+0000 7ff272fc8640 1 -- 172.21.15.148:0/2391944343 shutdown_connections 2024-03-29T04:11:28.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.459+0000 7ff272fc8640 1 -- 172.21.15.148:0/2391944343 wait complete. 2024-03-29T04:11:28.449 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-29T04:11:28.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.570+0000 7f7a5bf60640 1 Processor -- start 2024-03-29T04:11:28.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.571+0000 7f7a5bf60640 1 -- start start 2024-03-29T04:11:28.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.572+0000 7f7a5bf60640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7a540b3710 0x7f7a540b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.572+0000 7f7a5bf60640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a5406b5b0 0x7f7a5406ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.572+0000 7f7a5bf60640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a5406dd30 con 0x7f7a54158c30 2024-03-29T04:11:28.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.572+0000 7f7a5bf60640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f7a5406e010 con 0x7f7a540b3710 2024-03-29T04:11:28.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.572+0000 7f7a5bf60640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7a5406e2f0 con 0x7f7a5406b5b0 2024-03-29T04:11:28.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.572+0000 7f7a59cfc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7a54158c30 0x7f7a5415f480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50716/0 (socket says 172.21.15.148:50716) 2024-03-29T04:11:28.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.572+0000 7f7a59cfc640 1 -- 172.21.15.148:0/397237578 learned_addr learned my addr 172.21.15.148:0/397237578 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:28.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.572+0000 7f7a5a4fd640 1 --2- 172.21.15.148:0/397237578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a5406b5b0 0x7f7a5406ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.572+0000 7f7a594fb640 1 --2- 172.21.15.148:0/397237578 >> v2:172.21.15.157:3300/0 conn(0x7f7a540b3710 0x7f7a540b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.572+0000 7f7a58cfa640 1 -- 172.21.15.148:0/397237578 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1766398613 0 0) 0x7f7a5406dd30 con 0x7f7a54158c30 2024-03-29T04:11:28.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.572+0000 7f7a58cfa640 1 -- 172.21.15.148:0/397237578 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a40003520 con 0x7f7a54158c30 2024-03-29T04:11:28.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.572+0000 7f7a594fb640 1 -- 172.21.15.148:0/397237578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a5406b5b0 msgr2=0x7f7a5406ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:28.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.572+0000 7f7a594fb640 1 --2- 172.21.15.148:0/397237578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a5406b5b0 0x7f7a5406ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.572+0000 7f7a594fb640 1 -- 172.21.15.148:0/397237578 >> v1:172.21.15.148:6789/0 conn(0x7f7a54158c30 legacy=0x7f7a5415f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.572+0000 7f7a594fb640 1 -- 172.21.15.148:0/397237578 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5406e5d0 con 0x7f7a540b3710 2024-03-29T04:11:28.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.573+0000 7f7a594fb640 1 --2- 172.21.15.148:0/397237578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a540b3710 0x7f7a540b3b90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7a500097a0 tx=0x7f7a5002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77ce0e4812e0bbf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:28.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.573+0000 7f7a58cfa640 1 -- 172.21.15.148:0/397237578 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a500315a0 con 0x7f7a540b3710 2024-03-29T04:11:28.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.573+0000 7f7a58cfa640 1 -- 172.21.15.148:0/397237578 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a50037070 con 0x7f7a540b3710 2024-03-29T04:11:28.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.573+0000 7f7a58cfa640 1 -- 172.21.15.148:0/397237578 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a50031940 con 0x7f7a540b3710 2024-03-29T04:11:28.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.574+0000 7f7a5bf60640 1 -- 172.21.15.148:0/397237578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a540b3710 msgr2=0x7f7a540b3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.574+0000 7f7a5bf60640 1 --2- 172.21.15.148:0/397237578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a540b3710 0x7f7a540b3b90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7a500097a0 tx=0x7f7a5002b840 comp rx=0 tx=0).stop 2024-03-29T04:11:28.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.574+0000 7f7a5bf60640 1 -- 172.21.15.148:0/397237578 shutdown_connections 2024-03-29T04:11:28.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.574+0000 7f7a5bf60640 1 --2- 172.21.15.148:0/397237578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a5406b5b0 0x7f7a5406ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.574+0000 7f7a5bf60640 1 --2- 172.21.15.148:0/397237578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a540b3710 0x7f7a540b3b90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.574+0000 7f7a5bf60640 1 -- 172.21.15.148:0/397237578 >> 172.21.15.148:0/397237578 conn(0x7f7a540d0700 msgr2=0x7f7a540d0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:28.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.574+0000 7f7a5bf60640 1 -- 172.21.15.148:0/397237578 shutdown_connections 2024-03-29T04:11:28.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.574+0000 7f7a5bf60640 1 -- 172.21.15.148:0/397237578 wait complete. 2024-03-29T04:11:28.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.575+0000 7f7a5bf60640 1 Processor -- start 2024-03-29T04:11:28.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.575+0000 7f7a5bf60640 1 -- start start 2024-03-29T04:11:28.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.576+0000 7f7a5bf60640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a5406b5b0 0x7f7a541494d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.576+0000 7f7a5bf60640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a540b3710 0x7f7a540ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.576+0000 7f7a5bf60640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a54158c30 0x7f7a5415dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.576+0000 7f7a5bf60640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7a540b4400 con 0x7f7a540b3710 2024-03-29T04:11:28.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.576+0000 7f7a5bf60640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5406da00 con 0x7f7a5406b5b0 2024-03-29T04:11:28.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.576+0000 7f7a5bf60640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7a540b4580 con 0x7f7a54158c30 2024-03-29T04:11:28.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.576+0000 7f7a594fb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a540b3710 0x7f7a540ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.576+0000 7f7a594fb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a540b3710 0x7f7a540ce100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39314/0 (socket says 172.21.15.148:39314) 2024-03-29T04:11:28.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.576+0000 7f7a594fb640 1 -- 172.21.15.148:0/1195313938 learned_addr learned my addr 172.21.15.148:0/1195313938 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:28.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.576+0000 7f7a59cfc640 1 --2- 172.21.15.148:0/1195313938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a5406b5b0 0x7f7a541494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.576+0000 7f7a594fb640 1 -- 172.21.15.148:0/1195313938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a54158c30 msgr2=0x7f7a5415dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.576+0000 7f7a594fb640 1 --2- 172.21.15.148:0/1195313938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a54158c30 0x7f7a5415dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.576+0000 7f7a594fb640 1 -- 172.21.15.148:0/1195313938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a5406b5b0 msgr2=0x7f7a541494d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.576+0000 7f7a594fb640 1 --2- 172.21.15.148:0/1195313938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a5406b5b0 0x7f7a541494d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.576+0000 7f7a594fb640 1 -- 172.21.15.148:0/1195313938 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a54149a10 con 0x7f7a540b3710 2024-03-29T04:11:28.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.576+0000 7f7a594fb640 1 --2- 172.21.15.148:0/1195313938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a540b3710 0x7f7a540ce100 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7a50034040 tx=0x7f7a500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:28.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.576+0000 7f7a4a7fc640 1 -- 172.21.15.148:0/1195313938 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a50032630 con 0x7f7a540b3710 2024-03-29T04:11:28.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.576+0000 7f7a4a7fc640 1 -- 172.21.15.148:0/1195313938 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a500327f0 con 0x7f7a540b3710 2024-03-29T04:11:28.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.576+0000 7f7a5bf60640 1 -- 172.21.15.148:0/1195313938 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a540cebc0 con 0x7f7a540b3710 2024-03-29T04:11:28.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.576+0000 7f7a4a7fc640 1 -- 172.21.15.148:0/1195313938 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a50032be0 con 0x7f7a540b3710 2024-03-29T04:11:28.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.576+0000 7f7a5bf60640 1 -- 172.21.15.148:0/1195313938 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a541593a0 con 0x7f7a540b3710 2024-03-29T04:11:28.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.577+0000 7f7a5bf60640 1 -- 172.21.15.148:0/1195313938 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a540744a0 con 0x7f7a540b3710 2024-03-29T04:11:28.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.578+0000 7f7a4a7fc640 1 -- 172.21.15.148:0/1195313938 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7a5003b3f0 con 0x7f7a540b3710 2024-03-29T04:11:28.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.578+0000 7f7a4a7fc640 1 --2- 172.21.15.148:0/1195313938 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7a3806c530 0x7f7a3806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.578+0000 7f7a4a7fc640 1 -- 172.21.15.148:0/1195313938 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f7a5002f030 con 0x7f7a540b3710 2024-03-29T04:11:28.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.578+0000 7f7a59cfc640 1 --2- 172.21.15.148:0/1195313938 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7a3806c530 0x7f7a3806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.579+0000 7f7a59cfc640 1 --2- 172.21.15.148:0/1195313938 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7a3806c530 0x7f7a3806e9f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7a540cf3b0 tx=0x7f7a44009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:28.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.582+0000 7f7a4a7fc640 1 -- 172.21.15.148:0/1195313938 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7a50040070 con 0x7f7a540b3710 2024-03-29T04:11:28.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.823+0000 7f7a5bf60640 1 -- 172.21.15.148:0/1195313938 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f7a540d0fd0 con 0x7f7a540b3710 2024-03-29T04:11:28.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.827+0000 7f7a4a7fc640 1 -- 172.21.15.148:0/1195313938 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f7a540d0fd0 con 0x7f7a540b3710 2024-03-29T04:11:28.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.830+0000 7f7a5bf60640 1 -- 172.21.15.148:0/1195313938 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7a3806c530 msgr2=0x7f7a3806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.830+0000 7f7a5bf60640 1 --2- 172.21.15.148:0/1195313938 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7a3806c530 0x7f7a3806e9f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7a540cf3b0 tx=0x7f7a44009040 comp rx=0 tx=0).stop 2024-03-29T04:11:28.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.830+0000 7f7a5bf60640 1 -- 172.21.15.148:0/1195313938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a540b3710 msgr2=0x7f7a540ce100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.830+0000 7f7a5bf60640 1 --2- 172.21.15.148:0/1195313938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a540b3710 0x7f7a540ce100 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7a50034040 tx=0x7f7a500046c0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.830+0000 7f7a5bf60640 1 -- 172.21.15.148:0/1195313938 shutdown_connections 2024-03-29T04:11:28.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.830+0000 7f7a5bf60640 1 --2- 172.21.15.148:0/1195313938 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7a3806c530 0x7f7a3806e9f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.830+0000 7f7a5bf60640 1 --2- 172.21.15.148:0/1195313938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a54158c30 0x7f7a5415dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.830+0000 7f7a5bf60640 1 --2- 172.21.15.148:0/1195313938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a540b3710 0x7f7a540ce100 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.830+0000 7f7a5bf60640 1 --2- 172.21.15.148:0/1195313938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a5406b5b0 0x7f7a541494d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.830+0000 7f7a5bf60640 1 -- 172.21.15.148:0/1195313938 >> 172.21.15.148:0/1195313938 conn(0x7f7a540d0700 msgr2=0x7f7a540b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:28.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.830+0000 7f7a5bf60640 1 -- 172.21.15.148:0/1195313938 shutdown_connections 2024-03-29T04:11:28.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.830+0000 7f7a5bf60640 1 -- 172.21.15.148:0/1195313938 wait complete. 2024-03-29T04:11:28.819 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-29T04:11:28.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.942+0000 7f9b8daeb640 1 Processor -- start 2024-03-29T04:11:28.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.942+0000 7f9b8daeb640 1 -- start start 2024-03-29T04:11:28.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.943+0000 7f9b8daeb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9b880ce610 0x7f9b8815d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.943+0000 7f9b8daeb640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b880cea40 con 0x7f9b880cf780 2024-03-29T04:11:28.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.943+0000 7f9b8daeb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b8806da30 con 0x7f9b8806ccd0 2024-03-29T04:11:28.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.943+0000 7f9b8daeb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9b8806dd10 con 0x7f9b880ce610 2024-03-29T04:11:28.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.943+0000 7f9b877fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9b880cf780 0x7f9b8806c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50732/0 (socket says 172.21.15.148:50732) 2024-03-29T04:11:28.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.943+0000 7f9b877fe640 1 -- 172.21.15.148:0/1237323368 learned_addr learned my addr 172.21.15.148:0/1237323368 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:28.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.943+0000 7f9b867fc640 1 -- 172.21.15.148:0/1237323368 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 325497763 0 0) 0x7f9b880cea40 con 0x7f9b880cf780 2024-03-29T04:11:28.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.943+0000 7f9b867fc640 1 -- 172.21.15.148:0/1237323368 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b70003520 con 0x7f9b880cf780 2024-03-29T04:11:28.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.943+0000 7f9b87fff640 1 --2- 172.21.15.148:0/1237323368 >> v2:172.21.15.157:3300/0 conn(0x7f9b880ce610 0x7f9b8815d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.943+0000 7f9b867fc640 1 -- 172.21.15.148:0/1237323368 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4051434257 0 0) 0x7f9b70003520 con 0x7f9b880cf780 2024-03-29T04:11:28.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.943+0000 7f9b867fc640 1 -- 172.21.15.148:0/1237323368 >> v2:172.21.15.157:3300/0 conn(0x7f9b880ce610 msgr2=0x7f9b8815d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:28.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.943+0000 7f9b867fc640 1 --2- 172.21.15.148:0/1237323368 >> v2:172.21.15.157:3300/0 conn(0x7f9b880ce610 0x7f9b8815d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.943+0000 7f9b867fc640 1 -- 172.21.15.148:0/1237323368 >> v1:172.21.15.157:6789/0 conn(0x7f9b8806ccd0 legacy=0x7f9b880b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.943+0000 7f9b867fc640 1 -- 172.21.15.148:0/1237323368 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8806dff0 con 0x7f9b880cf780 2024-03-29T04:11:28.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.943+0000 7f9b867fc640 1 -- 172.21.15.148:0/1237323368 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f9b78001670 con 0x7f9b880cf780 2024-03-29T04:11:28.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.944+0000 7f9b867fc640 1 -- 172.21.15.148:0/1237323368 >> v1:172.21.15.148:6789/0 conn(0x7f9b880cf780 legacy=0x7f9b8806c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.944+0000 7f9b867fc640 1 --2- 172.21.15.148:0/1237323368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b700039b0 0x7f9b70003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.944+0000 7f9b867fc640 1 --2- 172.21.15.148:0/1237323368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b70004800 0x7f9b70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.944+0000 7f9b867fc640 1 --2- 172.21.15.148:0/1237323368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b700051c0 0x7f9b7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.944+0000 7f9b867fc640 1 -- 172.21.15.148:0/1237323368 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8806dd10 con 0x7f9b700051c0 2024-03-29T04:11:28.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.944+0000 7f9b867fc640 1 -- 172.21.15.148:0/1237323368 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7000c160 con 0x7f9b70004800 2024-03-29T04:11:28.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.944+0000 7f9b867fc640 1 -- 172.21.15.148:0/1237323368 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9b7000c360 con 0x7f9b700039b0 2024-03-29T04:11:28.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.944+0000 7f9b87fff640 1 --2- 172.21.15.148:0/1237323368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b700051c0 0x7f9b7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.944+0000 7f9b86ffd640 1 --2- 172.21.15.148:0/1237323368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b70004800 0x7f9b70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.944+0000 7f9b877fe640 1 --2- 172.21.15.148:0/1237323368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b700039b0 0x7f9b70003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.944+0000 7f9b87fff640 1 -- 172.21.15.148:0/1237323368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b700039b0 msgr2=0x7f9b70003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:28.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.944+0000 7f9b87fff640 1 --2- 172.21.15.148:0/1237323368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b700039b0 0x7f9b70003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.944+0000 7f9b87fff640 1 -- 172.21.15.148:0/1237323368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b70004800 msgr2=0x7f9b70004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:28.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.944+0000 7f9b87fff640 1 --2- 172.21.15.148:0/1237323368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b70004800 0x7f9b70004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.944+0000 7f9b87fff640 1 -- 172.21.15.148:0/1237323368 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b70003520 con 0x7f9b700051c0 2024-03-29T04:11:28.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.944+0000 7f9b877fe640 1 --2- 172.21.15.148:0/1237323368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b700039b0 0x7f9b70003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:28.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.945+0000 7f9b87fff640 1 -- 172.21.15.148:0/1237323368 reap_dead start 2024-03-29T04:11:28.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.945+0000 7f9b87fff640 1 --2- 172.21.15.148:0/1237323368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b700051c0 0x7f9b7000b9c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9b80009fc0 tx=0x7f9b70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf29c733a30b235 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:28.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.945+0000 7f9b867fc640 1 -- 172.21.15.148:0/1237323368 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b80009c10 con 0x7f9b700051c0 2024-03-29T04:11:28.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.945+0000 7f9b867fc640 1 -- 172.21.15.148:0/1237323368 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b8002cca0 con 0x7f9b700051c0 2024-03-29T04:11:28.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.945+0000 7f9b867fc640 1 -- 172.21.15.148:0/1237323368 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b80003bd0 con 0x7f9b700051c0 2024-03-29T04:11:28.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.945+0000 7f9b8daeb640 1 -- 172.21.15.148:0/1237323368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b700051c0 msgr2=0x7f9b7000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.945+0000 7f9b8daeb640 1 --2- 172.21.15.148:0/1237323368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b700051c0 0x7f9b7000b9c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9b80009fc0 tx=0x7f9b70005830 comp rx=0 tx=0).stop 2024-03-29T04:11:28.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.945+0000 7f9b8daeb640 1 -- 172.21.15.148:0/1237323368 shutdown_connections 2024-03-29T04:11:28.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.945+0000 7f9b8daeb640 1 --2- 172.21.15.148:0/1237323368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b700051c0 0x7f9b7000b9c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.945+0000 7f9b8daeb640 1 -- 172.21.15.148:0/1237323368 >> 172.21.15.148:0/1237323368 conn(0x7f9b880cfed0 msgr2=0x7f9b880d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:28.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.945+0000 7f9b8daeb640 1 -- 172.21.15.148:0/1237323368 shutdown_connections 2024-03-29T04:11:28.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.945+0000 7f9b8daeb640 1 -- 172.21.15.148:0/1237323368 wait complete. 2024-03-29T04:11:28.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.947+0000 7f9b8daeb640 1 Processor -- start 2024-03-29T04:11:28.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.947+0000 7f9b8daeb640 1 -- start start 2024-03-29T04:11:28.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.947+0000 7f9b8daeb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b700051c0 0x7f9b880cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.947+0000 7f9b8daeb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b88072ed0 0x7f9b8814ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.947+0000 7f9b8daeb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b8815e780 0x7f9b8815ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.947+0000 7f9b8daeb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8806d5c0 con 0x7f9b8815e780 2024-03-29T04:11:28.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.947+0000 7f9b8daeb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8806f340 con 0x7f9b700051c0 2024-03-29T04:11:28.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.947+0000 7f9b8daeb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9b8806d740 con 0x7f9b88072ed0 2024-03-29T04:11:28.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.948+0000 7f9b87fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b8815e780 0x7f9b8815ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.948+0000 7f9b87fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b8815e780 0x7f9b8815ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39332/0 (socket says 172.21.15.148:39332) 2024-03-29T04:11:28.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.948+0000 7f9b87fff640 1 -- 172.21.15.148:0/2353941658 learned_addr learned my addr 172.21.15.148:0/2353941658 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:28.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.948+0000 7f9b877fe640 1 --2- 172.21.15.148:0/2353941658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b700051c0 0x7f9b880cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.948+0000 7f9b87fff640 1 -- 172.21.15.148:0/2353941658 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b88072ed0 msgr2=0x7f9b8814ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.948+0000 7f9b87fff640 1 --2- 172.21.15.148:0/2353941658 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b88072ed0 0x7f9b8814ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.948+0000 7f9b87fff640 1 -- 172.21.15.148:0/2353941658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b700051c0 msgr2=0x7f9b880cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.948+0000 7f9b87fff640 1 --2- 172.21.15.148:0/2353941658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b700051c0 0x7f9b880cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.948+0000 7f9b87fff640 1 -- 172.21.15.148:0/2353941658 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b88149df0 con 0x7f9b8815e780 2024-03-29T04:11:28.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.948+0000 7f9b87fff640 1 --2- 172.21.15.148:0/2353941658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b8815e780 0x7f9b8815ec00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9b8000e200 tx=0x7f9b80009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:28.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.948+0000 7f9b6ffff640 1 -- 172.21.15.148:0/2353941658 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b80003990 con 0x7f9b8815e780 2024-03-29T04:11:28.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.948+0000 7f9b6ffff640 1 -- 172.21.15.148:0/2353941658 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b80044070 con 0x7f9b8815e780 2024-03-29T04:11:28.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.948+0000 7f9b8daeb640 1 -- 172.21.15.148:0/2353941658 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b8814bc40 con 0x7f9b8815e780 2024-03-29T04:11:28.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.948+0000 7f9b6ffff640 1 -- 172.21.15.148:0/2353941658 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b80003d90 con 0x7f9b8815e780 2024-03-29T04:11:28.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.948+0000 7f9b8daeb640 1 -- 172.21.15.148:0/2353941658 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b8814a370 con 0x7f9b8815e780 2024-03-29T04:11:28.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.949+0000 7f9b8daeb640 1 -- 172.21.15.148:0/2353941658 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b8815f140 con 0x7f9b8815e780 2024-03-29T04:11:28.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.950+0000 7f9b6ffff640 1 -- 172.21.15.148:0/2353941658 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9b80005d10 con 0x7f9b8815e780 2024-03-29T04:11:28.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.950+0000 7f9b6ffff640 1 --2- 172.21.15.148:0/2353941658 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9b7406c560 0x7f9b7406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.950+0000 7f9b6ffff640 1 -- 172.21.15.148:0/2353941658 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f9b800adb60 con 0x7f9b8815e780 2024-03-29T04:11:28.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.950+0000 7f9b877fe640 1 --2- 172.21.15.148:0/2353941658 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9b7406c560 0x7f9b7406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.951+0000 7f9b877fe640 1 --2- 172.21.15.148:0/2353941658 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9b7406c560 0x7f9b7406ea20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9b78001eb0 tx=0x7f9b78000ed0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:28.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:28.954+0000 7f9b6ffff640 1 -- 172.21.15.148:0/2353941658 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9b8007c3d0 con 0x7f9b8815e780 2024-03-29T04:11:29.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.197+0000 7f9b8daeb640 1 -- 172.21.15.148:0/2353941658 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f9b880687e0 con 0x7f9b8815e780 2024-03-29T04:11:29.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.201+0000 7f9b6ffff640 1 -- 172.21.15.148:0/2353941658 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7f9b8007bcd0 con 0x7f9b8815e780 2024-03-29T04:11:29.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.203+0000 7f9b8daeb640 1 -- 172.21.15.148:0/2353941658 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9b7406c560 msgr2=0x7f9b7406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.203+0000 7f9b8daeb640 1 --2- 172.21.15.148:0/2353941658 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9b7406c560 0x7f9b7406ea20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9b78001eb0 tx=0x7f9b78000ed0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.203+0000 7f9b8daeb640 1 -- 172.21.15.148:0/2353941658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b8815e780 msgr2=0x7f9b8815ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.203+0000 7f9b8daeb640 1 --2- 172.21.15.148:0/2353941658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b8815e780 0x7f9b8815ec00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9b8000e200 tx=0x7f9b80009750 comp rx=0 tx=0).stop 2024-03-29T04:11:29.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.203+0000 7f9b8daeb640 1 -- 172.21.15.148:0/2353941658 shutdown_connections 2024-03-29T04:11:29.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.203+0000 7f9b8daeb640 1 --2- 172.21.15.148:0/2353941658 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9b7406c560 0x7f9b7406ea20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.203+0000 7f9b8daeb640 1 --2- 172.21.15.148:0/2353941658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b8815e780 0x7f9b8815ec00 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.203+0000 7f9b8daeb640 1 --2- 172.21.15.148:0/2353941658 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9b88072ed0 0x7f9b8814ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.203+0000 7f9b8daeb640 1 --2- 172.21.15.148:0/2353941658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b700051c0 0x7f9b880cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.203+0000 7f9b8daeb640 1 -- 172.21.15.148:0/2353941658 >> 172.21.15.148:0/2353941658 conn(0x7f9b880cfed0 msgr2=0x7f9b8815a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:29.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.203+0000 7f9b8daeb640 1 -- 172.21.15.148:0/2353941658 shutdown_connections 2024-03-29T04:11:29.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.203+0000 7f9b8daeb640 1 -- 172.21.15.148:0/2353941658 wait complete. 2024-03-29T04:11:29.193 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-29T04:11:29.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.316+0000 7fc021722640 1 Processor -- start 2024-03-29T04:11:29.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.316+0000 7fc021722640 1 -- start start 2024-03-29T04:11:29.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.317+0000 7fc021722640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc01c0d3430 0x7fc01c0cf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.317+0000 7fc021722640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc01c0b3ba0 con 0x7fc01c0cf8a0 2024-03-29T04:11:29.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.317+0000 7fc021722640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc01c0b3e80 con 0x7fc01c0cef80 2024-03-29T04:11:29.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.317+0000 7fc021722640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc01c15ec20 con 0x7fc01c0d3430 2024-03-29T04:11:29.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.317+0000 7fc01b7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc01c0cf8a0 0x7fc01c0b4c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50740/0 (socket says 172.21.15.148:50740) 2024-03-29T04:11:29.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.317+0000 7fc01b7fe640 1 -- 172.21.15.148:0/2532151819 learned_addr learned my addr 172.21.15.148:0/2532151819 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:29.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.317+0000 7fc01a7fc640 1 --2- 172.21.15.148:0/2532151819 >> v2:172.21.15.157:3300/0 conn(0x7fc01c0d3430 0x7fc01c0cf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.317+0000 7fc019ffb640 1 -- 172.21.15.148:0/2532151819 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2059013920 0 0) 0x7fc01c0b3ba0 con 0x7fc01c0cf8a0 2024-03-29T04:11:29.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.317+0000 7fc019ffb640 1 -- 172.21.15.148:0/2532151819 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc004003520 con 0x7fc01c0cf8a0 2024-03-29T04:11:29.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.317+0000 7fc019ffb640 1 -- 172.21.15.148:0/2532151819 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2620918985 0 0) 0x7fc004003520 con 0x7fc01c0cf8a0 2024-03-29T04:11:29.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.317+0000 7fc01a7fc640 1 -- 172.21.15.148:0/2532151819 >> v1:172.21.15.157:6789/0 conn(0x7fc01c0cef80 legacy=0x7fc01c0b7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.317+0000 7fc01a7fc640 1 -- 172.21.15.148:0/2532151819 >> v1:172.21.15.148:6789/0 conn(0x7fc01c0cf8a0 legacy=0x7fc01c0b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.317+0000 7fc01a7fc640 1 -- 172.21.15.148:0/2532151819 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01c0b4160 con 0x7fc01c0d3430 2024-03-29T04:11:29.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.318+0000 7fc01a7fc640 1 --2- 172.21.15.148:0/2532151819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc01c0d3430 0x7fc01c0cf360 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc0100097a0 tx=0x7fc01002b680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2c09c535baf013f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:29.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.318+0000 7fc019ffb640 1 -- 172.21.15.148:0/2532151819 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc010037070 con 0x7fc01c0d3430 2024-03-29T04:11:29.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.318+0000 7fc019ffb640 1 -- 172.21.15.148:0/2532151819 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc010030510 con 0x7fc01c0d3430 2024-03-29T04:11:29.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.318+0000 7fc019ffb640 1 -- 172.21.15.148:0/2532151819 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0140019f0 con 0x7fc01c0d3430 2024-03-29T04:11:29.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.319+0000 7fc021722640 1 -- 172.21.15.148:0/2532151819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc01c0d3430 msgr2=0x7fc01c0cf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.319+0000 7fc021722640 1 --2- 172.21.15.148:0/2532151819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc01c0d3430 0x7fc01c0cf360 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc0100097a0 tx=0x7fc01002b680 comp rx=0 tx=0).stop 2024-03-29T04:11:29.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.319+0000 7fc021722640 1 -- 172.21.15.148:0/2532151819 shutdown_connections 2024-03-29T04:11:29.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.319+0000 7fc021722640 1 --2- 172.21.15.148:0/2532151819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc01c0d3430 0x7fc01c0cf360 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.319+0000 7fc021722640 1 -- 172.21.15.148:0/2532151819 >> 172.21.15.148:0/2532151819 conn(0x7fc01c0d4300 msgr2=0x7fc01c0d4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:29.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.319+0000 7fc021722640 1 -- 172.21.15.148:0/2532151819 shutdown_connections 2024-03-29T04:11:29.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.319+0000 7fc021722640 1 -- 172.21.15.148:0/2532151819 wait complete. 2024-03-29T04:11:29.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.320+0000 7fc021722640 1 Processor -- start 2024-03-29T04:11:29.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.320+0000 7fc021722640 1 -- start start 2024-03-29T04:11:29.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.321+0000 7fc021722640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc01c0cef80 0x7fc01c155360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.321+0000 7fc021722640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc01c0cf8a0 0x7fc01c0cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.321+0000 7fc021722640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc01c0d3430 0x7fc01c15f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.321+0000 7fc021722640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c0d26b0 con 0x7fc01c0cef80 2024-03-29T04:11:29.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.321+0000 7fc021722640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c0b47b0 con 0x7fc01c0cf8a0 2024-03-29T04:11:29.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.321+0000 7fc021722640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc01c0b3830 con 0x7fc01c0d3430 2024-03-29T04:11:29.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.321+0000 7fc01affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc01c0cef80 0x7fc01c155360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.321+0000 7fc01affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc01c0cef80 0x7fc01c155360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39346/0 (socket says 172.21.15.148:39346) 2024-03-29T04:11:29.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.321+0000 7fc01affd640 1 -- 172.21.15.148:0/1520750482 learned_addr learned my addr 172.21.15.148:0/1520750482 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:29.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.321+0000 7fc01a7fc640 1 --2- 172.21.15.148:0/1520750482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc01c0cf8a0 0x7fc01c0cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.321+0000 7fc01b7fe640 1 --2- 172.21.15.148:0/1520750482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc01c0d3430 0x7fc01c15f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.321+0000 7fc01affd640 1 -- 172.21.15.148:0/1520750482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc01c0d3430 msgr2=0x7fc01c15f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.321+0000 7fc01affd640 1 --2- 172.21.15.148:0/1520750482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc01c0d3430 0x7fc01c15f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.321+0000 7fc01affd640 1 -- 172.21.15.148:0/1520750482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc01c0cf8a0 msgr2=0x7fc01c0cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.321+0000 7fc01affd640 1 --2- 172.21.15.148:0/1520750482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc01c0cf8a0 0x7fc01c0cd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.321+0000 7fc01affd640 1 -- 172.21.15.148:0/1520750482 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01c0cd670 con 0x7fc01c0cef80 2024-03-29T04:11:29.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.321+0000 7fc01b7fe640 1 --2- 172.21.15.148:0/1520750482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc01c0d3430 0x7fc01c15f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:29.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.321+0000 7fc01affd640 1 --2- 172.21.15.148:0/1520750482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc01c0cef80 0x7fc01c155360 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00c7b0 tx=0x7fc00c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:29.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.322+0000 7fc0037fe640 1 -- 172.21.15.148:0/1520750482 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c017070 con 0x7fc01c0cef80 2024-03-29T04:11:29.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.322+0000 7fc0037fe640 1 -- 172.21.15.148:0/1520750482 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc00c005290 con 0x7fc01c0cef80 2024-03-29T04:11:29.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.322+0000 7fc021722640 1 -- 172.21.15.148:0/1520750482 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc01c160740 con 0x7fc01c0cef80 2024-03-29T04:11:29.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.322+0000 7fc0037fe640 1 -- 172.21.15.148:0/1520750482 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c00a880 con 0x7fc01c0cef80 2024-03-29T04:11:29.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.322+0000 7fc021722640 1 -- 172.21.15.148:0/1520750482 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc01c15ef60 con 0x7fc01c0cef80 2024-03-29T04:11:29.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.323+0000 7fc021722640 1 -- 172.21.15.148:0/1520750482 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc01c160c60 con 0x7fc01c0cef80 2024-03-29T04:11:29.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.323+0000 7fc0037fe640 1 -- 172.21.15.148:0/1520750482 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc00c01a070 con 0x7fc01c0cef80 2024-03-29T04:11:29.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.323+0000 7fc0037fe640 1 --2- 172.21.15.148:0/1520750482 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc00806c560 0x7fc00806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.323+0000 7fc0037fe640 1 -- 172.21.15.148:0/1520750482 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fc00c08a9c0 con 0x7fc01c0cef80 2024-03-29T04:11:29.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.323+0000 7fc01a7fc640 1 --2- 172.21.15.148:0/1520750482 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc00806c560 0x7fc00806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.324+0000 7fc01a7fc640 1 --2- 172.21.15.148:0/1520750482 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc00806c560 0x7fc00806ea20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc010034000 tx=0x7fc010008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:29.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.328+0000 7fc0037fe640 1 -- 172.21.15.148:0/1520750482 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc01c160c60 con 0x7fc01c0cef80 2024-03-29T04:11:29.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.572+0000 7fc021722640 1 -- 172.21.15.148:0/1520750482 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fc01c15e5b0 con 0x7fc01c0cef80 2024-03-29T04:11:29.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.576+0000 7fc0037fe640 1 -- 172.21.15.148:0/1520750482 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7fc01c15e5b0 con 0x7fc01c0cef80 2024-03-29T04:11:29.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.579+0000 7fc021722640 1 -- 172.21.15.148:0/1520750482 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc00806c560 msgr2=0x7fc00806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.579+0000 7fc021722640 1 --2- 172.21.15.148:0/1520750482 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc00806c560 0x7fc00806ea20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc010034000 tx=0x7fc010008030 comp rx=0 tx=0).stop 2024-03-29T04:11:29.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.579+0000 7fc021722640 1 -- 172.21.15.148:0/1520750482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc01c0cef80 msgr2=0x7fc01c155360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.579+0000 7fc021722640 1 --2- 172.21.15.148:0/1520750482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc01c0cef80 0x7fc01c155360 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00c7b0 tx=0x7fc00c00cc70 comp rx=0 tx=0).stop 2024-03-29T04:11:29.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.579+0000 7fc021722640 1 -- 172.21.15.148:0/1520750482 shutdown_connections 2024-03-29T04:11:29.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.579+0000 7fc021722640 1 --2- 172.21.15.148:0/1520750482 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc00806c560 0x7fc00806ea20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.579+0000 7fc021722640 1 --2- 172.21.15.148:0/1520750482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc01c0d3430 0x7fc01c15f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.579+0000 7fc021722640 1 --2- 172.21.15.148:0/1520750482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc01c0cf8a0 0x7fc01c0cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.579+0000 7fc021722640 1 --2- 172.21.15.148:0/1520750482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc01c0cef80 0x7fc01c155360 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.579+0000 7fc021722640 1 -- 172.21.15.148:0/1520750482 >> 172.21.15.148:0/1520750482 conn(0x7fc01c0d4300 msgr2=0x7fc01c06ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:29.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.579+0000 7fc021722640 1 -- 172.21.15.148:0/1520750482 shutdown_connections 2024-03-29T04:11:29.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.579+0000 7fc021722640 1 -- 172.21.15.148:0/1520750482 wait complete. 2024-03-29T04:11:29.570 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:11:29.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.691+0000 7efd9ae55640 1 Processor -- start 2024-03-29T04:11:29.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.691+0000 7efd9ae55640 1 -- start start 2024-03-29T04:11:29.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.692+0000 7efd9ae55640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efd9415d3e0 0x7efd940cd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.692+0000 7efd9ae55640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd940cd650 0x7efd9406ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.692+0000 7efd9ae55640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd940cfa20 con 0x7efd940afce0 2024-03-29T04:11:29.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.692+0000 7efd9ae55640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7efd9406b5b0 con 0x7efd9415d3e0 2024-03-29T04:11:29.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.692+0000 7efd9ae55640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efd940cdaa0 con 0x7efd940cd650 2024-03-29T04:11:29.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.692+0000 7efd993f2640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd940cd650 0x7efd9406ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.692+0000 7efd93fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efd9415d3e0 0x7efd940cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.692+0000 7efd93fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efd9415d3e0 0x7efd940cd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:57634/0 (socket says 172.21.15.148:57634) 2024-03-29T04:11:29.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.692+0000 7efd98bf1640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7efd940afce0 0x7efd940cf0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.148:58314/0 (socket says 172.21.15.148:58314) 2024-03-29T04:11:29.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.692+0000 7efd93fff640 1 -- 172.21.15.148:0/3707772665 learned_addr learned my addr 172.21.15.148:0/3707772665 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:29.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.692+0000 7efd93fff640 1 -- 172.21.15.148:0/3707772665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd940cd650 msgr2=0x7efd9406ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:29.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.692+0000 7efd93fff640 1 --2- 172.21.15.148:0/3707772665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd940cd650 0x7efd9406ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.692+0000 7efd93fff640 1 -- 172.21.15.148:0/3707772665 >> v1:172.21.15.157:6789/0 conn(0x7efd940afce0 legacy=0x7efd940cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.692+0000 7efd93fff640 1 -- 172.21.15.148:0/3707772665 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd940cfd00 con 0x7efd9415d3e0 2024-03-29T04:11:29.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.692+0000 7efd937fe640 1 -- 172.21.15.148:0/3707772665 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1977275901 0 0) 0x7efd940cfa20 con 0x7efd940afce0 2024-03-29T04:11:29.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.692+0000 7efd993f2640 1 --2- 172.21.15.148:0/3707772665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd940cd650 0x7efd9406ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:11:29.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.693+0000 7efd93fff640 1 --2- 172.21.15.148:0/3707772665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd9415d3e0 0x7efd940cd110 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efd80009870 tx=0x7efd8002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab0a91d0d9aed60d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:29.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.693+0000 7efd937fe640 1 -- 172.21.15.148:0/3707772665 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80034070 con 0x7efd9415d3e0 2024-03-29T04:11:29.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.693+0000 7efd937fe640 1 -- 172.21.15.148:0/3707772665 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd80030840 con 0x7efd9415d3e0 2024-03-29T04:11:29.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.693+0000 7efd937fe640 1 -- 172.21.15.148:0/3707772665 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80030c10 con 0x7efd9415d3e0 2024-03-29T04:11:29.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.694+0000 7efd9ae55640 1 -- 172.21.15.148:0/3707772665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd9415d3e0 msgr2=0x7efd940cd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.694+0000 7efd9ae55640 1 --2- 172.21.15.148:0/3707772665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd9415d3e0 0x7efd940cd110 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efd80009870 tx=0x7efd8002b980 comp rx=0 tx=0).stop 2024-03-29T04:11:29.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.694+0000 7efd9ae55640 1 -- 172.21.15.148:0/3707772665 shutdown_connections 2024-03-29T04:11:29.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.694+0000 7efd9ae55640 1 --2- 172.21.15.148:0/3707772665 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd940cd650 0x7efd9406ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.694+0000 7efd9ae55640 1 --2- 172.21.15.148:0/3707772665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd9415d3e0 0x7efd940cd110 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.694+0000 7efd9ae55640 1 -- 172.21.15.148:0/3707772665 >> 172.21.15.148:0/3707772665 conn(0x7efd940d0700 msgr2=0x7efd940d0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:29.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.694+0000 7efd9ae55640 1 -- 172.21.15.148:0/3707772665 shutdown_connections 2024-03-29T04:11:29.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.694+0000 7efd9ae55640 1 -- 172.21.15.148:0/3707772665 wait complete. 2024-03-29T04:11:29.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.695+0000 7efd9ae55640 1 Processor -- start 2024-03-29T04:11:29.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.695+0000 7efd9ae55640 1 -- start start 2024-03-29T04:11:29.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.696+0000 7efd9ae55640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd940afce0 0x7efd9415ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.696+0000 7efd9ae55640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd940cd650 0x7efd940b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.696+0000 7efd9ae55640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd9415d3e0 0x7efd940b4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.696+0000 7efd9ae55640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efd940d02c0 con 0x7efd940afce0 2024-03-29T04:11:29.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.696+0000 7efd9ae55640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efd940cf7c0 con 0x7efd9415d3e0 2024-03-29T04:11:29.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.696+0000 7efd9ae55640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efd940b00c0 con 0x7efd940cd650 2024-03-29T04:11:29.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.696+0000 7efd98bf1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd940afce0 0x7efd9415ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.696+0000 7efd98bf1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd940afce0 0x7efd9415ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39352/0 (socket says 172.21.15.148:39352) 2024-03-29T04:11:29.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.696+0000 7efd98bf1640 1 -- 172.21.15.148:0/1925103205 learned_addr learned my addr 172.21.15.148:0/1925103205 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:29.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.696+0000 7efd98bf1640 1 -- 172.21.15.148:0/1925103205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd940cd650 msgr2=0x7efd940b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.696+0000 7efd993f2640 1 --2- 172.21.15.148:0/1925103205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd9415d3e0 0x7efd940b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.696+0000 7efd93fff640 1 --2- 172.21.15.148:0/1925103205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd940cd650 0x7efd940b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.696+0000 7efd98bf1640 1 --2- 172.21.15.148:0/1925103205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd940cd650 0x7efd940b41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.696+0000 7efd98bf1640 1 -- 172.21.15.148:0/1925103205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd9415d3e0 msgr2=0x7efd940b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.696+0000 7efd98bf1640 1 --2- 172.21.15.148:0/1925103205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd9415d3e0 0x7efd940b4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.696+0000 7efd98bf1640 1 -- 172.21.15.148:0/1925103205 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd940b4d10 con 0x7efd940afce0 2024-03-29T04:11:29.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.696+0000 7efd993f2640 1 --2- 172.21.15.148:0/1925103205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd9415d3e0 0x7efd940b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:29.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.696+0000 7efd98bf1640 1 --2- 172.21.15.148:0/1925103205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd940afce0 0x7efd9415ff00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7efd8800c500 tx=0x7efd8800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:29.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.697+0000 7efd917fa640 1 -- 172.21.15.148:0/1925103205 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd88018070 con 0x7efd940afce0 2024-03-29T04:11:29.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.697+0000 7efd917fa640 1 -- 172.21.15.148:0/1925103205 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd88007580 con 0x7efd940afce0 2024-03-29T04:11:29.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.697+0000 7efd917fa640 1 -- 172.21.15.148:0/1925103205 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd88004a20 con 0x7efd940afce0 2024-03-29T04:11:29.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.697+0000 7efd9ae55640 1 -- 172.21.15.148:0/1925103205 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd940b4ff0 con 0x7efd940afce0 2024-03-29T04:11:29.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.697+0000 7efd9ae55640 1 -- 172.21.15.148:0/1925103205 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd9406dbe0 con 0x7efd940afce0 2024-03-29T04:11:29.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.698+0000 7efd9ae55640 1 -- 172.21.15.148:0/1925103205 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd940687e0 con 0x7efd940afce0 2024-03-29T04:11:29.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.698+0000 7efd917fa640 1 -- 172.21.15.148:0/1925103205 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efd8801b070 con 0x7efd940afce0 2024-03-29T04:11:29.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.699+0000 7efd917fa640 1 --2- 172.21.15.148:0/1925103205 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efd7806c530 0x7efd7806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.699+0000 7efd93fff640 1 --2- 172.21.15.148:0/1925103205 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efd7806c530 0x7efd7806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.699+0000 7efd917fa640 1 -- 172.21.15.148:0/1925103205 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7efd8808a820 con 0x7efd940afce0 2024-03-29T04:11:29.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.699+0000 7efd93fff640 1 --2- 172.21.15.148:0/1925103205 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efd7806c530 0x7efd7806e9f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efd80036000 tx=0x7efd80008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:29.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.703+0000 7efd917fa640 1 -- 172.21.15.148:0/1925103205 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efd88059090 con 0x7efd940afce0 2024-03-29T04:11:29.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.878+0000 7efd9ae55640 1 -- 172.21.15.148:0/1925103205 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd940cca10 con 0x7efd940afce0 2024-03-29T04:11:29.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.879+0000 7efd917fa640 1 -- 172.21.15.148:0/1925103205 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+13176 (secure 0 0 0) 0x7efd88058990 con 0x7efd940afce0 2024-03-29T04:11:29.857 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:11:29.858 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":35,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:11:26.957376+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:11:29.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.880+0000 7efd9ae55640 1 -- 172.21.15.148:0/1925103205 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efd7806c530 msgr2=0x7efd7806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.880+0000 7efd9ae55640 1 --2- 172.21.15.148:0/1925103205 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efd7806c530 0x7efd7806e9f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efd80036000 tx=0x7efd80008030 comp rx=0 tx=0).stop 2024-03-29T04:11:29.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.880+0000 7efd9ae55640 1 -- 172.21.15.148:0/1925103205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd940afce0 msgr2=0x7efd9415ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.880+0000 7efd9ae55640 1 --2- 172.21.15.148:0/1925103205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd940afce0 0x7efd9415ff00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7efd8800c500 tx=0x7efd8800c9c0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.881+0000 7efd9ae55640 1 -- 172.21.15.148:0/1925103205 shutdown_connections 2024-03-29T04:11:29.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.881+0000 7efd9ae55640 1 --2- 172.21.15.148:0/1925103205 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efd7806c530 0x7efd7806e9f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.881+0000 7efd9ae55640 1 --2- 172.21.15.148:0/1925103205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd9415d3e0 0x7efd940b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.881+0000 7efd9ae55640 1 --2- 172.21.15.148:0/1925103205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efd940cd650 0x7efd940b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.881+0000 7efd9ae55640 1 --2- 172.21.15.148:0/1925103205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd940afce0 0x7efd9415ff00 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.881+0000 7efd9ae55640 1 -- 172.21.15.148:0/1925103205 >> 172.21.15.148:0/1925103205 conn(0x7efd940d0700 msgr2=0x7efd940d0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:29.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.881+0000 7efd9ae55640 1 -- 172.21.15.148:0/1925103205 shutdown_connections 2024-03-29T04:11:29.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.881+0000 7efd9ae55640 1 -- 172.21.15.148:0/1925103205 wait complete. 2024-03-29T04:11:29.871 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:47.898351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:11:29.871 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:11:29.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.998+0000 7f3cf99a8640 1 Processor -- start 2024-03-29T04:11:29.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.998+0000 7f3cf99a8640 1 -- start start 2024-03-29T04:11:29.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.999+0000 7f3cf99a8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3cf40d07b0 0x7f3cf415e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.999+0000 7f3cf99a8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cf4141760 con 0x7f3cf40d03d0 2024-03-29T04:11:29.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.999+0000 7f3cf99a8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cf415f160 con 0x7f3cf4143ad0 2024-03-29T04:11:29.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.999+0000 7f3cf99a8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f3cf40d0c30 con 0x7f3cf40d07b0 2024-03-29T04:11:29.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.999+0000 7f3cf2ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3cf40d03d0 0x7f3cf41626a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50748/0 (socket says 172.21.15.148:50748) 2024-03-29T04:11:29.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.999+0000 7f3cf2ffd640 1 -- 172.21.15.148:0/1057540760 learned_addr learned my addr 172.21.15.148:0/1057540760 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:29.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.999+0000 7f3cf27fc640 1 --2- 172.21.15.148:0/1057540760 >> v2:172.21.15.157:3300/0 conn(0x7f3cf40d07b0 0x7f3cf415e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.999+0000 7f3cf1ffb640 1 -- 172.21.15.148:0/1057540760 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1849699627 0 0) 0x7f3ce4001490 con 0x7f3cf40d03d0 2024-03-29T04:11:29.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.999+0000 7f3cf1ffb640 1 -- 172.21.15.148:0/1057540760 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3cdc003520 con 0x7f3cf40d03d0 2024-03-29T04:11:29.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.999+0000 7f3cf1ffb640 1 -- 172.21.15.148:0/1057540760 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1216834783 0 0) 0x7f3cdc003520 con 0x7f3cf40d03d0 2024-03-29T04:11:29.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.999+0000 7f3cf1ffb640 1 -- 172.21.15.148:0/1057540760 >> v2:172.21.15.157:3300/0 conn(0x7f3cf40d07b0 msgr2=0x7f3cf415e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:29.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.999+0000 7f3cf1ffb640 1 --2- 172.21.15.148:0/1057540760 >> v2:172.21.15.157:3300/0 conn(0x7f3cf40d07b0 0x7f3cf415e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.999+0000 7f3cf1ffb640 1 -- 172.21.15.148:0/1057540760 >> v1:172.21.15.157:6789/0 conn(0x7f3cf4143ad0 legacy=0x7f3cf4072f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.999+0000 7f3cf1ffb640 1 -- 172.21.15.148:0/1057540760 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf415f440 con 0x7f3cf40d03d0 2024-03-29T04:11:29.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.999+0000 7f3cf1ffb640 1 -- 172.21.15.148:0/1057540760 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f3ce4001bc0 con 0x7f3cf40d03d0 2024-03-29T04:11:29.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:29.999+0000 7f3cf1ffb640 1 -- 172.21.15.148:0/1057540760 >> v1:172.21.15.148:6789/0 conn(0x7f3cf40d03d0 legacy=0x7f3cf41626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.000+0000 7f3cf1ffb640 1 --2- 172.21.15.148:0/1057540760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cdc0039b0 0x7f3cdc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.000+0000 7f3cf1ffb640 1 --2- 172.21.15.148:0/1057540760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cdc004800 0x7f3cdc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.000+0000 7f3cf1ffb640 1 --2- 172.21.15.148:0/1057540760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cdc0051c0 0x7f3cdc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.000+0000 7f3cf1ffb640 1 -- 172.21.15.148:0/1057540760 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf40d0c30 con 0x7f3cdc0039b0 2024-03-29T04:11:29.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.000+0000 7f3cf1ffb640 1 -- 172.21.15.148:0/1057540760 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc00c060 con 0x7f3cdc004800 2024-03-29T04:11:29.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.000+0000 7f3cf1ffb640 1 -- 172.21.15.148:0/1057540760 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3cdc00c260 con 0x7f3cdc0051c0 2024-03-29T04:11:29.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.000+0000 7f3cf2ffd640 1 --2- 172.21.15.148:0/1057540760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cdc0039b0 0x7f3cdc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.000+0000 7f3cf27fc640 1 --2- 172.21.15.148:0/1057540760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cdc004800 0x7f3cdc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.000+0000 7f3cf37fe640 1 --2- 172.21.15.148:0/1057540760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cdc0051c0 0x7f3cdc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.000+0000 7f3cf2ffd640 1 -- 172.21.15.148:0/1057540760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cdc0051c0 msgr2=0x7f3cdc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:29.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.000+0000 7f3cf2ffd640 1 --2- 172.21.15.148:0/1057540760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cdc0051c0 0x7f3cdc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.000+0000 7f3cf2ffd640 1 -- 172.21.15.148:0/1057540760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cdc004800 msgr2=0x7f3cdc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:29.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.000+0000 7f3cf2ffd640 1 --2- 172.21.15.148:0/1057540760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cdc004800 0x7f3cdc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.000+0000 7f3cf2ffd640 1 -- 172.21.15.148:0/1057540760 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cdc003520 con 0x7f3cdc0039b0 2024-03-29T04:11:29.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.000+0000 7f3cf37fe640 1 --2- 172.21.15.148:0/1057540760 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cdc0051c0 0x7f3cdc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:29.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.000+0000 7f3cf37fe640 1 -- 172.21.15.148:0/1057540760 reap_dead start 2024-03-29T04:11:29.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.001+0000 7f3cf2ffd640 1 --2- 172.21.15.148:0/1057540760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cdc0039b0 0x7f3cdc003dd0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400aa50 tx=0x7f3cdc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf6575123e45065 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:29.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.001+0000 7f3cf1ffb640 1 -- 172.21.15.148:0/1057540760 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4038070 con 0x7f3cdc0039b0 2024-03-29T04:11:29.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.001+0000 7f3cf1ffb640 1 -- 172.21.15.148:0/1057540760 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ce4033650 con 0x7f3cdc0039b0 2024-03-29T04:11:29.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.001+0000 7f3cf1ffb640 1 -- 172.21.15.148:0/1057540760 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce400a670 con 0x7f3cdc0039b0 2024-03-29T04:11:29.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.001+0000 7f3cf99a8640 1 -- 172.21.15.148:0/1057540760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cdc0039b0 msgr2=0x7f3cdc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.001+0000 7f3cf99a8640 1 --2- 172.21.15.148:0/1057540760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cdc0039b0 0x7f3cdc003dd0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400aa50 tx=0x7f3cdc005830 comp rx=0 tx=0).stop 2024-03-29T04:11:29.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.001+0000 7f3cf99a8640 1 -- 172.21.15.148:0/1057540760 shutdown_connections 2024-03-29T04:11:29.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.001+0000 7f3cf99a8640 1 --2- 172.21.15.148:0/1057540760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cdc0039b0 0x7f3cdc003dd0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.001+0000 7f3cf99a8640 1 -- 172.21.15.148:0/1057540760 >> 172.21.15.148:0/1057540760 conn(0x7f3cf4072430 msgr2=0x7f3cf4072840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:29.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.001+0000 7f3cf99a8640 1 -- 172.21.15.148:0/1057540760 shutdown_connections 2024-03-29T04:11:29.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.001+0000 7f3cf99a8640 1 -- 172.21.15.148:0/1057540760 wait complete. 2024-03-29T04:11:29.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.002+0000 7f3cf99a8640 1 Processor -- start 2024-03-29T04:11:29.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.003+0000 7f3cf99a8640 1 -- start start 2024-03-29T04:11:29.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.003+0000 7f3cf99a8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cdc0039b0 0x7f3cf41405a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.003+0000 7f3cf99a8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cf41610e0 0x7f3cf4161560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.003+0000 7f3cf99a8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cf40b5b50 0x7f3cf40b5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.003+0000 7f3cf99a8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf4143f40 con 0x7f3cf41610e0 2024-03-29T04:11:29.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.003+0000 7f3cf99a8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf4162180 con 0x7f3cf40b5b50 2024-03-29T04:11:29.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.003+0000 7f3cf99a8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3cf41440c0 con 0x7f3cdc0039b0 2024-03-29T04:11:29.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.003+0000 7f3cf27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cf41610e0 0x7f3cf4161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.003+0000 7f3cf27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cf41610e0 0x7f3cf4161560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39372/0 (socket says 172.21.15.148:39372) 2024-03-29T04:11:29.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.003+0000 7f3cf27fc640 1 -- 172.21.15.148:0/344775021 learned_addr learned my addr 172.21.15.148:0/344775021 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:29.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.004+0000 7f3cf37fe640 1 --2- 172.21.15.148:0/344775021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cf40b5b50 0x7f3cf40b5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.004+0000 7f3cf2ffd640 1 --2- 172.21.15.148:0/344775021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cdc0039b0 0x7f3cf41405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.004+0000 7f3cf27fc640 1 -- 172.21.15.148:0/344775021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cdc0039b0 msgr2=0x7f3cf41405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.004+0000 7f3cf27fc640 1 --2- 172.21.15.148:0/344775021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cdc0039b0 0x7f3cf41405a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.004+0000 7f3cf27fc640 1 -- 172.21.15.148:0/344775021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cf40b5b50 msgr2=0x7f3cf40b5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.004+0000 7f3cf27fc640 1 --2- 172.21.15.148:0/344775021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cf40b5b50 0x7f3cf40b5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.004+0000 7f3cf27fc640 1 -- 172.21.15.148:0/344775021 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf41e7fa0 con 0x7f3cf41610e0 2024-03-29T04:11:29.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.004+0000 7f3cf27fc640 1 --2- 172.21.15.148:0/344775021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cf41610e0 0x7f3cf4161560 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3ce80116d0 tx=0x7f3ce8011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:29.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.004+0000 7f3cdbfff640 1 -- 172.21.15.148:0/344775021 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce800f9c0 con 0x7f3cf41610e0 2024-03-29T04:11:29.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.004+0000 7f3cdbfff640 1 -- 172.21.15.148:0/344775021 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ce800fb60 con 0x7f3cf41610e0 2024-03-29T04:11:29.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.004+0000 7f3cf99a8640 1 -- 172.21.15.148:0/344775021 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cf41e8280 con 0x7f3cf41610e0 2024-03-29T04:11:29.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.004+0000 7f3cdbfff640 1 -- 172.21.15.148:0/344775021 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8015460 con 0x7f3cf41610e0 2024-03-29T04:11:29.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.004+0000 7f3cf99a8640 1 -- 172.21.15.148:0/344775021 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cf40b6510 con 0x7f3cf41610e0 2024-03-29T04:11:29.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.005+0000 7f3cf99a8640 1 -- 172.21.15.148:0/344775021 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cf4068850 con 0x7f3cf41610e0 2024-03-29T04:11:29.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.006+0000 7f3cdbfff640 1 -- 172.21.15.148:0/344775021 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3ce8015600 con 0x7f3cf41610e0 2024-03-29T04:11:29.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.006+0000 7f3cdbfff640 1 --2- 172.21.15.148:0/344775021 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3ce006c530 0x7f3ce006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.006+0000 7f3cdbfff640 1 -- 172.21.15.148:0/344775021 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f3ce808bea0 con 0x7f3cf41610e0 2024-03-29T04:11:29.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.006+0000 7f3cf2ffd640 1 --2- 172.21.15.148:0/344775021 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3ce006c530 0x7f3ce006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.006+0000 7f3cf2ffd640 1 --2- 172.21.15.148:0/344775021 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3ce006c530 0x7f3ce006e9f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400aa20 tx=0x7f3ce4030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:29.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.010+0000 7f3cdbfff640 1 -- 172.21.15.148:0/344775021 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3ce8019030 con 0x7f3cf41610e0 2024-03-29T04:11:30.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.185+0000 7f3cf99a8640 1 -- 172.21.15.148:0/344775021 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3cf40ca410 con 0x7f3cf41610e0 2024-03-29T04:11:30.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.186+0000 7f3cdbfff640 1 -- 172.21.15.148:0/344775021 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7f3ce805a680 con 0x7f3cf41610e0 2024-03-29T04:11:30.165 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-29T04:11:30.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.187+0000 7f3cf99a8640 1 -- 172.21.15.148:0/344775021 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3ce006c530 msgr2=0x7f3ce006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:30.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.187+0000 7f3cf99a8640 1 --2- 172.21.15.148:0/344775021 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3ce006c530 0x7f3ce006e9f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400aa20 tx=0x7f3ce4030000 comp rx=0 tx=0).stop 2024-03-29T04:11:30.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.187+0000 7f3cf99a8640 1 -- 172.21.15.148:0/344775021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cf41610e0 msgr2=0x7f3cf4161560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:30.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.187+0000 7f3cf99a8640 1 --2- 172.21.15.148:0/344775021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cf41610e0 0x7f3cf4161560 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3ce80116d0 tx=0x7f3ce8011b90 comp rx=0 tx=0).stop 2024-03-29T04:11:30.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.187+0000 7f3cf99a8640 1 -- 172.21.15.148:0/344775021 shutdown_connections 2024-03-29T04:11:30.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.187+0000 7f3cf99a8640 1 --2- 172.21.15.148:0/344775021 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3ce006c530 0x7f3ce006e9f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.187+0000 7f3cf99a8640 1 --2- 172.21.15.148:0/344775021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cf40b5b50 0x7f3cf40b5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.188+0000 7f3cf99a8640 1 --2- 172.21.15.148:0/344775021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cf41610e0 0x7f3cf4161560 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.188+0000 7f3cf99a8640 1 --2- 172.21.15.148:0/344775021 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cdc0039b0 0x7f3cf41405a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.188+0000 7f3cf99a8640 1 -- 172.21.15.148:0/344775021 >> 172.21.15.148:0/344775021 conn(0x7f3cf4072430 msgr2=0x7f3cf40ceba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:30.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.188+0000 7f3cf99a8640 1 -- 172.21.15.148:0/344775021 shutdown_connections 2024-03-29T04:11:30.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.188+0000 7f3cf99a8640 1 -- 172.21.15.148:0/344775021 wait complete. 2024-03-29T04:11:30.178 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-29T04:11:30.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.299+0000 7f6178d0e640 1 Processor -- start 2024-03-29T04:11:30.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.299+0000 7f6178d0e640 1 -- start start 2024-03-29T04:11:30.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.300+0000 7f6178d0e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f617415b060 0x7f61740b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:30.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.300+0000 7f6178d0e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f617406c5c0 0x7f617406ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:30.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.300+0000 7f6178d0e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61740b3630 con 0x7f617415f6d0 2024-03-29T04:11:30.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.300+0000 7f6178d0e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f61740b3910 con 0x7f617415b060 2024-03-29T04:11:30.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.300+0000 7f6178d0e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f61740b3bf0 con 0x7f617406c5c0 2024-03-29T04:11:30.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.301+0000 7f617259c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f617415b060 0x7f61740b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:30.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.301+0000 7f617259c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f617415b060 0x7f61740b2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39376/0 (socket says 172.21.15.148:39376) 2024-03-29T04:11:30.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.301+0000 7f617259c640 1 -- 172.21.15.148:0/313802369 learned_addr learned my addr 172.21.15.148:0/313802369 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:30.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.301+0000 7f6172d9d640 1 --2- 172.21.15.148:0/313802369 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f617406c5c0 0x7f617406ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:30.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.301+0000 7f617259c640 1 -- 172.21.15.148:0/313802369 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f617406c5c0 msgr2=0x7f617406ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:30.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.301+0000 7f617259c640 1 --2- 172.21.15.148:0/313802369 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f617406c5c0 0x7f617406ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.301+0000 7f617259c640 1 -- 172.21.15.148:0/313802369 >> v1:172.21.15.157:6789/0 conn(0x7f617415f6d0 legacy=0x7f61740b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:30.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.301+0000 7f617259c640 1 -- 172.21.15.148:0/313802369 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61740b3ed0 con 0x7f617415b060 2024-03-29T04:11:30.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.302+0000 7f617259c640 1 --2- 172.21.15.148:0/313802369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f617415b060 0x7f61740b2ee0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f616c009870 tx=0x7f616c02b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef927df936fa5c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:30.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.302+0000 7f617159a640 1 -- 172.21.15.148:0/313802369 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c038070 con 0x7f617415b060 2024-03-29T04:11:30.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.302+0000 7f617159a640 1 -- 172.21.15.148:0/313802369 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f616c031740 con 0x7f617415b060 2024-03-29T04:11:30.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.302+0000 7f617159a640 1 -- 172.21.15.148:0/313802369 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c02f050 con 0x7f617415b060 2024-03-29T04:11:30.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.302+0000 7f6178d0e640 1 -- 172.21.15.148:0/313802369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f617415b060 msgr2=0x7f61740b2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:30.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.302+0000 7f6178d0e640 1 --2- 172.21.15.148:0/313802369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f617415b060 0x7f61740b2ee0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f616c009870 tx=0x7f616c02b7e0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.302+0000 7f6178d0e640 1 -- 172.21.15.148:0/313802369 shutdown_connections 2024-03-29T04:11:30.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.302+0000 7f6178d0e640 1 --2- 172.21.15.148:0/313802369 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f617406c5c0 0x7f617406ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.302+0000 7f6178d0e640 1 --2- 172.21.15.148:0/313802369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f617415b060 0x7f61740b2ee0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.302+0000 7f6178d0e640 1 -- 172.21.15.148:0/313802369 >> 172.21.15.148:0/313802369 conn(0x7f61740cfed0 msgr2=0x7f61740d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:30.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.302+0000 7f6178d0e640 1 -- 172.21.15.148:0/313802369 shutdown_connections 2024-03-29T04:11:30.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.302+0000 7f6178d0e640 1 -- 172.21.15.148:0/313802369 wait complete. 2024-03-29T04:11:30.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.304+0000 7f6178d0e640 1 Processor -- start 2024-03-29T04:11:30.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.304+0000 7f6178d0e640 1 -- start start 2024-03-29T04:11:30.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.304+0000 7f6178d0e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f617406c5c0 0x7f61741f4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:30.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.304+0000 7f6178d0e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f617415b060 0x7f61740c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:30.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.304+0000 7f6178d0e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f617415f6d0 0x7f61740c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:30.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.304+0000 7f6178d0e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f617415b440 con 0x7f617406c5c0 2024-03-29T04:11:30.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.304+0000 7f6178d0e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f617415fb30 con 0x7f617415f6d0 2024-03-29T04:11:30.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.304+0000 7f6178d0e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f61740cd530 con 0x7f617415b060 2024-03-29T04:11:30.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.305+0000 7f617259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f617406c5c0 0x7f61741f4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:30.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.305+0000 7f617259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f617406c5c0 0x7f61741f4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39378/0 (socket says 172.21.15.148:39378) 2024-03-29T04:11:30.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.305+0000 7f617259c640 1 -- 172.21.15.148:0/1337021320 learned_addr learned my addr 172.21.15.148:0/1337021320 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:30.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.305+0000 7f6172d9d640 1 --2- 172.21.15.148:0/1337021320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f617415f6d0 0x7f61740c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:30.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.305+0000 7f6171d9b640 1 --2- 172.21.15.148:0/1337021320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f617415b060 0x7f61740c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:30.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.305+0000 7f617259c640 1 -- 172.21.15.148:0/1337021320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f617415b060 msgr2=0x7f61740c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:30.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.305+0000 7f617259c640 1 --2- 172.21.15.148:0/1337021320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f617415b060 0x7f61740c6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.305+0000 7f617259c640 1 -- 172.21.15.148:0/1337021320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f617415f6d0 msgr2=0x7f61740c71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:30.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.305+0000 7f617259c640 1 --2- 172.21.15.148:0/1337021320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f617415f6d0 0x7f61740c71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.305+0000 7f617259c640 1 -- 172.21.15.148:0/1337021320 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61741f2730 con 0x7f617406c5c0 2024-03-29T04:11:30.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.305+0000 7f617259c640 1 --2- 172.21.15.148:0/1337021320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f617406c5c0 0x7f61741f4fc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f616c009870 tx=0x7f616c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:30.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.305+0000 7f6162ffd640 1 -- 172.21.15.148:0/1337021320 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c042070 con 0x7f617406c5c0 2024-03-29T04:11:30.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.305+0000 7f6162ffd640 1 -- 172.21.15.148:0/1337021320 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f616c033c10 con 0x7f617406c5c0 2024-03-29T04:11:30.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.305+0000 7f6178d0e640 1 -- 172.21.15.148:0/1337021320 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61741f2a10 con 0x7f617406c5c0 2024-03-29T04:11:30.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.305+0000 7f6162ffd640 1 -- 172.21.15.148:0/1337021320 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c038040 con 0x7f617406c5c0 2024-03-29T04:11:30.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.305+0000 7f6178d0e640 1 -- 172.21.15.148:0/1337021320 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61741f2fd0 con 0x7f617406c5c0 2024-03-29T04:11:30.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.306+0000 7f6178d0e640 1 -- 172.21.15.148:0/1337021320 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61740721e0 con 0x7f617406c5c0 2024-03-29T04:11:30.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.307+0000 7f6162ffd640 1 -- 172.21.15.148:0/1337021320 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f616c045070 con 0x7f617406c5c0 2024-03-29T04:11:30.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.307+0000 7f6162ffd640 1 --2- 172.21.15.148:0/1337021320 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f615406c530 0x7f615406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:30.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.307+0000 7f6162ffd640 1 -- 172.21.15.148:0/1337021320 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f616c0ad2b0 con 0x7f617406c5c0 2024-03-29T04:11:30.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.307+0000 7f6171d9b640 1 --2- 172.21.15.148:0/1337021320 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f615406c530 0x7f615406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:30.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.308+0000 7f6171d9b640 1 --2- 172.21.15.148:0/1337021320 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f615406c530 0x7f615406e9f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f61740c63d0 tx=0x7f61680094d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:30.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.311+0000 7f6162ffd640 1 -- 172.21.15.148:0/1337021320 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f616c04c030 con 0x7f617406c5c0 2024-03-29T04:11:30.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.486+0000 7f6178d0e640 1 -- 172.21.15.148:0/1337021320 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f61740c4ea0 con 0x7f617406c5c0 2024-03-29T04:11:30.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.486+0000 7f6162ffd640 1 -- 172.21.15.148:0/1337021320 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) ==== 84+0+65 (secure 0 0 0) 0x7f61740c4ea0 con 0x7f617406c5c0 2024-03-29T04:11:30.465 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:11:30.466 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2024-03-29T04:11:30.466 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-03-29T04:11:30.466 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2024-03-29T04:11:30.466 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2024-03-29T04:11:30.466 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:30.466 INFO:teuthology.orchestra.run.smithi148.stdout:] 2024-03-29T04:11:30.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.488+0000 7f6178d0e640 1 -- 172.21.15.148:0/1337021320 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f615406c530 msgr2=0x7f615406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:30.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.488+0000 7f6178d0e640 1 --2- 172.21.15.148:0/1337021320 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f615406c530 0x7f615406e9f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f61740c63d0 tx=0x7f61680094d0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.488+0000 7f6178d0e640 1 -- 172.21.15.148:0/1337021320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f617406c5c0 msgr2=0x7f61741f4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:30.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.488+0000 7f6178d0e640 1 --2- 172.21.15.148:0/1337021320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f617406c5c0 0x7f61741f4fc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f616c009870 tx=0x7f616c0046c0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.489+0000 7f6178d0e640 1 -- 172.21.15.148:0/1337021320 shutdown_connections 2024-03-29T04:11:30.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.489+0000 7f6178d0e640 1 --2- 172.21.15.148:0/1337021320 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f615406c530 0x7f615406e9f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.489+0000 7f6178d0e640 1 --2- 172.21.15.148:0/1337021320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f617415f6d0 0x7f61740c71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.489+0000 7f6178d0e640 1 --2- 172.21.15.148:0/1337021320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f617415b060 0x7f61740c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.489+0000 7f6178d0e640 1 --2- 172.21.15.148:0/1337021320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f617406c5c0 0x7f61741f4fc0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.489+0000 7f6178d0e640 1 -- 172.21.15.148:0/1337021320 >> 172.21.15.148:0/1337021320 conn(0x7f61740cfed0 msgr2=0x7f61740cf0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:30.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.489+0000 7f6178d0e640 1 -- 172.21.15.148:0/1337021320 shutdown_connections 2024-03-29T04:11:30.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.489+0000 7f6178d0e640 1 -- 172.21.15.148:0/1337021320 wait complete. 2024-03-29T04:11:30.478 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-29T04:11:30.478 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-29T04:11:30.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.605+0000 7fcd8c014640 1 Processor -- start 2024-03-29T04:11:30.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.605+0000 7fcd8c014640 1 -- start start 2024-03-29T04:11:30.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.606+0000 7fcd8c014640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcd84158bb0 0x7fcd840ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:30.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.606+0000 7fcd8c014640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd840681c0 con 0x7fcd8415ea60 2024-03-29T04:11:30.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.606+0000 7fcd8c014640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd8415f600 con 0x7fcd840ca100 2024-03-29T04:11:30.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.606+0000 7fcd8c014640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fcd8415f8e0 con 0x7fcd84158bb0 2024-03-29T04:11:30.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.606+0000 7fcd89db0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcd84158bb0 0x7fcd840ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:30.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.606+0000 7fcd895af640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fcd8415ea60 0x7fcd840c99f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50756/0 (socket says 172.21.15.148:50756) 2024-03-29T04:11:30.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.606+0000 7fcd895af640 1 -- 172.21.15.148:0/4265088018 learned_addr learned my addr 172.21.15.148:0/4265088018 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:30.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.606+0000 7fcd89db0640 1 -- 172.21.15.148:0/4265088018 >> v1:172.21.15.157:6789/0 conn(0x7fcd840ca100 legacy=0x7fcd840ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:30.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.606+0000 7fcd89db0640 1 -- 172.21.15.148:0/4265088018 >> v1:172.21.15.148:6789/0 conn(0x7fcd8415ea60 legacy=0x7fcd840c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:30.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.606+0000 7fcd89db0640 1 -- 172.21.15.148:0/4265088018 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd840cc2e0 con 0x7fcd84158bb0 2024-03-29T04:11:30.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.606+0000 7fcd88dae640 1 -- 172.21.15.148:0/4265088018 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1157203368 0 0) 0x7fcd840681c0 con 0x7fcd8415ea60 2024-03-29T04:11:30.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.607+0000 7fcd89db0640 1 --2- 172.21.15.148:0/4265088018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd84158bb0 0x7fcd840ccae0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fcd800097a0 tx=0x7fcd8002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cccd5c8049b7298 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:30.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.607+0000 7fcd88dae640 1 -- 172.21.15.148:0/4265088018 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd80034070 con 0x7fcd84158bb0 2024-03-29T04:11:30.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.607+0000 7fcd88dae640 1 -- 172.21.15.148:0/4265088018 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd80030610 con 0x7fcd84158bb0 2024-03-29T04:11:30.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.607+0000 7fcd88dae640 1 -- 172.21.15.148:0/4265088018 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd800309e0 con 0x7fcd84158bb0 2024-03-29T04:11:30.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.607+0000 7fcd8c014640 1 -- 172.21.15.148:0/4265088018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd84158bb0 msgr2=0x7fcd840ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:30.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.607+0000 7fcd8c014640 1 --2- 172.21.15.148:0/4265088018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd84158bb0 0x7fcd840ccae0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fcd800097a0 tx=0x7fcd8002b960 comp rx=0 tx=0).stop 2024-03-29T04:11:30.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.607+0000 7fcd8c014640 1 -- 172.21.15.148:0/4265088018 shutdown_connections 2024-03-29T04:11:30.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.607+0000 7fcd8c014640 1 --2- 172.21.15.148:0/4265088018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd84158bb0 0x7fcd840ccae0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.608+0000 7fcd8c014640 1 -- 172.21.15.148:0/4265088018 >> 172.21.15.148:0/4265088018 conn(0x7fcd840cd140 msgr2=0x7fcd840cd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:30.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.608+0000 7fcd8c014640 1 -- 172.21.15.148:0/4265088018 shutdown_connections 2024-03-29T04:11:30.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.608+0000 7fcd8c014640 1 -- 172.21.15.148:0/4265088018 wait complete. 2024-03-29T04:11:30.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.609+0000 7fcd8c014640 1 Processor -- start 2024-03-29T04:11:30.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.609+0000 7fcd8c014640 1 -- start start 2024-03-29T04:11:30.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.609+0000 7fcd8c014640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd840ca100 0x7fcd8414aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:30.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.609+0000 7fcd8c014640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd84158bb0 0x7fcd841e7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:30.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.609+0000 7fcd8c014640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd8415ea60 0x7fcd840b5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:30.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.609+0000 7fcd8c014640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcd840cc080 con 0x7fcd84158bb0 2024-03-29T04:11:30.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.609+0000 7fcd8c014640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcd840cbf00 con 0x7fcd840ca100 2024-03-29T04:11:30.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.609+0000 7fcd8c014640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcd8415ef10 con 0x7fcd8415ea60 2024-03-29T04:11:30.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.610+0000 7fcd895af640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd84158bb0 0x7fcd841e7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:30.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.610+0000 7fcd895af640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd84158bb0 0x7fcd841e7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39396/0 (socket says 172.21.15.148:39396) 2024-03-29T04:11:30.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.610+0000 7fcd895af640 1 -- 172.21.15.148:0/4169082937 learned_addr learned my addr 172.21.15.148:0/4169082937 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:30.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.610+0000 7fcd89db0640 1 --2- 172.21.15.148:0/4169082937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd840ca100 0x7fcd8414aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:30.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.610+0000 7fcd8a5b1640 1 --2- 172.21.15.148:0/4169082937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd8415ea60 0x7fcd840b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:30.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.610+0000 7fcd895af640 1 -- 172.21.15.148:0/4169082937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd8415ea60 msgr2=0x7fcd840b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:30.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.610+0000 7fcd895af640 1 --2- 172.21.15.148:0/4169082937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd8415ea60 0x7fcd840b5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.610+0000 7fcd895af640 1 -- 172.21.15.148:0/4169082937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd840ca100 msgr2=0x7fcd8414aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:30.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.610+0000 7fcd895af640 1 --2- 172.21.15.148:0/4169082937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd840ca100 0x7fcd8414aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.610+0000 7fcd895af640 1 -- 172.21.15.148:0/4169082937 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd841e88f0 con 0x7fcd84158bb0 2024-03-29T04:11:30.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.610+0000 7fcd8a5b1640 1 --2- 172.21.15.148:0/4169082937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd8415ea60 0x7fcd840b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:30.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.610+0000 7fcd895af640 1 --2- 172.21.15.148:0/4169082937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd84158bb0 0x7fcd841e7e50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fcd7400c460 tx=0x7fcd7400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:30.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.610+0000 7fcd7a7fc640 1 -- 172.21.15.148:0/4169082937 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd74005ce0 con 0x7fcd84158bb0 2024-03-29T04:11:30.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.610+0000 7fcd7a7fc640 1 -- 172.21.15.148:0/4169082937 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd74007070 con 0x7fcd84158bb0 2024-03-29T04:11:30.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.610+0000 7fcd8c014640 1 -- 172.21.15.148:0/4169082937 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd841e8bd0 con 0x7fcd84158bb0 2024-03-29T04:11:30.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.610+0000 7fcd7a7fc640 1 -- 172.21.15.148:0/4169082937 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd74007460 con 0x7fcd84158bb0 2024-03-29T04:11:30.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.610+0000 7fcd8c014640 1 -- 172.21.15.148:0/4169082937 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd841e9190 con 0x7fcd84158bb0 2024-03-29T04:11:30.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.611+0000 7fcd8c014640 1 -- 172.21.15.148:0/4169082937 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd84068740 con 0x7fcd84158bb0 2024-03-29T04:11:30.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.612+0000 7fcd7a7fc640 1 -- 172.21.15.148:0/4169082937 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcd74007600 con 0x7fcd84158bb0 2024-03-29T04:11:30.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.612+0000 7fcd7a7fc640 1 --2- 172.21.15.148:0/4169082937 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fcd6806c530 0x7fcd6806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:30.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.612+0000 7fcd7a7fc640 1 -- 172.21.15.148:0/4169082937 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fcd7408b160 con 0x7fcd84158bb0 2024-03-29T04:11:30.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.612+0000 7fcd89db0640 1 --2- 172.21.15.148:0/4169082937 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fcd6806c530 0x7fcd6806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:30.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.612+0000 7fcd89db0640 1 --2- 172.21.15.148:0/4169082937 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fcd6806c530 0x7fcd6806e9f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcd80036000 tx=0x7fcd8003b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:30.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.616+0000 7fcd7a7fc640 1 -- 172.21.15.148:0/4169082937 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcd74059940 con 0x7fcd84158bb0 2024-03-29T04:11:30.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:30.790+0000 7fcd8c014640 1 -- 172.21.15.148:0/4169082937 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fcd84067c90 con 0x7fcd84158bb0 2024-03-29T04:11:31.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.579+0000 7fcd7a7fc640 1 -- 172.21.15.148:0/4169082937 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) ==== 138+0+0 (secure 0 0 0) 0x7fcd74059240 con 0x7fcd84158bb0 2024-03-29T04:11:31.558 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2024-03-29T04:11:31.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.582+0000 7fcd8c014640 1 -- 172.21.15.148:0/4169082937 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fcd6806c530 msgr2=0x7fcd6806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:31.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.582+0000 7fcd8c014640 1 --2- 172.21.15.148:0/4169082937 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fcd6806c530 0x7fcd6806e9f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcd80036000 tx=0x7fcd8003b040 comp rx=0 tx=0).stop 2024-03-29T04:11:31.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.582+0000 7fcd8c014640 1 -- 172.21.15.148:0/4169082937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd84158bb0 msgr2=0x7fcd841e7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:31.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.582+0000 7fcd8c014640 1 --2- 172.21.15.148:0/4169082937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd84158bb0 0x7fcd841e7e50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fcd7400c460 tx=0x7fcd7400c920 comp rx=0 tx=0).stop 2024-03-29T04:11:31.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.583+0000 7fcd8c014640 1 -- 172.21.15.148:0/4169082937 shutdown_connections 2024-03-29T04:11:31.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.583+0000 7fcd8c014640 1 --2- 172.21.15.148:0/4169082937 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fcd6806c530 0x7fcd6806e9f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:31.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.583+0000 7fcd8c014640 1 --2- 172.21.15.148:0/4169082937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd8415ea60 0x7fcd840b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:31.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.583+0000 7fcd8c014640 1 --2- 172.21.15.148:0/4169082937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd84158bb0 0x7fcd841e7e50 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:31.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.583+0000 7fcd8c014640 1 --2- 172.21.15.148:0/4169082937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd840ca100 0x7fcd8414aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:31.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.583+0000 7fcd8c014640 1 -- 172.21.15.148:0/4169082937 >> 172.21.15.148:0/4169082937 conn(0x7fcd840cd140 msgr2=0x7fcd84161fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:31.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.584+0000 7fcd8c014640 1 -- 172.21.15.148:0/4169082937 shutdown_connections 2024-03-29T04:11:31.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.584+0000 7fcd8c014640 1 -- 172.21.15.148:0/4169082937 wait complete. 2024-03-29T04:11:31.577 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-29T04:11:31.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.707+0000 7eff7ba4e640 1 Processor -- start 2024-03-29T04:11:31.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.708+0000 7eff7ba4e640 1 -- start start 2024-03-29T04:11:31.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.708+0000 7eff7ba4e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7eff74147cb0 0x7eff740713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:31.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.708+0000 7eff7ba4e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff7415e800 con 0x7eff7406f9e0 2024-03-29T04:11:31.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.708+0000 7eff7ba4e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff7406fef0 con 0x7eff740d05e0 2024-03-29T04:11:31.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.708+0000 7eff7ba4e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7eff740701d0 con 0x7eff74147cb0 2024-03-29T04:11:31.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.709+0000 7eff78fe9640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7eff7406f9e0 0x7eff740cfed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50762/0 (socket says 172.21.15.148:50762) 2024-03-29T04:11:31.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.709+0000 7eff78fe9640 1 -- 172.21.15.148:0/66135089 learned_addr learned my addr 172.21.15.148:0/66135089 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:31.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.709+0000 7eff797ea640 1 --2- 172.21.15.148:0/66135089 >> v2:172.21.15.157:3300/0 conn(0x7eff74147cb0 0x7eff740713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:31.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.709+0000 7eff6bfff640 1 -- 172.21.15.148:0/66135089 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3545968095 0 0) 0x7eff7415e800 con 0x7eff7406f9e0 2024-03-29T04:11:31.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.709+0000 7eff6bfff640 1 -- 172.21.15.148:0/66135089 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff60003520 con 0x7eff7406f9e0 2024-03-29T04:11:31.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.709+0000 7eff6bfff640 1 -- 172.21.15.148:0/66135089 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1898873921 0 0) 0x7eff60003520 con 0x7eff7406f9e0 2024-03-29T04:11:31.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.709+0000 7eff6bfff640 1 -- 172.21.15.148:0/66135089 >> v2:172.21.15.157:3300/0 conn(0x7eff74147cb0 msgr2=0x7eff740713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:31.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.709+0000 7eff6bfff640 1 --2- 172.21.15.148:0/66135089 >> v2:172.21.15.157:3300/0 conn(0x7eff74147cb0 0x7eff740713f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:31.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.709+0000 7eff6bfff640 1 -- 172.21.15.148:0/66135089 >> v1:172.21.15.157:6789/0 conn(0x7eff740d05e0 legacy=0x7eff740cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:31.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.709+0000 7eff6bfff640 1 -- 172.21.15.148:0/66135089 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff740704b0 con 0x7eff7406f9e0 2024-03-29T04:11:31.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.709+0000 7eff6bfff640 1 -- 172.21.15.148:0/66135089 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7eff64001670 con 0x7eff7406f9e0 2024-03-29T04:11:31.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.709+0000 7eff6bfff640 1 -- 172.21.15.148:0/66135089 >> v1:172.21.15.148:6789/0 conn(0x7eff7406f9e0 legacy=0x7eff740cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:31.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.709+0000 7eff6bfff640 1 --2- 172.21.15.148:0/66135089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff600039b0 0x7eff60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:31.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.709+0000 7eff6bfff640 1 --2- 172.21.15.148:0/66135089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff60004800 0x7eff60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:31.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.709+0000 7eff6bfff640 1 --2- 172.21.15.148:0/66135089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff600051c0 0x7eff6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:31.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.709+0000 7eff6bfff640 1 -- 172.21.15.148:0/66135089 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7eff740701d0 con 0x7eff60004800 2024-03-29T04:11:31.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.709+0000 7eff6bfff640 1 -- 172.21.15.148:0/66135089 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7eff6000c060 con 0x7eff600051c0 2024-03-29T04:11:31.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.709+0000 7eff6bfff640 1 -- 172.21.15.148:0/66135089 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7eff6000c260 con 0x7eff600039b0 2024-03-29T04:11:31.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.709+0000 7eff78fe9640 1 --2- 172.21.15.148:0/66135089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff60004800 0x7eff60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:31.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.709+0000 7eff797ea640 1 --2- 172.21.15.148:0/66135089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff600039b0 0x7eff60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:31.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.709+0000 7eff79feb640 1 --2- 172.21.15.148:0/66135089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff600051c0 0x7eff6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:31.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.710+0000 7eff78fe9640 1 -- 172.21.15.148:0/66135089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff600039b0 msgr2=0x7eff60003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:31.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.710+0000 7eff78fe9640 1 --2- 172.21.15.148:0/66135089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff600039b0 0x7eff60003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:31.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.710+0000 7eff78fe9640 1 -- 172.21.15.148:0/66135089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff600051c0 msgr2=0x7eff6000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:31.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.710+0000 7eff78fe9640 1 --2- 172.21.15.148:0/66135089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff600051c0 0x7eff6000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:31.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.710+0000 7eff79feb640 1 -- 172.21.15.148:0/66135089 reap_dead start 2024-03-29T04:11:31.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.710+0000 7eff78fe9640 1 -- 172.21.15.148:0/66135089 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff60003520 con 0x7eff60004800 2024-03-29T04:11:31.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.710+0000 7eff78fe9640 1 --2- 172.21.15.148:0/66135089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff60004800 0x7eff60004c80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7eff6400c210 tx=0x7eff60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df019ec84abbf4b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:31.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.710+0000 7eff6bfff640 1 -- 172.21.15.148:0/66135089 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff64007cf0 con 0x7eff60004800 2024-03-29T04:11:31.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.710+0000 7eff6bfff640 1 -- 172.21.15.148:0/66135089 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff64030070 con 0x7eff60004800 2024-03-29T04:11:31.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.710+0000 7eff6bfff640 1 -- 172.21.15.148:0/66135089 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff64005d50 con 0x7eff60004800 2024-03-29T04:11:31.692 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:11:31.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.711+0000 7eff7ba4e640 1 -- 172.21.15.148:0/66135089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff60004800 msgr2=0x7eff60004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:31.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.711+0000 7eff7ba4e640 1 --2- 172.21.15.148:0/66135089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff60004800 0x7eff60004c80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7eff6400c210 tx=0x7eff60005830 comp rx=0 tx=0).stop 2024-03-29T04:11:31.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.711+0000 7eff7ba4e640 1 -- 172.21.15.148:0/66135089 shutdown_connections 2024-03-29T04:11:31.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.711+0000 7eff7ba4e640 1 --2- 172.21.15.148:0/66135089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff60004800 0x7eff60004c80 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:31.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.711+0000 7eff7ba4e640 1 -- 172.21.15.148:0/66135089 >> 172.21.15.148:0/66135089 conn(0x7eff740d2300 msgr2=0x7eff740d26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:31.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.711+0000 7eff7ba4e640 1 -- 172.21.15.148:0/66135089 shutdown_connections 2024-03-29T04:11:31.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.711+0000 7eff7ba4e640 1 -- 172.21.15.148:0/66135089 wait complete. 2024-03-29T04:11:31.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.712+0000 7eff7ba4e640 1 Processor -- start 2024-03-29T04:11:31.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.712+0000 7eff7ba4e640 1 -- start start 2024-03-29T04:11:31.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.713+0000 7eff7ba4e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff60004800 0x7eff740b1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:31.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.713+0000 7eff7ba4e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff7406d8f0 0x7eff740af130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:31.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.713+0000 7eff7ba4e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff740af670 0x7eff740cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:31.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.713+0000 7eff7ba4e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7eff740d0050 con 0x7eff740af670 2024-03-29T04:11:31.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.713+0000 7eff7ba4e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7eff740cfed0 con 0x7eff7406d8f0 2024-03-29T04:11:31.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.713+0000 7eff7ba4e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7eff740d01d0 con 0x7eff60004800 2024-03-29T04:11:31.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.713+0000 7eff79feb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff740af670 0x7eff740cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:31.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.713+0000 7eff79feb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff740af670 0x7eff740cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39410/0 (socket says 172.21.15.148:39410) 2024-03-29T04:11:31.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.713+0000 7eff79feb640 1 -- 172.21.15.148:0/1548403472 learned_addr learned my addr 172.21.15.148:0/1548403472 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:31.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.713+0000 7eff797ea640 1 --2- 172.21.15.148:0/1548403472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff60004800 0x7eff740b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:31.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.713+0000 7eff78fe9640 1 --2- 172.21.15.148:0/1548403472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff7406d8f0 0x7eff740af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:31.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.713+0000 7eff79feb640 1 -- 172.21.15.148:0/1548403472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff60004800 msgr2=0x7eff740b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:31.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.713+0000 7eff79feb640 1 --2- 172.21.15.148:0/1548403472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff60004800 0x7eff740b1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:31.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.713+0000 7eff79feb640 1 -- 172.21.15.148:0/1548403472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff7406d8f0 msgr2=0x7eff740af130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:31.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.713+0000 7eff79feb640 1 --2- 172.21.15.148:0/1548403472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff7406d8f0 0x7eff740af130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:31.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.713+0000 7eff79feb640 1 -- 172.21.15.148:0/1548403472 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff7406dd80 con 0x7eff740af670 2024-03-29T04:11:31.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.713+0000 7eff78fe9640 1 --2- 172.21.15.148:0/1548403472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff7406d8f0 0x7eff740af130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:31.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.714+0000 7eff79feb640 1 --2- 172.21.15.148:0/1548403472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff740af670 0x7eff740cb5a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7eff740d2ea0 tx=0x7eff6c00fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:31.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.714+0000 7eff69ffb640 1 -- 172.21.15.148:0/1548403472 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c01a070 con 0x7eff740af670 2024-03-29T04:11:31.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.714+0000 7eff69ffb640 1 -- 172.21.15.148:0/1548403472 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff6c006d60 con 0x7eff740af670 2024-03-29T04:11:31.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.714+0000 7eff7ba4e640 1 -- 172.21.15.148:0/1548403472 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff7406e060 con 0x7eff740af670 2024-03-29T04:11:31.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.714+0000 7eff69ffb640 1 -- 172.21.15.148:0/1548403472 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c003bb0 con 0x7eff740af670 2024-03-29T04:11:31.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.714+0000 7eff7ba4e640 1 -- 172.21.15.148:0/1548403472 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff740b61e0 con 0x7eff740af670 2024-03-29T04:11:31.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.715+0000 7eff7ba4e640 1 -- 172.21.15.148:0/1548403472 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff74068850 con 0x7eff740af670 2024-03-29T04:11:31.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.715+0000 7eff69ffb640 1 -- 172.21.15.148:0/1548403472 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7eff6c01d070 con 0x7eff740af670 2024-03-29T04:11:31.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.716+0000 7eff69ffb640 1 --2- 172.21.15.148:0/1548403472 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7eff5806c530 0x7eff5806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:31.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.716+0000 7eff69ffb640 1 -- 172.21.15.148:0/1548403472 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5619+0+0 (secure 0 0 0) 0x7eff6c08abc0 con 0x7eff740af670 2024-03-29T04:11:31.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.716+0000 7eff797ea640 1 --2- 172.21.15.148:0/1548403472 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7eff5806c530 0x7eff5806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:31.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.716+0000 7eff797ea640 1 --2- 172.21.15.148:0/1548403472 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7eff5806c530 0x7eff5806e9f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7eff70009a60 tx=0x7eff70008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:31.699 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:11:31.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.721+0000 7eff69ffb640 1 -- 172.21.15.148:0/1548403472 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7eff6c059230 con 0x7eff740af670 2024-03-29T04:11:31.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:31.903+0000 7eff7ba4e640 1 -- 172.21.15.148:0/1548403472 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7eff74071c30 con 0x7eff740af670 2024-03-29T04:11:32.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.584+0000 7eff69ffb640 1 -- 172.21.15.148:0/1548403472 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) ==== 170+0+0 (secure 0 0 0) 0x7eff6c058b30 con 0x7eff740af670 2024-03-29T04:11:32.563 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2024-03-29T04:11:32.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.587+0000 7eff7ba4e640 1 -- 172.21.15.148:0/1548403472 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7eff5806c530 msgr2=0x7eff5806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:32.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.587+0000 7eff7ba4e640 1 --2- 172.21.15.148:0/1548403472 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7eff5806c530 0x7eff5806e9f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7eff70009a60 tx=0x7eff70008040 comp rx=0 tx=0).stop 2024-03-29T04:11:32.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.587+0000 7eff7ba4e640 1 -- 172.21.15.148:0/1548403472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff740af670 msgr2=0x7eff740cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:32.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.587+0000 7eff7ba4e640 1 --2- 172.21.15.148:0/1548403472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff740af670 0x7eff740cb5a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7eff740d2ea0 tx=0x7eff6c00fc00 comp rx=0 tx=0).stop 2024-03-29T04:11:32.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.588+0000 7eff7ba4e640 1 -- 172.21.15.148:0/1548403472 shutdown_connections 2024-03-29T04:11:32.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.588+0000 7eff7ba4e640 1 --2- 172.21.15.148:0/1548403472 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7eff5806c530 0x7eff5806e9f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:32.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.588+0000 7eff7ba4e640 1 --2- 172.21.15.148:0/1548403472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff740af670 0x7eff740cb5a0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:32.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.588+0000 7eff7ba4e640 1 --2- 172.21.15.148:0/1548403472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff7406d8f0 0x7eff740af130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:32.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.588+0000 7eff7ba4e640 1 --2- 172.21.15.148:0/1548403472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff60004800 0x7eff740b1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:32.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.588+0000 7eff7ba4e640 1 -- 172.21.15.148:0/1548403472 >> 172.21.15.148:0/1548403472 conn(0x7eff740d2300 msgr2=0x7eff74160810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:32.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.589+0000 7eff7ba4e640 1 -- 172.21.15.148:0/1548403472 shutdown_connections 2024-03-29T04:11:32.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.590+0000 7eff7ba4e640 1 -- 172.21.15.148:0/1548403472 wait complete. 2024-03-29T04:11:32.592 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-29T04:11:32.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.719+0000 7f5d2fa55640 1 Processor -- start 2024-03-29T04:11:32.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.719+0000 7f5d2fa55640 1 -- start start 2024-03-29T04:11:32.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.720+0000 7f5d2fa55640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5d2806ccd0 0x7f5d280b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:32.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.720+0000 7f5d2fa55640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d280b3d60 con 0x7f5d28073120 2024-03-29T04:11:32.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.720+0000 7f5d2fa55640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d280b34c0 con 0x7f5d28072d40 2024-03-29T04:11:32.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.720+0000 7f5d2fa55640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f5d2815ee60 con 0x7f5d2806ccd0 2024-03-29T04:11:32.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.720+0000 7f5d2dff2640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5d2806ccd0 0x7f5d280b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:32.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.720+0000 7f5d2cff0640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5d28073120 0x7f5d2806c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50774/0 (socket says 172.21.15.148:50774) 2024-03-29T04:11:32.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.720+0000 7f5d2cff0640 1 -- 172.21.15.148:0/3073278028 learned_addr learned my addr 172.21.15.148:0/3073278028 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:32.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.721+0000 7f5d2dff2640 1 -- 172.21.15.148:0/3073278028 >> v1:172.21.15.157:6789/0 conn(0x7f5d28072d40 legacy=0x7f5d2815b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:32.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.721+0000 7f5d2dff2640 1 -- 172.21.15.148:0/3073278028 >> v1:172.21.15.148:6789/0 conn(0x7f5d28073120 legacy=0x7f5d2806c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:32.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.721+0000 7f5d2dff2640 1 -- 172.21.15.148:0/3073278028 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d280b37a0 con 0x7f5d2806ccd0 2024-03-29T04:11:32.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.721+0000 7f5d27fff640 1 -- 172.21.15.148:0/3073278028 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2575296140 0 0) 0x7f5d280b3d60 con 0x7f5d28073120 2024-03-29T04:11:32.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.721+0000 7f5d2dff2640 1 --2- 172.21.15.148:0/3073278028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d2806ccd0 0x7f5d280b2e70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5d2815efe0 tx=0x7f5d2002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ade46ada26be5ec server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:32.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.722+0000 7f5d27fff640 1 -- 172.21.15.148:0/3073278028 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20034070 con 0x7f5d2806ccd0 2024-03-29T04:11:32.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.722+0000 7f5d27fff640 1 -- 172.21.15.148:0/3073278028 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d20030610 con 0x7f5d2806ccd0 2024-03-29T04:11:32.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.722+0000 7f5d27fff640 1 -- 172.21.15.148:0/3073278028 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d200309e0 con 0x7f5d2806ccd0 2024-03-29T04:11:32.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.722+0000 7f5d2fa55640 1 -- 172.21.15.148:0/3073278028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d2806ccd0 msgr2=0x7f5d280b2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:32.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.722+0000 7f5d2fa55640 1 --2- 172.21.15.148:0/3073278028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d2806ccd0 0x7f5d280b2e70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5d2815efe0 tx=0x7f5d2002b960 comp rx=0 tx=0).stop 2024-03-29T04:11:32.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.722+0000 7f5d2fa55640 1 -- 172.21.15.148:0/3073278028 shutdown_connections 2024-03-29T04:11:32.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.722+0000 7f5d2fa55640 1 --2- 172.21.15.148:0/3073278028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d2806ccd0 0x7f5d280b2e70 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:32.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.722+0000 7f5d2fa55640 1 -- 172.21.15.148:0/3073278028 >> 172.21.15.148:0/3073278028 conn(0x7f5d280d00a0 msgr2=0x7f5d280d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:32.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.722+0000 7f5d2fa55640 1 -- 172.21.15.148:0/3073278028 shutdown_connections 2024-03-29T04:11:32.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.722+0000 7f5d2fa55640 1 -- 172.21.15.148:0/3073278028 wait complete. 2024-03-29T04:11:32.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.723+0000 7f5d2fa55640 1 Processor -- start 2024-03-29T04:11:32.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.724+0000 7f5d2fa55640 1 -- start start 2024-03-29T04:11:32.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.724+0000 7f5d2fa55640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d2806ccd0 0x7f5d280cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:32.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.724+0000 7f5d2fa55640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d28072d40 0x7f5d28155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:32.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.724+0000 7f5d2fa55640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d28073120 0x7f5d2813c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:32.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.724+0000 7f5d2fa55640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2815bf70 con 0x7f5d28072d40 2024-03-29T04:11:32.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.724+0000 7f5d2fa55640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2815bdf0 con 0x7f5d28073120 2024-03-29T04:11:32.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.724+0000 7f5d2fa55640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5d281412d0 con 0x7f5d2806ccd0 2024-03-29T04:11:32.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.724+0000 7f5d2cff0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d28072d40 0x7f5d28155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:32.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.724+0000 7f5d2cff0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d28072d40 0x7f5d28155e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39430/0 (socket says 172.21.15.148:39430) 2024-03-29T04:11:32.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.724+0000 7f5d2cff0640 1 -- 172.21.15.148:0/1687000207 learned_addr learned my addr 172.21.15.148:0/1687000207 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:32.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.725+0000 7f5d2dff2640 1 --2- 172.21.15.148:0/1687000207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d28073120 0x7f5d2813c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:32.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.725+0000 7f5d2d7f1640 1 --2- 172.21.15.148:0/1687000207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d2806ccd0 0x7f5d280cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:32.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.725+0000 7f5d2cff0640 1 -- 172.21.15.148:0/1687000207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d2806ccd0 msgr2=0x7f5d280cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:32.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.725+0000 7f5d2cff0640 1 --2- 172.21.15.148:0/1687000207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d2806ccd0 0x7f5d280cf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:32.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.725+0000 7f5d2cff0640 1 -- 172.21.15.148:0/1687000207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d28073120 msgr2=0x7f5d2813c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:32.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.725+0000 7f5d2cff0640 1 --2- 172.21.15.148:0/1687000207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d28073120 0x7f5d2813c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:32.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.725+0000 7f5d2cff0640 1 -- 172.21.15.148:0/1687000207 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2813d960 con 0x7f5d28072d40 2024-03-29T04:11:32.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.725+0000 7f5d2d7f1640 1 --2- 172.21.15.148:0/1687000207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d2806ccd0 0x7f5d280cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:32.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.725+0000 7f5d2dff2640 1 --2- 172.21.15.148:0/1687000207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d28073120 0x7f5d2813c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:32.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.725+0000 7f5d2cff0640 1 --2- 172.21.15.148:0/1687000207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d28072d40 0x7f5d28155e00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00c3f0 tx=0x7f5d1c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:32.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.725+0000 7f5d25ffb640 1 -- 172.21.15.148:0/1687000207 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c0074e0 con 0x7f5d28072d40 2024-03-29T04:11:32.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.725+0000 7f5d25ffb640 1 -- 172.21.15.148:0/1687000207 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d1c0047e0 con 0x7f5d28072d40 2024-03-29T04:11:32.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.725+0000 7f5d2fa55640 1 -- 172.21.15.148:0/1687000207 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d28156340 con 0x7f5d28072d40 2024-03-29T04:11:32.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.725+0000 7f5d25ffb640 1 -- 172.21.15.148:0/1687000207 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c004bd0 con 0x7f5d28072d40 2024-03-29T04:11:32.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.725+0000 7f5d2fa55640 1 -- 172.21.15.148:0/1687000207 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d2813d680 con 0x7f5d28072d40 2024-03-29T04:11:32.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.726+0000 7f5d2fa55640 1 -- 172.21.15.148:0/1687000207 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d28068850 con 0x7f5d28072d40 2024-03-29T04:11:32.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.726+0000 7f5d25ffb640 1 -- 172.21.15.148:0/1687000207 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5d1c011070 con 0x7f5d28072d40 2024-03-29T04:11:32.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.727+0000 7f5d25ffb640 1 --2- 172.21.15.148:0/1687000207 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5d1406c530 0x7f5d1406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:32.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.727+0000 7f5d2d7f1640 1 --2- 172.21.15.148:0/1687000207 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5d1406c530 0x7f5d1406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:32.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.727+0000 7f5d25ffb640 1 -- 172.21.15.148:0/1687000207 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6001+0+0 (secure 0 0 0) 0x7f5d1c08b3b0 con 0x7f5d28072d40 2024-03-29T04:11:32.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.727+0000 7f5d2d7f1640 1 --2- 172.21.15.148:0/1687000207 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5d1406c530 0x7f5d1406e9f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5d180012e0 tx=0x7f5d18009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:32.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.732+0000 7f5d25ffb640 1 -- 172.21.15.148:0/1687000207 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5d1c01a2b0 con 0x7f5d28072d40 2024-03-29T04:11:32.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:32.949+0000 7f5d2fa55640 1 -- 172.21.15.148:0/1687000207 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5d2813cf30 con 0x7f5d28072d40 2024-03-29T04:11:33.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.586+0000 7f5d25ffb640 1 -- 172.21.15.148:0/1687000207 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v15) ==== 172+0+0 (secure 0 0 0) 0x7f5d1c059930 con 0x7f5d28072d40 2024-03-29T04:11:33.565 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 5 and data pool 6 2024-03-29T04:11:33.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.592+0000 7f5d2fa55640 1 -- 172.21.15.148:0/1687000207 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5d1406c530 msgr2=0x7f5d1406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:33.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.592+0000 7f5d2fa55640 1 --2- 172.21.15.148:0/1687000207 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5d1406c530 0x7f5d1406e9f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5d180012e0 tx=0x7f5d18009040 comp rx=0 tx=0).stop 2024-03-29T04:11:33.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.592+0000 7f5d2fa55640 1 -- 172.21.15.148:0/1687000207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d28072d40 msgr2=0x7f5d28155e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:33.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.592+0000 7f5d2fa55640 1 --2- 172.21.15.148:0/1687000207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d28072d40 0x7f5d28155e00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00c3f0 tx=0x7f5d1c00c8b0 comp rx=0 tx=0).stop 2024-03-29T04:11:33.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.592+0000 7f5d2fa55640 1 -- 172.21.15.148:0/1687000207 shutdown_connections 2024-03-29T04:11:33.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.592+0000 7f5d2fa55640 1 --2- 172.21.15.148:0/1687000207 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5d1406c530 0x7f5d1406e9f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:33.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.592+0000 7f5d2fa55640 1 --2- 172.21.15.148:0/1687000207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d28073120 0x7f5d2813c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:33.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.592+0000 7f5d2fa55640 1 --2- 172.21.15.148:0/1687000207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d28072d40 0x7f5d28155e00 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:33.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.592+0000 7f5d2fa55640 1 --2- 172.21.15.148:0/1687000207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d2806ccd0 0x7f5d280cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:33.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.592+0000 7f5d2fa55640 1 -- 172.21.15.148:0/1687000207 >> 172.21.15.148:0/1687000207 conn(0x7f5d280d00a0 msgr2=0x7f5d2815b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:33.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.596+0000 7f5d2fa55640 1 -- 172.21.15.148:0/1687000207 shutdown_connections 2024-03-29T04:11:33.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.596+0000 7f5d2fa55640 1 -- 172.21.15.148:0/1687000207 wait complete. 2024-03-29T04:11:33.605 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:11:33.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.729+0000 7f52d253d640 1 Processor -- start 2024-03-29T04:11:33.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.729+0000 7f52d253d640 1 -- start start 2024-03-29T04:11:33.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.729+0000 7f52d253d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f52cc15e3e0 0x7f52cc0d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:33.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.729+0000 7f52d253d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52cc0ba030 0x7f52cc0d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:33.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.729+0000 7f52d253d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f52cc0d10d0 0x7f52cc0d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:33.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.729+0000 7f52d253d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f52cc0d3840 con 0x7f52cc0d10d0 2024-03-29T04:11:33.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.729+0000 7f52d253d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f52cc15e7c0 con 0x7f52cc15e3e0 2024-03-29T04:11:33.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.729+0000 7f52d253d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f52cc0d2c30 con 0x7f52cc0ba030 2024-03-29T04:11:33.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.729+0000 7f52d0ada640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f52cc0d10d0 0x7f52cc0d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:33.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.730+0000 7f52d0ada640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f52cc0d10d0 0x7f52cc0d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39444/0 (socket says 172.21.15.148:39444) 2024-03-29T04:11:33.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.730+0000 7f52d0ada640 1 -- 172.21.15.148:0/2352492314 learned_addr learned my addr 172.21.15.148:0/2352492314 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:33.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.730+0000 7f52cb7fe640 1 --2- 172.21.15.148:0/2352492314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52cc0ba030 0x7f52cc0d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:33.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.730+0000 7f52cbfff640 1 --2- 172.21.15.148:0/2352492314 >> v2:172.21.15.157:3300/0 conn(0x7f52cc15e3e0 0x7f52cc0d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:33.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.730+0000 7f52d0ada640 1 -- 172.21.15.148:0/2352492314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52cc0ba030 msgr2=0x7f52cc0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:33.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.730+0000 7f52d0ada640 1 --2- 172.21.15.148:0/2352492314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52cc0ba030 0x7f52cc0d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:33.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.730+0000 7f52d0ada640 1 -- 172.21.15.148:0/2352492314 >> v2:172.21.15.157:3300/0 conn(0x7f52cc15e3e0 msgr2=0x7f52cc0d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:33.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.730+0000 7f52d0ada640 1 --2- 172.21.15.148:0/2352492314 >> v2:172.21.15.157:3300/0 conn(0x7f52cc15e3e0 0x7f52cc0d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:33.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.730+0000 7f52d0ada640 1 -- 172.21.15.148:0/2352492314 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52cc0d2f10 con 0x7f52cc0d10d0 2024-03-29T04:11:33.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.730+0000 7f52cb7fe640 1 --2- 172.21.15.148:0/2352492314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52cc0ba030 0x7f52cc0d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:33.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.730+0000 7f52cbfff640 1 --2- 172.21.15.148:0/2352492314 >> v2:172.21.15.157:3300/0 conn(0x7f52cc15e3e0 0x7f52cc0d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:33.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.731+0000 7f52d0ada640 1 --2- 172.21.15.148:0/2352492314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52cc0d10d0 0x7f52cc0d1580 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f52cc0b4eb0 tx=0x7f52c402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd62499aca170b61 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:33.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.731+0000 7f52caffd640 1 -- 172.21.15.148:0/2352492314 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c402cb00 con 0x7f52cc0d10d0 2024-03-29T04:11:33.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.731+0000 7f52caffd640 1 -- 172.21.15.148:0/2352492314 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c402cca0 con 0x7f52cc0d10d0 2024-03-29T04:11:33.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.731+0000 7f52caffd640 1 -- 172.21.15.148:0/2352492314 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c4034610 con 0x7f52cc0d10d0 2024-03-29T04:11:33.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.731+0000 7f52d253d640 1 -- 172.21.15.148:0/2352492314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52cc0d10d0 msgr2=0x7f52cc0d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:33.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.731+0000 7f52d253d640 1 --2- 172.21.15.148:0/2352492314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52cc0d10d0 0x7f52cc0d1580 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f52cc0b4eb0 tx=0x7f52c402bf60 comp rx=0 tx=0).stop 2024-03-29T04:11:33.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.731+0000 7f52d253d640 1 -- 172.21.15.148:0/2352492314 shutdown_connections 2024-03-29T04:11:33.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.731+0000 7f52d253d640 1 --2- 172.21.15.148:0/2352492314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52cc0d10d0 0x7f52cc0d1580 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:33.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.731+0000 7f52d253d640 1 --2- 172.21.15.148:0/2352492314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52cc0ba030 0x7f52cc0d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:33.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.731+0000 7f52d253d640 1 --2- 172.21.15.148:0/2352492314 >> v2:172.21.15.157:3300/0 conn(0x7f52cc15e3e0 0x7f52cc0d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:33.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.731+0000 7f52d253d640 1 -- 172.21.15.148:0/2352492314 >> 172.21.15.148:0/2352492314 conn(0x7f52cc0b6210 msgr2=0x7f52cc0b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:33.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.731+0000 7f52d253d640 1 -- 172.21.15.148:0/2352492314 shutdown_connections 2024-03-29T04:11:33.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.731+0000 7f52d253d640 1 -- 172.21.15.148:0/2352492314 wait complete. 2024-03-29T04:11:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.732+0000 7f52d253d640 1 Processor -- start 2024-03-29T04:11:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.733+0000 7f52d253d640 1 -- start start 2024-03-29T04:11:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.733+0000 7f52d253d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52cc0ba030 0x7f52cc1614e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.733+0000 7f52d253d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52cc0d10d0 0x7f52cc0cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.733+0000 7f52d253d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52cc15e3e0 0x7f52cc15ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.733+0000 7f52d253d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc0d2870 con 0x7f52cc0ba030 2024-03-29T04:11:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.733+0000 7f52d253d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc0d34d0 con 0x7f52cc0d10d0 2024-03-29T04:11:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.733+0000 7f52d253d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f52cc0d29f0 con 0x7f52cc15e3e0 2024-03-29T04:11:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.733+0000 7f52cbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52cc0ba030 0x7f52cc1614e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.733+0000 7f52cbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52cc0ba030 0x7f52cc1614e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39448/0 (socket says 172.21.15.148:39448) 2024-03-29T04:11:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.733+0000 7f52cbfff640 1 -- 172.21.15.148:0/2845965719 learned_addr learned my addr 172.21.15.148:0/2845965719 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.734+0000 7f52cb7fe640 1 --2- 172.21.15.148:0/2845965719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52cc0d10d0 0x7f52cc0cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.734+0000 7f52d0ada640 1 --2- 172.21.15.148:0/2845965719 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52cc15e3e0 0x7f52cc15ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.734+0000 7f52cbfff640 1 -- 172.21.15.148:0/2845965719 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52cc15e3e0 msgr2=0x7f52cc15ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.734+0000 7f52cbfff640 1 --2- 172.21.15.148:0/2845965719 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52cc15e3e0 0x7f52cc15ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.734+0000 7f52cbfff640 1 -- 172.21.15.148:0/2845965719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52cc0d10d0 msgr2=0x7f52cc0cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.734+0000 7f52cbfff640 1 --2- 172.21.15.148:0/2845965719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52cc0d10d0 0x7f52cc0cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.734+0000 7f52cbfff640 1 -- 172.21.15.148:0/2845965719 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52cc0ccba0 con 0x7f52cc0ba030 2024-03-29T04:11:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.734+0000 7f52d0ada640 1 --2- 172.21.15.148:0/2845965719 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52cc15e3e0 0x7f52cc15ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.734+0000 7f52cb7fe640 1 --2- 172.21.15.148:0/2845965719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52cc0d10d0 0x7f52cc0cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.734+0000 7f52cbfff640 1 --2- 172.21.15.148:0/2845965719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52cc0ba030 0x7f52cc1614e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f52bc00d7d0 tx=0x7f52bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.734+0000 7f52c8ff9640 1 -- 172.21.15.148:0/2845965719 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc004030 con 0x7f52cc0ba030 2024-03-29T04:11:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.734+0000 7f52c8ff9640 1 -- 172.21.15.148:0/2845965719 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52bc0041d0 con 0x7f52cc0ba030 2024-03-29T04:11:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.734+0000 7f52c8ff9640 1 -- 172.21.15.148:0/2845965719 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc0096a0 con 0x7f52cc0ba030 2024-03-29T04:11:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.734+0000 7f52d253d640 1 -- 172.21.15.148:0/2845965719 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52cc0cce80 con 0x7f52cc0ba030 2024-03-29T04:11:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.734+0000 7f52d253d640 1 -- 172.21.15.148:0/2845965719 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52cc0cd440 con 0x7f52cc0ba030 2024-03-29T04:11:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.735+0000 7f52d253d640 1 -- 172.21.15.148:0/2845965719 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52cc0c8650 con 0x7f52cc0ba030 2024-03-29T04:11:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.735+0000 7f52c8ff9640 1 -- 172.21.15.148:0/2845965719 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f52bc009840 con 0x7f52cc0ba030 2024-03-29T04:11:33.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.736+0000 7f52c8ff9640 1 --2- 172.21.15.148:0/2845965719 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f52b806c530 0x7f52b806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:33.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.736+0000 7f52cb7fe640 1 --2- 172.21.15.148:0/2845965719 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f52b806c530 0x7f52b806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:33.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.736+0000 7f52c8ff9640 1 -- 172.21.15.148:0/2845965719 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f52bc08bb20 con 0x7f52cc0ba030 2024-03-29T04:11:33.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.736+0000 7f52cb7fe640 1 --2- 172.21.15.148:0/2845965719 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f52b806c530 0x7f52b806e9f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f52c0006fd0 tx=0x7f52c0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:33.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.740+0000 7f52c8ff9640 1 -- 172.21.15.148:0/2845965719 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f52cc0c8650 con 0x7f52cc0ba030 2024-03-29T04:11:33.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.918+0000 7f52d253d640 1 -- 172.21.15.148:0/2845965719 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52cc0d3b20 con 0x7f52cc0ba030 2024-03-29T04:11:33.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.920+0000 7f52c8ff9640 1 -- 172.21.15.148:0/2845965719 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16967 (secure 0 0 0) 0x7f52bc05a040 con 0x7f52cc0ba030 2024-03-29T04:11:33.899 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:11:33.899 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":38,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:11:32.950071+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:11:30.794790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T04:11:31.907100+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:11:33.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.922+0000 7f52d253d640 1 -- 172.21.15.148:0/2845965719 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f52b806c530 msgr2=0x7f52b806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:33.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.922+0000 7f52d253d640 1 --2- 172.21.15.148:0/2845965719 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f52b806c530 0x7f52b806e9f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f52c0006fd0 tx=0x7f52c0008040 comp rx=0 tx=0).stop 2024-03-29T04:11:33.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.922+0000 7f52d253d640 1 -- 172.21.15.148:0/2845965719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52cc0ba030 msgr2=0x7f52cc1614e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:33.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.922+0000 7f52d253d640 1 --2- 172.21.15.148:0/2845965719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52cc0ba030 0x7f52cc1614e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f52bc00d7d0 tx=0x7f52bc00dc90 comp rx=0 tx=0).stop 2024-03-29T04:11:33.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.922+0000 7f52d253d640 1 -- 172.21.15.148:0/2845965719 shutdown_connections 2024-03-29T04:11:33.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.922+0000 7f52d253d640 1 --2- 172.21.15.148:0/2845965719 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f52b806c530 0x7f52b806e9f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:33.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.922+0000 7f52d253d640 1 --2- 172.21.15.148:0/2845965719 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52cc15e3e0 0x7f52cc15ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:33.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.922+0000 7f52d253d640 1 --2- 172.21.15.148:0/2845965719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52cc0d10d0 0x7f52cc0cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:33.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.922+0000 7f52d253d640 1 --2- 172.21.15.148:0/2845965719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52cc0ba030 0x7f52cc1614e0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:33.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.922+0000 7f52d253d640 1 -- 172.21.15.148:0/2845965719 >> 172.21.15.148:0/2845965719 conn(0x7f52cc0b6210 msgr2=0x7f52cc0c7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:33.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.922+0000 7f52d253d640 1 -- 172.21.15.148:0/2845965719 shutdown_connections 2024-03-29T04:11:33.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:33.922+0000 7f52d253d640 1 -- 172.21.15.148:0/2845965719 wait complete. 2024-03-29T04:11:33.912 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:11:34.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.034+0000 7fee94160640 1 Processor -- start 2024-03-29T04:11:34.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.035+0000 7fee94160640 1 -- start start 2024-03-29T04:11:34.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.035+0000 7fee94160640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee8c0d17e0 0x7fee8c0d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.035+0000 7fee94160640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee8c15e7c0 con 0x7fee8c0ba030 2024-03-29T04:11:34.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.035+0000 7fee94160640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee8c0d2c30 con 0x7fee8c15e3e0 2024-03-29T04:11:34.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.035+0000 7fee94160640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fee8c0d3840 con 0x7fee8c0d17e0 2024-03-29T04:11:34.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.035+0000 7fee916fb640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fee8c0ba030 0x7fee8c0d10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50776/0 (socket says 172.21.15.148:50776) 2024-03-29T04:11:34.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.035+0000 7fee916fb640 1 -- 172.21.15.148:0/1198644857 learned_addr learned my addr 172.21.15.148:0/1198644857 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:34.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.036+0000 7fee926fd640 1 --2- 172.21.15.148:0/1198644857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee8c0d17e0 0x7fee8c0d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.036+0000 7fee90efa640 1 -- 172.21.15.148:0/1198644857 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1526811494 0 0) 0x7fee8c15e7c0 con 0x7fee8c0ba030 2024-03-29T04:11:34.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.036+0000 7fee90efa640 1 -- 172.21.15.148:0/1198644857 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee74003520 con 0x7fee8c0ba030 2024-03-29T04:11:34.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.036+0000 7fee90efa640 1 -- 172.21.15.148:0/1198644857 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1739453951 0 0) 0x7fee74003520 con 0x7fee8c0ba030 2024-03-29T04:11:34.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.036+0000 7fee90efa640 1 -- 172.21.15.148:0/1198644857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee8c0d17e0 msgr2=0x7fee8c0d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:34.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.036+0000 7fee90efa640 1 --2- 172.21.15.148:0/1198644857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee8c0d17e0 0x7fee8c0d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.036+0000 7fee90efa640 1 -- 172.21.15.148:0/1198644857 >> v1:172.21.15.157:6789/0 conn(0x7fee8c15e3e0 legacy=0x7fee8c0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:34.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.036+0000 7fee90efa640 1 -- 172.21.15.148:0/1198644857 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8c0d2f10 con 0x7fee8c0ba030 2024-03-29T04:11:34.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.036+0000 7fee90efa640 1 -- 172.21.15.148:0/1198644857 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7fee80001670 con 0x7fee8c0ba030 2024-03-29T04:11:34.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.036+0000 7fee90efa640 1 -- 172.21.15.148:0/1198644857 >> v1:172.21.15.148:6789/0 conn(0x7fee8c0ba030 legacy=0x7fee8c0d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:34.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.036+0000 7fee90efa640 1 --2- 172.21.15.148:0/1198644857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee740039e0 0x7fee74003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.036+0000 7fee90efa640 1 --2- 172.21.15.148:0/1198644857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee74004800 0x7fee74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.036+0000 7fee90efa640 1 --2- 172.21.15.148:0/1198644857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee8c0d17e0 0x7fee740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.036+0000 7fee90efa640 1 -- 172.21.15.148:0/1198644857 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c0d3840 con 0x7fee74004800 2024-03-29T04:11:34.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.036+0000 7fee90efa640 1 -- 172.21.15.148:0/1198644857 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fee7400bb30 con 0x7fee740039e0 2024-03-29T04:11:34.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.036+0000 7fee90efa640 1 -- 172.21.15.148:0/1198644857 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fee7400bd30 con 0x7fee8c0d17e0 2024-03-29T04:11:34.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.036+0000 7fee916fb640 1 --2- 172.21.15.148:0/1198644857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee74004800 0x7fee74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.036+0000 7fee926fd640 1 --2- 172.21.15.148:0/1198644857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee8c0d17e0 0x7fee740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.036+0000 7fee91efc640 1 --2- 172.21.15.148:0/1198644857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee740039e0 0x7fee74003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.036+0000 7fee916fb640 1 -- 172.21.15.148:0/1198644857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee8c0d17e0 msgr2=0x7fee740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:34.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.036+0000 7fee916fb640 1 --2- 172.21.15.148:0/1198644857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee8c0d17e0 0x7fee740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.036+0000 7fee916fb640 1 -- 172.21.15.148:0/1198644857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee740039e0 msgr2=0x7fee74003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:34.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.036+0000 7fee916fb640 1 --2- 172.21.15.148:0/1198644857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee740039e0 0x7fee74003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.036+0000 7fee916fb640 1 -- 172.21.15.148:0/1198644857 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee74003520 con 0x7fee74004800 2024-03-29T04:11:34.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.036+0000 7fee926fd640 1 --2- 172.21.15.148:0/1198644857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee8c0d17e0 0x7fee740051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:34.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.037+0000 7fee916fb640 1 --2- 172.21.15.148:0/1198644857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee74004800 0x7fee74004c80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fee8000c210 tx=0x7fee74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0d0fd940e6b506 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:34.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.037+0000 7fee90efa640 1 -- 172.21.15.148:0/1198644857 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80007cf0 con 0x7fee74004800 2024-03-29T04:11:34.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.037+0000 7fee90efa640 1 -- 172.21.15.148:0/1198644857 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee80030070 con 0x7fee74004800 2024-03-29T04:11:34.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.037+0000 7fee90efa640 1 -- 172.21.15.148:0/1198644857 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80005d30 con 0x7fee74004800 2024-03-29T04:11:34.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.038+0000 7fee94160640 1 -- 172.21.15.148:0/1198644857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee74004800 msgr2=0x7fee74004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:34.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.038+0000 7fee94160640 1 --2- 172.21.15.148:0/1198644857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee74004800 0x7fee74004c80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fee8000c210 tx=0x7fee74005830 comp rx=0 tx=0).stop 2024-03-29T04:11:34.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.038+0000 7fee926fd640 1 -- 172.21.15.148:0/1198644857 reap_dead start 2024-03-29T04:11:34.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.038+0000 7fee94160640 1 -- 172.21.15.148:0/1198644857 shutdown_connections 2024-03-29T04:11:34.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.038+0000 7fee94160640 1 -- 172.21.15.148:0/1198644857 >> 172.21.15.148:0/1198644857 conn(0x7fee8c0b6210 msgr2=0x7fee8c0b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:34.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.038+0000 7fee94160640 1 -- 172.21.15.148:0/1198644857 shutdown_connections 2024-03-29T04:11:34.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.038+0000 7fee94160640 1 -- 172.21.15.148:0/1198644857 wait complete. 2024-03-29T04:11:34.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.039+0000 7fee94160640 1 Processor -- start 2024-03-29T04:11:34.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.039+0000 7fee94160640 1 -- start start 2024-03-29T04:11:34.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.040+0000 7fee94160640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee74004800 0x7fee8c1f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.040+0000 7fee94160640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee8c0cb860 0x7fee8c156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.040+0000 7fee94160640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee8c1557b0 0x7fee8c155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.040+0000 7fee94160640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c0d3cd0 con 0x7fee8c1557b0 2024-03-29T04:11:34.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.040+0000 7fee94160640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c0d34d0 con 0x7fee8c0cb860 2024-03-29T04:11:34.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.040+0000 7fee94160640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fee8c0d3e50 con 0x7fee74004800 2024-03-29T04:11:34.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.040+0000 7fee926fd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee8c1557b0 0x7fee8c155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.040+0000 7fee926fd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee8c1557b0 0x7fee8c155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39476/0 (socket says 172.21.15.148:39476) 2024-03-29T04:11:34.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.040+0000 7fee926fd640 1 -- 172.21.15.148:0/3849214160 learned_addr learned my addr 172.21.15.148:0/3849214160 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:34.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.040+0000 7fee91efc640 1 --2- 172.21.15.148:0/3849214160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee74004800 0x7fee8c1f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.040+0000 7fee916fb640 1 --2- 172.21.15.148:0/3849214160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee8c0cb860 0x7fee8c156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.040+0000 7fee926fd640 1 -- 172.21.15.148:0/3849214160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee74004800 msgr2=0x7fee8c1f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:34.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.040+0000 7fee926fd640 1 --2- 172.21.15.148:0/3849214160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee74004800 0x7fee8c1f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.040+0000 7fee926fd640 1 -- 172.21.15.148:0/3849214160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee8c0cb860 msgr2=0x7fee8c156a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:34.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.040+0000 7fee926fd640 1 --2- 172.21.15.148:0/3849214160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee8c0cb860 0x7fee8c156a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.040+0000 7fee926fd640 1 -- 172.21.15.148:0/3849214160 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8c156170 con 0x7fee8c1557b0 2024-03-29T04:11:34.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.040+0000 7fee916fb640 1 --2- 172.21.15.148:0/3849214160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee8c0cb860 0x7fee8c156a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:34.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.040+0000 7fee926fd640 1 --2- 172.21.15.148:0/3849214160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee8c1557b0 0x7fee8c155c30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fee8c0cb390 tx=0x7fee7c011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:34.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.040+0000 7fee8a7fc640 1 -- 172.21.15.148:0/3849214160 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c003610 con 0x7fee8c1557b0 2024-03-29T04:11:34.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.040+0000 7fee8a7fc640 1 -- 172.21.15.148:0/3849214160 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee7c01b070 con 0x7fee8c1557b0 2024-03-29T04:11:34.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.040+0000 7fee8a7fc640 1 -- 172.21.15.148:0/3849214160 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c003a10 con 0x7fee8c1557b0 2024-03-29T04:11:34.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.040+0000 7fee94160640 1 -- 172.21.15.148:0/3849214160 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee8c0cc070 con 0x7fee8c1557b0 2024-03-29T04:11:34.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.040+0000 7fee94160640 1 -- 172.21.15.148:0/3849214160 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee8c0cc630 con 0x7fee8c1557b0 2024-03-29T04:11:34.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.041+0000 7fee94160640 1 -- 172.21.15.148:0/3849214160 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee8c0687e0 con 0x7fee8c1557b0 2024-03-29T04:11:34.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.042+0000 7fee8a7fc640 1 -- 172.21.15.148:0/3849214160 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fee7c003bb0 con 0x7fee8c1557b0 2024-03-29T04:11:34.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.042+0000 7fee8a7fc640 1 --2- 172.21.15.148:0/3849214160 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fee7806c530 0x7fee7806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.042+0000 7fee8a7fc640 1 -- 172.21.15.148:0/3849214160 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fee7c08b5f0 con 0x7fee8c1557b0 2024-03-29T04:11:34.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.042+0000 7fee91efc640 1 --2- 172.21.15.148:0/3849214160 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fee7806c530 0x7fee7806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.043+0000 7fee91efc640 1 --2- 172.21.15.148:0/3849214160 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fee7806c530 0x7fee7806e9f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fee84007fd0 tx=0x7fee84009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:34.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.046+0000 7fee8a7fc640 1 -- 172.21.15.148:0/3849214160 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fee7c024020 con 0x7fee8c1557b0 2024-03-29T04:11:34.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.222+0000 7fee94160640 1 -- 172.21.15.148:0/3849214160 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fee8c0b77b0 con 0x7fee8c1557b0 2024-03-29T04:11:34.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.223+0000 7fee8a7fc640 1 -- 172.21.15.148:0/3849214160 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16967 (secure 0 0 0) 0x7fee7c059a80 con 0x7fee8c1557b0 2024-03-29T04:11:34.202 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:11:34.202 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":38,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:11:32.950071+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:11:30.794790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T04:11:31.907100+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:11:34.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.225+0000 7fee94160640 1 -- 172.21.15.148:0/3849214160 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fee7806c530 msgr2=0x7fee7806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:34.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.225+0000 7fee94160640 1 --2- 172.21.15.148:0/3849214160 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fee7806c530 0x7fee7806e9f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fee84007fd0 tx=0x7fee84009040 comp rx=0 tx=0).stop 2024-03-29T04:11:34.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.225+0000 7fee94160640 1 -- 172.21.15.148:0/3849214160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee8c1557b0 msgr2=0x7fee8c155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:34.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.225+0000 7fee94160640 1 --2- 172.21.15.148:0/3849214160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee8c1557b0 0x7fee8c155c30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fee8c0cb390 tx=0x7fee7c011920 comp rx=0 tx=0).stop 2024-03-29T04:11:34.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.225+0000 7fee94160640 1 -- 172.21.15.148:0/3849214160 shutdown_connections 2024-03-29T04:11:34.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.225+0000 7fee94160640 1 --2- 172.21.15.148:0/3849214160 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fee7806c530 0x7fee7806e9f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.225+0000 7fee94160640 1 --2- 172.21.15.148:0/3849214160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee8c1557b0 0x7fee8c155c30 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.225+0000 7fee94160640 1 --2- 172.21.15.148:0/3849214160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee8c0cb860 0x7fee8c156a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.225+0000 7fee94160640 1 --2- 172.21.15.148:0/3849214160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee74004800 0x7fee8c1f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.225+0000 7fee94160640 1 -- 172.21.15.148:0/3849214160 >> 172.21.15.148:0/3849214160 conn(0x7fee8c0b6210 msgr2=0x7fee8c0d10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:34.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.225+0000 7fee94160640 1 -- 172.21.15.148:0/3849214160 shutdown_connections 2024-03-29T04:11:34.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.225+0000 7fee94160640 1 -- 172.21.15.148:0/3849214160 wait complete. 2024-03-29T04:11:34.215 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-29T04:11:34.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.339+0000 7fa7555b5640 1 Processor -- start 2024-03-29T04:11:34.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.339+0000 7fa7555b5640 1 -- start start 2024-03-29T04:11:34.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.340+0000 7fa7555b5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa75015e3c0 0x7fa750161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.340+0000 7fa7555b5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa7500b2f00 0x7fa7501624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.340+0000 7fa7555b5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa7500b3430 0x7fa7500d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.340+0000 7fa7555b5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fa7500cf960 con 0x7fa7500b2f00 2024-03-29T04:11:34.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.340+0000 7fa7555b5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa7500cf330 con 0x7fa7500b3430 2024-03-29T04:11:34.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.340+0000 7fa7555b5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa750162cf0 con 0x7fa75015e3c0 2024-03-29T04:11:34.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.340+0000 7fa747fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa7500b2f00 0x7fa7501624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.340+0000 7fa747fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa7500b2f00 0x7fa7501624b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39484/0 (socket says 172.21.15.148:39484) 2024-03-29T04:11:34.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.340+0000 7fa747fff640 1 -- 172.21.15.148:0/979450630 learned_addr learned my addr 172.21.15.148:0/979450630 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:34.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.340+0000 7fa74f7fe640 1 --2- 172.21.15.148:0/979450630 >> v2:172.21.15.157:3300/0 conn(0x7fa7500b3430 0x7fa7500d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.340+0000 7fa74effd640 1 --2- 172.21.15.148:0/979450630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa75015e3c0 0x7fa750161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.340+0000 7fa747fff640 1 -- 172.21.15.148:0/979450630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa75015e3c0 msgr2=0x7fa750161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:34.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.340+0000 7fa747fff640 1 --2- 172.21.15.148:0/979450630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa75015e3c0 0x7fa750161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.340+0000 7fa747fff640 1 -- 172.21.15.148:0/979450630 >> v2:172.21.15.157:3300/0 conn(0x7fa7500b3430 msgr2=0x7fa7500d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:34.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.340+0000 7fa747fff640 1 --2- 172.21.15.148:0/979450630 >> v2:172.21.15.157:3300/0 conn(0x7fa7500b3430 0x7fa7500d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.340+0000 7fa747fff640 1 -- 172.21.15.148:0/979450630 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7500cf5f0 con 0x7fa7500b2f00 2024-03-29T04:11:34.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.341+0000 7fa747fff640 1 --2- 172.21.15.148:0/979450630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7500b2f00 0x7fa7501624b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa7500cf0c0 tx=0x7fa73802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4585bd38087da819 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:34.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.341+0000 7fa74e7fc640 1 -- 172.21.15.148:0/979450630 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa738035020 con 0x7fa7500b2f00 2024-03-29T04:11:34.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.341+0000 7fa74e7fc640 1 -- 172.21.15.148:0/979450630 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa738031870 con 0x7fa7500b2f00 2024-03-29T04:11:34.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.341+0000 7fa74e7fc640 1 -- 172.21.15.148:0/979450630 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa738031c40 con 0x7fa7500b2f00 2024-03-29T04:11:34.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.342+0000 7fa7555b5640 1 -- 172.21.15.148:0/979450630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7500b2f00 msgr2=0x7fa7501624b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:34.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.342+0000 7fa7555b5640 1 --2- 172.21.15.148:0/979450630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7500b2f00 0x7fa7501624b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa7500cf0c0 tx=0x7fa73802b910 comp rx=0 tx=0).stop 2024-03-29T04:11:34.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.342+0000 7fa7555b5640 1 -- 172.21.15.148:0/979450630 shutdown_connections 2024-03-29T04:11:34.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.342+0000 7fa7555b5640 1 --2- 172.21.15.148:0/979450630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7500b2f00 0x7fa7501624b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.342+0000 7fa7555b5640 1 --2- 172.21.15.148:0/979450630 >> v2:172.21.15.157:3300/0 conn(0x7fa7500b3430 0x7fa7500d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.342+0000 7fa7555b5640 1 --2- 172.21.15.148:0/979450630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa75015e3c0 0x7fa750161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.342+0000 7fa7555b5640 1 -- 172.21.15.148:0/979450630 >> 172.21.15.148:0/979450630 conn(0x7fa7500b6430 msgr2=0x7fa7500d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:34.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.342+0000 7fa7555b5640 1 -- 172.21.15.148:0/979450630 shutdown_connections 2024-03-29T04:11:34.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.342+0000 7fa7555b5640 1 -- 172.21.15.148:0/979450630 wait complete. 2024-03-29T04:11:34.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.343+0000 7fa7555b5640 1 Processor -- start 2024-03-29T04:11:34.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.343+0000 7fa7555b5640 1 -- start start 2024-03-29T04:11:34.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.343+0000 7fa7555b5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7500b3430 0x7fa750160c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.343+0000 7fa7555b5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa75015e3c0 0x7fa75015c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.343+0000 7fa7555b5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7500cdb00 0x7fa7501e7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.343+0000 7fa7555b5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa7500b39a0 con 0x7fa75015e3c0 2024-03-29T04:11:34.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.343+0000 7fa7555b5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa75015e7a0 con 0x7fa7500b3430 2024-03-29T04:11:34.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.343+0000 7fa7555b5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa7500b3b20 con 0x7fa7500cdb00 2024-03-29T04:11:34.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.344+0000 7fa747fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa75015e3c0 0x7fa75015c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.344+0000 7fa747fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa75015e3c0 0x7fa75015c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39498/0 (socket says 172.21.15.148:39498) 2024-03-29T04:11:34.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.344+0000 7fa747fff640 1 -- 172.21.15.148:0/1001305276 learned_addr learned my addr 172.21.15.148:0/1001305276 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:34.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.344+0000 7fa747fff640 1 -- 172.21.15.148:0/1001305276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7500cdb00 msgr2=0x7fa7501e7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:34.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.344+0000 7fa74effd640 1 --2- 172.21.15.148:0/1001305276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7500b3430 0x7fa750160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.344+0000 7fa747fff640 1 --2- 172.21.15.148:0/1001305276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7500cdb00 0x7fa7501e7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.344+0000 7fa747fff640 1 -- 172.21.15.148:0/1001305276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7500b3430 msgr2=0x7fa750160c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:34.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.344+0000 7fa747fff640 1 --2- 172.21.15.148:0/1001305276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7500b3430 0x7fa750160c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.344+0000 7fa747fff640 1 -- 172.21.15.148:0/1001305276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa750160470 con 0x7fa75015e3c0 2024-03-29T04:11:34.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.344+0000 7fa747fff640 1 --2- 172.21.15.148:0/1001305276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa75015e3c0 0x7fa75015c880 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa738000c00 tx=0x7fa7380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:34.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.344+0000 7fa7477fe640 1 -- 172.21.15.148:0/1001305276 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa738035040 con 0x7fa75015e3c0 2024-03-29T04:11:34.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.344+0000 7fa7477fe640 1 -- 172.21.15.148:0/1001305276 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa738030c80 con 0x7fa75015e3c0 2024-03-29T04:11:34.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.344+0000 7fa7555b5640 1 -- 172.21.15.148:0/1001305276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7501e89b0 con 0x7fa75015e3c0 2024-03-29T04:11:34.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.344+0000 7fa7477fe640 1 -- 172.21.15.148:0/1001305276 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7380456a0 con 0x7fa75015e3c0 2024-03-29T04:11:34.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.344+0000 7fa7555b5640 1 -- 172.21.15.148:0/1001305276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7501e8f70 con 0x7fa75015e3c0 2024-03-29T04:11:34.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.345+0000 7fa7555b5640 1 -- 172.21.15.148:0/1001305276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa750068850 con 0x7fa75015e3c0 2024-03-29T04:11:34.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.346+0000 7fa7477fe640 1 -- 172.21.15.148:0/1001305276 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa738038020 con 0x7fa75015e3c0 2024-03-29T04:11:34.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.346+0000 7fa7477fe640 1 --2- 172.21.15.148:0/1001305276 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa73006c530 0x7fa73006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.346+0000 7fa74effd640 1 --2- 172.21.15.148:0/1001305276 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa73006c530 0x7fa73006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.346+0000 7fa7477fe640 1 -- 172.21.15.148:0/1001305276 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fa7380ada40 con 0x7fa75015e3c0 2024-03-29T04:11:34.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.347+0000 7fa74effd640 1 --2- 172.21.15.148:0/1001305276 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa73006c530 0x7fa73006e9f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa750161380 tx=0x7fa740008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:34.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.351+0000 7fa7477fe640 1 -- 172.21.15.148:0/1001305276 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa73807bf60 con 0x7fa75015e3c0 2024-03-29T04:11:34.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.561+0000 7fa7555b5640 1 -- 172.21.15.148:0/1001305276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa7500743c0 con 0x7fa75015e3c0 2024-03-29T04:11:34.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.601+0000 7fa7477fe640 1 -- 172.21.15.148:0/1001305276 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7fa7380ade20 con 0x7fa75015e3c0 2024-03-29T04:11:34.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.604+0000 7fa7555b5640 1 -- 172.21.15.148:0/1001305276 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa73006c530 msgr2=0x7fa73006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:34.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.604+0000 7fa7555b5640 1 --2- 172.21.15.148:0/1001305276 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa73006c530 0x7fa73006e9f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa750161380 tx=0x7fa740008040 comp rx=0 tx=0).stop 2024-03-29T04:11:34.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.604+0000 7fa7555b5640 1 -- 172.21.15.148:0/1001305276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa75015e3c0 msgr2=0x7fa75015c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:34.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.604+0000 7fa7555b5640 1 --2- 172.21.15.148:0/1001305276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa75015e3c0 0x7fa75015c880 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa738000c00 tx=0x7fa7380046c0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.604+0000 7fa7555b5640 1 -- 172.21.15.148:0/1001305276 shutdown_connections 2024-03-29T04:11:34.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.604+0000 7fa7555b5640 1 --2- 172.21.15.148:0/1001305276 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa73006c530 0x7fa73006e9f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.604+0000 7fa7555b5640 1 --2- 172.21.15.148:0/1001305276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7500cdb00 0x7fa7501e7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.604+0000 7fa7555b5640 1 --2- 172.21.15.148:0/1001305276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa75015e3c0 0x7fa75015c880 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.604+0000 7fa7555b5640 1 --2- 172.21.15.148:0/1001305276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7500b3430 0x7fa750160c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.604+0000 7fa7555b5640 1 -- 172.21.15.148:0/1001305276 >> 172.21.15.148:0/1001305276 conn(0x7fa7500b6430 msgr2=0x7fa7500d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:34.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.604+0000 7fa7555b5640 1 -- 172.21.15.148:0/1001305276 shutdown_connections 2024-03-29T04:11:34.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.604+0000 7fa7555b5640 1 -- 172.21.15.148:0/1001305276 wait complete. 2024-03-29T04:11:34.653 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-29T04:11:34.653 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-29T04:11:34.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.780+0000 7fcdb6360640 1 Processor -- start 2024-03-29T04:11:34.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.780+0000 7fcdb6360640 1 -- start start 2024-03-29T04:11:34.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.781+0000 7fcdb6360640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdb015e3c0 0x7fcdb00c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.781+0000 7fcdb6360640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdb0162ec0 con 0x7fcdb00cf290 2024-03-29T04:11:34.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.781+0000 7fcdb6360640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdb00ce4d0 con 0x7fcdb0162860 2024-03-29T04:11:34.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.781+0000 7fcdb6360640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcdb00cee00 con 0x7fcdb015e3c0 2024-03-29T04:11:34.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.781+0000 7fcdaf7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fcdb00cf290 0x7fcdb0162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50786/0 (socket says 172.21.15.148:50786) 2024-03-29T04:11:34.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.781+0000 7fcdaf7fe640 1 -- 172.21.15.148:0/715514210 learned_addr learned my addr 172.21.15.148:0/715514210 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:34.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.781+0000 7fcdaeffd640 1 -- 172.21.15.148:0/715514210 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2250312792 0 0) 0x7fcdb0162ec0 con 0x7fcdb00cf290 2024-03-29T04:11:34.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.781+0000 7fcdaeffd640 1 -- 172.21.15.148:0/715514210 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd98003520 con 0x7fcdb00cf290 2024-03-29T04:11:34.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.781+0000 7fcdaffff640 1 --2- 172.21.15.148:0/715514210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdb015e3c0 0x7fcdb00c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.781+0000 7fcdaeffd640 1 -- 172.21.15.148:0/715514210 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1266572405 0 0) 0x7fcd98003520 con 0x7fcdb00cf290 2024-03-29T04:11:34.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.781+0000 7fcdaeffd640 1 -- 172.21.15.148:0/715514210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdb015e3c0 msgr2=0x7fcdb00c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:34.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.781+0000 7fcdaeffd640 1 --2- 172.21.15.148:0/715514210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdb015e3c0 0x7fcdb00c99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.781+0000 7fcdaeffd640 1 -- 172.21.15.148:0/715514210 >> v1:172.21.15.157:6789/0 conn(0x7fcdb0162860 legacy=0x7fcdb00cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:34.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.781+0000 7fcdaeffd640 1 -- 172.21.15.148:0/715514210 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb00ce7b0 con 0x7fcdb00cf290 2024-03-29T04:11:34.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.781+0000 7fcdaeffd640 1 -- 172.21.15.148:0/715514210 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7fcda0001670 con 0x7fcdb00cf290 2024-03-29T04:11:34.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.781+0000 7fcdaeffd640 1 -- 172.21.15.148:0/715514210 >> v1:172.21.15.148:6789/0 conn(0x7fcdb00cf290 legacy=0x7fcdb0162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:34.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.781+0000 7fcdaeffd640 1 --2- 172.21.15.148:0/715514210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd980039e0 0x7fcd98003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.781+0000 7fcdaeffd640 1 --2- 172.21.15.148:0/715514210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdb015e3c0 0x7fcd98004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.782+0000 7fcdaeffd640 1 --2- 172.21.15.148:0/715514210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd98004d40 0x7fcd980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.782+0000 7fcdaeffd640 1 -- 172.21.15.148:0/715514210 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb00cee00 con 0x7fcd980039e0 2024-03-29T04:11:34.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.782+0000 7fcdaeffd640 1 -- 172.21.15.148:0/715514210 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9800ba70 con 0x7fcd98004d40 2024-03-29T04:11:34.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.782+0000 7fcdaeffd640 1 -- 172.21.15.148:0/715514210 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcd9800bc70 con 0x7fcdb015e3c0 2024-03-29T04:11:34.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.782+0000 7fcdaffff640 1 --2- 172.21.15.148:0/715514210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd980039e0 0x7fcd98003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.782+0000 7fcdaf7fe640 1 --2- 172.21.15.148:0/715514210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdb015e3c0 0x7fcd98004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.782+0000 7fcdb48fd640 1 --2- 172.21.15.148:0/715514210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd98004d40 0x7fcd980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.782+0000 7fcdaffff640 1 -- 172.21.15.148:0/715514210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdb015e3c0 msgr2=0x7fcd98004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:34.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.782+0000 7fcdaffff640 1 --2- 172.21.15.148:0/715514210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdb015e3c0 0x7fcd98004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.782+0000 7fcdaffff640 1 -- 172.21.15.148:0/715514210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd98004d40 msgr2=0x7fcd980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:34.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.782+0000 7fcdaffff640 1 --2- 172.21.15.148:0/715514210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd98004d40 0x7fcd980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.782+0000 7fcdaffff640 1 -- 172.21.15.148:0/715514210 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd98003520 con 0x7fcd980039e0 2024-03-29T04:11:34.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.783+0000 7fcdaffff640 1 --2- 172.21.15.148:0/715514210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd980039e0 0x7fcd98003e00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009a90 tx=0x7fcd98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09af60238b1db1a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:34.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.783+0000 7fcdaeffd640 1 -- 172.21.15.148:0/715514210 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4039070 con 0x7fcd980039e0 2024-03-29T04:11:34.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.783+0000 7fcdaeffd640 1 -- 172.21.15.148:0/715514210 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcda402c680 con 0x7fcd980039e0 2024-03-29T04:11:34.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.783+0000 7fcdaeffd640 1 -- 172.21.15.148:0/715514210 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4009690 con 0x7fcd980039e0 2024-03-29T04:11:34.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.783+0000 7fcdb6360640 1 -- 172.21.15.148:0/715514210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd980039e0 msgr2=0x7fcd98003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:34.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.783+0000 7fcdb6360640 1 --2- 172.21.15.148:0/715514210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd980039e0 0x7fcd98003e00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009a90 tx=0x7fcd98005830 comp rx=0 tx=0).stop 2024-03-29T04:11:34.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.783+0000 7fcdb48fd640 1 -- 172.21.15.148:0/715514210 reap_dead start 2024-03-29T04:11:34.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.783+0000 7fcdb6360640 1 -- 172.21.15.148:0/715514210 shutdown_connections 2024-03-29T04:11:34.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.783+0000 7fcdb6360640 1 -- 172.21.15.148:0/715514210 >> 172.21.15.148:0/715514210 conn(0x7fcdb00cd160 msgr2=0x7fcdb00cd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:34.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.783+0000 7fcdb6360640 1 -- 172.21.15.148:0/715514210 shutdown_connections 2024-03-29T04:11:34.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.783+0000 7fcdb6360640 1 -- 172.21.15.148:0/715514210 wait complete. 2024-03-29T04:11:34.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.784+0000 7fcdb6360640 1 Processor -- start 2024-03-29T04:11:34.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.785+0000 7fcdb6360640 1 -- start start 2024-03-29T04:11:34.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.785+0000 7fcdb6360640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd980039e0 0x7fcdb0152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.785+0000 7fcdb6360640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdb015af60 0x7fcdb015b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.785+0000 7fcdb6360640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb01f7720 0x7fcdb01f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.785+0000 7fcdb6360640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb015e920 con 0x7fcdb01f7720 2024-03-29T04:11:34.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.785+0000 7fcdb6360640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb015e7a0 con 0x7fcd980039e0 2024-03-29T04:11:34.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.785+0000 7fcdb6360640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcdb00cc760 con 0x7fcdb015af60 2024-03-29T04:11:34.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.786+0000 7fcdb48fd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb01f7720 0x7fcdb01f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.786+0000 7fcdb48fd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb01f7720 0x7fcdb01f7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39518/0 (socket says 172.21.15.148:39518) 2024-03-29T04:11:34.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.786+0000 7fcdb48fd640 1 -- 172.21.15.148:0/2919086285 learned_addr learned my addr 172.21.15.148:0/2919086285 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:34.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.786+0000 7fcdaf7fe640 1 --2- 172.21.15.148:0/2919086285 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdb015af60 0x7fcdb015b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.786+0000 7fcdaffff640 1 --2- 172.21.15.148:0/2919086285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd980039e0 0x7fcdb0152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.786+0000 7fcdb48fd640 1 -- 172.21.15.148:0/2919086285 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdb015af60 msgr2=0x7fcdb015b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:34.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.786+0000 7fcdb48fd640 1 --2- 172.21.15.148:0/2919086285 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdb015af60 0x7fcdb015b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.786+0000 7fcdb48fd640 1 -- 172.21.15.148:0/2919086285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd980039e0 msgr2=0x7fcdb0152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:34.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.786+0000 7fcdb48fd640 1 --2- 172.21.15.148:0/2919086285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd980039e0 0x7fcdb0152220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.786+0000 7fcdb48fd640 1 -- 172.21.15.148:0/2919086285 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb0155fe0 con 0x7fcdb01f7720 2024-03-29T04:11:34.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.786+0000 7fcdaffff640 1 --2- 172.21.15.148:0/2919086285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd980039e0 0x7fcdb0152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:34.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.786+0000 7fcdb48fd640 1 --2- 172.21.15.148:0/2919086285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb01f7720 0x7fcdb01f7ba0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0162c40 tx=0x7fcda800fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:34.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.786+0000 7fcdacff9640 1 -- 172.21.15.148:0/2919086285 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda801a070 con 0x7fcdb01f7720 2024-03-29T04:11:34.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.786+0000 7fcdacff9640 1 -- 172.21.15.148:0/2919086285 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcda8006d60 con 0x7fcdb01f7720 2024-03-29T04:11:34.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.786+0000 7fcdb6360640 1 -- 172.21.15.148:0/2919086285 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdb0152760 con 0x7fcdb01f7720 2024-03-29T04:11:34.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.786+0000 7fcdacff9640 1 -- 172.21.15.148:0/2919086285 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda8003b90 con 0x7fcdb01f7720 2024-03-29T04:11:34.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.786+0000 7fcdb6360640 1 -- 172.21.15.148:0/2919086285 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdb01f5390 con 0x7fcdb01f7720 2024-03-29T04:11:34.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.787+0000 7fcdb6360640 1 -- 172.21.15.148:0/2919086285 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdb0068850 con 0x7fcdb01f7720 2024-03-29T04:11:34.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.788+0000 7fcdacff9640 1 -- 172.21.15.148:0/2919086285 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcda801d070 con 0x7fcdb01f7720 2024-03-29T04:11:34.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.788+0000 7fcdacff9640 1 --2- 172.21.15.148:0/2919086285 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fcd9c06c530 0x7fcd9c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.788+0000 7fcdaffff640 1 --2- 172.21.15.148:0/2919086285 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fcd9c06c530 0x7fcd9c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.788+0000 7fcdacff9640 1 -- 172.21.15.148:0/2919086285 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fcda808ad50 con 0x7fcdb01f7720 2024-03-29T04:11:34.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.789+0000 7fcdaffff640 1 --2- 172.21.15.148:0/2919086285 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fcd9c06c530 0x7fcd9c06e9f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcda4036000 tx=0x7fcda4030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:34.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:34.792+0000 7fcdacff9640 1 -- 172.21.15.148:0/2919086285 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcda808e030 con 0x7fcdb01f7720 2024-03-29T04:11:34.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.012+0000 7fcdb6360640 1 -- 172.21.15.148:0/2919086285 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fcdb0067c90 con 0x7fcdb01f7720 2024-03-29T04:11:35.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.610+0000 7fcdacff9640 1 -- 172.21.15.148:0/2919086285 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7fcdb0155fe0 con 0x7fcdb01f7720 2024-03-29T04:11:35.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.612+0000 7fcdb6360640 1 -- 172.21.15.148:0/2919086285 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fcd9c06c530 msgr2=0x7fcd9c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:35.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.612+0000 7fcdb6360640 1 --2- 172.21.15.148:0/2919086285 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fcd9c06c530 0x7fcd9c06e9f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcda4036000 tx=0x7fcda4030000 comp rx=0 tx=0).stop 2024-03-29T04:11:35.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.612+0000 7fcdb6360640 1 -- 172.21.15.148:0/2919086285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb01f7720 msgr2=0x7fcdb01f7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:35.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.612+0000 7fcdb6360640 1 --2- 172.21.15.148:0/2919086285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb01f7720 0x7fcdb01f7ba0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0162c40 tx=0x7fcda800fd40 comp rx=0 tx=0).stop 2024-03-29T04:11:35.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.612+0000 7fcdb6360640 1 -- 172.21.15.148:0/2919086285 shutdown_connections 2024-03-29T04:11:35.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.612+0000 7fcdb6360640 1 --2- 172.21.15.148:0/2919086285 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fcd9c06c530 0x7fcd9c06e9f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.612+0000 7fcdb6360640 1 --2- 172.21.15.148:0/2919086285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb01f7720 0x7fcdb01f7ba0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.612+0000 7fcdb6360640 1 --2- 172.21.15.148:0/2919086285 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcdb015af60 0x7fcdb015b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.612+0000 7fcdb6360640 1 --2- 172.21.15.148:0/2919086285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd980039e0 0x7fcdb0152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.612+0000 7fcdb6360640 1 -- 172.21.15.148:0/2919086285 >> 172.21.15.148:0/2919086285 conn(0x7fcdb00cd160 msgr2=0x7fcdb006c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:35.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.612+0000 7fcdb6360640 1 -- 172.21.15.148:0/2919086285 shutdown_connections 2024-03-29T04:11:35.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.612+0000 7fcdb6360640 1 -- 172.21.15.148:0/2919086285 wait complete. 2024-03-29T04:11:35.602 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-29T04:11:35.602 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:11:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.725+0000 7effa03d4640 1 Processor -- start 2024-03-29T04:11:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.725+0000 7effa03d4640 1 -- start start 2024-03-29T04:11:35.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.726+0000 7effa03d4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff980b05c0 0x7eff9806e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:35.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.726+0000 7effa03d4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7eff980b5e90 0x7eff9806ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:35.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.726+0000 7effa03d4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff9806b930 con 0x7eff9815e380 2024-03-29T04:11:35.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.726+0000 7effa03d4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7eff9806bc10 con 0x7eff980b5e90 2024-03-29T04:11:35.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.726+0000 7effa03d4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7eff9806bef0 con 0x7eff980b05c0 2024-03-29T04:11:35.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.726+0000 7eff9d96f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff980b05c0 0x7eff9806e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:35.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.727+0000 7eff9e971640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7eff980b5e90 0x7eff9806ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:35.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.727+0000 7eff9d96f640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff980b05c0 0x7eff9806e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.148:54728/0 (socket says 172.21.15.148:54728) 2024-03-29T04:11:35.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.727+0000 7eff9e971640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7eff980b5e90 0x7eff9806ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:36060/0 (socket says 172.21.15.148:36060) 2024-03-29T04:11:35.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.727+0000 7eff9d96f640 1 -- 172.21.15.148:0/4175250576 learned_addr learned my addr 172.21.15.148:0/4175250576 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:35.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.727+0000 7eff9d96f640 1 -- 172.21.15.148:0/4175250576 >> v2:172.21.15.157:3300/0 conn(0x7eff980b5e90 msgr2=0x7eff9806ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:35.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.727+0000 7eff9d96f640 1 --2- 172.21.15.148:0/4175250576 >> v2:172.21.15.157:3300/0 conn(0x7eff980b5e90 0x7eff9806ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.727+0000 7eff9d96f640 1 -- 172.21.15.148:0/4175250576 >> v1:172.21.15.157:6789/0 conn(0x7eff9815e380 legacy=0x7eff980afeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:35.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.727+0000 7eff9d96f640 1 -- 172.21.15.148:0/4175250576 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff9806c1d0 con 0x7eff980b05c0 2024-03-29T04:11:35.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.727+0000 7eff9d16e640 1 -- 172.21.15.148:0/4175250576 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 636972570 0 0) 0x7eff9806b930 con 0x7eff9815e380 2024-03-29T04:11:35.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.727+0000 7eff9e971640 1 --2- 172.21.15.148:0/4175250576 >> v2:172.21.15.157:3300/0 conn(0x7eff980b5e90 0x7eff9806ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:11:35.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.728+0000 7eff9d96f640 1 --2- 172.21.15.148:0/4175250576 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff980b05c0 0x7eff9806e6d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7eff94009870 tx=0x7eff9402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ab712e2a79764bd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:35.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.728+0000 7eff9d16e640 1 -- 172.21.15.148:0/4175250576 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff94035020 con 0x7eff980b05c0 2024-03-29T04:11:35.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.728+0000 7eff9d16e640 1 -- 172.21.15.148:0/4175250576 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff94031640 con 0x7eff980b05c0 2024-03-29T04:11:35.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.728+0000 7eff9d16e640 1 -- 172.21.15.148:0/4175250576 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9402d040 con 0x7eff980b05c0 2024-03-29T04:11:35.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.728+0000 7effa03d4640 1 -- 172.21.15.148:0/4175250576 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff980b05c0 msgr2=0x7eff9806e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:35.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.728+0000 7effa03d4640 1 --2- 172.21.15.148:0/4175250576 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff980b05c0 0x7eff9806e6d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7eff94009870 tx=0x7eff9402b8f0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.729+0000 7effa03d4640 1 -- 172.21.15.148:0/4175250576 shutdown_connections 2024-03-29T04:11:35.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.729+0000 7effa03d4640 1 --2- 172.21.15.148:0/4175250576 >> v2:172.21.15.157:3300/0 conn(0x7eff980b5e90 0x7eff9806ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.729+0000 7effa03d4640 1 --2- 172.21.15.148:0/4175250576 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff980b05c0 0x7eff9806e6d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.729+0000 7effa03d4640 1 -- 172.21.15.148:0/4175250576 >> 172.21.15.148:0/4175250576 conn(0x7eff980b2c60 msgr2=0x7eff980b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:35.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.729+0000 7effa03d4640 1 -- 172.21.15.148:0/4175250576 shutdown_connections 2024-03-29T04:11:35.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.729+0000 7effa03d4640 1 -- 172.21.15.148:0/4175250576 wait complete. 2024-03-29T04:11:35.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.730+0000 7effa03d4640 1 Processor -- start 2024-03-29T04:11:35.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.730+0000 7effa03d4640 1 -- start start 2024-03-29T04:11:35.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.730+0000 7effa03d4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff980b05c0 0x7eff981456a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:35.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.730+0000 7effa03d4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff980b5e90 0x7eff98158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:35.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.730+0000 7effa03d4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff9815e380 0x7eff98144e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:35.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.730+0000 7effa03d4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7eff980b4fa0 con 0x7eff980b5e90 2024-03-29T04:11:35.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.730+0000 7effa03d4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7eff9806b5b0 con 0x7eff9815e380 2024-03-29T04:11:35.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.730+0000 7effa03d4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7eff980b5120 con 0x7eff980b05c0 2024-03-29T04:11:35.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.731+0000 7eff9d96f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff980b5e90 0x7eff98158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:35.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.731+0000 7eff9d96f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff980b5e90 0x7eff98158bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42252/0 (socket says 172.21.15.148:42252) 2024-03-29T04:11:35.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.731+0000 7eff9d96f640 1 -- 172.21.15.148:0/1505507975 learned_addr learned my addr 172.21.15.148:0/1505507975 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:35.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.731+0000 7eff9e971640 1 --2- 172.21.15.148:0/1505507975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff9815e380 0x7eff98144e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:35.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.731+0000 7eff9e170640 1 --2- 172.21.15.148:0/1505507975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff980b05c0 0x7eff981456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:35.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.731+0000 7eff9d96f640 1 -- 172.21.15.148:0/1505507975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff980b05c0 msgr2=0x7eff981456a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:35.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.731+0000 7eff9d96f640 1 --2- 172.21.15.148:0/1505507975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff980b05c0 0x7eff981456a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.731+0000 7eff9d96f640 1 -- 172.21.15.148:0/1505507975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff9815e380 msgr2=0x7eff98144e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:35.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.731+0000 7eff9d96f640 1 --2- 172.21.15.148:0/1505507975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff9815e380 0x7eff98144e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.731+0000 7eff9d96f640 1 -- 172.21.15.148:0/1505507975 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff981590f0 con 0x7eff980b5e90 2024-03-29T04:11:35.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.731+0000 7eff9e170640 1 --2- 172.21.15.148:0/1505507975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff980b05c0 0x7eff981456a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:35.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.731+0000 7eff9d96f640 1 --2- 172.21.15.148:0/1505507975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff980b5e90 0x7eff98158bb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7eff940090b0 tx=0x7eff940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:35.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.731+0000 7eff8effd640 1 -- 172.21.15.148:0/1505507975 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff94040070 con 0x7eff980b5e90 2024-03-29T04:11:35.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.731+0000 7eff8effd640 1 -- 172.21.15.148:0/1505507975 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff940399f0 con 0x7eff980b5e90 2024-03-29T04:11:35.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.731+0000 7effa03d4640 1 -- 172.21.15.148:0/1505507975 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff98145cc0 con 0x7eff980b5e90 2024-03-29T04:11:35.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.731+0000 7eff8effd640 1 -- 172.21.15.148:0/1505507975 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff94035040 con 0x7eff980b5e90 2024-03-29T04:11:35.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.731+0000 7effa03d4640 1 -- 172.21.15.148:0/1505507975 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff98146280 con 0x7eff980b5e90 2024-03-29T04:11:35.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.733+0000 7effa03d4640 1 -- 172.21.15.148:0/1505507975 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff980687e0 con 0x7eff980b5e90 2024-03-29T04:11:35.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.733+0000 7eff8effd640 1 -- 172.21.15.148:0/1505507975 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7eff94038020 con 0x7eff980b5e90 2024-03-29T04:11:35.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.733+0000 7eff8effd640 1 --2- 172.21.15.148:0/1505507975 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7eff7c06c530 0x7eff7c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:35.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.733+0000 7eff9e170640 1 --2- 172.21.15.148:0/1505507975 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7eff7c06c530 0x7eff7c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:35.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.733+0000 7eff8effd640 1 -- 172.21.15.148:0/1505507975 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7eff940ad1c0 con 0x7eff980b5e90 2024-03-29T04:11:35.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.734+0000 7eff9e170640 1 --2- 172.21.15.148:0/1505507975 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7eff7c06c530 0x7eff7c06e9f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7eff88000f60 tx=0x7eff88009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:35.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.738+0000 7eff8effd640 1 -- 172.21.15.148:0/1505507975 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7eff9407b5c0 con 0x7eff980b5e90 2024-03-29T04:11:35.941 INFO:tasks.ceph.mds.a.smithi148.stderr:2024-03-29T04:11:35.963+0000 7fe2025a2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T04:11:35.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.965+0000 7effa03d4640 1 -- 172.21.15.148:0/1505507975 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff980674d0 con 0x7eff980b5e90 2024-03-29T04:11:35.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.966+0000 7eff8effd640 1 -- 172.21.15.148:0/1505507975 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5190 (secure 0 0 0) 0x7eff9407aec0 con 0x7eff980b5e90 2024-03-29T04:11:35.945 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:11:35.945 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2618617956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2618617956},{"type":"v1","addr":"172.21.15.157:6834","nonce":2618617956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5260,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1177855619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1177855619},{"type":"v1","addr":"172.21.15.157:6835","nonce":1177855619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5562,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3610463670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3610463670},{"type":"v1","addr":"172.21.15.148:6835","nonce":3610463670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:11:32.950391+0000","modified":"2024-03-29T04:11:35.013562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5583},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5583":{"gid":5583,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/473448013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":473448013},{"type":"v1","addr":"172.21.15.148:6837","nonce":473448013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5583,"qdb_cluster":[5583]},"id":2}]} 2024-03-29T04:11:35.945 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 17 2024-03-29T04:11:35.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.968+0000 7effa03d4640 1 -- 172.21.15.148:0/1505507975 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7eff7c06c530 msgr2=0x7eff7c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:35.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.968+0000 7effa03d4640 1 --2- 172.21.15.148:0/1505507975 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7eff7c06c530 0x7eff7c06e9f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7eff88000f60 tx=0x7eff88009040 comp rx=0 tx=0).stop 2024-03-29T04:11:35.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.968+0000 7effa03d4640 1 -- 172.21.15.148:0/1505507975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff980b5e90 msgr2=0x7eff98158bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:35.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.968+0000 7effa03d4640 1 --2- 172.21.15.148:0/1505507975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff980b5e90 0x7eff98158bb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7eff940090b0 tx=0x7eff940046c0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.968+0000 7effa03d4640 1 -- 172.21.15.148:0/1505507975 shutdown_connections 2024-03-29T04:11:35.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.968+0000 7effa03d4640 1 --2- 172.21.15.148:0/1505507975 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7eff7c06c530 0x7eff7c06e9f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.968+0000 7effa03d4640 1 --2- 172.21.15.148:0/1505507975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff9815e380 0x7eff98144e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.968+0000 7effa03d4640 1 --2- 172.21.15.148:0/1505507975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff980b5e90 0x7eff98158bb0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.968+0000 7effa03d4640 1 --2- 172.21.15.148:0/1505507975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff980b05c0 0x7eff981456a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.968+0000 7effa03d4640 1 -- 172.21.15.148:0/1505507975 >> 172.21.15.148:0/1505507975 conn(0x7eff980b2c60 msgr2=0x7eff980ad8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:35.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.968+0000 7effa03d4640 1 -- 172.21.15.148:0/1505507975 shutdown_connections 2024-03-29T04:11:35.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:35.968+0000 7effa03d4640 1 -- 172.21.15.148:0/1505507975 wait complete. 2024-03-29T04:11:35.958 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:11:36.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.081+0000 7fa49713d640 1 Processor -- start 2024-03-29T04:11:36.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.081+0000 7fa49713d640 1 -- start start 2024-03-29T04:11:36.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.082+0000 7fa49713d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa49015e3e0 0x7fa4900d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:36.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.082+0000 7fa49713d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa4900ba030 0x7fa4900d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:36.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.082+0000 7fa49713d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa49015e7c0 con 0x7fa4900d10d0 2024-03-29T04:11:36.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.082+0000 7fa49713d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fa4900d3840 con 0x7fa49015e3e0 2024-03-29T04:11:36.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.082+0000 7fa49713d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa4900d2c30 con 0x7fa4900ba030 2024-03-29T04:11:36.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.082+0000 7fa494ed9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa49015e3e0 0x7fa4900d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:36.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.082+0000 7fa4956da640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa4900d10d0 0x7fa4900b72d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40628/0 (socket says 172.21.15.148:40628) 2024-03-29T04:11:36.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.082+0000 7fa4956da640 1 -- 172.21.15.148:0/3935286860 learned_addr learned my addr 172.21.15.148:0/3935286860 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:36.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.083+0000 7fa48ffff640 1 --2- 172.21.15.148:0/3935286860 >> v2:172.21.15.157:3300/0 conn(0x7fa4900ba030 0x7fa4900d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:36.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.083+0000 7fa494ed9640 1 -- 172.21.15.148:0/3935286860 >> v2:172.21.15.157:3300/0 conn(0x7fa4900ba030 msgr2=0x7fa4900d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:36.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.083+0000 7fa494ed9640 1 --2- 172.21.15.148:0/3935286860 >> v2:172.21.15.157:3300/0 conn(0x7fa4900ba030 0x7fa4900d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.083+0000 7fa494ed9640 1 -- 172.21.15.148:0/3935286860 >> v1:172.21.15.148:6789/0 conn(0x7fa4900d10d0 legacy=0x7fa4900b72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:36.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.083+0000 7fa494ed9640 1 -- 172.21.15.148:0/3935286860 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4900d2f10 con 0x7fa49015e3e0 2024-03-29T04:11:36.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.083+0000 7fa48f7fe640 1 -- 172.21.15.148:0/3935286860 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 20665136 0 0) 0x7fa49015e7c0 con 0x7fa4900d10d0 2024-03-29T04:11:36.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.083+0000 7fa494ed9640 1 --2- 172.21.15.148:0/3935286860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa49015e3e0 0x7fa4900d3cd0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa4900d05d0 tx=0x7fa48002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c3bf85979701fd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:36.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.084+0000 7fa48f7fe640 1 -- 172.21.15.148:0/3935286860 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480034070 con 0x7fa49015e3e0 2024-03-29T04:11:36.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.084+0000 7fa48f7fe640 1 -- 172.21.15.148:0/3935286860 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa480030610 con 0x7fa49015e3e0 2024-03-29T04:11:36.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.084+0000 7fa48f7fe640 1 -- 172.21.15.148:0/3935286860 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4800309e0 con 0x7fa49015e3e0 2024-03-29T04:11:36.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.084+0000 7fa49713d640 1 -- 172.21.15.148:0/3935286860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa49015e3e0 msgr2=0x7fa4900d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:36.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.084+0000 7fa49713d640 1 --2- 172.21.15.148:0/3935286860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa49015e3e0 0x7fa4900d3cd0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa4900d05d0 tx=0x7fa48002b960 comp rx=0 tx=0).stop 2024-03-29T04:11:36.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.084+0000 7fa49713d640 1 -- 172.21.15.148:0/3935286860 shutdown_connections 2024-03-29T04:11:36.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.084+0000 7fa49713d640 1 --2- 172.21.15.148:0/3935286860 >> v2:172.21.15.157:3300/0 conn(0x7fa4900ba030 0x7fa4900d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.084+0000 7fa49713d640 1 --2- 172.21.15.148:0/3935286860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa49015e3e0 0x7fa4900d3cd0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.084+0000 7fa49713d640 1 -- 172.21.15.148:0/3935286860 >> 172.21.15.148:0/3935286860 conn(0x7fa4900b6210 msgr2=0x7fa4900b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:36.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.084+0000 7fa49713d640 1 -- 172.21.15.148:0/3935286860 shutdown_connections 2024-03-29T04:11:36.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.084+0000 7fa49713d640 1 -- 172.21.15.148:0/3935286860 wait complete. 2024-03-29T04:11:36.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.085+0000 7fa49713d640 1 Processor -- start 2024-03-29T04:11:36.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.085+0000 7fa49713d640 1 -- start start 2024-03-29T04:11:36.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.086+0000 7fa49713d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4900ba030 0x7fa4901f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:36.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.086+0000 7fa49713d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4900d10d0 0x7fa490159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:36.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.086+0000 7fa49713d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa49015e3e0 0x7fa4901f5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:36.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.086+0000 7fa49713d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa4900d2870 con 0x7fa4900ba030 2024-03-29T04:11:36.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.086+0000 7fa49713d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa4900d34d0 con 0x7fa49015e3e0 2024-03-29T04:11:36.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.086+0000 7fa49713d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa4900d29f0 con 0x7fa4900d10d0 2024-03-29T04:11:36.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.086+0000 7fa494ed9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4900ba030 0x7fa4901f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:36.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.086+0000 7fa494ed9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4900ba030 0x7fa4901f5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42274/0 (socket says 172.21.15.148:42274) 2024-03-29T04:11:36.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.086+0000 7fa494ed9640 1 -- 172.21.15.148:0/3201776502 learned_addr learned my addr 172.21.15.148:0/3201776502 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:36.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.087+0000 7fa4956da640 1 --2- 172.21.15.148:0/3201776502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa49015e3e0 0x7fa4901f5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:36.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.087+0000 7fa494ed9640 1 -- 172.21.15.148:0/3201776502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4900d10d0 msgr2=0x7fa490159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:36.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.087+0000 7fa48ffff640 1 --2- 172.21.15.148:0/3201776502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4900d10d0 0x7fa490159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:36.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.087+0000 7fa494ed9640 1 --2- 172.21.15.148:0/3201776502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4900d10d0 0x7fa490159180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.087+0000 7fa494ed9640 1 -- 172.21.15.148:0/3201776502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa49015e3e0 msgr2=0x7fa4901f5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:36.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.087+0000 7fa494ed9640 1 --2- 172.21.15.148:0/3201776502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa49015e3e0 0x7fa4901f5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.087+0000 7fa494ed9640 1 -- 172.21.15.148:0/3201776502 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4900cb480 con 0x7fa4900ba030 2024-03-29T04:11:36.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.087+0000 7fa494ed9640 1 --2- 172.21.15.148:0/3201776502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4900ba030 0x7fa4901f5cb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa480000c00 tx=0x7fa4800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:36.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.087+0000 7fa48d7fa640 1 -- 172.21.15.148:0/3201776502 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480034040 con 0x7fa4900ba030 2024-03-29T04:11:36.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.087+0000 7fa48d7fa640 1 -- 172.21.15.148:0/3201776502 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa48003aaa0 con 0x7fa4900ba030 2024-03-29T04:11:36.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.087+0000 7fa49713d640 1 -- 172.21.15.148:0/3201776502 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4900cb720 con 0x7fa4900ba030 2024-03-29T04:11:36.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.087+0000 7fa48d7fa640 1 -- 172.21.15.148:0/3201776502 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4800443f0 con 0x7fa4900ba030 2024-03-29T04:11:36.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.087+0000 7fa49713d640 1 -- 172.21.15.148:0/3201776502 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4901f5950 con 0x7fa4900ba030 2024-03-29T04:11:36.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.088+0000 7fa49713d640 1 -- 172.21.15.148:0/3201776502 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4900687e0 con 0x7fa4900ba030 2024-03-29T04:11:36.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.089+0000 7fa48d7fa640 1 -- 172.21.15.148:0/3201776502 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa48004b060 con 0x7fa4900ba030 2024-03-29T04:11:36.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.089+0000 7fa48d7fa640 1 --2- 172.21.15.148:0/3201776502 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa47c06c560 0x7fa47c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:36.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.089+0000 7fa48d7fa640 1 -- 172.21.15.148:0/3201776502 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fa4800ad630 con 0x7fa4900ba030 2024-03-29T04:11:36.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.089+0000 7fa48ffff640 1 --2- 172.21.15.148:0/3201776502 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa47c06c560 0x7fa47c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:36.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.089+0000 7fa48ffff640 1 --2- 172.21.15.148:0/3201776502 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa47c06c560 0x7fa47c06ea20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa4900c9a60 tx=0x7fa484008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:36.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.093+0000 7fa48d7fa640 1 -- 172.21.15.148:0/3201776502 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa48003f070 con 0x7fa4900ba030 2024-03-29T04:11:36.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.276+0000 7fa49713d640 1 -- 172.21.15.148:0/3201776502 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4900b93c0 con 0x7fa4900ba030 2024-03-29T04:11:36.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.277+0000 7fa48d7fa640 1 -- 172.21.15.148:0/3201776502 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+16954 (secure 0 0 0) 0x7fa48004b350 con 0x7fa4900ba030 2024-03-29T04:11:36.258 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:11:36.258 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":39,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:11:34.596103+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:11:30.794790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T04:11:31.907100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:11:36.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.279+0000 7fa49713d640 1 -- 172.21.15.148:0/3201776502 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa47c06c560 msgr2=0x7fa47c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:36.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.279+0000 7fa49713d640 1 --2- 172.21.15.148:0/3201776502 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa47c06c560 0x7fa47c06ea20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa4900c9a60 tx=0x7fa484008040 comp rx=0 tx=0).stop 2024-03-29T04:11:36.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.279+0000 7fa49713d640 1 -- 172.21.15.148:0/3201776502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4900ba030 msgr2=0x7fa4901f5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:36.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.279+0000 7fa49713d640 1 --2- 172.21.15.148:0/3201776502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4900ba030 0x7fa4901f5cb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa480000c00 tx=0x7fa4800046c0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.280+0000 7fa49713d640 1 -- 172.21.15.148:0/3201776502 shutdown_connections 2024-03-29T04:11:36.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.280+0000 7fa49713d640 1 --2- 172.21.15.148:0/3201776502 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa47c06c560 0x7fa47c06ea20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.280+0000 7fa49713d640 1 --2- 172.21.15.148:0/3201776502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa49015e3e0 0x7fa4901f5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.280+0000 7fa49713d640 1 --2- 172.21.15.148:0/3201776502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4900d10d0 0x7fa490159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.280+0000 7fa49713d640 1 --2- 172.21.15.148:0/3201776502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4900ba030 0x7fa4901f5cb0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.280+0000 7fa49713d640 1 -- 172.21.15.148:0/3201776502 >> 172.21.15.148:0/3201776502 conn(0x7fa4900b6210 msgr2=0x7fa4900cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:36.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.280+0000 7fa49713d640 1 -- 172.21.15.148:0/3201776502 shutdown_connections 2024-03-29T04:11:36.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.280+0000 7fa49713d640 1 -- 172.21.15.148:0/3201776502 wait complete. 2024-03-29T04:11:36.270 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-29T04:11:36.270 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:11:36.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.393+0000 7f62f3bba640 1 Processor -- start 2024-03-29T04:11:36.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.393+0000 7f62f3bba640 1 -- start start 2024-03-29T04:11:36.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.394+0000 7f62f3bba640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62ec15e3e0 0x7f62ec0d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:36.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.394+0000 7f62f3bba640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f62ec0ba030 0x7f62ec0d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:36.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.394+0000 7f62f3bba640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62ec15e7c0 con 0x7f62ec0d10d0 2024-03-29T04:11:36.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.394+0000 7f62f3bba640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f62ec0d3840 con 0x7f62ec0ba030 2024-03-29T04:11:36.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.394+0000 7f62f3bba640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f62ec0d2c30 con 0x7f62ec15e3e0 2024-03-29T04:11:36.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.394+0000 7f62f1956640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62ec15e3e0 0x7f62ec0d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:36.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.394+0000 7f62f1155640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f62ec0ba030 0x7f62ec0d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:36.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.394+0000 7f62f1155640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f62ec0ba030 0x7f62ec0d2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:36092/0 (socket says 172.21.15.148:36092) 2024-03-29T04:11:36.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.394+0000 7f62f1155640 1 -- 172.21.15.148:0/4140531799 learned_addr learned my addr 172.21.15.148:0/4140531799 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:36.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.395+0000 7f62f1155640 1 -- 172.21.15.148:0/4140531799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62ec15e3e0 msgr2=0x7f62ec0d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:36.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.395+0000 7f62f1155640 1 --2- 172.21.15.148:0/4140531799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62ec15e3e0 0x7f62ec0d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.395+0000 7f62f1155640 1 -- 172.21.15.148:0/4140531799 >> v1:172.21.15.157:6789/0 conn(0x7f62ec0d10d0 legacy=0x7f62ec0b72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:36.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.395+0000 7f62f1155640 1 -- 172.21.15.148:0/4140531799 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ec0d2f10 con 0x7f62ec0ba030 2024-03-29T04:11:36.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.395+0000 7f62f0954640 1 -- 172.21.15.148:0/4140531799 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 669099489 0 0) 0x7f62ec15e7c0 con 0x7f62ec0d10d0 2024-03-29T04:11:36.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.396+0000 7f62f1155640 1 --2- 172.21.15.148:0/4140531799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62ec0ba030 0x7f62ec0d2330 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009870 tx=0x7f62dc02ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c82b536f883b8a8b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:36.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.396+0000 7f62f0954640 1 -- 172.21.15.148:0/4140531799 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc02f840 con 0x7f62ec0ba030 2024-03-29T04:11:36.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.396+0000 7f62f0954640 1 -- 172.21.15.148:0/4140531799 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62dc02f9e0 con 0x7f62ec0ba030 2024-03-29T04:11:36.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.396+0000 7f62f0954640 1 -- 172.21.15.148:0/4140531799 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc02fdb0 con 0x7f62ec0ba030 2024-03-29T04:11:36.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.397+0000 7f62f3bba640 1 -- 172.21.15.148:0/4140531799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62ec0ba030 msgr2=0x7f62ec0d2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:36.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.397+0000 7f62f3bba640 1 --2- 172.21.15.148:0/4140531799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62ec0ba030 0x7f62ec0d2330 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009870 tx=0x7f62dc02ba20 comp rx=0 tx=0).stop 2024-03-29T04:11:36.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.397+0000 7f62f3bba640 1 -- 172.21.15.148:0/4140531799 shutdown_connections 2024-03-29T04:11:36.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.397+0000 7f62f3bba640 1 --2- 172.21.15.148:0/4140531799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62ec0ba030 0x7f62ec0d2330 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.397+0000 7f62f3bba640 1 --2- 172.21.15.148:0/4140531799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62ec15e3e0 0x7f62ec0d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.397+0000 7f62f3bba640 1 -- 172.21.15.148:0/4140531799 >> 172.21.15.148:0/4140531799 conn(0x7f62ec0b6210 msgr2=0x7f62ec0b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:36.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.397+0000 7f62f3bba640 1 -- 172.21.15.148:0/4140531799 shutdown_connections 2024-03-29T04:11:36.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.397+0000 7f62f3bba640 1 -- 172.21.15.148:0/4140531799 wait complete. 2024-03-29T04:11:36.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.398+0000 7f62f3bba640 1 Processor -- start 2024-03-29T04:11:36.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.398+0000 7f62f3bba640 1 -- start start 2024-03-29T04:11:36.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.399+0000 7f62f3bba640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62ec0ba030 0x7f62ec15cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:36.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.399+0000 7f62f3bba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62ec0d10d0 0x7f62ec161140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:36.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.399+0000 7f62f3bba640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62ec15e3e0 0x7f62ec15ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:36.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.399+0000 7f62f3bba640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec0d2870 con 0x7f62ec0ba030 2024-03-29T04:11:36.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.399+0000 7f62f3bba640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec0d34d0 con 0x7f62ec0d10d0 2024-03-29T04:11:36.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.399+0000 7f62f3bba640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f62ec0d29f0 con 0x7f62ec15e3e0 2024-03-29T04:11:36.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.399+0000 7f62f1956640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62ec0ba030 0x7f62ec15cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:36.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.399+0000 7f62f1956640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62ec0ba030 0x7f62ec15cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42280/0 (socket says 172.21.15.148:42280) 2024-03-29T04:11:36.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.399+0000 7f62f1956640 1 -- 172.21.15.148:0/1665072043 learned_addr learned my addr 172.21.15.148:0/1665072043 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:36.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.399+0000 7f62f1155640 1 --2- 172.21.15.148:0/1665072043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62ec0d10d0 0x7f62ec161140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:36.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.399+0000 7f62f2157640 1 --2- 172.21.15.148:0/1665072043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62ec15e3e0 0x7f62ec15ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:36.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.399+0000 7f62f1956640 1 -- 172.21.15.148:0/1665072043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62ec15e3e0 msgr2=0x7f62ec15ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:36.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.399+0000 7f62f1956640 1 --2- 172.21.15.148:0/1665072043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62ec15e3e0 0x7f62ec15ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.399+0000 7f62f1956640 1 -- 172.21.15.148:0/1665072043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62ec0d10d0 msgr2=0x7f62ec161140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:36.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.399+0000 7f62f1956640 1 --2- 172.21.15.148:0/1665072043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62ec0d10d0 0x7f62ec161140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.399+0000 7f62f1956640 1 -- 172.21.15.148:0/1665072043 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ec15bcd0 con 0x7f62ec0ba030 2024-03-29T04:11:36.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.399+0000 7f62f1956640 1 --2- 172.21.15.148:0/1665072043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62ec0ba030 0x7f62ec15cd70 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f62e400efc0 tx=0x7f62e400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:36.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.399+0000 7f62e27fc640 1 -- 172.21.15.148:0/1665072043 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4018020 con 0x7f62ec0ba030 2024-03-29T04:11:36.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.399+0000 7f62e27fc640 1 -- 172.21.15.148:0/1665072043 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e4014480 con 0x7f62ec0ba030 2024-03-29T04:11:36.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.399+0000 7f62f3bba640 1 -- 172.21.15.148:0/1665072043 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62ec15bfb0 con 0x7f62ec0ba030 2024-03-29T04:11:36.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.400+0000 7f62e27fc640 1 -- 172.21.15.148:0/1665072043 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4014850 con 0x7f62ec0ba030 2024-03-29T04:11:36.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.400+0000 7f62f3bba640 1 -- 172.21.15.148:0/1665072043 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62ec15c570 con 0x7f62ec0ba030 2024-03-29T04:11:36.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.400+0000 7f62f3bba640 1 -- 172.21.15.148:0/1665072043 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62ec0687e0 con 0x7f62ec0ba030 2024-03-29T04:11:36.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.401+0000 7f62e27fc640 1 -- 172.21.15.148:0/1665072043 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f62e4005d10 con 0x7f62ec0ba030 2024-03-29T04:11:36.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.401+0000 7f62e27fc640 1 --2- 172.21.15.148:0/1665072043 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f62d006c530 0x7f62d006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:36.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.401+0000 7f62e27fc640 1 -- 172.21.15.148:0/1665072043 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f62e408c5b0 con 0x7f62ec0ba030 2024-03-29T04:11:36.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.402+0000 7f62f1155640 1 --2- 172.21.15.148:0/1665072043 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f62d006c530 0x7f62d006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:36.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.402+0000 7f62f1155640 1 --2- 172.21.15.148:0/1665072043 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f62d006c530 0x7f62d006e9f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f62dc036000 tx=0x7f62dc008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:36.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.406+0000 7f62e27fc640 1 -- 172.21.15.148:0/1665072043 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f62e4022030 con 0x7f62ec0ba030 2024-03-29T04:11:36.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.585+0000 7f62f3bba640 1 -- 172.21.15.148:0/1665072043 --> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f62ec0cb180 con 0x7f62d006c530 2024-03-29T04:11:36.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.590+0000 7f62e27fc640 1 -- 172.21.15.148:0/1665072043 <== mgr.4113 v2:172.21.15.148:6832/3271445075 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240027 (secure 0 0 0) 0x7f62ec0cb180 con 0x7f62d006c530 2024-03-29T04:11:36.572 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:11:36.574 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-03-29T04:11:36.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.593+0000 7f62f3bba640 1 -- 172.21.15.148:0/1665072043 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f62d006c530 msgr2=0x7f62d006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:36.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.593+0000 7f62f3bba640 1 --2- 172.21.15.148:0/1665072043 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f62d006c530 0x7f62d006e9f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f62dc036000 tx=0x7f62dc008030 comp rx=0 tx=0).stop 2024-03-29T04:11:36.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.593+0000 7f62f3bba640 1 -- 172.21.15.148:0/1665072043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62ec0ba030 msgr2=0x7f62ec15cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:36.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.593+0000 7f62f3bba640 1 --2- 172.21.15.148:0/1665072043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62ec0ba030 0x7f62ec15cd70 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f62e400efc0 tx=0x7f62e400c490 comp rx=0 tx=0).stop 2024-03-29T04:11:36.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.593+0000 7f62f3bba640 1 -- 172.21.15.148:0/1665072043 shutdown_connections 2024-03-29T04:11:36.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.593+0000 7f62f3bba640 1 --2- 172.21.15.148:0/1665072043 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f62d006c530 0x7f62d006e9f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.593+0000 7f62f3bba640 1 --2- 172.21.15.148:0/1665072043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62ec15e3e0 0x7f62ec15ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.593+0000 7f62f3bba640 1 --2- 172.21.15.148:0/1665072043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62ec0d10d0 0x7f62ec161140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.593+0000 7f62f3bba640 1 --2- 172.21.15.148:0/1665072043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62ec0ba030 0x7f62ec15cd70 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.593+0000 7f62f3bba640 1 -- 172.21.15.148:0/1665072043 >> 172.21.15.148:0/1665072043 conn(0x7f62ec0b6210 msgr2=0x7f62ec0b77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:36.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.594+0000 7f62f3bba640 1 -- 172.21.15.148:0/1665072043 shutdown_connections 2024-03-29T04:11:36.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.594+0000 7f62f3bba640 1 -- 172.21.15.148:0/1665072043 wait complete. 2024-03-29T04:11:36.584 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-03-29T04:11:35.900868+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237132,"kb_used_data":4320,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737588,"statfs":{"total":773094113280,"available":772851290112,"internally_reserved":0,"allocated":4423680,"data_stored":2545090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.319933"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.583191+0000","last_change":"2024-03-29T04:11:31.583191+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.583191+0000","last_undegraded":"2024-03-29T04:11:31.583191+0000","last_fullsized":"2024-03-29T04:11:31.583191+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.588394+0000","last_change":"2024-03-29T04:11:31.588394+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.588394+0000","last_undegraded":"2024-03-29T04:11:31.588394+0000","last_fullsized":"2024-03-29T04:11:31.588394+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.591020+0000","last_change":"2024-03-29T04:11:31.591020+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.591020+0000","last_undegraded":"2024-03-29T04:11:31.591020+0000","last_fullsized":"2024-03-29T04:11:31.591020+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.583097+0000","last_change":"2024-03-29T04:11:31.583097+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.583097+0000","last_undegraded":"2024-03-29T04:11:31.583097+0000","last_fullsized":"2024-03-29T04:11:31.583097+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.585232+0000","last_change":"2024-03-29T04:11:31.585232+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.585232+0000","last_undegraded":"2024-03-29T04:11:31.585232+0000","last_fullsized":"2024-03-29T04:11:31.585232+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.587179+0000","last_change":"2024-03-29T04:11:31.587179+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.587179+0000","last_undegraded":"2024-03-29T04:11:31.587179+0000","last_fullsized":"2024-03-29T04:11:31.587179+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.591029+0000","last_change":"2024-03-29T04:11:31.591029+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.591029+0000","last_undegraded":"2024-03-29T04:11:31.591029+0000","last_fullsized":"2024-03-29T04:11:31.591029+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.583148+0000","last_change":"2024-03-29T04:11:31.583148+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.583148+0000","last_undegraded":"2024-03-29T04:11:31.583148+0000","last_fullsized":"2024-03-29T04:11:31.583148+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.586125+0000","last_change":"2024-03-29T04:11:31.586125+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.586125+0000","last_undegraded":"2024-03-29T04:11:31.586125+0000","last_fullsized":"2024-03-29T04:11:31.586125+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.585369+0000","last_change":"2024-03-29T04:11:31.585369+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.585369+0000","last_undegraded":"2024-03-29T04:11:31.585369+0000","last_fullsized":"2024-03-29T04:11:31.585369+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.585174+0000","last_change":"2024-03-29T04:11:31.585174+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.585174+0000","last_undegraded":"2024-03-29T04:11:31.585174+0000","last_fullsized":"2024-03-29T04:11:31.585174+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.585788+0000","last_change":"2024-03-29T04:11:31.585788+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.585788+0000","last_undegraded":"2024-03-29T04:11:31.585788+0000","last_fullsized":"2024-03-29T04:11:31.585788+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.585730+0000","last_change":"2024-03-29T04:11:31.585730+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.585730+0000","last_undegraded":"2024-03-29T04:11:31.585730+0000","last_fullsized":"2024-03-29T04:11:31.585730+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.591061+0000","last_change":"2024-03-29T04:11:31.591061+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.591061+0000","last_undegraded":"2024-03-29T04:11:31.591061+0000","last_fullsized":"2024-03-29T04:11:31.591061+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.583091+0000","last_change":"2024-03-29T04:11:31.583091+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.583091+0000","last_undegraded":"2024-03-29T04:11:31.583091+0000","last_fullsized":"2024-03-29T04:11:31.583091+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.587179+0000","last_change":"2024-03-29T04:11:31.587179+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.587179+0000","last_undegraded":"2024-03-29T04:11:31.587179+0000","last_fullsized":"2024-03-29T04:11:31.587179+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.585723+0000","last_change":"2024-03-29T04:11:31.585723+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.585723+0000","last_undegraded":"2024-03-29T04:11:31.585723+0000","last_fullsized":"2024-03-29T04:11:31.585723+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.583097+0000","last_change":"2024-03-29T04:11:31.583097+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.583097+0000","last_undegraded":"2024-03-29T04:11:31.583097+0000","last_fullsized":"2024-03-29T04:11:31.583097+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.584993+0000","last_change":"2024-03-29T04:11:31.584993+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.584993+0000","last_undegraded":"2024-03-29T04:11:31.584993+0000","last_fullsized":"2024-03-29T04:11:31.584993+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.591301+0000","last_change":"2024-03-29T04:11:31.591301+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.591301+0000","last_undegraded":"2024-03-29T04:11:31.591301+0000","last_fullsized":"2024-03-29T04:11:31.591301+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.584820+0000","last_change":"2024-03-29T04:11:31.584820+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.584820+0000","last_undegraded":"2024-03-29T04:11:31.584820+0000","last_fullsized":"2024-03-29T04:11:31.584820+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.584192+0000","last_change":"2024-03-29T04:11:31.584192+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.584192+0000","last_undegraded":"2024-03-29T04:11:31.584192+0000","last_fullsized":"2024-03-29T04:11:31.584192+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.587161+0000","last_change":"2024-03-29T04:11:31.587161+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.587161+0000","last_undegraded":"2024-03-29T04:11:31.587161+0000","last_fullsized":"2024-03-29T04:11:31.587161+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.584192+0000","last_change":"2024-03-29T04:11:31.584192+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.584192+0000","last_undegraded":"2024-03-29T04:11:31.584192+0000","last_fullsized":"2024-03-29T04:11:31.584192+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.585260+0000","last_change":"2024-03-29T04:11:31.585260+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.585260+0000","last_undegraded":"2024-03-29T04:11:31.585260+0000","last_fullsized":"2024-03-29T04:11:31.585260+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.584195+0000","last_change":"2024-03-29T04:11:31.584195+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.584195+0000","last_undegraded":"2024-03-29T04:11:31.584195+0000","last_fullsized":"2024-03-29T04:11:31.584195+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.588220+0000","last_change":"2024-03-29T04:11:31.588220+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.588220+0000","last_undegraded":"2024-03-29T04:11:31.588220+0000","last_fullsized":"2024-03-29T04:11:31.588220+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.588370+0000","last_change":"2024-03-29T04:11:31.588370+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.588370+0000","last_undegraded":"2024-03-29T04:11:31.588370+0000","last_fullsized":"2024-03-29T04:11:31.588370+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.583807+0000","last_change":"2024-03-29T04:11:31.583807+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.583807+0000","last_undegraded":"2024-03-29T04:11:31.583807+0000","last_fullsized":"2024-03-29T04:11:31.583807+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.583135+0000","last_change":"2024-03-29T04:11:31.583135+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.583135+0000","last_undegraded":"2024-03-29T04:11:31.583135+0000","last_fullsized":"2024-03-29T04:11:31.583135+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.585704+0000","last_change":"2024-03-29T04:11:31.585704+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.585704+0000","last_undegraded":"2024-03-29T04:11:31.585704+0000","last_fullsized":"2024-03-29T04:11:31.585704+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.587152+0000","last_change":"2024-03-29T04:11:31.587152+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.587152+0000","last_undegraded":"2024-03-29T04:11:31.587152+0000","last_fullsized":"2024-03-29T04:11:31.587152+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":201,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-29T04:11:31.587612+0000","last_change":"2024-03-29T04:09:49.002570+0000","last_active":"2024-03-29T04:11:31.587612+0000","last_peered":"2024-03-29T04:11:31.587612+0000","last_clean":"2024-03-29T04:11:31.587612+0000","last_became_active":"2024-03-29T04:09:49.002235+0000","last_became_peered":"2024-03-29T04:09:49.002235+0000","last_unstale":"2024-03-29T04:11:31.587612+0000","last_undegraded":"2024-03-29T04:11:31.587612+0000","last_fullsized":"2024-03-29T04:11:31.587612+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:47.900733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:47.900733+0000","last_clean_scrub_stamp":"2024-03-29T04:09:47.900733+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:44:04.608265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":759360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":401408,"data_stored":168992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":401408,"data_stored":168992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":175201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":168992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":168992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":168992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":765569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:11:36.588 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:11:36.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.747+0000 7f5f281d4640 1 Processor -- start 2024-03-29T04:11:36.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.747+0000 7f5f281d4640 1 -- start start 2024-03-29T04:11:36.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.748+0000 7f5f281d4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5f20070e20 0x7f5f20071200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:36.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.748+0000 7f5f281d4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f200d0af0 con 0x7f5f200b7150 2024-03-29T04:11:36.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.748+0000 7f5f281d4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f20071740 con 0x7f5f200b2ff0 2024-03-29T04:11:36.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.748+0000 7f5f281d4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5f200b50a0 con 0x7f5f20070e20 2024-03-29T04:11:36.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.748+0000 7f5f26771640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5f200b7150 0x7f5f200d03e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40642/0 (socket says 172.21.15.148:40642) 2024-03-29T04:11:36.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.748+0000 7f5f26771640 1 -- 172.21.15.148:0/245561581 learned_addr learned my addr 172.21.15.148:0/245561581 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:36.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.748+0000 7f5f24f6e640 1 -- 172.21.15.148:0/245561581 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3040629618 0 0) 0x7f5f200d0af0 con 0x7f5f200b7150 2024-03-29T04:11:36.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.748+0000 7f5f24f6e640 1 -- 172.21.15.148:0/245561581 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f0c003520 con 0x7f5f200b7150 2024-03-29T04:11:36.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.748+0000 7f5f25f70640 1 --2- 172.21.15.148:0/245561581 >> v2:172.21.15.157:3300/0 conn(0x7f5f20070e20 0x7f5f20071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:36.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.748+0000 7f5f24f6e640 1 -- 172.21.15.148:0/245561581 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3300918969 0 0) 0x7f5f0c003520 con 0x7f5f200b7150 2024-03-29T04:11:36.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.748+0000 7f5f24f6e640 1 -- 172.21.15.148:0/245561581 >> v2:172.21.15.157:3300/0 conn(0x7f5f20070e20 msgr2=0x7f5f20071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:36.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.748+0000 7f5f24f6e640 1 --2- 172.21.15.148:0/245561581 >> v2:172.21.15.157:3300/0 conn(0x7f5f20070e20 0x7f5f20071200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.748+0000 7f5f24f6e640 1 -- 172.21.15.148:0/245561581 >> v1:172.21.15.157:6789/0 conn(0x7f5f200b2ff0 legacy=0x7f5f200cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:36.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.748+0000 7f5f24f6e640 1 -- 172.21.15.148:0/245561581 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f200b5380 con 0x7f5f200b7150 2024-03-29T04:11:36.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.748+0000 7f5f24f6e640 1 -- 172.21.15.148:0/245561581 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f5f180019f0 con 0x7f5f200b7150 2024-03-29T04:11:36.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.748+0000 7f5f24f6e640 1 -- 172.21.15.148:0/245561581 >> v1:172.21.15.148:6789/0 conn(0x7f5f200b7150 legacy=0x7f5f200d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:36.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.748+0000 7f5f24f6e640 1 --2- 172.21.15.148:0/245561581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f0c0039b0 0x7f5f0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:36.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.748+0000 7f5f24f6e640 1 --2- 172.21.15.148:0/245561581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f0c004800 0x7f5f0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:36.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.748+0000 7f5f24f6e640 1 --2- 172.21.15.148:0/245561581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f0c0051c0 0x7f5f0c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:36.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.748+0000 7f5f24f6e640 1 -- 172.21.15.148:0/245561581 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5f200b50a0 con 0x7f5f0c0051c0 2024-03-29T04:11:36.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.748+0000 7f5f24f6e640 1 -- 172.21.15.148:0/245561581 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0c00c160 con 0x7f5f0c004800 2024-03-29T04:11:36.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.748+0000 7f5f24f6e640 1 -- 172.21.15.148:0/245561581 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5f0c00c360 con 0x7f5f0c0039b0 2024-03-29T04:11:36.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.749+0000 7f5f26771640 1 --2- 172.21.15.148:0/245561581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f0c0051c0 0x7f5f0c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:36.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.749+0000 7f5f25f70640 1 --2- 172.21.15.148:0/245561581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f0c0039b0 0x7f5f0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:36.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.749+0000 7f5f2576f640 1 --2- 172.21.15.148:0/245561581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f0c004800 0x7f5f0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:36.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.749+0000 7f5f26771640 1 -- 172.21.15.148:0/245561581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f0c0039b0 msgr2=0x7f5f0c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:36.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.749+0000 7f5f26771640 1 --2- 172.21.15.148:0/245561581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f0c0039b0 0x7f5f0c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.749+0000 7f5f26771640 1 -- 172.21.15.148:0/245561581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f0c004800 msgr2=0x7f5f0c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:36.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.749+0000 7f5f26771640 1 --2- 172.21.15.148:0/245561581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f0c004800 0x7f5f0c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.749+0000 7f5f26771640 1 -- 172.21.15.148:0/245561581 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f0c003520 con 0x7f5f0c0051c0 2024-03-29T04:11:36.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.749+0000 7f5f26771640 1 -- 172.21.15.148:0/245561581 reap_dead start 2024-03-29T04:11:36.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.750+0000 7f5f26771640 1 --2- 172.21.15.148:0/245561581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f0c0051c0 0x7f5f0c00b9c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800a850 tx=0x7f5f0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b0f9064e9641baa server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:36.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.750+0000 7f5f24f6e640 1 -- 172.21.15.148:0/245561581 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1800a4a0 con 0x7f5f0c0051c0 2024-03-29T04:11:36.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.750+0000 7f5f24f6e640 1 -- 172.21.15.148:0/245561581 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f18034650 con 0x7f5f0c0051c0 2024-03-29T04:11:36.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.750+0000 7f5f24f6e640 1 -- 172.21.15.148:0/245561581 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f18034a40 con 0x7f5f0c0051c0 2024-03-29T04:11:36.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.750+0000 7f5f281d4640 1 -- 172.21.15.148:0/245561581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f0c0051c0 msgr2=0x7f5f0c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:36.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.750+0000 7f5f281d4640 1 --2- 172.21.15.148:0/245561581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f0c0051c0 0x7f5f0c00b9c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800a850 tx=0x7f5f0c005830 comp rx=0 tx=0).stop 2024-03-29T04:11:36.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.750+0000 7f5f281d4640 1 -- 172.21.15.148:0/245561581 shutdown_connections 2024-03-29T04:11:36.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.750+0000 7f5f281d4640 1 --2- 172.21.15.148:0/245561581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f0c0051c0 0x7f5f0c00b9c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.750+0000 7f5f281d4640 1 -- 172.21.15.148:0/245561581 >> 172.21.15.148:0/245561581 conn(0x7f5f200d2100 msgr2=0x7f5f200d2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:36.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.750+0000 7f5f281d4640 1 -- 172.21.15.148:0/245561581 shutdown_connections 2024-03-29T04:11:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.750+0000 7f5f281d4640 1 -- 172.21.15.148:0/245561581 wait complete. 2024-03-29T04:11:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.751+0000 7f5f281d4640 1 Processor -- start 2024-03-29T04:11:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.751+0000 7f5f281d4640 1 -- start start 2024-03-29T04:11:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.752+0000 7f5f281d4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f0c0051c0 0x7f5f201f13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.752+0000 7f5f281d4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f201f4f10 0x7f5f20153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.752+0000 7f5f281d4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f20154310 0x7f5f200cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.752+0000 7f5f281d4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5f200b5c20 con 0x7f5f0c0051c0 2024-03-29T04:11:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.752+0000 7f5f281d4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5f200b5940 con 0x7f5f20154310 2024-03-29T04:11:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.752+0000 7f5f281d4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5f200ce560 con 0x7f5f201f4f10 2024-03-29T04:11:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.752+0000 7f5f25f70640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f0c0051c0 0x7f5f201f13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.752+0000 7f5f25f70640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f0c0051c0 0x7f5f201f13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42302/0 (socket says 172.21.15.148:42302) 2024-03-29T04:11:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.752+0000 7f5f25f70640 1 -- 172.21.15.148:0/2092021727 learned_addr learned my addr 172.21.15.148:0/2092021727 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.752+0000 7f5f26771640 1 --2- 172.21.15.148:0/2092021727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f20154310 0x7f5f200cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.752+0000 7f5f2576f640 1 --2- 172.21.15.148:0/2092021727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f201f4f10 0x7f5f20153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.753+0000 7f5f25f70640 1 -- 172.21.15.148:0/2092021727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f201f4f10 msgr2=0x7f5f20153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:36.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.753+0000 7f5f25f70640 1 --2- 172.21.15.148:0/2092021727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f201f4f10 0x7f5f20153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.753+0000 7f5f25f70640 1 -- 172.21.15.148:0/2092021727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f20154310 msgr2=0x7f5f200cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:36.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.753+0000 7f5f25f70640 1 --2- 172.21.15.148:0/2092021727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f20154310 0x7f5f200cfc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.753+0000 7f5f25f70640 1 -- 172.21.15.148:0/2092021727 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f201f3cd0 con 0x7f5f0c0051c0 2024-03-29T04:11:36.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.753+0000 7f5f25f70640 1 --2- 172.21.15.148:0/2092021727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f0c0051c0 0x7f5f201f13d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c011910 tx=0x7f5f1c011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:36.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.753+0000 7f5f167fc640 1 -- 172.21.15.148:0/2092021727 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c00f9d0 con 0x7f5f0c0051c0 2024-03-29T04:11:36.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.753+0000 7f5f167fc640 1 -- 172.21.15.148:0/2092021727 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f1c00fb70 con 0x7f5f0c0051c0 2024-03-29T04:11:36.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.753+0000 7f5f281d4640 1 -- 172.21.15.148:0/2092021727 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f2006caa0 con 0x7f5f0c0051c0 2024-03-29T04:11:36.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.753+0000 7f5f167fc640 1 -- 172.21.15.148:0/2092021727 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c014440 con 0x7f5f0c0051c0 2024-03-29T04:11:36.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.753+0000 7f5f281d4640 1 -- 172.21.15.148:0/2092021727 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f2006d060 con 0x7f5f0c0051c0 2024-03-29T04:11:36.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.754+0000 7f5f281d4640 1 -- 172.21.15.148:0/2092021727 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f200687e0 con 0x7f5f0c0051c0 2024-03-29T04:11:36.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.754+0000 7f5f167fc640 1 -- 172.21.15.148:0/2092021727 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5f1c012040 con 0x7f5f0c0051c0 2024-03-29T04:11:36.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.755+0000 7f5f167fc640 1 --2- 172.21.15.148:0/2092021727 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5f0406c530 0x7f5f0406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:36.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.755+0000 7f5f167fc640 1 -- 172.21.15.148:0/2092021727 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f5f1c08bdd0 con 0x7f5f0c0051c0 2024-03-29T04:11:36.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.755+0000 7f5f2576f640 1 --2- 172.21.15.148:0/2092021727 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5f0406c530 0x7f5f0406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:36.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.755+0000 7f5f2576f640 1 --2- 172.21.15.148:0/2092021727 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5f0406c530 0x7f5f0406e9f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5f10000c00 tx=0x7f5f10009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:36.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.759+0000 7f5f167fc640 1 -- 172.21.15.148:0/2092021727 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5f1c019030 con 0x7f5f0c0051c0 2024-03-29T04:11:36.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.945+0000 7f5f281d4640 1 -- 172.21.15.148:0/2092021727 --> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5f20074a20 con 0x7f5f0406c530 2024-03-29T04:11:36.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.956+0000 7f5f167fc640 1 -- 172.21.15.148:0/2092021727 <== mgr.4113 v2:172.21.15.148:6832/3271445075 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240027 (secure 0 0 0) 0x7f5f20074a20 con 0x7f5f0406c530 2024-03-29T04:11:36.937 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:11:36.938 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-03-29T04:11:36.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.959+0000 7f5f281d4640 1 -- 172.21.15.148:0/2092021727 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5f0406c530 msgr2=0x7f5f0406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:36.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.959+0000 7f5f281d4640 1 --2- 172.21.15.148:0/2092021727 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5f0406c530 0x7f5f0406e9f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5f10000c00 tx=0x7f5f10009040 comp rx=0 tx=0).stop 2024-03-29T04:11:36.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.959+0000 7f5f281d4640 1 -- 172.21.15.148:0/2092021727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f0c0051c0 msgr2=0x7f5f201f13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:36.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.959+0000 7f5f281d4640 1 --2- 172.21.15.148:0/2092021727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f0c0051c0 0x7f5f201f13d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c011910 tx=0x7f5f1c011dd0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.959+0000 7f5f281d4640 1 -- 172.21.15.148:0/2092021727 shutdown_connections 2024-03-29T04:11:36.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.959+0000 7f5f281d4640 1 --2- 172.21.15.148:0/2092021727 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5f0406c530 0x7f5f0406e9f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.959+0000 7f5f281d4640 1 --2- 172.21.15.148:0/2092021727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f20154310 0x7f5f200cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.959+0000 7f5f281d4640 1 --2- 172.21.15.148:0/2092021727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f201f4f10 0x7f5f20153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.959+0000 7f5f281d4640 1 --2- 172.21.15.148:0/2092021727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f0c0051c0 0x7f5f201f13d0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.959+0000 7f5f281d4640 1 -- 172.21.15.148:0/2092021727 >> 172.21.15.148:0/2092021727 conn(0x7f5f200d2100 msgr2=0x7f5f2015b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:36.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.959+0000 7f5f281d4640 1 -- 172.21.15.148:0/2092021727 shutdown_connections 2024-03-29T04:11:36.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:36.959+0000 7f5f281d4640 1 -- 172.21.15.148:0/2092021727 wait complete. 2024-03-29T04:11:36.949 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-03-29T04:11:35.900868+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237132,"kb_used_data":4320,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737588,"statfs":{"total":773094113280,"available":772851290112,"internally_reserved":0,"allocated":4423680,"data_stored":2545090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.319933"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.583191+0000","last_change":"2024-03-29T04:11:31.583191+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.583191+0000","last_undegraded":"2024-03-29T04:11:31.583191+0000","last_fullsized":"2024-03-29T04:11:31.583191+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.588394+0000","last_change":"2024-03-29T04:11:31.588394+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.588394+0000","last_undegraded":"2024-03-29T04:11:31.588394+0000","last_fullsized":"2024-03-29T04:11:31.588394+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.591020+0000","last_change":"2024-03-29T04:11:31.591020+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.591020+0000","last_undegraded":"2024-03-29T04:11:31.591020+0000","last_fullsized":"2024-03-29T04:11:31.591020+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.583097+0000","last_change":"2024-03-29T04:11:31.583097+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.583097+0000","last_undegraded":"2024-03-29T04:11:31.583097+0000","last_fullsized":"2024-03-29T04:11:31.583097+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.585232+0000","last_change":"2024-03-29T04:11:31.585232+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.585232+0000","last_undegraded":"2024-03-29T04:11:31.585232+0000","last_fullsized":"2024-03-29T04:11:31.585232+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.587179+0000","last_change":"2024-03-29T04:11:31.587179+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.587179+0000","last_undegraded":"2024-03-29T04:11:31.587179+0000","last_fullsized":"2024-03-29T04:11:31.587179+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.591029+0000","last_change":"2024-03-29T04:11:31.591029+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.591029+0000","last_undegraded":"2024-03-29T04:11:31.591029+0000","last_fullsized":"2024-03-29T04:11:31.591029+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.583148+0000","last_change":"2024-03-29T04:11:31.583148+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.583148+0000","last_undegraded":"2024-03-29T04:11:31.583148+0000","last_fullsized":"2024-03-29T04:11:31.583148+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.586125+0000","last_change":"2024-03-29T04:11:31.586125+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.586125+0000","last_undegraded":"2024-03-29T04:11:31.586125+0000","last_fullsized":"2024-03-29T04:11:31.586125+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.585369+0000","last_change":"2024-03-29T04:11:31.585369+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.585369+0000","last_undegraded":"2024-03-29T04:11:31.585369+0000","last_fullsized":"2024-03-29T04:11:31.585369+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.585174+0000","last_change":"2024-03-29T04:11:31.585174+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.585174+0000","last_undegraded":"2024-03-29T04:11:31.585174+0000","last_fullsized":"2024-03-29T04:11:31.585174+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.585788+0000","last_change":"2024-03-29T04:11:31.585788+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.585788+0000","last_undegraded":"2024-03-29T04:11:31.585788+0000","last_fullsized":"2024-03-29T04:11:31.585788+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.585730+0000","last_change":"2024-03-29T04:11:31.585730+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.585730+0000","last_undegraded":"2024-03-29T04:11:31.585730+0000","last_fullsized":"2024-03-29T04:11:31.585730+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.591061+0000","last_change":"2024-03-29T04:11:31.591061+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.591061+0000","last_undegraded":"2024-03-29T04:11:31.591061+0000","last_fullsized":"2024-03-29T04:11:31.591061+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.583091+0000","last_change":"2024-03-29T04:11:31.583091+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.583091+0000","last_undegraded":"2024-03-29T04:11:31.583091+0000","last_fullsized":"2024-03-29T04:11:31.583091+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.587179+0000","last_change":"2024-03-29T04:11:31.587179+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.587179+0000","last_undegraded":"2024-03-29T04:11:31.587179+0000","last_fullsized":"2024-03-29T04:11:31.587179+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.585723+0000","last_change":"2024-03-29T04:11:31.585723+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.585723+0000","last_undegraded":"2024-03-29T04:11:31.585723+0000","last_fullsized":"2024-03-29T04:11:31.585723+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.583097+0000","last_change":"2024-03-29T04:11:31.583097+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.583097+0000","last_undegraded":"2024-03-29T04:11:31.583097+0000","last_fullsized":"2024-03-29T04:11:31.583097+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.584993+0000","last_change":"2024-03-29T04:11:31.584993+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.584993+0000","last_undegraded":"2024-03-29T04:11:31.584993+0000","last_fullsized":"2024-03-29T04:11:31.584993+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.591301+0000","last_change":"2024-03-29T04:11:31.591301+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.591301+0000","last_undegraded":"2024-03-29T04:11:31.591301+0000","last_fullsized":"2024-03-29T04:11:31.591301+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.584820+0000","last_change":"2024-03-29T04:11:31.584820+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.584820+0000","last_undegraded":"2024-03-29T04:11:31.584820+0000","last_fullsized":"2024-03-29T04:11:31.584820+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.584192+0000","last_change":"2024-03-29T04:11:31.584192+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.584192+0000","last_undegraded":"2024-03-29T04:11:31.584192+0000","last_fullsized":"2024-03-29T04:11:31.584192+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.587161+0000","last_change":"2024-03-29T04:11:31.587161+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.587161+0000","last_undegraded":"2024-03-29T04:11:31.587161+0000","last_fullsized":"2024-03-29T04:11:31.587161+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.584192+0000","last_change":"2024-03-29T04:11:31.584192+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.584192+0000","last_undegraded":"2024-03-29T04:11:31.584192+0000","last_fullsized":"2024-03-29T04:11:31.584192+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.585260+0000","last_change":"2024-03-29T04:11:31.585260+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.585260+0000","last_undegraded":"2024-03-29T04:11:31.585260+0000","last_fullsized":"2024-03-29T04:11:31.585260+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.584195+0000","last_change":"2024-03-29T04:11:31.584195+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.584195+0000","last_undegraded":"2024-03-29T04:11:31.584195+0000","last_fullsized":"2024-03-29T04:11:31.584195+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.588220+0000","last_change":"2024-03-29T04:11:31.588220+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.588220+0000","last_undegraded":"2024-03-29T04:11:31.588220+0000","last_fullsized":"2024-03-29T04:11:31.588220+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.588370+0000","last_change":"2024-03-29T04:11:31.588370+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.588370+0000","last_undegraded":"2024-03-29T04:11:31.588370+0000","last_fullsized":"2024-03-29T04:11:31.588370+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.583807+0000","last_change":"2024-03-29T04:11:31.583807+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.583807+0000","last_undegraded":"2024-03-29T04:11:31.583807+0000","last_fullsized":"2024-03-29T04:11:31.583807+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.583135+0000","last_change":"2024-03-29T04:11:31.583135+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.583135+0000","last_undegraded":"2024-03-29T04:11:31.583135+0000","last_fullsized":"2024-03-29T04:11:31.583135+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.585704+0000","last_change":"2024-03-29T04:11:31.585704+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.585704+0000","last_undegraded":"2024-03-29T04:11:31.585704+0000","last_fullsized":"2024-03-29T04:11:31.585704+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.587152+0000","last_change":"2024-03-29T04:11:31.587152+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.587152+0000","last_undegraded":"2024-03-29T04:11:31.587152+0000","last_fullsized":"2024-03-29T04:11:31.587152+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":201,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-29T04:11:31.587612+0000","last_change":"2024-03-29T04:09:49.002570+0000","last_active":"2024-03-29T04:11:31.587612+0000","last_peered":"2024-03-29T04:11:31.587612+0000","last_clean":"2024-03-29T04:11:31.587612+0000","last_became_active":"2024-03-29T04:09:49.002235+0000","last_became_peered":"2024-03-29T04:09:49.002235+0000","last_unstale":"2024-03-29T04:11:31.587612+0000","last_undegraded":"2024-03-29T04:11:31.587612+0000","last_fullsized":"2024-03-29T04:11:31.587612+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:47.900733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:47.900733+0000","last_clean_scrub_stamp":"2024-03-29T04:09:47.900733+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:44:04.608265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":759360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":401408,"data_stored":168992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":401408,"data_stored":168992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":175201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":168992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":168992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":168992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":765569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:11:36.953 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:11:37.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.110+0000 7f5ad76a5640 1 Processor -- start 2024-03-29T04:11:37.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.111+0000 7f5ad76a5640 1 -- start start 2024-03-29T04:11:37.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.111+0000 7f5ad76a5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5ad00d17e0 0x7f5ad00d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.111+0000 7f5ad76a5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ad015e7c0 con 0x7f5ad015e3e0 2024-03-29T04:11:37.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.111+0000 7f5ad76a5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ad00d2c30 con 0x7f5ad00ba030 2024-03-29T04:11:37.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.111+0000 7f5ad76a5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5ad00d3840 con 0x7f5ad00d17e0 2024-03-29T04:11:37.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.112+0000 7f5ad5441640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5ad015e3e0 0x7f5ad00d2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40656/0 (socket says 172.21.15.148:40656) 2024-03-29T04:11:37.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.112+0000 7f5ad5441640 1 -- 172.21.15.148:0/1255963962 learned_addr learned my addr 172.21.15.148:0/1255963962 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:37.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.112+0000 7f5ad5c42640 1 --2- 172.21.15.148:0/1255963962 >> v2:172.21.15.157:3300/0 conn(0x7f5ad00d17e0 0x7f5ad00d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.112+0000 7f5ac7fff640 1 -- 172.21.15.148:0/1255963962 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1550809730 0 0) 0x7f5ad015e7c0 con 0x7f5ad015e3e0 2024-03-29T04:11:37.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.112+0000 7f5ac7fff640 1 -- 172.21.15.148:0/1255963962 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5abc003520 con 0x7f5ad015e3e0 2024-03-29T04:11:37.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.112+0000 7f5ad5c42640 1 -- 172.21.15.148:0/1255963962 >> v1:172.21.15.157:6789/0 conn(0x7f5ad00ba030 legacy=0x7f5ad00d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.112+0000 7f5ac7fff640 1 -- 172.21.15.148:0/1255963962 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2008397302 0 0) 0x7f5abc003520 con 0x7f5ad015e3e0 2024-03-29T04:11:37.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.112+0000 7f5ad5c42640 1 -- 172.21.15.148:0/1255963962 >> v1:172.21.15.148:6789/0 conn(0x7f5ad015e3e0 legacy=0x7f5ad00d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.112+0000 7f5ad5c42640 1 -- 172.21.15.148:0/1255963962 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad00d2f10 con 0x7f5ad00d17e0 2024-03-29T04:11:37.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.113+0000 7f5ad5c42640 1 --2- 172.21.15.148:0/1255963962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ad00d17e0 0x7f5ad00d3cd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8009d90 tx=0x7f5ac802bd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6628ca6a679ec24 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:37.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.113+0000 7f5ac7fff640 1 -- 172.21.15.148:0/1255963962 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8039070 con 0x7f5ad00d17e0 2024-03-29T04:11:37.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.113+0000 7f5ac7fff640 1 -- 172.21.15.148:0/1255963962 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac802c830 con 0x7f5ad00d17e0 2024-03-29T04:11:37.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.113+0000 7f5ac7fff640 1 -- 172.21.15.148:0/1255963962 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc001670 con 0x7f5ad00d17e0 2024-03-29T04:11:37.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.114+0000 7f5ad76a5640 1 -- 172.21.15.148:0/1255963962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ad00d17e0 msgr2=0x7f5ad00d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.114+0000 7f5ad76a5640 1 --2- 172.21.15.148:0/1255963962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ad00d17e0 0x7f5ad00d3cd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8009d90 tx=0x7f5ac802bd00 comp rx=0 tx=0).stop 2024-03-29T04:11:37.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.114+0000 7f5ad76a5640 1 -- 172.21.15.148:0/1255963962 shutdown_connections 2024-03-29T04:11:37.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.114+0000 7f5ad76a5640 1 --2- 172.21.15.148:0/1255963962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ad00d17e0 0x7f5ad00d3cd0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.114+0000 7f5ad76a5640 1 -- 172.21.15.148:0/1255963962 >> 172.21.15.148:0/1255963962 conn(0x7f5ad00b6210 msgr2=0x7f5ad00b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:37.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.114+0000 7f5ad76a5640 1 -- 172.21.15.148:0/1255963962 shutdown_connections 2024-03-29T04:11:37.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.114+0000 7f5ad76a5640 1 -- 172.21.15.148:0/1255963962 wait complete. 2024-03-29T04:11:37.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.115+0000 7f5ad76a5640 1 Processor -- start 2024-03-29T04:11:37.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.115+0000 7f5ad76a5640 1 -- start start 2024-03-29T04:11:37.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.116+0000 7f5ad76a5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ad00ba030 0x7f5ad00cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.116+0000 7f5ad76a5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ad00d17e0 0x7f5ad0159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.116+0000 7f5ad76a5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ad015e3e0 0x7f5ad014d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.116+0000 7f5ad76a5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad0168f60 con 0x7f5ad015e3e0 2024-03-29T04:11:37.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.116+0000 7f5ad76a5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad00d34d0 con 0x7f5ad00d17e0 2024-03-29T04:11:37.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.116+0000 7f5ad76a5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5ad01690e0 con 0x7f5ad00ba030 2024-03-29T04:11:37.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.116+0000 7f5ad5c42640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ad015e3e0 0x7f5ad014d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.116+0000 7f5ad5c42640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ad015e3e0 0x7f5ad014d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42312/0 (socket says 172.21.15.148:42312) 2024-03-29T04:11:37.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.116+0000 7f5ad5c42640 1 -- 172.21.15.148:0/1275360389 learned_addr learned my addr 172.21.15.148:0/1275360389 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:37.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.116+0000 7f5ad4c40640 1 --2- 172.21.15.148:0/1275360389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ad00d17e0 0x7f5ad0159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.116+0000 7f5ad5441640 1 --2- 172.21.15.148:0/1275360389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ad00ba030 0x7f5ad00cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.116+0000 7f5ad5c42640 1 -- 172.21.15.148:0/1275360389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ad00ba030 msgr2=0x7f5ad00cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.116+0000 7f5ad5c42640 1 --2- 172.21.15.148:0/1275360389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ad00ba030 0x7f5ad00cd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.116+0000 7f5ad5c42640 1 -- 172.21.15.148:0/1275360389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ad00d17e0 msgr2=0x7f5ad0159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.116+0000 7f5ad5c42640 1 --2- 172.21.15.148:0/1275360389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ad00d17e0 0x7f5ad0159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.116+0000 7f5ad5c42640 1 -- 172.21.15.148:0/1275360389 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad00cda60 con 0x7f5ad015e3e0 2024-03-29T04:11:37.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.116+0000 7f5ad4c40640 1 --2- 172.21.15.148:0/1275360389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ad00d17e0 0x7f5ad0159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:37.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.116+0000 7f5ad5c42640 1 --2- 172.21.15.148:0/1275360389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ad015e3e0 0x7f5ad014d540 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8037040 tx=0x7f5ac8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:37.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.116+0000 7f5ac5ffb640 1 -- 172.21.15.148:0/1275360389 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8044020 con 0x7f5ad015e3e0 2024-03-29T04:11:37.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.116+0000 7f5ad76a5640 1 -- 172.21.15.148:0/1275360389 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad00cdd40 con 0x7f5ad015e3e0 2024-03-29T04:11:37.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.116+0000 7f5ac5ffb640 1 -- 172.21.15.148:0/1275360389 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac8034d10 con 0x7f5ad015e3e0 2024-03-29T04:11:37.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.116+0000 7f5ac5ffb640 1 -- 172.21.15.148:0/1275360389 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8039040 con 0x7f5ad015e3e0 2024-03-29T04:11:37.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.117+0000 7f5ad76a5640 1 -- 172.21.15.148:0/1275360389 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad015fa90 con 0x7f5ad015e3e0 2024-03-29T04:11:37.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.118+0000 7f5ad76a5640 1 -- 172.21.15.148:0/1275360389 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ad00687e0 con 0x7f5ad015e3e0 2024-03-29T04:11:37.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.118+0000 7f5ac5ffb640 1 -- 172.21.15.148:0/1275360389 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5ac8047020 con 0x7f5ad015e3e0 2024-03-29T04:11:37.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.118+0000 7f5ac5ffb640 1 --2- 172.21.15.148:0/1275360389 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5ab406c530 0x7f5ab406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.119+0000 7f5ac5ffb640 1 -- 172.21.15.148:0/1275360389 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f5ac80adfe0 con 0x7f5ad015e3e0 2024-03-29T04:11:37.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.119+0000 7f5ad5441640 1 --2- 172.21.15.148:0/1275360389 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5ab406c530 0x7f5ab406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.119+0000 7f5ad5441640 1 --2- 172.21.15.148:0/1275360389 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5ab406c530 0x7f5ab406e9f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5ad014e480 tx=0x7f5acc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:37.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.127+0000 7f5ac5ffb640 1 -- 172.21.15.148:0/1275360389 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5ac804e030 con 0x7f5ad015e3e0 2024-03-29T04:11:37.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.302+0000 7f5ad76a5640 1 -- 172.21.15.148:0/1275360389 --> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5ad015fc60 con 0x7f5ab406c530 2024-03-29T04:11:37.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.307+0000 7f5ac5ffb640 1 -- 172.21.15.148:0/1275360389 <== mgr.4113 v2:172.21.15.148:6832/3271445075 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240027 (secure 0 0 0) 0x7f5ad015fc60 con 0x7f5ab406c530 2024-03-29T04:11:37.290 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:11:37.291 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-03-29T04:11:37.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.310+0000 7f5ad76a5640 1 -- 172.21.15.148:0/1275360389 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5ab406c530 msgr2=0x7f5ab406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.310+0000 7f5ad76a5640 1 --2- 172.21.15.148:0/1275360389 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5ab406c530 0x7f5ab406e9f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5ad014e480 tx=0x7f5acc009040 comp rx=0 tx=0).stop 2024-03-29T04:11:37.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.310+0000 7f5ad76a5640 1 -- 172.21.15.148:0/1275360389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ad015e3e0 msgr2=0x7f5ad014d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.310+0000 7f5ad76a5640 1 --2- 172.21.15.148:0/1275360389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ad015e3e0 0x7f5ad014d540 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8037040 tx=0x7f5ac8004a40 comp rx=0 tx=0).stop 2024-03-29T04:11:37.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.310+0000 7f5ad76a5640 1 -- 172.21.15.148:0/1275360389 shutdown_connections 2024-03-29T04:11:37.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.310+0000 7f5ad76a5640 1 --2- 172.21.15.148:0/1275360389 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5ab406c530 0x7f5ab406e9f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.310+0000 7f5ad76a5640 1 --2- 172.21.15.148:0/1275360389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ad015e3e0 0x7f5ad014d540 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.310+0000 7f5ad76a5640 1 --2- 172.21.15.148:0/1275360389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ad00d17e0 0x7f5ad0159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.310+0000 7f5ad76a5640 1 --2- 172.21.15.148:0/1275360389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ad00ba030 0x7f5ad00cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.310+0000 7f5ad76a5640 1 -- 172.21.15.148:0/1275360389 >> 172.21.15.148:0/1275360389 conn(0x7f5ad00b6210 msgr2=0x7f5ad015e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:37.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.310+0000 7f5ad76a5640 1 -- 172.21.15.148:0/1275360389 shutdown_connections 2024-03-29T04:11:37.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.310+0000 7f5ad76a5640 1 -- 172.21.15.148:0/1275360389 wait complete. 2024-03-29T04:11:37.300 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-03-29T04:11:35.900868+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237132,"kb_used_data":4320,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737588,"statfs":{"total":773094113280,"available":772851290112,"internally_reserved":0,"allocated":4423680,"data_stored":2545090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.319933"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.583191+0000","last_change":"2024-03-29T04:11:31.583191+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.583191+0000","last_undegraded":"2024-03-29T04:11:31.583191+0000","last_fullsized":"2024-03-29T04:11:31.583191+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.588394+0000","last_change":"2024-03-29T04:11:31.588394+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.588394+0000","last_undegraded":"2024-03-29T04:11:31.588394+0000","last_fullsized":"2024-03-29T04:11:31.588394+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.591020+0000","last_change":"2024-03-29T04:11:31.591020+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.591020+0000","last_undegraded":"2024-03-29T04:11:31.591020+0000","last_fullsized":"2024-03-29T04:11:31.591020+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.583097+0000","last_change":"2024-03-29T04:11:31.583097+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.583097+0000","last_undegraded":"2024-03-29T04:11:31.583097+0000","last_fullsized":"2024-03-29T04:11:31.583097+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.585232+0000","last_change":"2024-03-29T04:11:31.585232+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.585232+0000","last_undegraded":"2024-03-29T04:11:31.585232+0000","last_fullsized":"2024-03-29T04:11:31.585232+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.587179+0000","last_change":"2024-03-29T04:11:31.587179+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.587179+0000","last_undegraded":"2024-03-29T04:11:31.587179+0000","last_fullsized":"2024-03-29T04:11:31.587179+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.591029+0000","last_change":"2024-03-29T04:11:31.591029+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.591029+0000","last_undegraded":"2024-03-29T04:11:31.591029+0000","last_fullsized":"2024-03-29T04:11:31.591029+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.583148+0000","last_change":"2024-03-29T04:11:31.583148+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.583148+0000","last_undegraded":"2024-03-29T04:11:31.583148+0000","last_fullsized":"2024-03-29T04:11:31.583148+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.586125+0000","last_change":"2024-03-29T04:11:31.586125+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.586125+0000","last_undegraded":"2024-03-29T04:11:31.586125+0000","last_fullsized":"2024-03-29T04:11:31.586125+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.585369+0000","last_change":"2024-03-29T04:11:31.585369+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.585369+0000","last_undegraded":"2024-03-29T04:11:31.585369+0000","last_fullsized":"2024-03-29T04:11:31.585369+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.585174+0000","last_change":"2024-03-29T04:11:31.585174+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.585174+0000","last_undegraded":"2024-03-29T04:11:31.585174+0000","last_fullsized":"2024-03-29T04:11:31.585174+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.585788+0000","last_change":"2024-03-29T04:11:31.585788+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.585788+0000","last_undegraded":"2024-03-29T04:11:31.585788+0000","last_fullsized":"2024-03-29T04:11:31.585788+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.585730+0000","last_change":"2024-03-29T04:11:31.585730+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.585730+0000","last_undegraded":"2024-03-29T04:11:31.585730+0000","last_fullsized":"2024-03-29T04:11:31.585730+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.591061+0000","last_change":"2024-03-29T04:11:31.591061+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.591061+0000","last_undegraded":"2024-03-29T04:11:31.591061+0000","last_fullsized":"2024-03-29T04:11:31.591061+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.583091+0000","last_change":"2024-03-29T04:11:31.583091+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.583091+0000","last_undegraded":"2024-03-29T04:11:31.583091+0000","last_fullsized":"2024-03-29T04:11:31.583091+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.587179+0000","last_change":"2024-03-29T04:11:31.587179+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.587179+0000","last_undegraded":"2024-03-29T04:11:31.587179+0000","last_fullsized":"2024-03-29T04:11:31.587179+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.585723+0000","last_change":"2024-03-29T04:11:31.585723+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.585723+0000","last_undegraded":"2024-03-29T04:11:31.585723+0000","last_fullsized":"2024-03-29T04:11:31.585723+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.583097+0000","last_change":"2024-03-29T04:11:31.583097+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.583097+0000","last_undegraded":"2024-03-29T04:11:31.583097+0000","last_fullsized":"2024-03-29T04:11:31.583097+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.584993+0000","last_change":"2024-03-29T04:11:31.584993+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.584993+0000","last_undegraded":"2024-03-29T04:11:31.584993+0000","last_fullsized":"2024-03-29T04:11:31.584993+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.591301+0000","last_change":"2024-03-29T04:11:31.591301+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.591301+0000","last_undegraded":"2024-03-29T04:11:31.591301+0000","last_fullsized":"2024-03-29T04:11:31.591301+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.584820+0000","last_change":"2024-03-29T04:11:31.584820+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.584820+0000","last_undegraded":"2024-03-29T04:11:31.584820+0000","last_fullsized":"2024-03-29T04:11:31.584820+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.584192+0000","last_change":"2024-03-29T04:11:31.584192+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.584192+0000","last_undegraded":"2024-03-29T04:11:31.584192+0000","last_fullsized":"2024-03-29T04:11:31.584192+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.587161+0000","last_change":"2024-03-29T04:11:31.587161+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.587161+0000","last_undegraded":"2024-03-29T04:11:31.587161+0000","last_fullsized":"2024-03-29T04:11:31.587161+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.584192+0000","last_change":"2024-03-29T04:11:31.584192+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.584192+0000","last_undegraded":"2024-03-29T04:11:31.584192+0000","last_fullsized":"2024-03-29T04:11:31.584192+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.585260+0000","last_change":"2024-03-29T04:11:31.585260+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.585260+0000","last_undegraded":"2024-03-29T04:11:31.585260+0000","last_fullsized":"2024-03-29T04:11:31.585260+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.584195+0000","last_change":"2024-03-29T04:11:31.584195+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.584195+0000","last_undegraded":"2024-03-29T04:11:31.584195+0000","last_fullsized":"2024-03-29T04:11:31.584195+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.588220+0000","last_change":"2024-03-29T04:11:31.588220+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.588220+0000","last_undegraded":"2024-03-29T04:11:31.588220+0000","last_fullsized":"2024-03-29T04:11:31.588220+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.588370+0000","last_change":"2024-03-29T04:11:31.588370+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.588370+0000","last_undegraded":"2024-03-29T04:11:31.588370+0000","last_fullsized":"2024-03-29T04:11:31.588370+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.583807+0000","last_change":"2024-03-29T04:11:31.583807+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.583807+0000","last_undegraded":"2024-03-29T04:11:31.583807+0000","last_fullsized":"2024-03-29T04:11:31.583807+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.583135+0000","last_change":"2024-03-29T04:11:31.583135+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.583135+0000","last_undegraded":"2024-03-29T04:11:31.583135+0000","last_fullsized":"2024-03-29T04:11:31.583135+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.585704+0000","last_change":"2024-03-29T04:11:31.585704+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.585704+0000","last_undegraded":"2024-03-29T04:11:31.585704+0000","last_fullsized":"2024-03-29T04:11:31.585704+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:11:32.580737+0000","last_change":"2024-03-29T04:11:32.580737+0000","last_active":"2024-03-29T04:11:32.580737+0000","last_peered":"2024-03-29T04:11:32.580737+0000","last_clean":"2024-03-29T04:11:32.580737+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:32.580737+0000","last_undegraded":"2024-03-29T04:11:32.580737+0000","last_fullsized":"2024-03-29T04:11:32.580737+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-03-29T04:11:31.587152+0000","last_change":"2024-03-29T04:11:31.587152+0000","last_active":"2024-03-29T04:11:31.578560+0000","last_peered":"2024-03-29T04:11:31.578560+0000","last_clean":"2024-03-29T04:11:31.578560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:11:31.587152+0000","last_undegraded":"2024-03-29T04:11:31.587152+0000","last_fullsized":"2024-03-29T04:11:31.587152+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":201,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-29T04:11:31.587612+0000","last_change":"2024-03-29T04:09:49.002570+0000","last_active":"2024-03-29T04:11:31.587612+0000","last_peered":"2024-03-29T04:11:31.587612+0000","last_clean":"2024-03-29T04:11:31.587612+0000","last_became_active":"2024-03-29T04:09:49.002235+0000","last_became_peered":"2024-03-29T04:09:49.002235+0000","last_unstale":"2024-03-29T04:11:31.587612+0000","last_undegraded":"2024-03-29T04:11:31.587612+0000","last_fullsized":"2024-03-29T04:11:31.587612+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:47.900733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:47.900733+0000","last_clean_scrub_stamp":"2024-03-29T04:09:47.900733+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:44:04.608265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":759360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":401408,"data_stored":168992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":401408,"data_stored":168992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":175201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":168992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":168992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":168992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":765569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:11:37.304 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3f is not active+clean 2024-03-29T04:11:37.304 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.304 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3e is not active+clean 2024-03-29T04:11:37.304 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.304 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3d is not active+clean 2024-03-29T04:11:37.304 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.304 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3c is not active+clean 2024-03-29T04:11:37.304 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.305 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3b is not active+clean 2024-03-29T04:11:37.305 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.305 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3a is not active+clean 2024-03-29T04:11:37.305 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.305 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.39 is not active+clean 2024-03-29T04:11:37.305 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.305 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.38 is not active+clean 2024-03-29T04:11:37.305 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.305 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.37 is not active+clean 2024-03-29T04:11:37.305 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.305 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.36 is not active+clean 2024-03-29T04:11:37.306 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.306 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2024-03-29T04:11:37.306 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.306 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.34 is not active+clean 2024-03-29T04:11:37.306 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.306 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.33 is not active+clean 2024-03-29T04:11:37.306 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.306 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.32 is not active+clean 2024-03-29T04:11:37.306 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.306 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2024-03-29T04:11:37.306 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.307 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.30 is not active+clean 2024-03-29T04:11:37.307 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.307 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2f is not active+clean 2024-03-29T04:11:37.307 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.307 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2e is not active+clean 2024-03-29T04:11:37.307 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.307 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2d is not active+clean 2024-03-29T04:11:37.307 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.307 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2c is not active+clean 2024-03-29T04:11:37.307 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.307 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2024-03-29T04:11:37.307 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.308 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2024-03-29T04:11:37.308 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.308 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.29 is not active+clean 2024-03-29T04:11:37.308 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.308 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.28 is not active+clean 2024-03-29T04:11:37.308 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.308 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.27 is not active+clean 2024-03-29T04:11:37.308 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.308 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.26 is not active+clean 2024-03-29T04:11:37.308 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.308 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.25 is not active+clean 2024-03-29T04:11:37.309 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.309 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.24 is not active+clean 2024-03-29T04:11:37.309 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.309 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.23 is not active+clean 2024-03-29T04:11:37.309 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.309 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.22 is not active+clean 2024-03-29T04:11:37.309 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.309 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.21 is not active+clean 2024-03-29T04:11:37.309 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.309 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.20 is not active+clean 2024-03-29T04:11:37.309 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.309 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2024-03-29T04:11:37.310 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.310 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-03-29T04:11:37.310 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.583191+0000', 'last_change': '2024-03-29T04:11:31.583191+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.583191+0000', 'last_undegraded': '2024-03-29T04:11:31.583191+0000', 'last_fullsized': '2024-03-29T04:11:31.583191+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-29T04:11:37.310 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2024-03-29T04:11:37.310 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.310 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-03-29T04:11:37.310 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.588394+0000', 'last_change': '2024-03-29T04:11:31.588394+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.588394+0000', 'last_undegraded': '2024-03-29T04:11:31.588394+0000', 'last_fullsized': '2024-03-29T04:11:31.588394+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-29T04:11:37.310 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2024-03-29T04:11:37.310 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.311 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-03-29T04:11:37.311 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.591020+0000', 'last_change': '2024-03-29T04:11:31.591020+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.591020+0000', 'last_undegraded': '2024-03-29T04:11:31.591020+0000', 'last_fullsized': '2024-03-29T04:11:31.591020+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-29T04:11:37.311 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2024-03-29T04:11:37.311 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.311 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-03-29T04:11:37.311 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.583097+0000', 'last_change': '2024-03-29T04:11:31.583097+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.583097+0000', 'last_undegraded': '2024-03-29T04:11:31.583097+0000', 'last_fullsized': '2024-03-29T04:11:31.583097+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-29T04:11:37.311 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2024-03-29T04:11:37.311 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.311 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-03-29T04:11:37.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.585232+0000', 'last_change': '2024-03-29T04:11:31.585232+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.585232+0000', 'last_undegraded': '2024-03-29T04:11:31.585232+0000', 'last_fullsized': '2024-03-29T04:11:31.585232+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T04:11:37.312 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2024-03-29T04:11:37.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.312 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-03-29T04:11:37.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.587179+0000', 'last_change': '2024-03-29T04:11:31.587179+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.587179+0000', 'last_undegraded': '2024-03-29T04:11:31.587179+0000', 'last_fullsized': '2024-03-29T04:11:31.587179+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-29T04:11:37.312 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2024-03-29T04:11:37.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.312 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-03-29T04:11:37.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.591029+0000', 'last_change': '2024-03-29T04:11:31.591029+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.591029+0000', 'last_undegraded': '2024-03-29T04:11:31.591029+0000', 'last_fullsized': '2024-03-29T04:11:31.591029+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-29T04:11:37.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2024-03-29T04:11:37.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-03-29T04:11:37.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.583148+0000', 'last_change': '2024-03-29T04:11:31.583148+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.583148+0000', 'last_undegraded': '2024-03-29T04:11:31.583148+0000', 'last_fullsized': '2024-03-29T04:11:31.583148+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-29T04:11:37.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2024-03-29T04:11:37.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-03-29T04:11:37.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.586125+0000', 'last_change': '2024-03-29T04:11:31.586125+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.586125+0000', 'last_undegraded': '2024-03-29T04:11:31.586125+0000', 'last_fullsized': '2024-03-29T04:11:31.586125+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-29T04:11:37.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2024-03-29T04:11:37.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-03-29T04:11:37.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.585369+0000', 'last_change': '2024-03-29T04:11:31.585369+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.585369+0000', 'last_undegraded': '2024-03-29T04:11:31.585369+0000', 'last_fullsized': '2024-03-29T04:11:31.585369+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T04:11:37.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2024-03-29T04:11:37.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-03-29T04:11:37.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.585174+0000', 'last_change': '2024-03-29T04:11:31.585174+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.585174+0000', 'last_undegraded': '2024-03-29T04:11:31.585174+0000', 'last_fullsized': '2024-03-29T04:11:31.585174+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T04:11:37.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2024-03-29T04:11:37.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-03-29T04:11:37.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.585788+0000', 'last_change': '2024-03-29T04:11:31.585788+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.585788+0000', 'last_undegraded': '2024-03-29T04:11:31.585788+0000', 'last_fullsized': '2024-03-29T04:11:31.585788+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-29T04:11:37.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2024-03-29T04:11:37.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-03-29T04:11:37.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.585730+0000', 'last_change': '2024-03-29T04:11:31.585730+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.585730+0000', 'last_undegraded': '2024-03-29T04:11:31.585730+0000', 'last_fullsized': '2024-03-29T04:11:31.585730+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-29T04:11:37.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2024-03-29T04:11:37.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-03-29T04:11:37.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.591061+0000', 'last_change': '2024-03-29T04:11:31.591061+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.591061+0000', 'last_undegraded': '2024-03-29T04:11:31.591061+0000', 'last_fullsized': '2024-03-29T04:11:31.591061+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-29T04:11:37.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2024-03-29T04:11:37.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-03-29T04:11:37.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.583091+0000', 'last_change': '2024-03-29T04:11:31.583091+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.583091+0000', 'last_undegraded': '2024-03-29T04:11:31.583091+0000', 'last_fullsized': '2024-03-29T04:11:31.583091+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-29T04:11:37.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2024-03-29T04:11:37.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-03-29T04:11:37.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.587179+0000', 'last_change': '2024-03-29T04:11:31.587179+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.587179+0000', 'last_undegraded': '2024-03-29T04:11:31.587179+0000', 'last_fullsized': '2024-03-29T04:11:31.587179+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-29T04:11:37.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2024-03-29T04:11:37.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-03-29T04:11:37.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.585723+0000', 'last_change': '2024-03-29T04:11:31.585723+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.585723+0000', 'last_undegraded': '2024-03-29T04:11:31.585723+0000', 'last_fullsized': '2024-03-29T04:11:31.585723+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T04:11:37.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2024-03-29T04:11:37.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-03-29T04:11:37.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.583097+0000', 'last_change': '2024-03-29T04:11:31.583097+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.583097+0000', 'last_undegraded': '2024-03-29T04:11:31.583097+0000', 'last_fullsized': '2024-03-29T04:11:31.583097+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-29T04:11:37.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2024-03-29T04:11:37.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-03-29T04:11:37.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.584993+0000', 'last_change': '2024-03-29T04:11:31.584993+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.584993+0000', 'last_undegraded': '2024-03-29T04:11:31.584993+0000', 'last_fullsized': '2024-03-29T04:11:31.584993+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-29T04:11:37.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2024-03-29T04:11:37.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-03-29T04:11:37.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.591301+0000', 'last_change': '2024-03-29T04:11:31.591301+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.591301+0000', 'last_undegraded': '2024-03-29T04:11:31.591301+0000', 'last_fullsized': '2024-03-29T04:11:31.591301+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-29T04:11:37.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2024-03-29T04:11:37.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-03-29T04:11:37.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.584820+0000', 'last_change': '2024-03-29T04:11:31.584820+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.584820+0000', 'last_undegraded': '2024-03-29T04:11:31.584820+0000', 'last_fullsized': '2024-03-29T04:11:31.584820+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-29T04:11:37.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2024-03-29T04:11:37.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-03-29T04:11:37.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.584192+0000', 'last_change': '2024-03-29T04:11:31.584192+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.584192+0000', 'last_undegraded': '2024-03-29T04:11:31.584192+0000', 'last_fullsized': '2024-03-29T04:11:31.584192+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-29T04:11:37.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2024-03-29T04:11:37.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-03-29T04:11:37.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.587161+0000', 'last_change': '2024-03-29T04:11:31.587161+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.587161+0000', 'last_undegraded': '2024-03-29T04:11:31.587161+0000', 'last_fullsized': '2024-03-29T04:11:31.587161+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-29T04:11:37.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2024-03-29T04:11:37.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-03-29T04:11:37.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.584192+0000', 'last_change': '2024-03-29T04:11:31.584192+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.584192+0000', 'last_undegraded': '2024-03-29T04:11:31.584192+0000', 'last_fullsized': '2024-03-29T04:11:31.584192+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-29T04:11:37.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2024-03-29T04:11:37.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-03-29T04:11:37.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.585260+0000', 'last_change': '2024-03-29T04:11:31.585260+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.585260+0000', 'last_undegraded': '2024-03-29T04:11:31.585260+0000', 'last_fullsized': '2024-03-29T04:11:31.585260+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T04:11:37.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2024-03-29T04:11:37.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-03-29T04:11:37.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.584195+0000', 'last_change': '2024-03-29T04:11:31.584195+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.584195+0000', 'last_undegraded': '2024-03-29T04:11:31.584195+0000', 'last_fullsized': '2024-03-29T04:11:31.584195+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-29T04:11:37.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2024-03-29T04:11:37.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-03-29T04:11:37.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.588220+0000', 'last_change': '2024-03-29T04:11:31.588220+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.588220+0000', 'last_undegraded': '2024-03-29T04:11:31.588220+0000', 'last_fullsized': '2024-03-29T04:11:31.588220+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-29T04:11:37.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2024-03-29T04:11:37.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-03-29T04:11:37.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.588370+0000', 'last_change': '2024-03-29T04:11:31.588370+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.588370+0000', 'last_undegraded': '2024-03-29T04:11:31.588370+0000', 'last_fullsized': '2024-03-29T04:11:31.588370+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-29T04:11:37.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2024-03-29T04:11:37.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-03-29T04:11:37.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.583807+0000', 'last_change': '2024-03-29T04:11:31.583807+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.583807+0000', 'last_undegraded': '2024-03-29T04:11:31.583807+0000', 'last_fullsized': '2024-03-29T04:11:31.583807+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-29T04:11:37.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2024-03-29T04:11:37.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-03-29T04:11:37.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.583135+0000', 'last_change': '2024-03-29T04:11:31.583135+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.583135+0000', 'last_undegraded': '2024-03-29T04:11:31.583135+0000', 'last_fullsized': '2024-03-29T04:11:31.583135+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-29T04:11:37.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2024-03-29T04:11:37.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-03-29T04:11:37.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.585704+0000', 'last_change': '2024-03-29T04:11:31.585704+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.585704+0000', 'last_undegraded': '2024-03-29T04:11:31.585704+0000', 'last_fullsized': '2024-03-29T04:11:31.585704+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-29T04:11:37.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2024-03-29T04:11:37.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T04:11:32.580737+0000', 'last_change': '2024-03-29T04:11:32.580737+0000', 'last_active': '2024-03-29T04:11:32.580737+0000', 'last_peered': '2024-03-29T04:11:32.580737+0000', 'last_clean': '2024-03-29T04:11:32.580737+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:32.580737+0000', 'last_undegraded': '2024-03-29T04:11:32.580737+0000', 'last_fullsized': '2024-03-29T04:11:32.580737+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:32.580737+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T04:11:37.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-03-29T04:11:37.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:11:31.587152+0000', 'last_change': '2024-03-29T04:11:31.587152+0000', 'last_active': '2024-03-29T04:11:31.578560+0000', 'last_peered': '2024-03-29T04:11:31.578560+0000', 'last_clean': '2024-03-29T04:11:31.578560+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:11:31.587152+0000', 'last_undegraded': '2024-03-29T04:11:31.587152+0000', 'last_fullsized': '2024-03-29T04:11:31.587152+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'last_clean_scrub_stamp': '2024-03-29T04:11:31.578560+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-29T04:11:37.323 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-29T04:11:37.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.461+0000 7fe8c307b640 1 Processor -- start 2024-03-29T04:11:37.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.462+0000 7fe8c307b640 1 -- start start 2024-03-29T04:11:37.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.462+0000 7fe8c307b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe8bc15e3e0 0x7fe8bc0d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.462+0000 7fe8c307b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe8bc0ba030 0x7fe8bc0d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.462+0000 7fe8c307b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8bc0d10d0 0x7fe8bc0d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.462+0000 7fe8c307b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fe8bc0d3840 con 0x7fe8bc15e3e0 2024-03-29T04:11:37.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.462+0000 7fe8c307b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe8bc15e7c0 con 0x7fe8bc0ba030 2024-03-29T04:11:37.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.462+0000 7fe8c307b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe8bc0d2c30 con 0x7fe8bc0d10d0 2024-03-29T04:11:37.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.462+0000 7fe8c0e17640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe8bc15e3e0 0x7fe8bc0d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.462+0000 7fe8c0e17640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe8bc15e3e0 0x7fe8bc0d3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42316/0 (socket says 172.21.15.148:42316) 2024-03-29T04:11:37.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.462+0000 7fe8c0e17640 1 -- 172.21.15.148:0/151413136 learned_addr learned my addr 172.21.15.148:0/151413136 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:37.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.463+0000 7fe8bbfff640 1 --2- 172.21.15.148:0/151413136 >> v2:172.21.15.157:3300/0 conn(0x7fe8bc0ba030 0x7fe8bc0d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.463+0000 7fe8c1618640 1 --2- 172.21.15.148:0/151413136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8bc0d10d0 0x7fe8bc0d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.463+0000 7fe8c0e17640 1 -- 172.21.15.148:0/151413136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8bc0d10d0 msgr2=0x7fe8bc0d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:37.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.463+0000 7fe8c0e17640 1 --2- 172.21.15.148:0/151413136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8bc0d10d0 0x7fe8bc0d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.463+0000 7fe8c0e17640 1 -- 172.21.15.148:0/151413136 >> v2:172.21.15.157:3300/0 conn(0x7fe8bc0ba030 msgr2=0x7fe8bc0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:37.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.463+0000 7fe8c0e17640 1 --2- 172.21.15.148:0/151413136 >> v2:172.21.15.157:3300/0 conn(0x7fe8bc0ba030 0x7fe8bc0d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.463+0000 7fe8c0e17640 1 -- 172.21.15.148:0/151413136 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8bc0d2f10 con 0x7fe8bc15e3e0 2024-03-29T04:11:37.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.464+0000 7fe8c0e17640 1 --2- 172.21.15.148:0/151413136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8bc15e3e0 0x7fe8bc0d3cd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0d05d0 tx=0x7fe8b002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c736252a4acfef5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:37.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.464+0000 7fe8bb7fe640 1 -- 172.21.15.148:0/151413136 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0034070 con 0x7fe8bc15e3e0 2024-03-29T04:11:37.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.464+0000 7fe8bb7fe640 1 -- 172.21.15.148:0/151413136 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8b0030860 con 0x7fe8bc15e3e0 2024-03-29T04:11:37.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.464+0000 7fe8bb7fe640 1 -- 172.21.15.148:0/151413136 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0030c30 con 0x7fe8bc15e3e0 2024-03-29T04:11:37.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.465+0000 7fe8c307b640 1 -- 172.21.15.148:0/151413136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8bc15e3e0 msgr2=0x7fe8bc0d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.465+0000 7fe8c307b640 1 --2- 172.21.15.148:0/151413136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8bc15e3e0 0x7fe8bc0d3cd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0d05d0 tx=0x7fe8b002b9a0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.465+0000 7fe8c307b640 1 -- 172.21.15.148:0/151413136 shutdown_connections 2024-03-29T04:11:37.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.465+0000 7fe8c307b640 1 --2- 172.21.15.148:0/151413136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8bc0d10d0 0x7fe8bc0d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.465+0000 7fe8c307b640 1 --2- 172.21.15.148:0/151413136 >> v2:172.21.15.157:3300/0 conn(0x7fe8bc0ba030 0x7fe8bc0d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.465+0000 7fe8c307b640 1 --2- 172.21.15.148:0/151413136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8bc15e3e0 0x7fe8bc0d3cd0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.465+0000 7fe8c307b640 1 -- 172.21.15.148:0/151413136 >> 172.21.15.148:0/151413136 conn(0x7fe8bc0b6210 msgr2=0x7fe8bc0b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:37.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.465+0000 7fe8c307b640 1 -- 172.21.15.148:0/151413136 shutdown_connections 2024-03-29T04:11:37.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.465+0000 7fe8c307b640 1 -- 172.21.15.148:0/151413136 wait complete. 2024-03-29T04:11:37.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.466+0000 7fe8c307b640 1 Processor -- start 2024-03-29T04:11:37.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.466+0000 7fe8c307b640 1 -- start start 2024-03-29T04:11:37.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.467+0000 7fe8c307b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8bc0ba030 0x7fe8bc153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.467+0000 7fe8c307b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8bc0d10d0 0x7fe8bc0cd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.467+0000 7fe8c307b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8bc15e3e0 0x7fe8bc0cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.467+0000 7fe8c307b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc0d2870 con 0x7fe8bc0ba030 2024-03-29T04:11:37.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.467+0000 7fe8c307b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc0d34d0 con 0x7fe8bc0d10d0 2024-03-29T04:11:37.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.467+0000 7fe8c307b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe8bc0d29f0 con 0x7fe8bc15e3e0 2024-03-29T04:11:37.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.467+0000 7fe8c0e17640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8bc0ba030 0x7fe8bc153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.467+0000 7fe8c0e17640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8bc0ba030 0x7fe8bc153bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42318/0 (socket says 172.21.15.148:42318) 2024-03-29T04:11:37.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.467+0000 7fe8c0e17640 1 -- 172.21.15.148:0/965401818 learned_addr learned my addr 172.21.15.148:0/965401818 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:37.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.467+0000 7fe8bbfff640 1 --2- 172.21.15.148:0/965401818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8bc0d10d0 0x7fe8bc0cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.467+0000 7fe8c1618640 1 --2- 172.21.15.148:0/965401818 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8bc15e3e0 0x7fe8bc0cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.467+0000 7fe8c0e17640 1 -- 172.21.15.148:0/965401818 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8bc15e3e0 msgr2=0x7fe8bc0cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.467+0000 7fe8c0e17640 1 --2- 172.21.15.148:0/965401818 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8bc15e3e0 0x7fe8bc0cdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.467+0000 7fe8c0e17640 1 -- 172.21.15.148:0/965401818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8bc0d10d0 msgr2=0x7fe8bc0cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.467+0000 7fe8c0e17640 1 --2- 172.21.15.148:0/965401818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8bc0d10d0 0x7fe8bc0cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.467+0000 7fe8c0e17640 1 -- 172.21.15.148:0/965401818 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8bc0cb710 con 0x7fe8bc0ba030 2024-03-29T04:11:37.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.467+0000 7fe8bbfff640 1 --2- 172.21.15.148:0/965401818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8bc0d10d0 0x7fe8bc0cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:37.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.467+0000 7fe8c0e17640 1 --2- 172.21.15.148:0/965401818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8bc0ba030 0x7fe8bc153bc0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe8b002b9e0 tx=0x7fe8b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:37.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.467+0000 7fe8b97fa640 1 -- 172.21.15.148:0/965401818 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0034040 con 0x7fe8bc0ba030 2024-03-29T04:11:37.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.467+0000 7fe8b97fa640 1 -- 172.21.15.148:0/965401818 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8b0039d90 con 0x7fe8bc0ba030 2024-03-29T04:11:37.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.467+0000 7fe8c307b640 1 -- 172.21.15.148:0/965401818 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8bc157b40 con 0x7fe8bc0ba030 2024-03-29T04:11:37.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.467+0000 7fe8b97fa640 1 -- 172.21.15.148:0/965401818 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0043610 con 0x7fe8bc0ba030 2024-03-29T04:11:37.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.468+0000 7fe8c307b640 1 -- 172.21.15.148:0/965401818 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8bc158100 con 0x7fe8bc0ba030 2024-03-29T04:11:37.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.468+0000 7fe8c307b640 1 -- 172.21.15.148:0/965401818 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8bc0687e0 con 0x7fe8bc0ba030 2024-03-29T04:11:37.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.469+0000 7fe8b97fa640 1 -- 172.21.15.148:0/965401818 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe8b0037070 con 0x7fe8bc0ba030 2024-03-29T04:11:37.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.469+0000 7fe8b97fa640 1 --2- 172.21.15.148:0/965401818 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe8a006c530 0x7fe8a006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.469+0000 7fe8b97fa640 1 -- 172.21.15.148:0/965401818 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fe8b00ad950 con 0x7fe8bc0ba030 2024-03-29T04:11:37.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.469+0000 7fe8bbfff640 1 --2- 172.21.15.148:0/965401818 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe8a006c530 0x7fe8a006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.470+0000 7fe8bbfff640 1 --2- 172.21.15.148:0/965401818 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe8a006c530 0x7fe8a006e9f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0b4880 tx=0x7fe8a8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:37.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.478+0000 7fe8b97fa640 1 -- 172.21.15.148:0/965401818 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe8bc0cb710 con 0x7fe8bc0ba030 2024-03-29T04:11:37.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.732+0000 7fe8c307b640 1 -- 172.21.15.148:0/965401818 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fe8bc0c7a30 con 0x7fe8bc0ba030 2024-03-29T04:11:37.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.732+0000 7fe8b97fa640 1 -- 172.21.15.148:0/965401818 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1109 (secure 0 0 0) 0x7fe8b003e070 con 0x7fe8bc0ba030 2024-03-29T04:11:37.712 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:11:37.712 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":122,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":8,"num_up_osds":8,"osd_up_since":1711685386,"num_in_osds":8,"osd_in_since":1711685377,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":242823168,"bytes_avail":772851290112,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":17,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5583}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-03-29T04:11:05.895774+0000","services":{}},"progress_events":{}} 2024-03-29T04:11:37.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.735+0000 7fe8c307b640 1 -- 172.21.15.148:0/965401818 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe8a006c530 msgr2=0x7fe8a006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.735+0000 7fe8c307b640 1 --2- 172.21.15.148:0/965401818 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe8a006c530 0x7fe8a006e9f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0b4880 tx=0x7fe8a8008040 comp rx=0 tx=0).stop 2024-03-29T04:11:37.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.735+0000 7fe8c307b640 1 -- 172.21.15.148:0/965401818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8bc0ba030 msgr2=0x7fe8bc153bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.735+0000 7fe8c307b640 1 --2- 172.21.15.148:0/965401818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8bc0ba030 0x7fe8bc153bc0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe8b002b9e0 tx=0x7fe8b00046c0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.735+0000 7fe8c307b640 1 -- 172.21.15.148:0/965401818 shutdown_connections 2024-03-29T04:11:37.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.736+0000 7fe8c307b640 1 --2- 172.21.15.148:0/965401818 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe8a006c530 0x7fe8a006e9f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.736+0000 7fe8c307b640 1 --2- 172.21.15.148:0/965401818 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8bc15e3e0 0x7fe8bc0cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.736+0000 7fe8c307b640 1 --2- 172.21.15.148:0/965401818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8bc0d10d0 0x7fe8bc0cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.736+0000 7fe8c307b640 1 --2- 172.21.15.148:0/965401818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8bc0ba030 0x7fe8bc153bc0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.736+0000 7fe8c307b640 1 -- 172.21.15.148:0/965401818 >> 172.21.15.148:0/965401818 conn(0x7fe8bc0b6210 msgr2=0x7fe8bc0ca6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:37.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.736+0000 7fe8c307b640 1 -- 172.21.15.148:0/965401818 shutdown_connections 2024-03-29T04:11:37.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.736+0000 7fe8c307b640 1 -- 172.21.15.148:0/965401818 wait complete. 2024-03-29T04:11:37.726 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-29T04:11:37.726 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:11:37.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.850+0000 7f7c7b3d5640 1 Processor -- start 2024-03-29T04:11:37.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.850+0000 7f7c7b3d5640 1 -- start start 2024-03-29T04:11:37.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.850+0000 7f7c7b3d5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c740ba030 0x7f7c740d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.850+0000 7f7c7b3d5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7c740d10d0 0x7f7c740d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.850+0000 7f7c7b3d5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c7415e7c0 con 0x7f7c7415e3e0 2024-03-29T04:11:37.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.850+0000 7f7c7b3d5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f7c740d3840 con 0x7f7c740d10d0 2024-03-29T04:11:37.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.850+0000 7f7c7b3d5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7c740d2c30 con 0x7f7c740ba030 2024-03-29T04:11:37.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.851+0000 7f7c79972640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7c740d10d0 0x7f7c740d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.851+0000 7f7c79972640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7c740d10d0 0x7f7c740d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42334/0 (socket says 172.21.15.148:42334) 2024-03-29T04:11:37.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.851+0000 7f7c79972640 1 -- 172.21.15.148:0/2807808873 learned_addr learned my addr 172.21.15.148:0/2807808873 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:37.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.851+0000 7f7c79972640 1 -- 172.21.15.148:0/2807808873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c740ba030 msgr2=0x7f7c740d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:37.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.851+0000 7f7c78970640 1 --2- 172.21.15.148:0/2807808873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c740ba030 0x7f7c740d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.851+0000 7f7c6bfff640 1 -- 172.21.15.148:0/2807808873 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 170722670 0 0) 0x7f7c7415e7c0 con 0x7f7c7415e3e0 2024-03-29T04:11:37.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.851+0000 7f7c79972640 1 --2- 172.21.15.148:0/2807808873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c740ba030 0x7f7c740d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.851+0000 7f7c79972640 1 -- 172.21.15.148:0/2807808873 >> v1:172.21.15.148:6789/0 conn(0x7f7c7415e3e0 legacy=0x7f7c740d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.851+0000 7f7c79972640 1 -- 172.21.15.148:0/2807808873 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c740d2f10 con 0x7f7c740d10d0 2024-03-29T04:11:37.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.851+0000 7f7c78970640 1 --2- 172.21.15.148:0/2807808873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c740ba030 0x7f7c740d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:37.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.852+0000 7f7c79972640 1 --2- 172.21.15.148:0/2807808873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c740d10d0 0x7f7c740d1580 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c009e50 tx=0x7f7c6c02bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3fa54b45fd330b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:37.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.852+0000 7f7c6bfff640 1 -- 172.21.15.148:0/2807808873 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c02cb90 con 0x7f7c740d10d0 2024-03-29T04:11:37.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.852+0000 7f7c6bfff640 1 -- 172.21.15.148:0/2807808873 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c6c02cd30 con 0x7f7c740d10d0 2024-03-29T04:11:37.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.852+0000 7f7c6bfff640 1 -- 172.21.15.148:0/2807808873 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c034610 con 0x7f7c740d10d0 2024-03-29T04:11:37.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.852+0000 7f7c7b3d5640 1 -- 172.21.15.148:0/2807808873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c740d10d0 msgr2=0x7f7c740d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.852+0000 7f7c7b3d5640 1 --2- 172.21.15.148:0/2807808873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c740d10d0 0x7f7c740d1580 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c009e50 tx=0x7f7c6c02bff0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.852+0000 7f7c7b3d5640 1 -- 172.21.15.148:0/2807808873 shutdown_connections 2024-03-29T04:11:37.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.852+0000 7f7c7b3d5640 1 --2- 172.21.15.148:0/2807808873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c740d10d0 0x7f7c740d1580 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.852+0000 7f7c7b3d5640 1 --2- 172.21.15.148:0/2807808873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c740ba030 0x7f7c740d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.852+0000 7f7c7b3d5640 1 -- 172.21.15.148:0/2807808873 >> 172.21.15.148:0/2807808873 conn(0x7f7c740b6210 msgr2=0x7f7c740b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:37.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.853+0000 7f7c7b3d5640 1 -- 172.21.15.148:0/2807808873 shutdown_connections 2024-03-29T04:11:37.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.853+0000 7f7c7b3d5640 1 -- 172.21.15.148:0/2807808873 wait complete. 2024-03-29T04:11:37.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.854+0000 7f7c7b3d5640 1 Processor -- start 2024-03-29T04:11:37.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.854+0000 7f7c7b3d5640 1 -- start start 2024-03-29T04:11:37.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.854+0000 7f7c7b3d5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c740ba030 0x7f7c7414f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.854+0000 7f7c7b3d5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c740d10d0 0x7f7c740c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.854+0000 7f7c7b3d5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c7415e3e0 0x7f7c740cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.854+0000 7f7c7b3d5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7c74167f50 con 0x7f7c7415e3e0 2024-03-29T04:11:37.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.854+0000 7f7c7b3d5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7c740d34d0 con 0x7f7c740d10d0 2024-03-29T04:11:37.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.854+0000 7f7c7b3d5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7c741680d0 con 0x7f7c740ba030 2024-03-29T04:11:37.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.854+0000 7f7c79972640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c7415e3e0 0x7f7c740cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.855+0000 7f7c79972640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c7415e3e0 0x7f7c740cd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42346/0 (socket says 172.21.15.148:42346) 2024-03-29T04:11:37.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.855+0000 7f7c79972640 1 -- 172.21.15.148:0/1838247887 learned_addr learned my addr 172.21.15.148:0/1838247887 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:37.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.855+0000 7f7c78970640 1 --2- 172.21.15.148:0/1838247887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c740d10d0 0x7f7c740c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.855+0000 7f7c79171640 1 --2- 172.21.15.148:0/1838247887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c740ba030 0x7f7c7414f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.855+0000 7f7c79972640 1 -- 172.21.15.148:0/1838247887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c740ba030 msgr2=0x7f7c7414f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.855+0000 7f7c79972640 1 --2- 172.21.15.148:0/1838247887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c740ba030 0x7f7c7414f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.855+0000 7f7c79972640 1 -- 172.21.15.148:0/1838247887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c740d10d0 msgr2=0x7f7c740c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.855+0000 7f7c79972640 1 --2- 172.21.15.148:0/1838247887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c740d10d0 0x7f7c740c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.855+0000 7f7c79972640 1 -- 172.21.15.148:0/1838247887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c7414f7a0 con 0x7f7c7415e3e0 2024-03-29T04:11:37.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.855+0000 7f7c79972640 1 --2- 172.21.15.148:0/1838247887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c7415e3e0 0x7f7c740cd970 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c009e50 tx=0x7f7c6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:37.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.855+0000 7f7c69ffb640 1 -- 172.21.15.148:0/1838247887 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c043020 con 0x7f7c7415e3e0 2024-03-29T04:11:37.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.855+0000 7f7c69ffb640 1 -- 172.21.15.148:0/1838247887 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c6c03e450 con 0x7f7c7415e3e0 2024-03-29T04:11:37.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.855+0000 7f7c7b3d5640 1 -- 172.21.15.148:0/1838247887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c740c9450 con 0x7f7c7415e3e0 2024-03-29T04:11:37.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.855+0000 7f7c69ffb640 1 -- 172.21.15.148:0/1838247887 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c009510 con 0x7f7c7415e3e0 2024-03-29T04:11:37.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.855+0000 7f7c7b3d5640 1 -- 172.21.15.148:0/1838247887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c741f7d60 con 0x7f7c7415e3e0 2024-03-29T04:11:37.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.856+0000 7f7c7b3d5640 1 -- 172.21.15.148:0/1838247887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c740b5780 con 0x7f7c7415e3e0 2024-03-29T04:11:37.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.857+0000 7f7c69ffb640 1 -- 172.21.15.148:0/1838247887 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7c6c046020 con 0x7f7c7415e3e0 2024-03-29T04:11:37.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.857+0000 7f7c69ffb640 1 --2- 172.21.15.148:0/1838247887 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7c5806c530 0x7f7c5806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.857+0000 7f7c69ffb640 1 -- 172.21.15.148:0/1838247887 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f7c6c0ae0b0 con 0x7f7c7415e3e0 2024-03-29T04:11:37.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.857+0000 7f7c79171640 1 --2- 172.21.15.148:0/1838247887 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7c5806c530 0x7f7c5806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.858+0000 7f7c79171640 1 --2- 172.21.15.148:0/1838247887 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7c5806c530 0x7f7c5806e9f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7c70007fd0 tx=0x7f7c70009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:37.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:37.862+0000 7f7c69ffb640 1 -- 172.21.15.148:0/1838247887 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7c740b5780 con 0x7f7c7415e3e0 2024-03-29T04:11:38.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:38.038+0000 7f7c7b3d5640 1 -- 172.21.15.148:0/1838247887 --> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7c740c7680 con 0x7f7c5806c530 2024-03-29T04:11:38.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:38.044+0000 7f7c69ffb640 1 -- 172.21.15.148:0/1838247887 <== mgr.4113 v2:172.21.15.148:6832/3271445075 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251957 (secure 0 0 0) 0x7f7c740c7680 con 0x7f7c5806c530 2024-03-29T04:11:38.025 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:11:38.027 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-03-29T04:11:38.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:38.047+0000 7f7c7b3d5640 1 -- 172.21.15.148:0/1838247887 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7c5806c530 msgr2=0x7f7c5806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:38.047+0000 7f7c7b3d5640 1 --2- 172.21.15.148:0/1838247887 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7c5806c530 0x7f7c5806e9f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7c70007fd0 tx=0x7f7c70009040 comp rx=0 tx=0).stop 2024-03-29T04:11:38.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:38.048+0000 7f7c7b3d5640 1 -- 172.21.15.148:0/1838247887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c7415e3e0 msgr2=0x7f7c740cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:38.048+0000 7f7c7b3d5640 1 --2- 172.21.15.148:0/1838247887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c7415e3e0 0x7f7c740cd970 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c009e50 tx=0x7f7c6c004a40 comp rx=0 tx=0).stop 2024-03-29T04:11:38.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:38.048+0000 7f7c7b3d5640 1 -- 172.21.15.148:0/1838247887 shutdown_connections 2024-03-29T04:11:38.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:38.048+0000 7f7c7b3d5640 1 --2- 172.21.15.148:0/1838247887 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7c5806c530 0x7f7c5806e9f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:38.048+0000 7f7c7b3d5640 1 --2- 172.21.15.148:0/1838247887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c7415e3e0 0x7f7c740cd970 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:38.048+0000 7f7c7b3d5640 1 --2- 172.21.15.148:0/1838247887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c740d10d0 0x7f7c740c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:38.048+0000 7f7c7b3d5640 1 --2- 172.21.15.148:0/1838247887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c740ba030 0x7f7c7414f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:38.048+0000 7f7c7b3d5640 1 -- 172.21.15.148:0/1838247887 >> 172.21.15.148:0/1838247887 conn(0x7f7c740b6210 msgr2=0x7f7c740b78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:38.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:38.048+0000 7f7c7b3d5640 1 -- 172.21.15.148:0/1838247887 shutdown_connections 2024-03-29T04:11:38.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:38.048+0000 7f7c7b3d5640 1 -- 172.21.15.148:0/1838247887 wait complete. 2024-03-29T04:11:38.037 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2024-03-29T04:11:37.901227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":140,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237572,"kb_used_data":4744,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737148,"statfs":{"total":773094113280,"available":772850839552,"internally_reserved":0,"allocated":4857856,"data_stored":2751372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001039"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.608970+0000","last_change":"2024-03-29T04:11:33.601967+0000","last_active":"2024-03-29T04:11:34.608970+0000","last_peered":"2024-03-29T04:11:34.608970+0000","last_clean":"2024-03-29T04:11:34.608970+0000","last_became_active":"2024-03-29T04:11:33.601685+0000","last_became_peered":"2024-03-29T04:11:33.601685+0000","last_unstale":"2024-03-29T04:11:34.608970+0000","last_undegraded":"2024-03-29T04:11:34.608970+0000","last_fullsized":"2024-03-29T04:11:34.608970+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:56:33.700043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.608594+0000","last_change":"2024-03-29T04:11:33.607738+0000","last_active":"2024-03-29T04:11:34.608594+0000","last_peered":"2024-03-29T04:11:34.608594+0000","last_clean":"2024-03-29T04:11:34.608594+0000","last_became_active":"2024-03-29T04:11:33.607354+0000","last_became_peered":"2024-03-29T04:11:33.607354+0000","last_unstale":"2024-03-29T04:11:34.608594+0000","last_undegraded":"2024-03-29T04:11:34.608594+0000","last_fullsized":"2024-03-29T04:11:34.608594+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:29:27.741744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.257662+0000","last_change":"2024-03-29T04:11:33.606837+0000","last_active":"2024-03-29T04:11:35.257662+0000","last_peered":"2024-03-29T04:11:35.257662+0000","last_clean":"2024-03-29T04:11:35.257662+0000","last_became_active":"2024-03-29T04:11:33.606578+0000","last_became_peered":"2024-03-29T04:11:33.606578+0000","last_unstale":"2024-03-29T04:11:35.257662+0000","last_undegraded":"2024-03-29T04:11:35.257662+0000","last_fullsized":"2024-03-29T04:11:35.257662+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:01:09.652503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.594452+0000","last_change":"2024-03-29T04:11:33.596572+0000","last_active":"2024-03-29T04:11:35.594452+0000","last_peered":"2024-03-29T04:11:35.594452+0000","last_clean":"2024-03-29T04:11:35.594452+0000","last_became_active":"2024-03-29T04:11:33.596233+0000","last_became_peered":"2024-03-29T04:11:33.596233+0000","last_unstale":"2024-03-29T04:11:35.594452+0000","last_undegraded":"2024-03-29T04:11:35.594452+0000","last_fullsized":"2024-03-29T04:11:35.594452+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:17:49.196438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.594319+0000","last_change":"2024-03-29T04:11:33.597215+0000","last_active":"2024-03-29T04:11:35.594319+0000","last_peered":"2024-03-29T04:11:35.594319+0000","last_clean":"2024-03-29T04:11:35.594319+0000","last_became_active":"2024-03-29T04:11:33.596936+0000","last_became_peered":"2024-03-29T04:11:33.596936+0000","last_unstale":"2024-03-29T04:11:35.594319+0000","last_undegraded":"2024-03-29T04:11:35.594319+0000","last_fullsized":"2024-03-29T04:11:35.594319+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:18:52.875329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.753302+0000","last_change":"2024-03-29T04:11:33.608169+0000","last_active":"2024-03-29T04:11:35.753302+0000","last_peered":"2024-03-29T04:11:35.753302+0000","last_clean":"2024-03-29T04:11:35.753302+0000","last_became_active":"2024-03-29T04:11:33.607890+0000","last_became_peered":"2024-03-29T04:11:33.607890+0000","last_unstale":"2024-03-29T04:11:35.753302+0000","last_undegraded":"2024-03-29T04:11:35.753302+0000","last_fullsized":"2024-03-29T04:11:35.753302+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:25:50.154821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.605547+0000","last_change":"2024-03-29T04:11:33.608074+0000","last_active":"2024-03-29T04:11:34.605547+0000","last_peered":"2024-03-29T04:11:34.605547+0000","last_clean":"2024-03-29T04:11:34.605547+0000","last_became_active":"2024-03-29T04:11:33.607538+0000","last_became_peered":"2024-03-29T04:11:33.607538+0000","last_unstale":"2024-03-29T04:11:34.605547+0000","last_undegraded":"2024-03-29T04:11:34.605547+0000","last_fullsized":"2024-03-29T04:11:34.605547+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:36:56.199586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.608351+0000","last_change":"2024-03-29T04:11:33.608747+0000","last_active":"2024-03-29T04:11:34.608351+0000","last_peered":"2024-03-29T04:11:34.608351+0000","last_clean":"2024-03-29T04:11:34.608351+0000","last_became_active":"2024-03-29T04:11:33.608459+0000","last_became_peered":"2024-03-29T04:11:33.608459+0000","last_unstale":"2024-03-29T04:11:34.608351+0000","last_undegraded":"2024-03-29T04:11:34.608351+0000","last_fullsized":"2024-03-29T04:11:34.608351+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:36:56.199586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.257834+0000","last_change":"2024-03-29T04:11:33.607342+0000","last_active":"2024-03-29T04:11:35.257834+0000","last_peered":"2024-03-29T04:11:35.257834+0000","last_clean":"2024-03-29T04:11:35.257834+0000","last_became_active":"2024-03-29T04:11:33.607095+0000","last_became_peered":"2024-03-29T04:11:33.607095+0000","last_unstale":"2024-03-29T04:11:35.257834+0000","last_undegraded":"2024-03-29T04:11:35.257834+0000","last_fullsized":"2024-03-29T04:11:35.257834+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:25:50.154821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.605840+0000","last_change":"2024-03-29T04:11:33.601045+0000","last_active":"2024-03-29T04:11:34.605840+0000","last_peered":"2024-03-29T04:11:34.605840+0000","last_clean":"2024-03-29T04:11:34.605840+0000","last_became_active":"2024-03-29T04:11:33.600739+0000","last_became_peered":"2024-03-29T04:11:33.600739+0000","last_unstale":"2024-03-29T04:11:34.605840+0000","last_undegraded":"2024-03-29T04:11:34.605840+0000","last_fullsized":"2024-03-29T04:11:34.605840+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:28:05.628657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606307+0000","last_change":"2024-03-29T04:11:33.605296+0000","last_active":"2024-03-29T04:11:34.606307+0000","last_peered":"2024-03-29T04:11:34.606307+0000","last_clean":"2024-03-29T04:11:34.606307+0000","last_became_active":"2024-03-29T04:11:33.605049+0000","last_became_peered":"2024-03-29T04:11:33.605049+0000","last_unstale":"2024-03-29T04:11:34.606307+0000","last_undegraded":"2024-03-29T04:11:34.606307+0000","last_fullsized":"2024-03-29T04:11:34.606307+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:35:44.357464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.257615+0000","last_change":"2024-03-29T04:11:33.606364+0000","last_active":"2024-03-29T04:11:35.257615+0000","last_peered":"2024-03-29T04:11:35.257615+0000","last_clean":"2024-03-29T04:11:35.257615+0000","last_became_active":"2024-03-29T04:11:33.606079+0000","last_became_peered":"2024-03-29T04:11:33.606079+0000","last_unstale":"2024-03-29T04:11:35.257615+0000","last_undegraded":"2024-03-29T04:11:35.257615+0000","last_fullsized":"2024-03-29T04:11:35.257615+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:02:25.169625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.605979+0000","last_change":"2024-03-29T04:11:33.610191+0000","last_active":"2024-03-29T04:11:34.605979+0000","last_peered":"2024-03-29T04:11:34.605979+0000","last_clean":"2024-03-29T04:11:34.605979+0000","last_became_active":"2024-03-29T04:11:33.609928+0000","last_became_peered":"2024-03-29T04:11:33.609928+0000","last_unstale":"2024-03-29T04:11:34.605979+0000","last_undegraded":"2024-03-29T04:11:34.605979+0000","last_fullsized":"2024-03-29T04:11:34.605979+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:29:27.741744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.605548+0000","last_change":"2024-03-29T04:11:33.596488+0000","last_active":"2024-03-29T04:11:34.605548+0000","last_peered":"2024-03-29T04:11:34.605548+0000","last_clean":"2024-03-29T04:11:34.605548+0000","last_became_active":"2024-03-29T04:11:33.595911+0000","last_became_peered":"2024-03-29T04:11:33.595911+0000","last_unstale":"2024-03-29T04:11:34.605548+0000","last_undegraded":"2024-03-29T04:11:34.605548+0000","last_fullsized":"2024-03-29T04:11:34.605548+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:29:32.925022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606216+0000","last_change":"2024-03-29T04:11:33.610876+0000","last_active":"2024-03-29T04:11:34.606216+0000","last_peered":"2024-03-29T04:11:34.606216+0000","last_clean":"2024-03-29T04:11:34.606216+0000","last_became_active":"2024-03-29T04:11:33.610588+0000","last_became_peered":"2024-03-29T04:11:33.610588+0000","last_unstale":"2024-03-29T04:11:34.606216+0000","last_undegraded":"2024-03-29T04:11:34.606216+0000","last_fullsized":"2024-03-29T04:11:34.606216+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:35:25.923249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254149+0000","last_change":"2024-03-29T04:11:33.604516+0000","last_active":"2024-03-29T04:11:36.254149+0000","last_peered":"2024-03-29T04:11:36.254149+0000","last_clean":"2024-03-29T04:11:36.254149+0000","last_became_active":"2024-03-29T04:11:33.604214+0000","last_became_peered":"2024-03-29T04:11:33.604214+0000","last_unstale":"2024-03-29T04:11:36.254149+0000","last_undegraded":"2024-03-29T04:11:36.254149+0000","last_fullsized":"2024-03-29T04:11:36.254149+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:59.417284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.255216+0000","last_change":"2024-03-29T04:11:33.605235+0000","last_active":"2024-03-29T04:11:36.255216+0000","last_peered":"2024-03-29T04:11:36.255216+0000","last_clean":"2024-03-29T04:11:36.255216+0000","last_became_active":"2024-03-29T04:11:33.604942+0000","last_became_peered":"2024-03-29T04:11:33.604942+0000","last_unstale":"2024-03-29T04:11:36.255216+0000","last_undegraded":"2024-03-29T04:11:36.255216+0000","last_fullsized":"2024-03-29T04:11:36.255216+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:29:27.741744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254748+0000","last_change":"2024-03-29T04:11:33.594629+0000","last_active":"2024-03-29T04:11:36.254748+0000","last_peered":"2024-03-29T04:11:36.254748+0000","last_clean":"2024-03-29T04:11:36.254748+0000","last_became_active":"2024-03-29T04:11:33.594319+0000","last_became_peered":"2024-03-29T04:11:33.594319+0000","last_unstale":"2024-03-29T04:11:36.254748+0000","last_undegraded":"2024-03-29T04:11:36.254748+0000","last_fullsized":"2024-03-29T04:11:36.254748+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:31:43.545143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.594477+0000","last_change":"2024-03-29T04:11:33.609128+0000","last_active":"2024-03-29T04:11:35.594477+0000","last_peered":"2024-03-29T04:11:35.594477+0000","last_clean":"2024-03-29T04:11:35.594477+0000","last_became_active":"2024-03-29T04:11:33.608867+0000","last_became_peered":"2024-03-29T04:11:33.608867+0000","last_unstale":"2024-03-29T04:11:35.594477+0000","last_undegraded":"2024-03-29T04:11:35.594477+0000","last_fullsized":"2024-03-29T04:11:35.594477+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:01:19.411576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254443+0000","last_change":"2024-03-29T04:11:33.595436+0000","last_active":"2024-03-29T04:11:36.254443+0000","last_peered":"2024-03-29T04:11:36.254443+0000","last_clean":"2024-03-29T04:11:36.254443+0000","last_became_active":"2024-03-29T04:11:33.595102+0000","last_became_peered":"2024-03-29T04:11:33.595102+0000","last_unstale":"2024-03-29T04:11:36.254443+0000","last_undegraded":"2024-03-29T04:11:36.254443+0000","last_fullsized":"2024-03-29T04:11:36.254443+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:22:19.094023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606115+0000","last_change":"2024-03-29T04:11:33.613521+0000","last_active":"2024-03-29T04:11:34.606115+0000","last_peered":"2024-03-29T04:11:34.606115+0000","last_clean":"2024-03-29T04:11:34.606115+0000","last_became_active":"2024-03-29T04:11:33.613254+0000","last_became_peered":"2024-03-29T04:11:33.613254+0000","last_unstale":"2024-03-29T04:11:34.606115+0000","last_undegraded":"2024-03-29T04:11:34.606115+0000","last_fullsized":"2024-03-29T04:11:34.606115+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:59:25.023691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.605619+0000","last_change":"2024-03-29T04:11:33.597653+0000","last_active":"2024-03-29T04:11:34.605619+0000","last_peered":"2024-03-29T04:11:34.605619+0000","last_clean":"2024-03-29T04:11:34.605619+0000","last_became_active":"2024-03-29T04:11:33.597358+0000","last_became_peered":"2024-03-29T04:11:33.597358+0000","last_unstale":"2024-03-29T04:11:34.605619+0000","last_undegraded":"2024-03-29T04:11:34.605619+0000","last_fullsized":"2024-03-29T04:11:34.605619+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:20:05.840636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254874+0000","last_change":"2024-03-29T04:11:33.606199+0000","last_active":"2024-03-29T04:11:36.254874+0000","last_peered":"2024-03-29T04:11:36.254874+0000","last_clean":"2024-03-29T04:11:36.254874+0000","last_became_active":"2024-03-29T04:11:33.605612+0000","last_became_peered":"2024-03-29T04:11:33.605612+0000","last_unstale":"2024-03-29T04:11:36.254874+0000","last_undegraded":"2024-03-29T04:11:36.254874+0000","last_fullsized":"2024-03-29T04:11:36.254874+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:11:06.528477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.594191+0000","last_change":"2024-03-29T04:11:33.600709+0000","last_active":"2024-03-29T04:11:35.594191+0000","last_peered":"2024-03-29T04:11:35.594191+0000","last_clean":"2024-03-29T04:11:35.594191+0000","last_became_active":"2024-03-29T04:11:33.600429+0000","last_became_peered":"2024-03-29T04:11:33.600429+0000","last_unstale":"2024-03-29T04:11:35.594191+0000","last_undegraded":"2024-03-29T04:11:35.594191+0000","last_fullsized":"2024-03-29T04:11:35.594191+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:13:05.923879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254684+0000","last_change":"2024-03-29T04:11:33.603241+0000","last_active":"2024-03-29T04:11:36.254684+0000","last_peered":"2024-03-29T04:11:36.254684+0000","last_clean":"2024-03-29T04:11:36.254684+0000","last_became_active":"2024-03-29T04:11:33.602697+0000","last_became_peered":"2024-03-29T04:11:33.602697+0000","last_unstale":"2024-03-29T04:11:36.254684+0000","last_undegraded":"2024-03-29T04:11:36.254684+0000","last_fullsized":"2024-03-29T04:11:36.254684+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:20:05.840636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.753394+0000","last_change":"2024-03-29T04:11:33.611316+0000","last_active":"2024-03-29T04:11:35.753394+0000","last_peered":"2024-03-29T04:11:35.753394+0000","last_clean":"2024-03-29T04:11:35.753394+0000","last_became_active":"2024-03-29T04:11:33.611022+0000","last_became_peered":"2024-03-29T04:11:33.611022+0000","last_unstale":"2024-03-29T04:11:35.753394+0000","last_undegraded":"2024-03-29T04:11:35.753394+0000","last_fullsized":"2024-03-29T04:11:35.753394+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:36:38.576501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254627+0000","last_change":"2024-03-29T04:11:33.600095+0000","last_active":"2024-03-29T04:11:36.254627+0000","last_peered":"2024-03-29T04:11:36.254627+0000","last_clean":"2024-03-29T04:11:36.254627+0000","last_became_active":"2024-03-29T04:11:33.594726+0000","last_became_peered":"2024-03-29T04:11:33.594726+0000","last_unstale":"2024-03-29T04:11:36.254627+0000","last_undegraded":"2024-03-29T04:11:36.254627+0000","last_fullsized":"2024-03-29T04:11:36.254627+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:29:32.925022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606529+0000","last_change":"2024-03-29T04:11:33.608301+0000","last_active":"2024-03-29T04:11:34.606529+0000","last_peered":"2024-03-29T04:11:34.606529+0000","last_clean":"2024-03-29T04:11:34.606529+0000","last_became_active":"2024-03-29T04:11:33.607462+0000","last_became_peered":"2024-03-29T04:11:33.607462+0000","last_unstale":"2024-03-29T04:11:34.606529+0000","last_undegraded":"2024-03-29T04:11:34.606529+0000","last_fullsized":"2024-03-29T04:11:34.606529+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:56:33.700043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254480+0000","last_change":"2024-03-29T04:11:33.613665+0000","last_active":"2024-03-29T04:11:36.254480+0000","last_peered":"2024-03-29T04:11:36.254480+0000","last_clean":"2024-03-29T04:11:36.254480+0000","last_became_active":"2024-03-29T04:11:33.613347+0000","last_became_peered":"2024-03-29T04:11:33.613347+0000","last_unstale":"2024-03-29T04:11:36.254480+0000","last_undegraded":"2024-03-29T04:11:36.254480+0000","last_fullsized":"2024-03-29T04:11:36.254480+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:44:30.880135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254350+0000","last_change":"2024-03-29T04:11:33.602018+0000","last_active":"2024-03-29T04:11:36.254350+0000","last_peered":"2024-03-29T04:11:36.254350+0000","last_clean":"2024-03-29T04:11:36.254350+0000","last_became_active":"2024-03-29T04:11:33.601071+0000","last_became_peered":"2024-03-29T04:11:33.601071+0000","last_unstale":"2024-03-29T04:11:36.254350+0000","last_undegraded":"2024-03-29T04:11:36.254350+0000","last_fullsized":"2024-03-29T04:11:36.254350+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:01:41.102786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254972+0000","last_change":"2024-03-29T04:11:33.606461+0000","last_active":"2024-03-29T04:11:36.254972+0000","last_peered":"2024-03-29T04:11:36.254972+0000","last_clean":"2024-03-29T04:11:36.254972+0000","last_became_active":"2024-03-29T04:11:33.605924+0000","last_became_peered":"2024-03-29T04:11:33.605924+0000","last_unstale":"2024-03-29T04:11:36.254972+0000","last_undegraded":"2024-03-29T04:11:36.254972+0000","last_fullsized":"2024-03-29T04:11:36.254972+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:12:14.577653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.753086+0000","last_change":"2024-03-29T04:11:33.608693+0000","last_active":"2024-03-29T04:11:35.753086+0000","last_peered":"2024-03-29T04:11:35.753086+0000","last_clean":"2024-03-29T04:11:35.753086+0000","last_became_active":"2024-03-29T04:11:33.608249+0000","last_became_peered":"2024-03-29T04:11:33.608249+0000","last_unstale":"2024-03-29T04:11:35.753086+0000","last_undegraded":"2024-03-29T04:11:35.753086+0000","last_fullsized":"2024-03-29T04:11:35.753086+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:29:04.401595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254948+0000","last_change":"2024-03-29T04:11:33.609887+0000","last_active":"2024-03-29T04:11:36.254948+0000","last_peered":"2024-03-29T04:11:36.254948+0000","last_clean":"2024-03-29T04:11:36.254948+0000","last_became_active":"2024-03-29T04:11:33.609619+0000","last_became_peered":"2024-03-29T04:11:33.609619+0000","last_unstale":"2024-03-29T04:11:36.254948+0000","last_undegraded":"2024-03-29T04:11:36.254948+0000","last_fullsized":"2024-03-29T04:11:36.254948+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:36:56.199586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254749+0000","last_change":"2024-03-29T04:11:32.605430+0000","last_active":"2024-03-29T04:11:36.254749+0000","last_peered":"2024-03-29T04:11:36.254749+0000","last_clean":"2024-03-29T04:11:36.254749+0000","last_became_active":"2024-03-29T04:11:32.605121+0000","last_became_peered":"2024-03-29T04:11:32.605121+0000","last_unstale":"2024-03-29T04:11:36.254749+0000","last_undegraded":"2024-03-29T04:11:36.254749+0000","last_fullsized":"2024-03-29T04:11:36.254749+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:01:18.409399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.753288+0000","last_change":"2024-03-29T04:11:33.611709+0000","last_active":"2024-03-29T04:11:35.753288+0000","last_peered":"2024-03-29T04:11:35.753288+0000","last_clean":"2024-03-29T04:11:35.753288+0000","last_became_active":"2024-03-29T04:11:33.611007+0000","last_became_peered":"2024-03-29T04:11:33.611007+0000","last_unstale":"2024-03-29T04:11:35.753288+0000","last_undegraded":"2024-03-29T04:11:35.753288+0000","last_fullsized":"2024-03-29T04:11:35.753288+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:59.417284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.257857+0000","last_change":"2024-03-29T04:11:32.600987+0000","last_active":"2024-03-29T04:11:35.257857+0000","last_peered":"2024-03-29T04:11:35.257857+0000","last_clean":"2024-03-29T04:11:35.257857+0000","last_became_active":"2024-03-29T04:11:32.600724+0000","last_became_peered":"2024-03-29T04:11:32.600724+0000","last_unstale":"2024-03-29T04:11:35.257857+0000","last_undegraded":"2024-03-29T04:11:35.257857+0000","last_fullsized":"2024-03-29T04:11:35.257857+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:32:39.355355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606170+0000","last_change":"2024-03-29T04:11:33.603227+0000","last_active":"2024-03-29T04:11:34.606170+0000","last_peered":"2024-03-29T04:11:34.606170+0000","last_clean":"2024-03-29T04:11:34.606170+0000","last_became_active":"2024-03-29T04:11:33.602949+0000","last_became_peered":"2024-03-29T04:11:33.602949+0000","last_unstale":"2024-03-29T04:11:34.606170+0000","last_undegraded":"2024-03-29T04:11:34.606170+0000","last_fullsized":"2024-03-29T04:11:34.606170+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:01:09.652503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.753341+0000","last_change":"2024-03-29T04:11:32.603767+0000","last_active":"2024-03-29T04:11:35.753341+0000","last_peered":"2024-03-29T04:11:35.753341+0000","last_clean":"2024-03-29T04:11:35.753341+0000","last_became_active":"2024-03-29T04:11:32.603505+0000","last_became_peered":"2024-03-29T04:11:32.603505+0000","last_unstale":"2024-03-29T04:11:35.753341+0000","last_undegraded":"2024-03-29T04:11:35.753341+0000","last_fullsized":"2024-03-29T04:11:35.753341+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:59:22.058596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.608853+0000","last_change":"2024-03-29T04:11:33.612683+0000","last_active":"2024-03-29T04:11:34.608853+0000","last_peered":"2024-03-29T04:11:34.608853+0000","last_clean":"2024-03-29T04:11:34.608853+0000","last_became_active":"2024-03-29T04:11:33.612368+0000","last_became_peered":"2024-03-29T04:11:33.612368+0000","last_unstale":"2024-03-29T04:11:34.608853+0000","last_undegraded":"2024-03-29T04:11:34.608853+0000","last_fullsized":"2024-03-29T04:11:34.608853+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:49:11.854530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.605447+0000","last_change":"2024-03-29T04:11:32.601060+0000","last_active":"2024-03-29T04:11:34.605447+0000","last_peered":"2024-03-29T04:11:34.605447+0000","last_clean":"2024-03-29T04:11:34.605447+0000","last_became_active":"2024-03-29T04:11:32.600832+0000","last_became_peered":"2024-03-29T04:11:32.600832+0000","last_unstale":"2024-03-29T04:11:34.605447+0000","last_undegraded":"2024-03-29T04:11:34.605447+0000","last_fullsized":"2024-03-29T04:11:34.605447+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:17:48.194261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.257657+0000","last_change":"2024-03-29T04:11:33.606101+0000","last_active":"2024-03-29T04:11:35.257657+0000","last_peered":"2024-03-29T04:11:35.257657+0000","last_clean":"2024-03-29T04:11:35.257657+0000","last_became_active":"2024-03-29T04:11:33.605416+0000","last_became_peered":"2024-03-29T04:11:33.605416+0000","last_unstale":"2024-03-29T04:11:35.257657+0000","last_undegraded":"2024-03-29T04:11:35.257657+0000","last_fullsized":"2024-03-29T04:11:35.257657+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:29:04.401595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606169+0000","last_change":"2024-03-29T04:11:32.598199+0000","last_active":"2024-03-29T04:11:34.606169+0000","last_peered":"2024-03-29T04:11:34.606169+0000","last_clean":"2024-03-29T04:11:34.606169+0000","last_became_active":"2024-03-29T04:11:32.597920+0000","last_became_peered":"2024-03-29T04:11:32.597920+0000","last_unstale":"2024-03-29T04:11:34.606169+0000","last_undegraded":"2024-03-29T04:11:34.606169+0000","last_fullsized":"2024-03-29T04:11:34.606169+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:22:18.091846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.605732+0000","last_change":"2024-03-29T04:11:33.615618+0000","last_active":"2024-03-29T04:11:34.605732+0000","last_peered":"2024-03-29T04:11:34.605732+0000","last_clean":"2024-03-29T04:11:34.605732+0000","last_became_active":"2024-03-29T04:11:33.615302+0000","last_became_peered":"2024-03-29T04:11:33.615302+0000","last_unstale":"2024-03-29T04:11:34.605732+0000","last_undegraded":"2024-03-29T04:11:34.605732+0000","last_fullsized":"2024-03-29T04:11:34.605732+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:12:14.577653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254568+0000","last_change":"2024-03-29T04:11:32.606202+0000","last_active":"2024-03-29T04:11:36.254568+0000","last_peered":"2024-03-29T04:11:36.254568+0000","last_clean":"2024-03-29T04:11:36.254568+0000","last_became_active":"2024-03-29T04:11:32.605956+0000","last_became_peered":"2024-03-29T04:11:32.605956+0000","last_unstale":"2024-03-29T04:11:36.254568+0000","last_undegraded":"2024-03-29T04:11:36.254568+0000","last_fullsized":"2024-03-29T04:11:36.254568+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:59:22.058596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.605711+0000","last_change":"2024-03-29T04:11:33.606680+0000","last_active":"2024-03-29T04:11:34.605711+0000","last_peered":"2024-03-29T04:11:34.605711+0000","last_clean":"2024-03-29T04:11:34.605711+0000","last_became_active":"2024-03-29T04:11:33.606425+0000","last_became_peered":"2024-03-29T04:11:33.606425+0000","last_unstale":"2024-03-29T04:11:34.605711+0000","last_undegraded":"2024-03-29T04:11:34.605711+0000","last_fullsized":"2024-03-29T04:11:34.605711+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:31:43.545143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.753034+0000","last_change":"2024-03-29T04:11:32.600094+0000","last_active":"2024-03-29T04:11:35.753034+0000","last_peered":"2024-03-29T04:11:35.753034+0000","last_clean":"2024-03-29T04:11:35.753034+0000","last_became_active":"2024-03-29T04:11:32.599709+0000","last_became_peered":"2024-03-29T04:11:32.599709+0000","last_unstale":"2024-03-29T04:11:35.753034+0000","last_undegraded":"2024-03-29T04:11:35.753034+0000","last_fullsized":"2024-03-29T04:11:35.753034+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:01:36.484198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.608456+0000","last_change":"2024-03-29T04:11:33.608241+0000","last_active":"2024-03-29T04:11:34.608456+0000","last_peered":"2024-03-29T04:11:34.608456+0000","last_clean":"2024-03-29T04:11:34.608456+0000","last_became_active":"2024-03-29T04:11:33.607755+0000","last_became_peered":"2024-03-29T04:11:33.607755+0000","last_unstale":"2024-03-29T04:11:34.608456+0000","last_undegraded":"2024-03-29T04:11:34.608456+0000","last_fullsized":"2024-03-29T04:11:34.608456+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:12:14.577653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.605080+0000","last_change":"2024-03-29T04:11:32.598819+0000","last_active":"2024-03-29T04:11:34.605080+0000","last_peered":"2024-03-29T04:11:34.605080+0000","last_clean":"2024-03-29T04:11:34.605080+0000","last_became_active":"2024-03-29T04:11:32.597774+0000","last_became_peered":"2024-03-29T04:11:32.597774+0000","last_unstale":"2024-03-29T04:11:34.605080+0000","last_undegraded":"2024-03-29T04:11:34.605080+0000","last_fullsized":"2024-03-29T04:11:34.605080+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:32:39.355355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.594486+0000","last_change":"2024-03-29T04:11:33.600732+0000","last_active":"2024-03-29T04:11:35.594486+0000","last_peered":"2024-03-29T04:11:35.594486+0000","last_clean":"2024-03-29T04:11:35.594486+0000","last_became_active":"2024-03-29T04:11:33.600472+0000","last_became_peered":"2024-03-29T04:11:33.600472+0000","last_unstale":"2024-03-29T04:11:35.594486+0000","last_undegraded":"2024-03-29T04:11:35.594486+0000","last_fullsized":"2024-03-29T04:11:35.594486+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:32:40.357532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.609013+0000","last_change":"2024-03-29T04:11:32.603200+0000","last_active":"2024-03-29T04:11:34.609013+0000","last_peered":"2024-03-29T04:11:34.609013+0000","last_clean":"2024-03-29T04:11:34.609013+0000","last_became_active":"2024-03-29T04:11:32.602912+0000","last_became_peered":"2024-03-29T04:11:32.602912+0000","last_unstale":"2024-03-29T04:11:34.609013+0000","last_undegraded":"2024-03-29T04:11:34.609013+0000","last_fullsized":"2024-03-29T04:11:34.609013+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:29:31.922845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.255118+0000","last_change":"2024-03-29T04:11:33.611218+0000","last_active":"2024-03-29T04:11:36.255118+0000","last_peered":"2024-03-29T04:11:36.255118+0000","last_clean":"2024-03-29T04:11:36.255118+0000","last_became_active":"2024-03-29T04:11:33.610963+0000","last_became_peered":"2024-03-29T04:11:33.610963+0000","last_unstale":"2024-03-29T04:11:36.255118+0000","last_undegraded":"2024-03-29T04:11:36.255118+0000","last_fullsized":"2024-03-29T04:11:36.255118+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:57:51.680570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606622+0000","last_change":"2024-03-29T04:11:32.603797+0000","last_active":"2024-03-29T04:11:34.606622+0000","last_peered":"2024-03-29T04:11:34.606622+0000","last_clean":"2024-03-29T04:11:34.606622+0000","last_became_active":"2024-03-29T04:11:32.603514+0000","last_became_peered":"2024-03-29T04:11:32.603514+0000","last_unstale":"2024-03-29T04:11:34.606622+0000","last_undegraded":"2024-03-29T04:11:34.606622+0000","last_fullsized":"2024-03-29T04:11:34.606622+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:01:40.100609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.257863+0000","last_change":"2024-03-29T04:11:33.604476+0000","last_active":"2024-03-29T04:11:35.257863+0000","last_peered":"2024-03-29T04:11:35.257863+0000","last_clean":"2024-03-29T04:11:35.257863+0000","last_became_active":"2024-03-29T04:11:33.604181+0000","last_became_peered":"2024-03-29T04:11:33.604181+0000","last_unstale":"2024-03-29T04:11:35.257863+0000","last_undegraded":"2024-03-29T04:11:35.257863+0000","last_fullsized":"2024-03-29T04:11:35.257863+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:36:38.576501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606500+0000","last_change":"2024-03-29T04:11:32.606176+0000","last_active":"2024-03-29T04:11:34.606500+0000","last_peered":"2024-03-29T04:11:34.606500+0000","last_clean":"2024-03-29T04:11:34.606500+0000","last_became_active":"2024-03-29T04:11:32.605915+0000","last_became_peered":"2024-03-29T04:11:32.605915+0000","last_unstale":"2024-03-29T04:11:34.606500+0000","last_undegraded":"2024-03-29T04:11:34.606500+0000","last_fullsized":"2024-03-29T04:11:34.606500+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:44:29.877958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606091+0000","last_change":"2024-03-29T04:11:33.596958+0000","last_active":"2024-03-29T04:11:34.606091+0000","last_peered":"2024-03-29T04:11:34.606091+0000","last_clean":"2024-03-29T04:11:34.606091+0000","last_became_active":"2024-03-29T04:11:33.596634+0000","last_became_peered":"2024-03-29T04:11:33.596634+0000","last_unstale":"2024-03-29T04:11:34.606091+0000","last_undegraded":"2024-03-29T04:11:34.606091+0000","last_fullsized":"2024-03-29T04:11:34.606091+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:02:25.169625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.608697+0000","last_change":"2024-03-29T04:11:32.607490+0000","last_active":"2024-03-29T04:11:34.608697+0000","last_peered":"2024-03-29T04:11:34.608697+0000","last_clean":"2024-03-29T04:11:34.608697+0000","last_became_active":"2024-03-29T04:11:32.607245+0000","last_became_peered":"2024-03-29T04:11:32.607245+0000","last_unstale":"2024-03-29T04:11:34.608697+0000","last_undegraded":"2024-03-29T04:11:34.608697+0000","last_fullsized":"2024-03-29T04:11:34.608697+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:44:29.877958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.257592+0000","last_change":"2024-03-29T04:11:33.606594+0000","last_active":"2024-03-29T04:11:35.257592+0000","last_peered":"2024-03-29T04:11:35.257592+0000","last_clean":"2024-03-29T04:11:35.257592+0000","last_became_active":"2024-03-29T04:11:33.605711+0000","last_became_peered":"2024-03-29T04:11:33.605711+0000","last_unstale":"2024-03-29T04:11:35.257592+0000","last_undegraded":"2024-03-29T04:11:35.257592+0000","last_fullsized":"2024-03-29T04:11:35.257592+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:31:43.545143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.608584+0000","last_change":"2024-03-29T04:11:32.599838+0000","last_active":"2024-03-29T04:11:34.608584+0000","last_peered":"2024-03-29T04:11:34.608584+0000","last_clean":"2024-03-29T04:11:34.608584+0000","last_became_active":"2024-03-29T04:11:32.599585+0000","last_became_peered":"2024-03-29T04:11:32.599585+0000","last_unstale":"2024-03-29T04:11:34.608584+0000","last_undegraded":"2024-03-29T04:11:34.608584+0000","last_fullsized":"2024-03-29T04:11:34.608584+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:22:18.091846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.608458+0000","last_change":"2024-03-29T04:11:33.602106+0000","last_active":"2024-03-29T04:11:34.608458+0000","last_peered":"2024-03-29T04:11:34.608458+0000","last_clean":"2024-03-29T04:11:34.608458+0000","last_became_active":"2024-03-29T04:11:33.601852+0000","last_became_peered":"2024-03-29T04:11:33.601852+0000","last_unstale":"2024-03-29T04:11:34.608458+0000","last_undegraded":"2024-03-29T04:11:34.608458+0000","last_fullsized":"2024-03-29T04:11:34.608458+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:57:51.680570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.752988+0000","last_change":"2024-03-29T04:11:32.598043+0000","last_active":"2024-03-29T04:11:35.752988+0000","last_peered":"2024-03-29T04:11:35.752988+0000","last_clean":"2024-03-29T04:11:35.752988+0000","last_became_active":"2024-03-29T04:11:32.597779+0000","last_became_peered":"2024-03-29T04:11:32.597779+0000","last_unstale":"2024-03-29T04:11:35.752988+0000","last_undegraded":"2024-03-29T04:11:35.752988+0000","last_fullsized":"2024-03-29T04:11:35.752988+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:43:22.219049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254529+0000","last_change":"2024-03-29T04:11:33.611724+0000","last_active":"2024-03-29T04:11:36.254529+0000","last_peered":"2024-03-29T04:11:36.254529+0000","last_clean":"2024-03-29T04:11:36.254529+0000","last_became_active":"2024-03-29T04:11:33.611420+0000","last_became_peered":"2024-03-29T04:11:33.611420+0000","last_unstale":"2024-03-29T04:11:36.254529+0000","last_undegraded":"2024-03-29T04:11:36.254529+0000","last_fullsized":"2024-03-29T04:11:36.254529+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:36:38.576501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254102+0000","last_change":"2024-03-29T04:11:32.607645+0000","last_active":"2024-03-29T04:11:36.254102+0000","last_peered":"2024-03-29T04:11:36.254102+0000","last_clean":"2024-03-29T04:11:36.254102+0000","last_became_active":"2024-03-29T04:11:32.607341+0000","last_became_peered":"2024-03-29T04:11:32.607341+0000","last_unstale":"2024-03-29T04:11:36.254102+0000","last_undegraded":"2024-03-29T04:11:36.254102+0000","last_fullsized":"2024-03-29T04:11:36.254102+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:17:48.194261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.608534+0000","last_change":"2024-03-29T04:11:33.602606+0000","last_active":"2024-03-29T04:11:34.608534+0000","last_peered":"2024-03-29T04:11:34.608534+0000","last_clean":"2024-03-29T04:11:34.608534+0000","last_became_active":"2024-03-29T04:11:33.602344+0000","last_became_peered":"2024-03-29T04:11:33.602344+0000","last_unstale":"2024-03-29T04:11:34.608534+0000","last_undegraded":"2024-03-29T04:11:34.608534+0000","last_fullsized":"2024-03-29T04:11:34.608534+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:28:05.628657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254105+0000","last_change":"2024-03-29T04:11:32.606796+0000","last_active":"2024-03-29T04:11:36.254105+0000","last_peered":"2024-03-29T04:11:36.254105+0000","last_clean":"2024-03-29T04:11:36.254105+0000","last_became_active":"2024-03-29T04:11:32.606544+0000","last_became_peered":"2024-03-29T04:11:32.606544+0000","last_unstale":"2024-03-29T04:11:36.254105+0000","last_undegraded":"2024-03-29T04:11:36.254105+0000","last_fullsized":"2024-03-29T04:11:36.254105+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:01:28.999548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.608595+0000","last_change":"2024-03-29T04:11:33.612827+0000","last_active":"2024-03-29T04:11:34.608595+0000","last_peered":"2024-03-29T04:11:34.608595+0000","last_clean":"2024-03-29T04:11:34.608595+0000","last_became_active":"2024-03-29T04:11:33.612576+0000","last_became_peered":"2024-03-29T04:11:33.612576+0000","last_unstale":"2024-03-29T04:11:34.608595+0000","last_undegraded":"2024-03-29T04:11:34.608595+0000","last_fullsized":"2024-03-29T04:11:34.608595+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:59:25.023691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606758+0000","last_change":"2024-03-29T04:11:32.599893+0000","last_active":"2024-03-29T04:11:34.606758+0000","last_peered":"2024-03-29T04:11:34.606758+0000","last_clean":"2024-03-29T04:11:34.606758+0000","last_became_active":"2024-03-29T04:11:32.599635+0000","last_became_peered":"2024-03-29T04:11:32.599635+0000","last_unstale":"2024-03-29T04:11:34.606758+0000","last_undegraded":"2024-03-29T04:11:34.606758+0000","last_fullsized":"2024-03-29T04:11:34.606758+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:59:22.058596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606451+0000","last_change":"2024-03-29T04:11:33.614834+0000","last_active":"2024-03-29T04:11:34.606451+0000","last_peered":"2024-03-29T04:11:34.606451+0000","last_clean":"2024-03-29T04:11:34.606451+0000","last_became_active":"2024-03-29T04:11:33.614431+0000","last_became_peered":"2024-03-29T04:11:33.614431+0000","last_unstale":"2024-03-29T04:11:34.606451+0000","last_undegraded":"2024-03-29T04:11:34.606451+0000","last_fullsized":"2024-03-29T04:11:34.606451+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:49:11.854530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606207+0000","last_change":"2024-03-29T04:11:32.598560+0000","last_active":"2024-03-29T04:11:34.606207+0000","last_peered":"2024-03-29T04:11:34.606207+0000","last_clean":"2024-03-29T04:11:34.606207+0000","last_became_active":"2024-03-29T04:11:32.598189+0000","last_became_peered":"2024-03-29T04:11:32.598189+0000","last_unstale":"2024-03-29T04:11:34.606207+0000","last_undegraded":"2024-03-29T04:11:34.606207+0000","last_fullsized":"2024-03-29T04:11:34.606207+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:52:01.808081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606155+0000","last_change":"2024-03-29T04:11:33.604399+0000","last_active":"2024-03-29T04:11:34.606155+0000","last_peered":"2024-03-29T04:11:34.606155+0000","last_clean":"2024-03-29T04:11:34.606155+0000","last_became_active":"2024-03-29T04:11:33.603948+0000","last_became_peered":"2024-03-29T04:11:33.603948+0000","last_unstale":"2024-03-29T04:11:34.606155+0000","last_undegraded":"2024-03-29T04:11:34.606155+0000","last_fullsized":"2024-03-29T04:11:34.606155+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:29:04.401595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.594406+0000","last_change":"2024-03-29T04:11:32.603790+0000","last_active":"2024-03-29T04:11:35.594406+0000","last_peered":"2024-03-29T04:11:35.594406+0000","last_clean":"2024-03-29T04:11:35.594406+0000","last_became_active":"2024-03-29T04:11:32.603488+0000","last_became_peered":"2024-03-29T04:11:32.603488+0000","last_unstale":"2024-03-29T04:11:35.594406+0000","last_undegraded":"2024-03-29T04:11:35.594406+0000","last_fullsized":"2024-03-29T04:11:35.594406+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:47:38.689753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.594551+0000","last_change":"2024-03-29T04:11:33.603706+0000","last_active":"2024-03-29T04:11:35.594551+0000","last_peered":"2024-03-29T04:11:35.594551+0000","last_clean":"2024-03-29T04:11:35.594551+0000","last_became_active":"2024-03-29T04:11:33.603435+0000","last_became_peered":"2024-03-29T04:11:33.603435+0000","last_unstale":"2024-03-29T04:11:35.594551+0000","last_undegraded":"2024-03-29T04:11:35.594551+0000","last_fullsized":"2024-03-29T04:11:35.594551+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:52:02.810258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.753141+0000","last_change":"2024-03-29T04:11:32.605114+0000","last_active":"2024-03-29T04:11:35.753141+0000","last_peered":"2024-03-29T04:11:35.753141+0000","last_clean":"2024-03-29T04:11:35.753141+0000","last_became_active":"2024-03-29T04:11:32.604860+0000","last_became_peered":"2024-03-29T04:11:32.604860+0000","last_unstale":"2024-03-29T04:11:35.753141+0000","last_undegraded":"2024-03-29T04:11:35.753141+0000","last_fullsized":"2024-03-29T04:11:35.753141+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:01:28.999548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254350+0000","last_change":"2024-03-29T04:11:33.607041+0000","last_active":"2024-03-29T04:11:36.254350+0000","last_peered":"2024-03-29T04:11:36.254350+0000","last_clean":"2024-03-29T04:11:36.254350+0000","last_became_active":"2024-03-29T04:11:33.606778+0000","last_became_peered":"2024-03-29T04:11:33.606778+0000","last_unstale":"2024-03-29T04:11:36.254350+0000","last_undegraded":"2024-03-29T04:11:36.254350+0000","last_fullsized":"2024-03-29T04:11:36.254350+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:25:50.154821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.594235+0000","last_change":"2024-03-29T04:11:32.597119+0000","last_active":"2024-03-29T04:11:35.594235+0000","last_peered":"2024-03-29T04:11:35.594235+0000","last_clean":"2024-03-29T04:11:35.594235+0000","last_became_active":"2024-03-29T04:11:32.596839+0000","last_became_peered":"2024-03-29T04:11:32.596839+0000","last_unstale":"2024-03-29T04:11:35.594235+0000","last_undegraded":"2024-03-29T04:11:35.594235+0000","last_fullsized":"2024-03-29T04:11:35.594235+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:47:00.502578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.257592+0000","last_change":"2024-03-29T04:11:33.604753+0000","last_active":"2024-03-29T04:11:35.257592+0000","last_peered":"2024-03-29T04:11:35.257592+0000","last_clean":"2024-03-29T04:11:35.257592+0000","last_became_active":"2024-03-29T04:11:33.604213+0000","last_became_peered":"2024-03-29T04:11:33.604213+0000","last_unstale":"2024-03-29T04:11:35.257592+0000","last_undegraded":"2024-03-29T04:11:35.257592+0000","last_fullsized":"2024-03-29T04:11:35.257592+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:20:05.840636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.594257+0000","last_change":"2024-03-29T04:11:32.594337+0000","last_active":"2024-03-29T04:11:35.594257+0000","last_peered":"2024-03-29T04:11:35.594257+0000","last_clean":"2024-03-29T04:11:35.594257+0000","last_became_active":"2024-03-29T04:11:32.594053+0000","last_became_peered":"2024-03-29T04:11:32.594053+0000","last_unstale":"2024-03-29T04:11:35.594257+0000","last_undegraded":"2024-03-29T04:11:35.594257+0000","last_fullsized":"2024-03-29T04:11:35.594257+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:40:57.685920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.605804+0000","last_change":"2024-03-29T04:11:33.609219+0000","last_active":"2024-03-29T04:11:34.605804+0000","last_peered":"2024-03-29T04:11:34.605804+0000","last_clean":"2024-03-29T04:11:34.605804+0000","last_became_active":"2024-03-29T04:11:33.608958+0000","last_became_peered":"2024-03-29T04:11:33.608958+0000","last_unstale":"2024-03-29T04:11:34.605804+0000","last_undegraded":"2024-03-29T04:11:34.605804+0000","last_fullsized":"2024-03-29T04:11:34.605804+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:57:51.680570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254949+0000","last_change":"2024-03-29T04:11:32.603484+0000","last_active":"2024-03-29T04:11:36.254949+0000","last_peered":"2024-03-29T04:11:36.254949+0000","last_clean":"2024-03-29T04:11:36.254949+0000","last_became_active":"2024-03-29T04:11:32.603152+0000","last_became_peered":"2024-03-29T04:11:32.603152+0000","last_unstale":"2024-03-29T04:11:36.254949+0000","last_undegraded":"2024-03-29T04:11:36.254949+0000","last_fullsized":"2024-03-29T04:11:36.254949+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:43:22.219049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.608651+0000","last_change":"2024-03-29T04:11:33.608025+0000","last_active":"2024-03-29T04:11:34.608651+0000","last_peered":"2024-03-29T04:11:34.608651+0000","last_clean":"2024-03-29T04:11:34.608651+0000","last_became_active":"2024-03-29T04:11:33.607394+0000","last_became_peered":"2024-03-29T04:11:33.607394+0000","last_unstale":"2024-03-29T04:11:34.608651+0000","last_undegraded":"2024-03-29T04:11:34.608651+0000","last_fullsized":"2024-03-29T04:11:34.608651+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:35:25.923249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.594162+0000","last_change":"2024-03-29T04:11:32.597603+0000","last_active":"2024-03-29T04:11:35.594162+0000","last_peered":"2024-03-29T04:11:35.594162+0000","last_clean":"2024-03-29T04:11:35.594162+0000","last_became_active":"2024-03-29T04:11:32.597340+0000","last_became_peered":"2024-03-29T04:11:32.597340+0000","last_unstale":"2024-03-29T04:11:35.594162+0000","last_undegraded":"2024-03-29T04:11:35.594162+0000","last_fullsized":"2024-03-29T04:11:35.594162+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:08:17.331786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.753174+0000","last_change":"2024-03-29T04:11:33.610154+0000","last_active":"2024-03-29T04:11:35.753174+0000","last_peered":"2024-03-29T04:11:35.753174+0000","last_clean":"2024-03-29T04:11:35.753174+0000","last_became_active":"2024-03-29T04:11:33.609894+0000","last_became_peered":"2024-03-29T04:11:33.609894+0000","last_unstale":"2024-03-29T04:11:35.753174+0000","last_undegraded":"2024-03-29T04:11:35.753174+0000","last_fullsized":"2024-03-29T04:11:35.753174+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:01:09.652503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606812+0000","last_change":"2024-03-29T04:11:32.605443+0000","last_active":"2024-03-29T04:11:34.606812+0000","last_peered":"2024-03-29T04:11:34.606812+0000","last_clean":"2024-03-29T04:11:34.606812+0000","last_became_active":"2024-03-29T04:11:32.605033+0000","last_became_peered":"2024-03-29T04:11:32.605033+0000","last_unstale":"2024-03-29T04:11:34.606812+0000","last_undegraded":"2024-03-29T04:11:34.606812+0000","last_fullsized":"2024-03-29T04:11:34.606812+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:49:50.242921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.255195+0000","last_change":"2024-03-29T04:11:33.611683+0000","last_active":"2024-03-29T04:11:36.255195+0000","last_peered":"2024-03-29T04:11:36.255195+0000","last_clean":"2024-03-29T04:11:36.255195+0000","last_became_active":"2024-03-29T04:11:33.611234+0000","last_became_peered":"2024-03-29T04:11:33.611234+0000","last_unstale":"2024-03-29T04:11:36.255195+0000","last_undegraded":"2024-03-29T04:11:36.255195+0000","last_fullsized":"2024-03-29T04:11:36.255195+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:28:05.628657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.594469+0000","last_change":"2024-03-29T04:11:32.598263+0000","last_active":"2024-03-29T04:11:35.594469+0000","last_peered":"2024-03-29T04:11:35.594469+0000","last_clean":"2024-03-29T04:11:35.594469+0000","last_became_active":"2024-03-29T04:11:32.597985+0000","last_became_peered":"2024-03-29T04:11:32.597985+0000","last_unstale":"2024-03-29T04:11:35.594469+0000","last_undegraded":"2024-03-29T04:11:35.594469+0000","last_fullsized":"2024-03-29T04:11:35.594469+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:44:05.826219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.608660+0000","last_change":"2024-03-29T04:11:33.605426+0000","last_active":"2024-03-29T04:11:34.608660+0000","last_peered":"2024-03-29T04:11:34.608660+0000","last_clean":"2024-03-29T04:11:34.608660+0000","last_became_active":"2024-03-29T04:11:33.605173+0000","last_became_peered":"2024-03-29T04:11:33.605173+0000","last_unstale":"2024-03-29T04:11:34.608660+0000","last_undegraded":"2024-03-29T04:11:34.608660+0000","last_fullsized":"2024-03-29T04:11:34.608660+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:35:44.357464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.257674+0000","last_change":"2024-03-29T04:11:32.599052+0000","last_active":"2024-03-29T04:11:35.257674+0000","last_peered":"2024-03-29T04:11:35.257674+0000","last_clean":"2024-03-29T04:11:35.257674+0000","last_became_active":"2024-03-29T04:11:32.598775+0000","last_became_peered":"2024-03-29T04:11:32.598775+0000","last_unstale":"2024-03-29T04:11:35.257674+0000","last_undegraded":"2024-03-29T04:11:35.257674+0000","last_fullsized":"2024-03-29T04:11:35.257674+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:52:01.808081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.257771+0000","last_change":"2024-03-29T04:11:33.605650+0000","last_active":"2024-03-29T04:11:35.257771+0000","last_peered":"2024-03-29T04:11:35.257771+0000","last_clean":"2024-03-29T04:11:35.257771+0000","last_became_active":"2024-03-29T04:11:33.605098+0000","last_became_peered":"2024-03-29T04:11:33.605098+0000","last_unstale":"2024-03-29T04:11:35.257771+0000","last_undegraded":"2024-03-29T04:11:35.257771+0000","last_fullsized":"2024-03-29T04:11:35.257771+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:59.417284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.257742+0000","last_change":"2024-03-29T04:11:32.601769+0000","last_active":"2024-03-29T04:11:35.257742+0000","last_peered":"2024-03-29T04:11:35.257742+0000","last_clean":"2024-03-29T04:11:35.257742+0000","last_became_active":"2024-03-29T04:11:32.601320+0000","last_became_peered":"2024-03-29T04:11:32.601320+0000","last_unstale":"2024-03-29T04:11:35.257742+0000","last_undegraded":"2024-03-29T04:11:35.257742+0000","last_fullsized":"2024-03-29T04:11:35.257742+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:25.262311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.753075+0000","last_change":"2024-03-29T04:11:33.611575+0000","last_active":"2024-03-29T04:11:35.753075+0000","last_peered":"2024-03-29T04:11:35.753075+0000","last_clean":"2024-03-29T04:11:35.753075+0000","last_became_active":"2024-03-29T04:11:33.611286+0000","last_became_peered":"2024-03-29T04:11:33.611286+0000","last_unstale":"2024-03-29T04:11:35.753075+0000","last_undegraded":"2024-03-29T04:11:35.753075+0000","last_fullsized":"2024-03-29T04:11:35.753075+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:02:25.169625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254422+0000","last_change":"2024-03-29T04:11:32.603804+0000","last_active":"2024-03-29T04:11:36.254422+0000","last_peered":"2024-03-29T04:11:36.254422+0000","last_clean":"2024-03-29T04:11:36.254422+0000","last_became_active":"2024-03-29T04:11:32.603518+0000","last_became_peered":"2024-03-29T04:11:32.603518+0000","last_unstale":"2024-03-29T04:11:36.254422+0000","last_undegraded":"2024-03-29T04:11:36.254422+0000","last_fullsized":"2024-03-29T04:11:36.254422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:18:51.873151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.605375+0000","last_change":"2024-03-29T04:11:33.601327+0000","last_active":"2024-03-29T04:11:34.605375+0000","last_peered":"2024-03-29T04:11:34.605375+0000","last_clean":"2024-03-29T04:11:34.605375+0000","last_became_active":"2024-03-29T04:11:33.601071+0000","last_became_peered":"2024-03-29T04:11:33.601071+0000","last_unstale":"2024-03-29T04:11:34.605375+0000","last_undegraded":"2024-03-29T04:11:34.605375+0000","last_fullsized":"2024-03-29T04:11:34.605375+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:44:30.880135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.605300+0000","last_change":"2024-03-29T04:11:32.601521+0000","last_active":"2024-03-29T04:11:34.605300+0000","last_peered":"2024-03-29T04:11:34.605300+0000","last_clean":"2024-03-29T04:11:34.605300+0000","last_became_active":"2024-03-29T04:11:32.601232+0000","last_became_peered":"2024-03-29T04:11:32.601232+0000","last_unstale":"2024-03-29T04:11:34.605300+0000","last_undegraded":"2024-03-29T04:11:34.605300+0000","last_fullsized":"2024-03-29T04:11:34.605300+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:01:18.409399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254948+0000","last_change":"2024-03-29T04:11:33.607992+0000","last_active":"2024-03-29T04:11:36.254948+0000","last_peered":"2024-03-29T04:11:36.254948+0000","last_clean":"2024-03-29T04:11:36.254948+0000","last_became_active":"2024-03-29T04:11:33.607683+0000","last_became_peered":"2024-03-29T04:11:33.607683+0000","last_unstale":"2024-03-29T04:11:36.254948+0000","last_undegraded":"2024-03-29T04:11:36.254948+0000","last_fullsized":"2024-03-29T04:11:36.254948+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:05.626784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.608319+0000","last_change":"2024-03-29T04:11:32.599142+0000","last_active":"2024-03-29T04:11:34.608319+0000","last_peered":"2024-03-29T04:11:34.608319+0000","last_clean":"2024-03-29T04:11:34.608319+0000","last_became_active":"2024-03-29T04:11:32.598890+0000","last_became_peered":"2024-03-29T04:11:32.598890+0000","last_unstale":"2024-03-29T04:11:34.608319+0000","last_undegraded":"2024-03-29T04:11:34.608319+0000","last_fullsized":"2024-03-29T04:11:34.608319+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:01:40.100609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606721+0000","last_change":"2024-03-29T04:11:33.608225+0000","last_active":"2024-03-29T04:11:34.606721+0000","last_peered":"2024-03-29T04:11:34.606721+0000","last_clean":"2024-03-29T04:11:34.606721+0000","last_became_active":"2024-03-29T04:11:33.607456+0000","last_became_peered":"2024-03-29T04:11:33.607456+0000","last_unstale":"2024-03-29T04:11:34.606721+0000","last_undegraded":"2024-03-29T04:11:34.606721+0000","last_fullsized":"2024-03-29T04:11:34.606721+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:16:14.586985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254899+0000","last_change":"2024-03-29T04:11:32.603558+0000","last_active":"2024-03-29T04:11:36.254899+0000","last_peered":"2024-03-29T04:11:36.254899+0000","last_clean":"2024-03-29T04:11:36.254899+0000","last_became_active":"2024-03-29T04:11:32.603256+0000","last_became_peered":"2024-03-29T04:11:32.603256+0000","last_unstale":"2024-03-29T04:11:36.254899+0000","last_undegraded":"2024-03-29T04:11:36.254899+0000","last_fullsized":"2024-03-29T04:11:36.254899+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:01:36.484198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":207,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254229+0000","last_change":"2024-03-29T04:09:49.002570+0000","last_active":"2024-03-29T04:11:36.254229+0000","last_peered":"2024-03-29T04:11:36.254229+0000","last_clean":"2024-03-29T04:11:36.254229+0000","last_became_active":"2024-03-29T04:09:49.002235+0000","last_became_peered":"2024-03-29T04:09:49.002235+0000","last_unstale":"2024-03-29T04:11:36.254229+0000","last_undegraded":"2024-03-29T04:11:36.254229+0000","last_fullsized":"2024-03-29T04:11:36.254229+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:47.900733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:47.900733+0000","last_clean_scrub_stamp":"2024-03-29T04:09:47.900733+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:44:04.608265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1060864,"data_stored":787777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":196309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31632,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340208,"statfs":{"total":96636764160,"available":96604372992,"internally_reserved":0,"allocated":458752,"data_stored":195839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31632,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340208,"statfs":{"total":96636764160,"available":96604372992,"internally_reserved":0,"allocated":458752,"data_stored":196331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":466944,"data_stored":197373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":195797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":195817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1048576,"data_stored":786129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:11:41.044 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:11:41.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.171+0000 7f51bcd24640 1 Processor -- start 2024-03-29T04:11:41.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.171+0000 7f51bcd24640 1 -- start start 2024-03-29T04:11:41.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.172+0000 7f51bcd24640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51b815e3e0 0x7f51b80d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:41.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.172+0000 7f51bcd24640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f51b80d10d0 0x7f51b80d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:41.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.172+0000 7f51bcd24640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51b815e7c0 con 0x7f51b80ba030 2024-03-29T04:11:41.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.172+0000 7f51bcd24640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f51b80d3840 con 0x7f51b80d10d0 2024-03-29T04:11:41.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.172+0000 7f51bcd24640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f51b80d2c30 con 0x7f51b815e3e0 2024-03-29T04:11:41.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.172+0000 7f51b6d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f51b80d10d0 0x7f51b80d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:41.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.172+0000 7f51b6d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f51b80d10d0 0x7f51b80d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42356/0 (socket says 172.21.15.148:42356) 2024-03-29T04:11:41.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.172+0000 7f51b6d9d640 1 -- 172.21.15.148:0/451878271 learned_addr learned my addr 172.21.15.148:0/451878271 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:41.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.173+0000 7f51b659c640 1 --2- 172.21.15.148:0/451878271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51b815e3e0 0x7f51b80d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:41.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.173+0000 7f51b6d9d640 1 -- 172.21.15.148:0/451878271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51b815e3e0 msgr2=0x7f51b80d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:41.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.173+0000 7f51b6d9d640 1 --2- 172.21.15.148:0/451878271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51b815e3e0 0x7f51b80d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:41.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.173+0000 7f51b6d9d640 1 -- 172.21.15.148:0/451878271 >> v1:172.21.15.157:6789/0 conn(0x7f51b80ba030 legacy=0x7f51b80d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:41.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.173+0000 7f51b6d9d640 1 -- 172.21.15.148:0/451878271 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b80d2f10 con 0x7f51b80d10d0 2024-03-29T04:11:41.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.173+0000 7f51b6d9d640 1 --2- 172.21.15.148:0/451878271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51b80d10d0 0x7f51b80d1580 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f51a8009e50 tx=0x7f51a802bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ee2627085fc621 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:41.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.174+0000 7f51b559a640 1 -- 172.21.15.148:0/451878271 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a802cb90 con 0x7f51b80d10d0 2024-03-29T04:11:41.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.174+0000 7f51b559a640 1 -- 172.21.15.148:0/451878271 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51a802cd30 con 0x7f51b80d10d0 2024-03-29T04:11:41.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.174+0000 7f51b559a640 1 -- 172.21.15.148:0/451878271 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8034610 con 0x7f51b80d10d0 2024-03-29T04:11:41.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.174+0000 7f51bcd24640 1 -- 172.21.15.148:0/451878271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51b80d10d0 msgr2=0x7f51b80d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:41.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.174+0000 7f51bcd24640 1 --2- 172.21.15.148:0/451878271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51b80d10d0 0x7f51b80d1580 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f51a8009e50 tx=0x7f51a802bff0 comp rx=0 tx=0).stop 2024-03-29T04:11:41.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.174+0000 7f51bcd24640 1 -- 172.21.15.148:0/451878271 shutdown_connections 2024-03-29T04:11:41.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.174+0000 7f51bcd24640 1 --2- 172.21.15.148:0/451878271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51b80d10d0 0x7f51b80d1580 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:41.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.174+0000 7f51bcd24640 1 --2- 172.21.15.148:0/451878271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51b815e3e0 0x7f51b80d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:41.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.174+0000 7f51bcd24640 1 -- 172.21.15.148:0/451878271 >> 172.21.15.148:0/451878271 conn(0x7f51b80b6210 msgr2=0x7f51b80b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:41.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.174+0000 7f51bcd24640 1 -- 172.21.15.148:0/451878271 shutdown_connections 2024-03-29T04:11:41.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.174+0000 7f51bcd24640 1 -- 172.21.15.148:0/451878271 wait complete. 2024-03-29T04:11:41.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.176+0000 7f51bcd24640 1 Processor -- start 2024-03-29T04:11:41.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.176+0000 7f51bcd24640 1 -- start start 2024-03-29T04:11:41.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.176+0000 7f51bcd24640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51b80ba030 0x7f51b815e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:41.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.176+0000 7f51bcd24640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51b80d10d0 0x7f51b81f5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:41.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.176+0000 7f51bcd24640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51b815e3e0 0x7f51b8155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:41.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.176+0000 7f51bcd24640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f51b8167f50 con 0x7f51b815e3e0 2024-03-29T04:11:41.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.176+0000 7f51bcd24640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f51b80d34d0 con 0x7f51b80d10d0 2024-03-29T04:11:41.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.176+0000 7f51bcd24640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f51b81680d0 con 0x7f51b80ba030 2024-03-29T04:11:41.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.176+0000 7f51b6d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51b815e3e0 0x7f51b8155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:41.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.176+0000 7f51b6d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51b815e3e0 0x7f51b8155a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42368/0 (socket says 172.21.15.148:42368) 2024-03-29T04:11:41.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.176+0000 7f51b6d9d640 1 -- 172.21.15.148:0/87357263 learned_addr learned my addr 172.21.15.148:0/87357263 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:41.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.176+0000 7f51b659c640 1 --2- 172.21.15.148:0/87357263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51b80ba030 0x7f51b815e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:41.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.176+0000 7f51b5d9b640 1 --2- 172.21.15.148:0/87357263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51b80d10d0 0x7f51b81f5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:41.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.177+0000 7f51b6d9d640 1 -- 172.21.15.148:0/87357263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51b80ba030 msgr2=0x7f51b815e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:41.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.177+0000 7f51b6d9d640 1 --2- 172.21.15.148:0/87357263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51b80ba030 0x7f51b815e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:41.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.177+0000 7f51b6d9d640 1 -- 172.21.15.148:0/87357263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51b80d10d0 msgr2=0x7f51b81f5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:41.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.177+0000 7f51b6d9d640 1 --2- 172.21.15.148:0/87357263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51b80d10d0 0x7f51b81f5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:41.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.177+0000 7f51b6d9d640 1 -- 172.21.15.148:0/87357263 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b815f070 con 0x7f51b815e3e0 2024-03-29T04:11:41.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.177+0000 7f51b6d9d640 1 --2- 172.21.15.148:0/87357263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51b815e3e0 0x7f51b8155a90 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f51a8009e50 tx=0x7f51a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:41.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.177+0000 7f51a6ffd640 1 -- 172.21.15.148:0/87357263 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8043020 con 0x7f51b815e3e0 2024-03-29T04:11:41.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.177+0000 7f51a6ffd640 1 -- 172.21.15.148:0/87357263 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51a803e450 con 0x7f51b815e3e0 2024-03-29T04:11:41.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.177+0000 7f51bcd24640 1 -- 172.21.15.148:0/87357263 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b80c8810 con 0x7f51b815e3e0 2024-03-29T04:11:41.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.177+0000 7f51a6ffd640 1 -- 172.21.15.148:0/87357263 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8009510 con 0x7f51b815e3e0 2024-03-29T04:11:41.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.177+0000 7f51bcd24640 1 -- 172.21.15.148:0/87357263 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b80c8dd0 con 0x7f51b815e3e0 2024-03-29T04:11:41.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.178+0000 7f51bcd24640 1 -- 172.21.15.148:0/87357263 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51b80687e0 con 0x7f51b815e3e0 2024-03-29T04:11:41.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.178+0000 7f51a6ffd640 1 -- 172.21.15.148:0/87357263 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f51a8046020 con 0x7f51b815e3e0 2024-03-29T04:11:41.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.179+0000 7f51a6ffd640 1 --2- 172.21.15.148:0/87357263 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f519806c530 0x7f519806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:41.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.179+0000 7f51a6ffd640 1 -- 172.21.15.148:0/87357263 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f51a80addc0 con 0x7f51b815e3e0 2024-03-29T04:11:41.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.179+0000 7f51b659c640 1 --2- 172.21.15.148:0/87357263 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f519806c530 0x7f519806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:41.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.179+0000 7f51b659c640 1 --2- 172.21.15.148:0/87357263 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f519806c530 0x7f519806e9f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f51b8157eb0 tx=0x7f51b0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:41.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.183+0000 7f51a6ffd640 1 -- 172.21.15.148:0/87357263 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f51a804d030 con 0x7f51b815e3e0 2024-03-29T04:11:41.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.359+0000 7f51bcd24640 1 -- 172.21.15.148:0/87357263 --> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f51b80b6930 con 0x7f519806c530 2024-03-29T04:11:41.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.370+0000 7f51a6ffd640 1 -- 172.21.15.148:0/87357263 <== mgr.4113 v2:172.21.15.148:6832/3271445075 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251957 (secure 0 0 0) 0x7f51b80b6930 con 0x7f519806c530 2024-03-29T04:11:41.352 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:11:41.354 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-03-29T04:11:41.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.374+0000 7f51bcd24640 1 -- 172.21.15.148:0/87357263 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f519806c530 msgr2=0x7f519806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:41.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.374+0000 7f51bcd24640 1 --2- 172.21.15.148:0/87357263 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f519806c530 0x7f519806e9f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f51b8157eb0 tx=0x7f51b0008040 comp rx=0 tx=0).stop 2024-03-29T04:11:41.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.374+0000 7f51bcd24640 1 -- 172.21.15.148:0/87357263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51b815e3e0 msgr2=0x7f51b8155a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:41.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.374+0000 7f51bcd24640 1 --2- 172.21.15.148:0/87357263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51b815e3e0 0x7f51b8155a90 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f51a8009e50 tx=0x7f51a8004a40 comp rx=0 tx=0).stop 2024-03-29T04:11:41.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.374+0000 7f51bcd24640 1 -- 172.21.15.148:0/87357263 shutdown_connections 2024-03-29T04:11:41.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.374+0000 7f51bcd24640 1 --2- 172.21.15.148:0/87357263 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f519806c530 0x7f519806e9f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:41.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.374+0000 7f51bcd24640 1 --2- 172.21.15.148:0/87357263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51b815e3e0 0x7f51b8155a90 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:41.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.374+0000 7f51bcd24640 1 --2- 172.21.15.148:0/87357263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51b80d10d0 0x7f51b81f5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:41.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.374+0000 7f51bcd24640 1 --2- 172.21.15.148:0/87357263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51b80ba030 0x7f51b815e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:41.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.374+0000 7f51bcd24640 1 -- 172.21.15.148:0/87357263 >> 172.21.15.148:0/87357263 conn(0x7f51b80b6210 msgr2=0x7f51b815b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:41.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.374+0000 7f51bcd24640 1 -- 172.21.15.148:0/87357263 shutdown_connections 2024-03-29T04:11:41.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.374+0000 7f51bcd24640 1 -- 172.21.15.148:0/87357263 wait complete. 2024-03-29T04:11:41.363 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2024-03-29T04:11:39.901609+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":140,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237572,"kb_used_data":4744,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737148,"statfs":{"total":773094113280,"available":772850839552,"internally_reserved":0,"allocated":4857856,"data_stored":2751372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.315943"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.608970+0000","last_change":"2024-03-29T04:11:33.601967+0000","last_active":"2024-03-29T04:11:34.608970+0000","last_peered":"2024-03-29T04:11:34.608970+0000","last_clean":"2024-03-29T04:11:34.608970+0000","last_became_active":"2024-03-29T04:11:33.601685+0000","last_became_peered":"2024-03-29T04:11:33.601685+0000","last_unstale":"2024-03-29T04:11:34.608970+0000","last_undegraded":"2024-03-29T04:11:34.608970+0000","last_fullsized":"2024-03-29T04:11:34.608970+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:56:33.700043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.608594+0000","last_change":"2024-03-29T04:11:33.607738+0000","last_active":"2024-03-29T04:11:34.608594+0000","last_peered":"2024-03-29T04:11:34.608594+0000","last_clean":"2024-03-29T04:11:34.608594+0000","last_became_active":"2024-03-29T04:11:33.607354+0000","last_became_peered":"2024-03-29T04:11:33.607354+0000","last_unstale":"2024-03-29T04:11:34.608594+0000","last_undegraded":"2024-03-29T04:11:34.608594+0000","last_fullsized":"2024-03-29T04:11:34.608594+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:29:27.741744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.257662+0000","last_change":"2024-03-29T04:11:33.606837+0000","last_active":"2024-03-29T04:11:35.257662+0000","last_peered":"2024-03-29T04:11:35.257662+0000","last_clean":"2024-03-29T04:11:35.257662+0000","last_became_active":"2024-03-29T04:11:33.606578+0000","last_became_peered":"2024-03-29T04:11:33.606578+0000","last_unstale":"2024-03-29T04:11:35.257662+0000","last_undegraded":"2024-03-29T04:11:35.257662+0000","last_fullsized":"2024-03-29T04:11:35.257662+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:01:09.652503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.594452+0000","last_change":"2024-03-29T04:11:33.596572+0000","last_active":"2024-03-29T04:11:35.594452+0000","last_peered":"2024-03-29T04:11:35.594452+0000","last_clean":"2024-03-29T04:11:35.594452+0000","last_became_active":"2024-03-29T04:11:33.596233+0000","last_became_peered":"2024-03-29T04:11:33.596233+0000","last_unstale":"2024-03-29T04:11:35.594452+0000","last_undegraded":"2024-03-29T04:11:35.594452+0000","last_fullsized":"2024-03-29T04:11:35.594452+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:17:49.196438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.594319+0000","last_change":"2024-03-29T04:11:33.597215+0000","last_active":"2024-03-29T04:11:35.594319+0000","last_peered":"2024-03-29T04:11:35.594319+0000","last_clean":"2024-03-29T04:11:35.594319+0000","last_became_active":"2024-03-29T04:11:33.596936+0000","last_became_peered":"2024-03-29T04:11:33.596936+0000","last_unstale":"2024-03-29T04:11:35.594319+0000","last_undegraded":"2024-03-29T04:11:35.594319+0000","last_fullsized":"2024-03-29T04:11:35.594319+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:18:52.875329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.753302+0000","last_change":"2024-03-29T04:11:33.608169+0000","last_active":"2024-03-29T04:11:35.753302+0000","last_peered":"2024-03-29T04:11:35.753302+0000","last_clean":"2024-03-29T04:11:35.753302+0000","last_became_active":"2024-03-29T04:11:33.607890+0000","last_became_peered":"2024-03-29T04:11:33.607890+0000","last_unstale":"2024-03-29T04:11:35.753302+0000","last_undegraded":"2024-03-29T04:11:35.753302+0000","last_fullsized":"2024-03-29T04:11:35.753302+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:25:50.154821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.605547+0000","last_change":"2024-03-29T04:11:33.608074+0000","last_active":"2024-03-29T04:11:34.605547+0000","last_peered":"2024-03-29T04:11:34.605547+0000","last_clean":"2024-03-29T04:11:34.605547+0000","last_became_active":"2024-03-29T04:11:33.607538+0000","last_became_peered":"2024-03-29T04:11:33.607538+0000","last_unstale":"2024-03-29T04:11:34.605547+0000","last_undegraded":"2024-03-29T04:11:34.605547+0000","last_fullsized":"2024-03-29T04:11:34.605547+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:36:56.199586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.608351+0000","last_change":"2024-03-29T04:11:33.608747+0000","last_active":"2024-03-29T04:11:34.608351+0000","last_peered":"2024-03-29T04:11:34.608351+0000","last_clean":"2024-03-29T04:11:34.608351+0000","last_became_active":"2024-03-29T04:11:33.608459+0000","last_became_peered":"2024-03-29T04:11:33.608459+0000","last_unstale":"2024-03-29T04:11:34.608351+0000","last_undegraded":"2024-03-29T04:11:34.608351+0000","last_fullsized":"2024-03-29T04:11:34.608351+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:36:56.199586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.257834+0000","last_change":"2024-03-29T04:11:33.607342+0000","last_active":"2024-03-29T04:11:35.257834+0000","last_peered":"2024-03-29T04:11:35.257834+0000","last_clean":"2024-03-29T04:11:35.257834+0000","last_became_active":"2024-03-29T04:11:33.607095+0000","last_became_peered":"2024-03-29T04:11:33.607095+0000","last_unstale":"2024-03-29T04:11:35.257834+0000","last_undegraded":"2024-03-29T04:11:35.257834+0000","last_fullsized":"2024-03-29T04:11:35.257834+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:25:50.154821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.605840+0000","last_change":"2024-03-29T04:11:33.601045+0000","last_active":"2024-03-29T04:11:34.605840+0000","last_peered":"2024-03-29T04:11:34.605840+0000","last_clean":"2024-03-29T04:11:34.605840+0000","last_became_active":"2024-03-29T04:11:33.600739+0000","last_became_peered":"2024-03-29T04:11:33.600739+0000","last_unstale":"2024-03-29T04:11:34.605840+0000","last_undegraded":"2024-03-29T04:11:34.605840+0000","last_fullsized":"2024-03-29T04:11:34.605840+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:28:05.628657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606307+0000","last_change":"2024-03-29T04:11:33.605296+0000","last_active":"2024-03-29T04:11:34.606307+0000","last_peered":"2024-03-29T04:11:34.606307+0000","last_clean":"2024-03-29T04:11:34.606307+0000","last_became_active":"2024-03-29T04:11:33.605049+0000","last_became_peered":"2024-03-29T04:11:33.605049+0000","last_unstale":"2024-03-29T04:11:34.606307+0000","last_undegraded":"2024-03-29T04:11:34.606307+0000","last_fullsized":"2024-03-29T04:11:34.606307+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:35:44.357464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.257615+0000","last_change":"2024-03-29T04:11:33.606364+0000","last_active":"2024-03-29T04:11:35.257615+0000","last_peered":"2024-03-29T04:11:35.257615+0000","last_clean":"2024-03-29T04:11:35.257615+0000","last_became_active":"2024-03-29T04:11:33.606079+0000","last_became_peered":"2024-03-29T04:11:33.606079+0000","last_unstale":"2024-03-29T04:11:35.257615+0000","last_undegraded":"2024-03-29T04:11:35.257615+0000","last_fullsized":"2024-03-29T04:11:35.257615+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:02:25.169625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.605979+0000","last_change":"2024-03-29T04:11:33.610191+0000","last_active":"2024-03-29T04:11:34.605979+0000","last_peered":"2024-03-29T04:11:34.605979+0000","last_clean":"2024-03-29T04:11:34.605979+0000","last_became_active":"2024-03-29T04:11:33.609928+0000","last_became_peered":"2024-03-29T04:11:33.609928+0000","last_unstale":"2024-03-29T04:11:34.605979+0000","last_undegraded":"2024-03-29T04:11:34.605979+0000","last_fullsized":"2024-03-29T04:11:34.605979+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:29:27.741744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.605548+0000","last_change":"2024-03-29T04:11:33.596488+0000","last_active":"2024-03-29T04:11:34.605548+0000","last_peered":"2024-03-29T04:11:34.605548+0000","last_clean":"2024-03-29T04:11:34.605548+0000","last_became_active":"2024-03-29T04:11:33.595911+0000","last_became_peered":"2024-03-29T04:11:33.595911+0000","last_unstale":"2024-03-29T04:11:34.605548+0000","last_undegraded":"2024-03-29T04:11:34.605548+0000","last_fullsized":"2024-03-29T04:11:34.605548+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:29:32.925022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606216+0000","last_change":"2024-03-29T04:11:33.610876+0000","last_active":"2024-03-29T04:11:34.606216+0000","last_peered":"2024-03-29T04:11:34.606216+0000","last_clean":"2024-03-29T04:11:34.606216+0000","last_became_active":"2024-03-29T04:11:33.610588+0000","last_became_peered":"2024-03-29T04:11:33.610588+0000","last_unstale":"2024-03-29T04:11:34.606216+0000","last_undegraded":"2024-03-29T04:11:34.606216+0000","last_fullsized":"2024-03-29T04:11:34.606216+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:35:25.923249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254149+0000","last_change":"2024-03-29T04:11:33.604516+0000","last_active":"2024-03-29T04:11:36.254149+0000","last_peered":"2024-03-29T04:11:36.254149+0000","last_clean":"2024-03-29T04:11:36.254149+0000","last_became_active":"2024-03-29T04:11:33.604214+0000","last_became_peered":"2024-03-29T04:11:33.604214+0000","last_unstale":"2024-03-29T04:11:36.254149+0000","last_undegraded":"2024-03-29T04:11:36.254149+0000","last_fullsized":"2024-03-29T04:11:36.254149+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:59.417284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.255216+0000","last_change":"2024-03-29T04:11:33.605235+0000","last_active":"2024-03-29T04:11:36.255216+0000","last_peered":"2024-03-29T04:11:36.255216+0000","last_clean":"2024-03-29T04:11:36.255216+0000","last_became_active":"2024-03-29T04:11:33.604942+0000","last_became_peered":"2024-03-29T04:11:33.604942+0000","last_unstale":"2024-03-29T04:11:36.255216+0000","last_undegraded":"2024-03-29T04:11:36.255216+0000","last_fullsized":"2024-03-29T04:11:36.255216+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:29:27.741744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254748+0000","last_change":"2024-03-29T04:11:33.594629+0000","last_active":"2024-03-29T04:11:36.254748+0000","last_peered":"2024-03-29T04:11:36.254748+0000","last_clean":"2024-03-29T04:11:36.254748+0000","last_became_active":"2024-03-29T04:11:33.594319+0000","last_became_peered":"2024-03-29T04:11:33.594319+0000","last_unstale":"2024-03-29T04:11:36.254748+0000","last_undegraded":"2024-03-29T04:11:36.254748+0000","last_fullsized":"2024-03-29T04:11:36.254748+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:31:43.545143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.594477+0000","last_change":"2024-03-29T04:11:33.609128+0000","last_active":"2024-03-29T04:11:35.594477+0000","last_peered":"2024-03-29T04:11:35.594477+0000","last_clean":"2024-03-29T04:11:35.594477+0000","last_became_active":"2024-03-29T04:11:33.608867+0000","last_became_peered":"2024-03-29T04:11:33.608867+0000","last_unstale":"2024-03-29T04:11:35.594477+0000","last_undegraded":"2024-03-29T04:11:35.594477+0000","last_fullsized":"2024-03-29T04:11:35.594477+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:01:19.411576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254443+0000","last_change":"2024-03-29T04:11:33.595436+0000","last_active":"2024-03-29T04:11:36.254443+0000","last_peered":"2024-03-29T04:11:36.254443+0000","last_clean":"2024-03-29T04:11:36.254443+0000","last_became_active":"2024-03-29T04:11:33.595102+0000","last_became_peered":"2024-03-29T04:11:33.595102+0000","last_unstale":"2024-03-29T04:11:36.254443+0000","last_undegraded":"2024-03-29T04:11:36.254443+0000","last_fullsized":"2024-03-29T04:11:36.254443+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:22:19.094023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606115+0000","last_change":"2024-03-29T04:11:33.613521+0000","last_active":"2024-03-29T04:11:34.606115+0000","last_peered":"2024-03-29T04:11:34.606115+0000","last_clean":"2024-03-29T04:11:34.606115+0000","last_became_active":"2024-03-29T04:11:33.613254+0000","last_became_peered":"2024-03-29T04:11:33.613254+0000","last_unstale":"2024-03-29T04:11:34.606115+0000","last_undegraded":"2024-03-29T04:11:34.606115+0000","last_fullsized":"2024-03-29T04:11:34.606115+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:59:25.023691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.605619+0000","last_change":"2024-03-29T04:11:33.597653+0000","last_active":"2024-03-29T04:11:34.605619+0000","last_peered":"2024-03-29T04:11:34.605619+0000","last_clean":"2024-03-29T04:11:34.605619+0000","last_became_active":"2024-03-29T04:11:33.597358+0000","last_became_peered":"2024-03-29T04:11:33.597358+0000","last_unstale":"2024-03-29T04:11:34.605619+0000","last_undegraded":"2024-03-29T04:11:34.605619+0000","last_fullsized":"2024-03-29T04:11:34.605619+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:20:05.840636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254874+0000","last_change":"2024-03-29T04:11:33.606199+0000","last_active":"2024-03-29T04:11:36.254874+0000","last_peered":"2024-03-29T04:11:36.254874+0000","last_clean":"2024-03-29T04:11:36.254874+0000","last_became_active":"2024-03-29T04:11:33.605612+0000","last_became_peered":"2024-03-29T04:11:33.605612+0000","last_unstale":"2024-03-29T04:11:36.254874+0000","last_undegraded":"2024-03-29T04:11:36.254874+0000","last_fullsized":"2024-03-29T04:11:36.254874+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:11:06.528477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.594191+0000","last_change":"2024-03-29T04:11:33.600709+0000","last_active":"2024-03-29T04:11:35.594191+0000","last_peered":"2024-03-29T04:11:35.594191+0000","last_clean":"2024-03-29T04:11:35.594191+0000","last_became_active":"2024-03-29T04:11:33.600429+0000","last_became_peered":"2024-03-29T04:11:33.600429+0000","last_unstale":"2024-03-29T04:11:35.594191+0000","last_undegraded":"2024-03-29T04:11:35.594191+0000","last_fullsized":"2024-03-29T04:11:35.594191+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:13:05.923879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254684+0000","last_change":"2024-03-29T04:11:33.603241+0000","last_active":"2024-03-29T04:11:36.254684+0000","last_peered":"2024-03-29T04:11:36.254684+0000","last_clean":"2024-03-29T04:11:36.254684+0000","last_became_active":"2024-03-29T04:11:33.602697+0000","last_became_peered":"2024-03-29T04:11:33.602697+0000","last_unstale":"2024-03-29T04:11:36.254684+0000","last_undegraded":"2024-03-29T04:11:36.254684+0000","last_fullsized":"2024-03-29T04:11:36.254684+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:20:05.840636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.753394+0000","last_change":"2024-03-29T04:11:33.611316+0000","last_active":"2024-03-29T04:11:35.753394+0000","last_peered":"2024-03-29T04:11:35.753394+0000","last_clean":"2024-03-29T04:11:35.753394+0000","last_became_active":"2024-03-29T04:11:33.611022+0000","last_became_peered":"2024-03-29T04:11:33.611022+0000","last_unstale":"2024-03-29T04:11:35.753394+0000","last_undegraded":"2024-03-29T04:11:35.753394+0000","last_fullsized":"2024-03-29T04:11:35.753394+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:36:38.576501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254627+0000","last_change":"2024-03-29T04:11:33.600095+0000","last_active":"2024-03-29T04:11:36.254627+0000","last_peered":"2024-03-29T04:11:36.254627+0000","last_clean":"2024-03-29T04:11:36.254627+0000","last_became_active":"2024-03-29T04:11:33.594726+0000","last_became_peered":"2024-03-29T04:11:33.594726+0000","last_unstale":"2024-03-29T04:11:36.254627+0000","last_undegraded":"2024-03-29T04:11:36.254627+0000","last_fullsized":"2024-03-29T04:11:36.254627+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:29:32.925022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606529+0000","last_change":"2024-03-29T04:11:33.608301+0000","last_active":"2024-03-29T04:11:34.606529+0000","last_peered":"2024-03-29T04:11:34.606529+0000","last_clean":"2024-03-29T04:11:34.606529+0000","last_became_active":"2024-03-29T04:11:33.607462+0000","last_became_peered":"2024-03-29T04:11:33.607462+0000","last_unstale":"2024-03-29T04:11:34.606529+0000","last_undegraded":"2024-03-29T04:11:34.606529+0000","last_fullsized":"2024-03-29T04:11:34.606529+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:56:33.700043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254480+0000","last_change":"2024-03-29T04:11:33.613665+0000","last_active":"2024-03-29T04:11:36.254480+0000","last_peered":"2024-03-29T04:11:36.254480+0000","last_clean":"2024-03-29T04:11:36.254480+0000","last_became_active":"2024-03-29T04:11:33.613347+0000","last_became_peered":"2024-03-29T04:11:33.613347+0000","last_unstale":"2024-03-29T04:11:36.254480+0000","last_undegraded":"2024-03-29T04:11:36.254480+0000","last_fullsized":"2024-03-29T04:11:36.254480+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:44:30.880135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254350+0000","last_change":"2024-03-29T04:11:33.602018+0000","last_active":"2024-03-29T04:11:36.254350+0000","last_peered":"2024-03-29T04:11:36.254350+0000","last_clean":"2024-03-29T04:11:36.254350+0000","last_became_active":"2024-03-29T04:11:33.601071+0000","last_became_peered":"2024-03-29T04:11:33.601071+0000","last_unstale":"2024-03-29T04:11:36.254350+0000","last_undegraded":"2024-03-29T04:11:36.254350+0000","last_fullsized":"2024-03-29T04:11:36.254350+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:01:41.102786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254972+0000","last_change":"2024-03-29T04:11:33.606461+0000","last_active":"2024-03-29T04:11:36.254972+0000","last_peered":"2024-03-29T04:11:36.254972+0000","last_clean":"2024-03-29T04:11:36.254972+0000","last_became_active":"2024-03-29T04:11:33.605924+0000","last_became_peered":"2024-03-29T04:11:33.605924+0000","last_unstale":"2024-03-29T04:11:36.254972+0000","last_undegraded":"2024-03-29T04:11:36.254972+0000","last_fullsized":"2024-03-29T04:11:36.254972+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:12:14.577653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.753086+0000","last_change":"2024-03-29T04:11:33.608693+0000","last_active":"2024-03-29T04:11:35.753086+0000","last_peered":"2024-03-29T04:11:35.753086+0000","last_clean":"2024-03-29T04:11:35.753086+0000","last_became_active":"2024-03-29T04:11:33.608249+0000","last_became_peered":"2024-03-29T04:11:33.608249+0000","last_unstale":"2024-03-29T04:11:35.753086+0000","last_undegraded":"2024-03-29T04:11:35.753086+0000","last_fullsized":"2024-03-29T04:11:35.753086+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:29:04.401595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254948+0000","last_change":"2024-03-29T04:11:33.609887+0000","last_active":"2024-03-29T04:11:36.254948+0000","last_peered":"2024-03-29T04:11:36.254948+0000","last_clean":"2024-03-29T04:11:36.254948+0000","last_became_active":"2024-03-29T04:11:33.609619+0000","last_became_peered":"2024-03-29T04:11:33.609619+0000","last_unstale":"2024-03-29T04:11:36.254948+0000","last_undegraded":"2024-03-29T04:11:36.254948+0000","last_fullsized":"2024-03-29T04:11:36.254948+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:36:56.199586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254749+0000","last_change":"2024-03-29T04:11:32.605430+0000","last_active":"2024-03-29T04:11:36.254749+0000","last_peered":"2024-03-29T04:11:36.254749+0000","last_clean":"2024-03-29T04:11:36.254749+0000","last_became_active":"2024-03-29T04:11:32.605121+0000","last_became_peered":"2024-03-29T04:11:32.605121+0000","last_unstale":"2024-03-29T04:11:36.254749+0000","last_undegraded":"2024-03-29T04:11:36.254749+0000","last_fullsized":"2024-03-29T04:11:36.254749+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:01:18.409399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.753288+0000","last_change":"2024-03-29T04:11:33.611709+0000","last_active":"2024-03-29T04:11:35.753288+0000","last_peered":"2024-03-29T04:11:35.753288+0000","last_clean":"2024-03-29T04:11:35.753288+0000","last_became_active":"2024-03-29T04:11:33.611007+0000","last_became_peered":"2024-03-29T04:11:33.611007+0000","last_unstale":"2024-03-29T04:11:35.753288+0000","last_undegraded":"2024-03-29T04:11:35.753288+0000","last_fullsized":"2024-03-29T04:11:35.753288+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:59.417284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.257857+0000","last_change":"2024-03-29T04:11:32.600987+0000","last_active":"2024-03-29T04:11:35.257857+0000","last_peered":"2024-03-29T04:11:35.257857+0000","last_clean":"2024-03-29T04:11:35.257857+0000","last_became_active":"2024-03-29T04:11:32.600724+0000","last_became_peered":"2024-03-29T04:11:32.600724+0000","last_unstale":"2024-03-29T04:11:35.257857+0000","last_undegraded":"2024-03-29T04:11:35.257857+0000","last_fullsized":"2024-03-29T04:11:35.257857+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:32:39.355355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606170+0000","last_change":"2024-03-29T04:11:33.603227+0000","last_active":"2024-03-29T04:11:34.606170+0000","last_peered":"2024-03-29T04:11:34.606170+0000","last_clean":"2024-03-29T04:11:34.606170+0000","last_became_active":"2024-03-29T04:11:33.602949+0000","last_became_peered":"2024-03-29T04:11:33.602949+0000","last_unstale":"2024-03-29T04:11:34.606170+0000","last_undegraded":"2024-03-29T04:11:34.606170+0000","last_fullsized":"2024-03-29T04:11:34.606170+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:01:09.652503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.753341+0000","last_change":"2024-03-29T04:11:32.603767+0000","last_active":"2024-03-29T04:11:35.753341+0000","last_peered":"2024-03-29T04:11:35.753341+0000","last_clean":"2024-03-29T04:11:35.753341+0000","last_became_active":"2024-03-29T04:11:32.603505+0000","last_became_peered":"2024-03-29T04:11:32.603505+0000","last_unstale":"2024-03-29T04:11:35.753341+0000","last_undegraded":"2024-03-29T04:11:35.753341+0000","last_fullsized":"2024-03-29T04:11:35.753341+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:59:22.058596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.608853+0000","last_change":"2024-03-29T04:11:33.612683+0000","last_active":"2024-03-29T04:11:34.608853+0000","last_peered":"2024-03-29T04:11:34.608853+0000","last_clean":"2024-03-29T04:11:34.608853+0000","last_became_active":"2024-03-29T04:11:33.612368+0000","last_became_peered":"2024-03-29T04:11:33.612368+0000","last_unstale":"2024-03-29T04:11:34.608853+0000","last_undegraded":"2024-03-29T04:11:34.608853+0000","last_fullsized":"2024-03-29T04:11:34.608853+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:49:11.854530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.605447+0000","last_change":"2024-03-29T04:11:32.601060+0000","last_active":"2024-03-29T04:11:34.605447+0000","last_peered":"2024-03-29T04:11:34.605447+0000","last_clean":"2024-03-29T04:11:34.605447+0000","last_became_active":"2024-03-29T04:11:32.600832+0000","last_became_peered":"2024-03-29T04:11:32.600832+0000","last_unstale":"2024-03-29T04:11:34.605447+0000","last_undegraded":"2024-03-29T04:11:34.605447+0000","last_fullsized":"2024-03-29T04:11:34.605447+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:17:48.194261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.257657+0000","last_change":"2024-03-29T04:11:33.606101+0000","last_active":"2024-03-29T04:11:35.257657+0000","last_peered":"2024-03-29T04:11:35.257657+0000","last_clean":"2024-03-29T04:11:35.257657+0000","last_became_active":"2024-03-29T04:11:33.605416+0000","last_became_peered":"2024-03-29T04:11:33.605416+0000","last_unstale":"2024-03-29T04:11:35.257657+0000","last_undegraded":"2024-03-29T04:11:35.257657+0000","last_fullsized":"2024-03-29T04:11:35.257657+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:29:04.401595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606169+0000","last_change":"2024-03-29T04:11:32.598199+0000","last_active":"2024-03-29T04:11:34.606169+0000","last_peered":"2024-03-29T04:11:34.606169+0000","last_clean":"2024-03-29T04:11:34.606169+0000","last_became_active":"2024-03-29T04:11:32.597920+0000","last_became_peered":"2024-03-29T04:11:32.597920+0000","last_unstale":"2024-03-29T04:11:34.606169+0000","last_undegraded":"2024-03-29T04:11:34.606169+0000","last_fullsized":"2024-03-29T04:11:34.606169+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:22:18.091846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.605732+0000","last_change":"2024-03-29T04:11:33.615618+0000","last_active":"2024-03-29T04:11:34.605732+0000","last_peered":"2024-03-29T04:11:34.605732+0000","last_clean":"2024-03-29T04:11:34.605732+0000","last_became_active":"2024-03-29T04:11:33.615302+0000","last_became_peered":"2024-03-29T04:11:33.615302+0000","last_unstale":"2024-03-29T04:11:34.605732+0000","last_undegraded":"2024-03-29T04:11:34.605732+0000","last_fullsized":"2024-03-29T04:11:34.605732+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:12:14.577653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254568+0000","last_change":"2024-03-29T04:11:32.606202+0000","last_active":"2024-03-29T04:11:36.254568+0000","last_peered":"2024-03-29T04:11:36.254568+0000","last_clean":"2024-03-29T04:11:36.254568+0000","last_became_active":"2024-03-29T04:11:32.605956+0000","last_became_peered":"2024-03-29T04:11:32.605956+0000","last_unstale":"2024-03-29T04:11:36.254568+0000","last_undegraded":"2024-03-29T04:11:36.254568+0000","last_fullsized":"2024-03-29T04:11:36.254568+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:59:22.058596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.605711+0000","last_change":"2024-03-29T04:11:33.606680+0000","last_active":"2024-03-29T04:11:34.605711+0000","last_peered":"2024-03-29T04:11:34.605711+0000","last_clean":"2024-03-29T04:11:34.605711+0000","last_became_active":"2024-03-29T04:11:33.606425+0000","last_became_peered":"2024-03-29T04:11:33.606425+0000","last_unstale":"2024-03-29T04:11:34.605711+0000","last_undegraded":"2024-03-29T04:11:34.605711+0000","last_fullsized":"2024-03-29T04:11:34.605711+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:31:43.545143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.753034+0000","last_change":"2024-03-29T04:11:32.600094+0000","last_active":"2024-03-29T04:11:35.753034+0000","last_peered":"2024-03-29T04:11:35.753034+0000","last_clean":"2024-03-29T04:11:35.753034+0000","last_became_active":"2024-03-29T04:11:32.599709+0000","last_became_peered":"2024-03-29T04:11:32.599709+0000","last_unstale":"2024-03-29T04:11:35.753034+0000","last_undegraded":"2024-03-29T04:11:35.753034+0000","last_fullsized":"2024-03-29T04:11:35.753034+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:01:36.484198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.608456+0000","last_change":"2024-03-29T04:11:33.608241+0000","last_active":"2024-03-29T04:11:34.608456+0000","last_peered":"2024-03-29T04:11:34.608456+0000","last_clean":"2024-03-29T04:11:34.608456+0000","last_became_active":"2024-03-29T04:11:33.607755+0000","last_became_peered":"2024-03-29T04:11:33.607755+0000","last_unstale":"2024-03-29T04:11:34.608456+0000","last_undegraded":"2024-03-29T04:11:34.608456+0000","last_fullsized":"2024-03-29T04:11:34.608456+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:12:14.577653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.605080+0000","last_change":"2024-03-29T04:11:32.598819+0000","last_active":"2024-03-29T04:11:34.605080+0000","last_peered":"2024-03-29T04:11:34.605080+0000","last_clean":"2024-03-29T04:11:34.605080+0000","last_became_active":"2024-03-29T04:11:32.597774+0000","last_became_peered":"2024-03-29T04:11:32.597774+0000","last_unstale":"2024-03-29T04:11:34.605080+0000","last_undegraded":"2024-03-29T04:11:34.605080+0000","last_fullsized":"2024-03-29T04:11:34.605080+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:32:39.355355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.594486+0000","last_change":"2024-03-29T04:11:33.600732+0000","last_active":"2024-03-29T04:11:35.594486+0000","last_peered":"2024-03-29T04:11:35.594486+0000","last_clean":"2024-03-29T04:11:35.594486+0000","last_became_active":"2024-03-29T04:11:33.600472+0000","last_became_peered":"2024-03-29T04:11:33.600472+0000","last_unstale":"2024-03-29T04:11:35.594486+0000","last_undegraded":"2024-03-29T04:11:35.594486+0000","last_fullsized":"2024-03-29T04:11:35.594486+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:32:40.357532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.609013+0000","last_change":"2024-03-29T04:11:32.603200+0000","last_active":"2024-03-29T04:11:34.609013+0000","last_peered":"2024-03-29T04:11:34.609013+0000","last_clean":"2024-03-29T04:11:34.609013+0000","last_became_active":"2024-03-29T04:11:32.602912+0000","last_became_peered":"2024-03-29T04:11:32.602912+0000","last_unstale":"2024-03-29T04:11:34.609013+0000","last_undegraded":"2024-03-29T04:11:34.609013+0000","last_fullsized":"2024-03-29T04:11:34.609013+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:29:31.922845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.255118+0000","last_change":"2024-03-29T04:11:33.611218+0000","last_active":"2024-03-29T04:11:36.255118+0000","last_peered":"2024-03-29T04:11:36.255118+0000","last_clean":"2024-03-29T04:11:36.255118+0000","last_became_active":"2024-03-29T04:11:33.610963+0000","last_became_peered":"2024-03-29T04:11:33.610963+0000","last_unstale":"2024-03-29T04:11:36.255118+0000","last_undegraded":"2024-03-29T04:11:36.255118+0000","last_fullsized":"2024-03-29T04:11:36.255118+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:57:51.680570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606622+0000","last_change":"2024-03-29T04:11:32.603797+0000","last_active":"2024-03-29T04:11:34.606622+0000","last_peered":"2024-03-29T04:11:34.606622+0000","last_clean":"2024-03-29T04:11:34.606622+0000","last_became_active":"2024-03-29T04:11:32.603514+0000","last_became_peered":"2024-03-29T04:11:32.603514+0000","last_unstale":"2024-03-29T04:11:34.606622+0000","last_undegraded":"2024-03-29T04:11:34.606622+0000","last_fullsized":"2024-03-29T04:11:34.606622+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:01:40.100609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.257863+0000","last_change":"2024-03-29T04:11:33.604476+0000","last_active":"2024-03-29T04:11:35.257863+0000","last_peered":"2024-03-29T04:11:35.257863+0000","last_clean":"2024-03-29T04:11:35.257863+0000","last_became_active":"2024-03-29T04:11:33.604181+0000","last_became_peered":"2024-03-29T04:11:33.604181+0000","last_unstale":"2024-03-29T04:11:35.257863+0000","last_undegraded":"2024-03-29T04:11:35.257863+0000","last_fullsized":"2024-03-29T04:11:35.257863+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:36:38.576501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606500+0000","last_change":"2024-03-29T04:11:32.606176+0000","last_active":"2024-03-29T04:11:34.606500+0000","last_peered":"2024-03-29T04:11:34.606500+0000","last_clean":"2024-03-29T04:11:34.606500+0000","last_became_active":"2024-03-29T04:11:32.605915+0000","last_became_peered":"2024-03-29T04:11:32.605915+0000","last_unstale":"2024-03-29T04:11:34.606500+0000","last_undegraded":"2024-03-29T04:11:34.606500+0000","last_fullsized":"2024-03-29T04:11:34.606500+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:44:29.877958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606091+0000","last_change":"2024-03-29T04:11:33.596958+0000","last_active":"2024-03-29T04:11:34.606091+0000","last_peered":"2024-03-29T04:11:34.606091+0000","last_clean":"2024-03-29T04:11:34.606091+0000","last_became_active":"2024-03-29T04:11:33.596634+0000","last_became_peered":"2024-03-29T04:11:33.596634+0000","last_unstale":"2024-03-29T04:11:34.606091+0000","last_undegraded":"2024-03-29T04:11:34.606091+0000","last_fullsized":"2024-03-29T04:11:34.606091+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:02:25.169625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.608697+0000","last_change":"2024-03-29T04:11:32.607490+0000","last_active":"2024-03-29T04:11:34.608697+0000","last_peered":"2024-03-29T04:11:34.608697+0000","last_clean":"2024-03-29T04:11:34.608697+0000","last_became_active":"2024-03-29T04:11:32.607245+0000","last_became_peered":"2024-03-29T04:11:32.607245+0000","last_unstale":"2024-03-29T04:11:34.608697+0000","last_undegraded":"2024-03-29T04:11:34.608697+0000","last_fullsized":"2024-03-29T04:11:34.608697+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:44:29.877958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.257592+0000","last_change":"2024-03-29T04:11:33.606594+0000","last_active":"2024-03-29T04:11:35.257592+0000","last_peered":"2024-03-29T04:11:35.257592+0000","last_clean":"2024-03-29T04:11:35.257592+0000","last_became_active":"2024-03-29T04:11:33.605711+0000","last_became_peered":"2024-03-29T04:11:33.605711+0000","last_unstale":"2024-03-29T04:11:35.257592+0000","last_undegraded":"2024-03-29T04:11:35.257592+0000","last_fullsized":"2024-03-29T04:11:35.257592+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:31:43.545143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.608584+0000","last_change":"2024-03-29T04:11:32.599838+0000","last_active":"2024-03-29T04:11:34.608584+0000","last_peered":"2024-03-29T04:11:34.608584+0000","last_clean":"2024-03-29T04:11:34.608584+0000","last_became_active":"2024-03-29T04:11:32.599585+0000","last_became_peered":"2024-03-29T04:11:32.599585+0000","last_unstale":"2024-03-29T04:11:34.608584+0000","last_undegraded":"2024-03-29T04:11:34.608584+0000","last_fullsized":"2024-03-29T04:11:34.608584+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:22:18.091846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.608458+0000","last_change":"2024-03-29T04:11:33.602106+0000","last_active":"2024-03-29T04:11:34.608458+0000","last_peered":"2024-03-29T04:11:34.608458+0000","last_clean":"2024-03-29T04:11:34.608458+0000","last_became_active":"2024-03-29T04:11:33.601852+0000","last_became_peered":"2024-03-29T04:11:33.601852+0000","last_unstale":"2024-03-29T04:11:34.608458+0000","last_undegraded":"2024-03-29T04:11:34.608458+0000","last_fullsized":"2024-03-29T04:11:34.608458+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:57:51.680570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.752988+0000","last_change":"2024-03-29T04:11:32.598043+0000","last_active":"2024-03-29T04:11:35.752988+0000","last_peered":"2024-03-29T04:11:35.752988+0000","last_clean":"2024-03-29T04:11:35.752988+0000","last_became_active":"2024-03-29T04:11:32.597779+0000","last_became_peered":"2024-03-29T04:11:32.597779+0000","last_unstale":"2024-03-29T04:11:35.752988+0000","last_undegraded":"2024-03-29T04:11:35.752988+0000","last_fullsized":"2024-03-29T04:11:35.752988+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:43:22.219049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254529+0000","last_change":"2024-03-29T04:11:33.611724+0000","last_active":"2024-03-29T04:11:36.254529+0000","last_peered":"2024-03-29T04:11:36.254529+0000","last_clean":"2024-03-29T04:11:36.254529+0000","last_became_active":"2024-03-29T04:11:33.611420+0000","last_became_peered":"2024-03-29T04:11:33.611420+0000","last_unstale":"2024-03-29T04:11:36.254529+0000","last_undegraded":"2024-03-29T04:11:36.254529+0000","last_fullsized":"2024-03-29T04:11:36.254529+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:36:38.576501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254102+0000","last_change":"2024-03-29T04:11:32.607645+0000","last_active":"2024-03-29T04:11:36.254102+0000","last_peered":"2024-03-29T04:11:36.254102+0000","last_clean":"2024-03-29T04:11:36.254102+0000","last_became_active":"2024-03-29T04:11:32.607341+0000","last_became_peered":"2024-03-29T04:11:32.607341+0000","last_unstale":"2024-03-29T04:11:36.254102+0000","last_undegraded":"2024-03-29T04:11:36.254102+0000","last_fullsized":"2024-03-29T04:11:36.254102+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:17:48.194261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.608534+0000","last_change":"2024-03-29T04:11:33.602606+0000","last_active":"2024-03-29T04:11:34.608534+0000","last_peered":"2024-03-29T04:11:34.608534+0000","last_clean":"2024-03-29T04:11:34.608534+0000","last_became_active":"2024-03-29T04:11:33.602344+0000","last_became_peered":"2024-03-29T04:11:33.602344+0000","last_unstale":"2024-03-29T04:11:34.608534+0000","last_undegraded":"2024-03-29T04:11:34.608534+0000","last_fullsized":"2024-03-29T04:11:34.608534+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:28:05.628657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254105+0000","last_change":"2024-03-29T04:11:32.606796+0000","last_active":"2024-03-29T04:11:36.254105+0000","last_peered":"2024-03-29T04:11:36.254105+0000","last_clean":"2024-03-29T04:11:36.254105+0000","last_became_active":"2024-03-29T04:11:32.606544+0000","last_became_peered":"2024-03-29T04:11:32.606544+0000","last_unstale":"2024-03-29T04:11:36.254105+0000","last_undegraded":"2024-03-29T04:11:36.254105+0000","last_fullsized":"2024-03-29T04:11:36.254105+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:01:28.999548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.608595+0000","last_change":"2024-03-29T04:11:33.612827+0000","last_active":"2024-03-29T04:11:34.608595+0000","last_peered":"2024-03-29T04:11:34.608595+0000","last_clean":"2024-03-29T04:11:34.608595+0000","last_became_active":"2024-03-29T04:11:33.612576+0000","last_became_peered":"2024-03-29T04:11:33.612576+0000","last_unstale":"2024-03-29T04:11:34.608595+0000","last_undegraded":"2024-03-29T04:11:34.608595+0000","last_fullsized":"2024-03-29T04:11:34.608595+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:59:25.023691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606758+0000","last_change":"2024-03-29T04:11:32.599893+0000","last_active":"2024-03-29T04:11:34.606758+0000","last_peered":"2024-03-29T04:11:34.606758+0000","last_clean":"2024-03-29T04:11:34.606758+0000","last_became_active":"2024-03-29T04:11:32.599635+0000","last_became_peered":"2024-03-29T04:11:32.599635+0000","last_unstale":"2024-03-29T04:11:34.606758+0000","last_undegraded":"2024-03-29T04:11:34.606758+0000","last_fullsized":"2024-03-29T04:11:34.606758+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:59:22.058596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606451+0000","last_change":"2024-03-29T04:11:33.614834+0000","last_active":"2024-03-29T04:11:34.606451+0000","last_peered":"2024-03-29T04:11:34.606451+0000","last_clean":"2024-03-29T04:11:34.606451+0000","last_became_active":"2024-03-29T04:11:33.614431+0000","last_became_peered":"2024-03-29T04:11:33.614431+0000","last_unstale":"2024-03-29T04:11:34.606451+0000","last_undegraded":"2024-03-29T04:11:34.606451+0000","last_fullsized":"2024-03-29T04:11:34.606451+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:49:11.854530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606207+0000","last_change":"2024-03-29T04:11:32.598560+0000","last_active":"2024-03-29T04:11:34.606207+0000","last_peered":"2024-03-29T04:11:34.606207+0000","last_clean":"2024-03-29T04:11:34.606207+0000","last_became_active":"2024-03-29T04:11:32.598189+0000","last_became_peered":"2024-03-29T04:11:32.598189+0000","last_unstale":"2024-03-29T04:11:34.606207+0000","last_undegraded":"2024-03-29T04:11:34.606207+0000","last_fullsized":"2024-03-29T04:11:34.606207+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:52:01.808081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606155+0000","last_change":"2024-03-29T04:11:33.604399+0000","last_active":"2024-03-29T04:11:34.606155+0000","last_peered":"2024-03-29T04:11:34.606155+0000","last_clean":"2024-03-29T04:11:34.606155+0000","last_became_active":"2024-03-29T04:11:33.603948+0000","last_became_peered":"2024-03-29T04:11:33.603948+0000","last_unstale":"2024-03-29T04:11:34.606155+0000","last_undegraded":"2024-03-29T04:11:34.606155+0000","last_fullsized":"2024-03-29T04:11:34.606155+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:29:04.401595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.594406+0000","last_change":"2024-03-29T04:11:32.603790+0000","last_active":"2024-03-29T04:11:35.594406+0000","last_peered":"2024-03-29T04:11:35.594406+0000","last_clean":"2024-03-29T04:11:35.594406+0000","last_became_active":"2024-03-29T04:11:32.603488+0000","last_became_peered":"2024-03-29T04:11:32.603488+0000","last_unstale":"2024-03-29T04:11:35.594406+0000","last_undegraded":"2024-03-29T04:11:35.594406+0000","last_fullsized":"2024-03-29T04:11:35.594406+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:47:38.689753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.594551+0000","last_change":"2024-03-29T04:11:33.603706+0000","last_active":"2024-03-29T04:11:35.594551+0000","last_peered":"2024-03-29T04:11:35.594551+0000","last_clean":"2024-03-29T04:11:35.594551+0000","last_became_active":"2024-03-29T04:11:33.603435+0000","last_became_peered":"2024-03-29T04:11:33.603435+0000","last_unstale":"2024-03-29T04:11:35.594551+0000","last_undegraded":"2024-03-29T04:11:35.594551+0000","last_fullsized":"2024-03-29T04:11:35.594551+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:52:02.810258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.753141+0000","last_change":"2024-03-29T04:11:32.605114+0000","last_active":"2024-03-29T04:11:35.753141+0000","last_peered":"2024-03-29T04:11:35.753141+0000","last_clean":"2024-03-29T04:11:35.753141+0000","last_became_active":"2024-03-29T04:11:32.604860+0000","last_became_peered":"2024-03-29T04:11:32.604860+0000","last_unstale":"2024-03-29T04:11:35.753141+0000","last_undegraded":"2024-03-29T04:11:35.753141+0000","last_fullsized":"2024-03-29T04:11:35.753141+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:01:28.999548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254350+0000","last_change":"2024-03-29T04:11:33.607041+0000","last_active":"2024-03-29T04:11:36.254350+0000","last_peered":"2024-03-29T04:11:36.254350+0000","last_clean":"2024-03-29T04:11:36.254350+0000","last_became_active":"2024-03-29T04:11:33.606778+0000","last_became_peered":"2024-03-29T04:11:33.606778+0000","last_unstale":"2024-03-29T04:11:36.254350+0000","last_undegraded":"2024-03-29T04:11:36.254350+0000","last_fullsized":"2024-03-29T04:11:36.254350+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:25:50.154821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.594235+0000","last_change":"2024-03-29T04:11:32.597119+0000","last_active":"2024-03-29T04:11:35.594235+0000","last_peered":"2024-03-29T04:11:35.594235+0000","last_clean":"2024-03-29T04:11:35.594235+0000","last_became_active":"2024-03-29T04:11:32.596839+0000","last_became_peered":"2024-03-29T04:11:32.596839+0000","last_unstale":"2024-03-29T04:11:35.594235+0000","last_undegraded":"2024-03-29T04:11:35.594235+0000","last_fullsized":"2024-03-29T04:11:35.594235+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:47:00.502578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.257592+0000","last_change":"2024-03-29T04:11:33.604753+0000","last_active":"2024-03-29T04:11:35.257592+0000","last_peered":"2024-03-29T04:11:35.257592+0000","last_clean":"2024-03-29T04:11:35.257592+0000","last_became_active":"2024-03-29T04:11:33.604213+0000","last_became_peered":"2024-03-29T04:11:33.604213+0000","last_unstale":"2024-03-29T04:11:35.257592+0000","last_undegraded":"2024-03-29T04:11:35.257592+0000","last_fullsized":"2024-03-29T04:11:35.257592+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:20:05.840636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.594257+0000","last_change":"2024-03-29T04:11:32.594337+0000","last_active":"2024-03-29T04:11:35.594257+0000","last_peered":"2024-03-29T04:11:35.594257+0000","last_clean":"2024-03-29T04:11:35.594257+0000","last_became_active":"2024-03-29T04:11:32.594053+0000","last_became_peered":"2024-03-29T04:11:32.594053+0000","last_unstale":"2024-03-29T04:11:35.594257+0000","last_undegraded":"2024-03-29T04:11:35.594257+0000","last_fullsized":"2024-03-29T04:11:35.594257+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:40:57.685920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.605804+0000","last_change":"2024-03-29T04:11:33.609219+0000","last_active":"2024-03-29T04:11:34.605804+0000","last_peered":"2024-03-29T04:11:34.605804+0000","last_clean":"2024-03-29T04:11:34.605804+0000","last_became_active":"2024-03-29T04:11:33.608958+0000","last_became_peered":"2024-03-29T04:11:33.608958+0000","last_unstale":"2024-03-29T04:11:34.605804+0000","last_undegraded":"2024-03-29T04:11:34.605804+0000","last_fullsized":"2024-03-29T04:11:34.605804+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:57:51.680570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254949+0000","last_change":"2024-03-29T04:11:32.603484+0000","last_active":"2024-03-29T04:11:36.254949+0000","last_peered":"2024-03-29T04:11:36.254949+0000","last_clean":"2024-03-29T04:11:36.254949+0000","last_became_active":"2024-03-29T04:11:32.603152+0000","last_became_peered":"2024-03-29T04:11:32.603152+0000","last_unstale":"2024-03-29T04:11:36.254949+0000","last_undegraded":"2024-03-29T04:11:36.254949+0000","last_fullsized":"2024-03-29T04:11:36.254949+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:43:22.219049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.608651+0000","last_change":"2024-03-29T04:11:33.608025+0000","last_active":"2024-03-29T04:11:34.608651+0000","last_peered":"2024-03-29T04:11:34.608651+0000","last_clean":"2024-03-29T04:11:34.608651+0000","last_became_active":"2024-03-29T04:11:33.607394+0000","last_became_peered":"2024-03-29T04:11:33.607394+0000","last_unstale":"2024-03-29T04:11:34.608651+0000","last_undegraded":"2024-03-29T04:11:34.608651+0000","last_fullsized":"2024-03-29T04:11:34.608651+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:35:25.923249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.594162+0000","last_change":"2024-03-29T04:11:32.597603+0000","last_active":"2024-03-29T04:11:35.594162+0000","last_peered":"2024-03-29T04:11:35.594162+0000","last_clean":"2024-03-29T04:11:35.594162+0000","last_became_active":"2024-03-29T04:11:32.597340+0000","last_became_peered":"2024-03-29T04:11:32.597340+0000","last_unstale":"2024-03-29T04:11:35.594162+0000","last_undegraded":"2024-03-29T04:11:35.594162+0000","last_fullsized":"2024-03-29T04:11:35.594162+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:08:17.331786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.753174+0000","last_change":"2024-03-29T04:11:33.610154+0000","last_active":"2024-03-29T04:11:35.753174+0000","last_peered":"2024-03-29T04:11:35.753174+0000","last_clean":"2024-03-29T04:11:35.753174+0000","last_became_active":"2024-03-29T04:11:33.609894+0000","last_became_peered":"2024-03-29T04:11:33.609894+0000","last_unstale":"2024-03-29T04:11:35.753174+0000","last_undegraded":"2024-03-29T04:11:35.753174+0000","last_fullsized":"2024-03-29T04:11:35.753174+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:01:09.652503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606812+0000","last_change":"2024-03-29T04:11:32.605443+0000","last_active":"2024-03-29T04:11:34.606812+0000","last_peered":"2024-03-29T04:11:34.606812+0000","last_clean":"2024-03-29T04:11:34.606812+0000","last_became_active":"2024-03-29T04:11:32.605033+0000","last_became_peered":"2024-03-29T04:11:32.605033+0000","last_unstale":"2024-03-29T04:11:34.606812+0000","last_undegraded":"2024-03-29T04:11:34.606812+0000","last_fullsized":"2024-03-29T04:11:34.606812+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:49:50.242921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.255195+0000","last_change":"2024-03-29T04:11:33.611683+0000","last_active":"2024-03-29T04:11:36.255195+0000","last_peered":"2024-03-29T04:11:36.255195+0000","last_clean":"2024-03-29T04:11:36.255195+0000","last_became_active":"2024-03-29T04:11:33.611234+0000","last_became_peered":"2024-03-29T04:11:33.611234+0000","last_unstale":"2024-03-29T04:11:36.255195+0000","last_undegraded":"2024-03-29T04:11:36.255195+0000","last_fullsized":"2024-03-29T04:11:36.255195+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:28:05.628657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.594469+0000","last_change":"2024-03-29T04:11:32.598263+0000","last_active":"2024-03-29T04:11:35.594469+0000","last_peered":"2024-03-29T04:11:35.594469+0000","last_clean":"2024-03-29T04:11:35.594469+0000","last_became_active":"2024-03-29T04:11:32.597985+0000","last_became_peered":"2024-03-29T04:11:32.597985+0000","last_unstale":"2024-03-29T04:11:35.594469+0000","last_undegraded":"2024-03-29T04:11:35.594469+0000","last_fullsized":"2024-03-29T04:11:35.594469+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:44:05.826219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.608660+0000","last_change":"2024-03-29T04:11:33.605426+0000","last_active":"2024-03-29T04:11:34.608660+0000","last_peered":"2024-03-29T04:11:34.608660+0000","last_clean":"2024-03-29T04:11:34.608660+0000","last_became_active":"2024-03-29T04:11:33.605173+0000","last_became_peered":"2024-03-29T04:11:33.605173+0000","last_unstale":"2024-03-29T04:11:34.608660+0000","last_undegraded":"2024-03-29T04:11:34.608660+0000","last_fullsized":"2024-03-29T04:11:34.608660+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:35:44.357464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.257674+0000","last_change":"2024-03-29T04:11:32.599052+0000","last_active":"2024-03-29T04:11:35.257674+0000","last_peered":"2024-03-29T04:11:35.257674+0000","last_clean":"2024-03-29T04:11:35.257674+0000","last_became_active":"2024-03-29T04:11:32.598775+0000","last_became_peered":"2024-03-29T04:11:32.598775+0000","last_unstale":"2024-03-29T04:11:35.257674+0000","last_undegraded":"2024-03-29T04:11:35.257674+0000","last_fullsized":"2024-03-29T04:11:35.257674+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:52:01.808081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.257771+0000","last_change":"2024-03-29T04:11:33.605650+0000","last_active":"2024-03-29T04:11:35.257771+0000","last_peered":"2024-03-29T04:11:35.257771+0000","last_clean":"2024-03-29T04:11:35.257771+0000","last_became_active":"2024-03-29T04:11:33.605098+0000","last_became_peered":"2024-03-29T04:11:33.605098+0000","last_unstale":"2024-03-29T04:11:35.257771+0000","last_undegraded":"2024-03-29T04:11:35.257771+0000","last_fullsized":"2024-03-29T04:11:35.257771+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:59.417284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.257742+0000","last_change":"2024-03-29T04:11:32.601769+0000","last_active":"2024-03-29T04:11:35.257742+0000","last_peered":"2024-03-29T04:11:35.257742+0000","last_clean":"2024-03-29T04:11:35.257742+0000","last_became_active":"2024-03-29T04:11:32.601320+0000","last_became_peered":"2024-03-29T04:11:32.601320+0000","last_unstale":"2024-03-29T04:11:35.257742+0000","last_undegraded":"2024-03-29T04:11:35.257742+0000","last_fullsized":"2024-03-29T04:11:35.257742+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:25.262311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:35.753075+0000","last_change":"2024-03-29T04:11:33.611575+0000","last_active":"2024-03-29T04:11:35.753075+0000","last_peered":"2024-03-29T04:11:35.753075+0000","last_clean":"2024-03-29T04:11:35.753075+0000","last_became_active":"2024-03-29T04:11:33.611286+0000","last_became_peered":"2024-03-29T04:11:33.611286+0000","last_unstale":"2024-03-29T04:11:35.753075+0000","last_undegraded":"2024-03-29T04:11:35.753075+0000","last_fullsized":"2024-03-29T04:11:35.753075+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:02:25.169625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254422+0000","last_change":"2024-03-29T04:11:32.603804+0000","last_active":"2024-03-29T04:11:36.254422+0000","last_peered":"2024-03-29T04:11:36.254422+0000","last_clean":"2024-03-29T04:11:36.254422+0000","last_became_active":"2024-03-29T04:11:32.603518+0000","last_became_peered":"2024-03-29T04:11:32.603518+0000","last_unstale":"2024-03-29T04:11:36.254422+0000","last_undegraded":"2024-03-29T04:11:36.254422+0000","last_fullsized":"2024-03-29T04:11:36.254422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:18:51.873151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.605375+0000","last_change":"2024-03-29T04:11:33.601327+0000","last_active":"2024-03-29T04:11:34.605375+0000","last_peered":"2024-03-29T04:11:34.605375+0000","last_clean":"2024-03-29T04:11:34.605375+0000","last_became_active":"2024-03-29T04:11:33.601071+0000","last_became_peered":"2024-03-29T04:11:33.601071+0000","last_unstale":"2024-03-29T04:11:34.605375+0000","last_undegraded":"2024-03-29T04:11:34.605375+0000","last_fullsized":"2024-03-29T04:11:34.605375+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:44:30.880135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.605300+0000","last_change":"2024-03-29T04:11:32.601521+0000","last_active":"2024-03-29T04:11:34.605300+0000","last_peered":"2024-03-29T04:11:34.605300+0000","last_clean":"2024-03-29T04:11:34.605300+0000","last_became_active":"2024-03-29T04:11:32.601232+0000","last_became_peered":"2024-03-29T04:11:32.601232+0000","last_unstale":"2024-03-29T04:11:34.605300+0000","last_undegraded":"2024-03-29T04:11:34.605300+0000","last_fullsized":"2024-03-29T04:11:34.605300+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:01:18.409399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254948+0000","last_change":"2024-03-29T04:11:33.607992+0000","last_active":"2024-03-29T04:11:36.254948+0000","last_peered":"2024-03-29T04:11:36.254948+0000","last_clean":"2024-03-29T04:11:36.254948+0000","last_became_active":"2024-03-29T04:11:33.607683+0000","last_became_peered":"2024-03-29T04:11:33.607683+0000","last_unstale":"2024-03-29T04:11:36.254948+0000","last_undegraded":"2024-03-29T04:11:36.254948+0000","last_fullsized":"2024-03-29T04:11:36.254948+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:05.626784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.608319+0000","last_change":"2024-03-29T04:11:32.599142+0000","last_active":"2024-03-29T04:11:34.608319+0000","last_peered":"2024-03-29T04:11:34.608319+0000","last_clean":"2024-03-29T04:11:34.608319+0000","last_became_active":"2024-03-29T04:11:32.598890+0000","last_became_peered":"2024-03-29T04:11:32.598890+0000","last_unstale":"2024-03-29T04:11:34.608319+0000","last_undegraded":"2024-03-29T04:11:34.608319+0000","last_fullsized":"2024-03-29T04:11:34.608319+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:01:40.100609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:34.606721+0000","last_change":"2024-03-29T04:11:33.608225+0000","last_active":"2024-03-29T04:11:34.606721+0000","last_peered":"2024-03-29T04:11:34.606721+0000","last_clean":"2024-03-29T04:11:34.606721+0000","last_became_active":"2024-03-29T04:11:33.607456+0000","last_became_peered":"2024-03-29T04:11:33.607456+0000","last_unstale":"2024-03-29T04:11:34.606721+0000","last_undegraded":"2024-03-29T04:11:34.606721+0000","last_fullsized":"2024-03-29T04:11:34.606721+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:32.580737+0000","last_clean_scrub_stamp":"2024-03-29T04:11:32.580737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:16:14.586985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254899+0000","last_change":"2024-03-29T04:11:32.603558+0000","last_active":"2024-03-29T04:11:36.254899+0000","last_peered":"2024-03-29T04:11:36.254899+0000","last_clean":"2024-03-29T04:11:36.254899+0000","last_became_active":"2024-03-29T04:11:32.603256+0000","last_became_peered":"2024-03-29T04:11:32.603256+0000","last_unstale":"2024-03-29T04:11:36.254899+0000","last_undegraded":"2024-03-29T04:11:36.254899+0000","last_fullsized":"2024-03-29T04:11:36.254899+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:11:31.578560+0000","last_clean_scrub_stamp":"2024-03-29T04:11:31.578560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:01:36.484198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":207,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:11:36.254229+0000","last_change":"2024-03-29T04:09:49.002570+0000","last_active":"2024-03-29T04:11:36.254229+0000","last_peered":"2024-03-29T04:11:36.254229+0000","last_clean":"2024-03-29T04:11:36.254229+0000","last_became_active":"2024-03-29T04:09:49.002235+0000","last_became_peered":"2024-03-29T04:09:49.002235+0000","last_unstale":"2024-03-29T04:11:36.254229+0000","last_undegraded":"2024-03-29T04:11:36.254229+0000","last_fullsized":"2024-03-29T04:11:36.254229+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:47.900733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:47.900733+0000","last_clean_scrub_stamp":"2024-03-29T04:09:47.900733+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:44:04.608265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1060864,"data_stored":787777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":196309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31632,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340208,"statfs":{"total":96636764160,"available":96604372992,"internally_reserved":0,"allocated":458752,"data_stored":195839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31632,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340208,"statfs":{"total":96636764160,"available":96604372992,"internally_reserved":0,"allocated":458752,"data_stored":196331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":466944,"data_stored":197373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":195797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":195817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1048576,"data_stored":786129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:11:41.368 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-29T04:11:41.368 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-29T04:11:41.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.535+0000 7f20c94fb640 1 Processor -- start 2024-03-29T04:11:41.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.535+0000 7f20c94fb640 1 -- start start 2024-03-29T04:11:41.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.536+0000 7f20c94fb640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f20c40cf830 0x7f20c4169170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:41.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.536+0000 7f20c94fb640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20c415ea40 con 0x7f20c40d2440 2024-03-29T04:11:41.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.536+0000 7f20c94fb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20c40b8500 con 0x7f20c415e660 2024-03-29T04:11:41.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.536+0000 7f20c94fb640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f20c40d40a0 con 0x7f20c40cf830 2024-03-29T04:11:41.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.536+0000 7f20c37fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f20c40cf830 0x7f20c4169170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:41.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.536+0000 7f20c37fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f20c40cf830 0x7f20c4169170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42376/0 (socket says 172.21.15.148:42376) 2024-03-29T04:11:41.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.536+0000 7f20ba7fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f20c40d2440 0x7f20c40cf120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40668/0 (socket says 172.21.15.148:40668) 2024-03-29T04:11:41.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.536+0000 7f20c37fe640 1 -- 172.21.15.148:0/2482689942 learned_addr learned my addr 172.21.15.148:0/2482689942 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:41.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.536+0000 7f20c37fe640 1 -- 172.21.15.148:0/2482689942 >> v1:172.21.15.157:6789/0 conn(0x7f20c415e660 legacy=0x7f20c40b9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:41.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.536+0000 7f20c37fe640 1 -- 172.21.15.148:0/2482689942 >> v1:172.21.15.148:6789/0 conn(0x7f20c40d2440 legacy=0x7f20c40cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:41.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.536+0000 7f20c37fe640 1 -- 172.21.15.148:0/2482689942 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c40d28d0 con 0x7f20c40cf830 2024-03-29T04:11:41.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.536+0000 7f20c27fc640 1 -- 172.21.15.148:0/2482689942 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3221113150 0 0) 0x7f20c415ea40 con 0x7f20c40d2440 2024-03-29T04:11:41.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.537+0000 7f20c37fe640 1 --2- 172.21.15.148:0/2482689942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c40cf830 0x7f20c4169170 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f20bc009e50 tx=0x7f20bc02c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90eeda8e1ee43741 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:41.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.537+0000 7f20c27fc640 1 -- 172.21.15.148:0/2482689942 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc02cbb0 con 0x7f20c40cf830 2024-03-29T04:11:41.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.537+0000 7f20c27fc640 1 -- 172.21.15.148:0/2482689942 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20bc02cd50 con 0x7f20c40cf830 2024-03-29T04:11:41.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.537+0000 7f20c27fc640 1 -- 172.21.15.148:0/2482689942 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc034610 con 0x7f20c40cf830 2024-03-29T04:11:41.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.538+0000 7f20c94fb640 1 -- 172.21.15.148:0/2482689942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c40cf830 msgr2=0x7f20c4169170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:41.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.538+0000 7f20c94fb640 1 --2- 172.21.15.148:0/2482689942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c40cf830 0x7f20c4169170 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f20bc009e50 tx=0x7f20bc02c010 comp rx=0 tx=0).stop 2024-03-29T04:11:41.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.538+0000 7f20c94fb640 1 -- 172.21.15.148:0/2482689942 shutdown_connections 2024-03-29T04:11:41.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.538+0000 7f20c94fb640 1 --2- 172.21.15.148:0/2482689942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c40cf830 0x7f20c4169170 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:41.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.538+0000 7f20c94fb640 1 -- 172.21.15.148:0/2482689942 >> 172.21.15.148:0/2482689942 conn(0x7f20c40d11f0 msgr2=0x7f20c40d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:41.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.538+0000 7f20c94fb640 1 -- 172.21.15.148:0/2482689942 shutdown_connections 2024-03-29T04:11:41.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.538+0000 7f20c94fb640 1 -- 172.21.15.148:0/2482689942 wait complete. 2024-03-29T04:11:41.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.539+0000 7f20c94fb640 1 Processor -- start 2024-03-29T04:11:41.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.539+0000 7f20c94fb640 1 -- start start 2024-03-29T04:11:41.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.540+0000 7f20c94fb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c40cf830 0x7f20c41572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:41.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.540+0000 7f20c94fb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20c415e660 0x7f20c40c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:41.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.540+0000 7f20c94fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c4146e50 0x7f20c40ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:41.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.540+0000 7f20c94fb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f20c40d3170 con 0x7f20c40cf830 2024-03-29T04:11:41.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.540+0000 7f20c94fb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f20c40d2e90 con 0x7f20c4146e50 2024-03-29T04:11:41.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.540+0000 7f20c94fb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f20c40d3450 con 0x7f20c415e660 2024-03-29T04:11:41.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.540+0000 7f20c2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c40cf830 0x7f20c41572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:41.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.540+0000 7f20c2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c40cf830 0x7f20c41572b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42386/0 (socket says 172.21.15.148:42386) 2024-03-29T04:11:41.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.540+0000 7f20c2ffd640 1 -- 172.21.15.148:0/480219902 learned_addr learned my addr 172.21.15.148:0/480219902 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:41.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.540+0000 7f20c2ffd640 1 -- 172.21.15.148:0/480219902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20c415e660 msgr2=0x7f20c40c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:41.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.540+0000 7f20c2ffd640 1 --2- 172.21.15.148:0/480219902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20c415e660 0x7f20c40c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:41.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.540+0000 7f20c2ffd640 1 -- 172.21.15.148:0/480219902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c4146e50 msgr2=0x7f20c40ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:41.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.540+0000 7f20c2ffd640 1 --2- 172.21.15.148:0/480219902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c4146e50 0x7f20c40ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:41.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.540+0000 7f20c2ffd640 1 -- 172.21.15.148:0/480219902 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c4156210 con 0x7f20c40cf830 2024-03-29T04:11:41.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.540+0000 7f20c2ffd640 1 --2- 172.21.15.148:0/480219902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c40cf830 0x7f20c41572b0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f20b400c450 tx=0x7f20b400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:41.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.540+0000 7f20bbfff640 1 -- 172.21.15.148:0/480219902 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b40052d0 con 0x7f20c40cf830 2024-03-29T04:11:41.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.540+0000 7f20bbfff640 1 -- 172.21.15.148:0/480219902 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20b4005470 con 0x7f20c40cf830 2024-03-29T04:11:41.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.540+0000 7f20bbfff640 1 -- 172.21.15.148:0/480219902 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b4012870 con 0x7f20c40cf830 2024-03-29T04:11:41.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.540+0000 7f20c94fb640 1 -- 172.21.15.148:0/480219902 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c41564f0 con 0x7f20c40cf830 2024-03-29T04:11:41.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.540+0000 7f20c94fb640 1 -- 172.21.15.148:0/480219902 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c4156ab0 con 0x7f20c40cf830 2024-03-29T04:11:41.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.542+0000 7f20c94fb640 1 -- 172.21.15.148:0/480219902 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20c4068850 con 0x7f20c40cf830 2024-03-29T04:11:41.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.542+0000 7f20bbfff640 1 -- 172.21.15.148:0/480219902 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f20b4012a10 con 0x7f20c40cf830 2024-03-29T04:11:41.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.542+0000 7f20bbfff640 1 --2- 172.21.15.148:0/480219902 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f20a406c560 0x7f20a406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:41.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.542+0000 7f20ba7fc640 1 --2- 172.21.15.148:0/480219902 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f20a406c560 0x7f20a406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:41.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.543+0000 7f20bbfff640 1 -- 172.21.15.148:0/480219902 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f20b408b480 con 0x7f20c40cf830 2024-03-29T04:11:41.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.543+0000 7f20ba7fc640 1 --2- 172.21.15.148:0/480219902 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f20a406c560 0x7f20a406ea20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f20c415d560 tx=0x7f20ac0078e0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:41.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.547+0000 7f20bbfff640 1 -- 172.21.15.148:0/480219902 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f20b4059910 con 0x7f20c40cf830 2024-03-29T04:11:41.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.895+0000 7f20c94fb640 1 -- 172.21.15.148:0/480219902 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f20c40b8820 con 0x7f20c40cf830 2024-03-29T04:11:41.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.897+0000 7f20bbfff640 1 -- 172.21.15.148:0/480219902 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f20b4059210 con 0x7f20c40cf830 2024-03-29T04:11:41.876 INFO:teuthology.orchestra.run.smithi148.stdout:[client.0] 2024-03-29T04:11:41.876 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQDgPgZmzmjtDBAAZ0mAEI3GHeThXHriL59lXg== 2024-03-29T04:11:41.877 INFO:teuthology.orchestra.run.smithi148.stdout: caps mds = "allow" 2024-03-29T04:11:41.877 INFO:teuthology.orchestra.run.smithi148.stdout: caps mon = "allow r" 2024-03-29T04:11:41.877 INFO:teuthology.orchestra.run.smithi148.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-29T04:11:41.877 INFO:teuthology.orchestra.run.smithi148.stderr:updated caps for client.0 2024-03-29T04:11:41.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.899+0000 7f20c94fb640 1 -- 172.21.15.148:0/480219902 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f20a406c560 msgr2=0x7f20a406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:41.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.899+0000 7f20c94fb640 1 --2- 172.21.15.148:0/480219902 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f20a406c560 0x7f20a406ea20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f20c415d560 tx=0x7f20ac0078e0 comp rx=0 tx=0).stop 2024-03-29T04:11:41.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.899+0000 7f20c94fb640 1 -- 172.21.15.148:0/480219902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c40cf830 msgr2=0x7f20c41572b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:41.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.899+0000 7f20c94fb640 1 --2- 172.21.15.148:0/480219902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c40cf830 0x7f20c41572b0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f20b400c450 tx=0x7f20b400c910 comp rx=0 tx=0).stop 2024-03-29T04:11:41.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.900+0000 7f20c94fb640 1 -- 172.21.15.148:0/480219902 shutdown_connections 2024-03-29T04:11:41.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.900+0000 7f20c94fb640 1 --2- 172.21.15.148:0/480219902 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f20a406c560 0x7f20a406ea20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:41.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.900+0000 7f20c94fb640 1 --2- 172.21.15.148:0/480219902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c4146e50 0x7f20c40ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:41.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.900+0000 7f20c94fb640 1 --2- 172.21.15.148:0/480219902 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20c415e660 0x7f20c40c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:41.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.900+0000 7f20c94fb640 1 --2- 172.21.15.148:0/480219902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c40cf830 0x7f20c41572b0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:41.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.900+0000 7f20c94fb640 1 -- 172.21.15.148:0/480219902 >> 172.21.15.148:0/480219902 conn(0x7f20c40d11f0 msgr2=0x7f20c40d4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:41.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.900+0000 7f20c94fb640 1 -- 172.21.15.148:0/480219902 shutdown_connections 2024-03-29T04:11:41.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:41.900+0000 7f20c94fb640 1 -- 172.21.15.148:0/480219902 wait complete. 2024-03-29T04:11:41.890 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:11:41.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.013+0000 7f896b242640 1 Processor -- start 2024-03-29T04:11:41.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.013+0000 7f896b242640 1 -- start start 2024-03-29T04:11:41.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.014+0000 7f896b242640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f89640ba030 0x7f89640d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:41.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.014+0000 7f896b242640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89640d10d0 0x7f89640d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:41.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.014+0000 7f896b242640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f896415e7c0 con 0x7f896415e3e0 2024-03-29T04:11:41.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.014+0000 7f896b242640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f89640d3840 con 0x7f89640ba030 2024-03-29T04:11:41.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.014+0000 7f896b242640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f89640d2c30 con 0x7f89640d10d0 2024-03-29T04:11:41.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.014+0000 7f8968fde640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f896415e3e0 0x7f89640d2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40670/0 (socket says 172.21.15.148:40670) 2024-03-29T04:11:41.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.014+0000 7f8968fde640 1 -- 172.21.15.148:0/1366842994 learned_addr learned my addr 172.21.15.148:0/1366842994 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:41.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.015+0000 7f8963fff640 1 --2- 172.21.15.148:0/1366842994 >> v2:172.21.15.157:3300/0 conn(0x7f89640ba030 0x7f89640d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:41.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.015+0000 7f89697df640 1 --2- 172.21.15.148:0/1366842994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89640d10d0 0x7f89640d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:41.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.015+0000 7f89637fe640 1 -- 172.21.15.148:0/1366842994 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2528168664 0 0) 0x7f896415e7c0 con 0x7f896415e3e0 2024-03-29T04:11:41.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.015+0000 7f89637fe640 1 -- 172.21.15.148:0/1366842994 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8954003520 con 0x7f896415e3e0 2024-03-29T04:11:41.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.015+0000 7f8963fff640 1 -- 172.21.15.148:0/1366842994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89640d10d0 msgr2=0x7f89640d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:41.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.015+0000 7f8963fff640 1 --2- 172.21.15.148:0/1366842994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89640d10d0 0x7f89640d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:41.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.015+0000 7f8963fff640 1 -- 172.21.15.148:0/1366842994 >> v1:172.21.15.148:6789/0 conn(0x7f896415e3e0 legacy=0x7f89640d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:41.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.015+0000 7f8963fff640 1 -- 172.21.15.148:0/1366842994 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89640d2f10 con 0x7f89640ba030 2024-03-29T04:11:41.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.015+0000 7f89697df640 1 --2- 172.21.15.148:0/1366842994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89640d10d0 0x7f89640d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:11:41.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.016+0000 7f8963fff640 1 --2- 172.21.15.148:0/1366842994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89640ba030 0x7f89640d3cd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f89500097a0 tx=0x7f895002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=121fb19bf50c7f56 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:41.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.016+0000 7f89637fe640 1 -- 172.21.15.148:0/1366842994 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8950034070 con 0x7f89640ba030 2024-03-29T04:11:41.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.016+0000 7f89637fe640 1 -- 172.21.15.148:0/1366842994 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8950030610 con 0x7f89640ba030 2024-03-29T04:11:41.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.016+0000 7f89637fe640 1 -- 172.21.15.148:0/1366842994 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89500309e0 con 0x7f89640ba030 2024-03-29T04:11:41.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.016+0000 7f896b242640 1 -- 172.21.15.148:0/1366842994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89640ba030 msgr2=0x7f89640d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:41.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.016+0000 7f896b242640 1 --2- 172.21.15.148:0/1366842994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89640ba030 0x7f89640d3cd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f89500097a0 tx=0x7f895002b960 comp rx=0 tx=0).stop 2024-03-29T04:11:41.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.017+0000 7f896b242640 1 -- 172.21.15.148:0/1366842994 shutdown_connections 2024-03-29T04:11:41.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.017+0000 7f896b242640 1 --2- 172.21.15.148:0/1366842994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89640d10d0 0x7f89640d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:41.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.017+0000 7f896b242640 1 --2- 172.21.15.148:0/1366842994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89640ba030 0x7f89640d3cd0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:41.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.017+0000 7f896b242640 1 -- 172.21.15.148:0/1366842994 >> 172.21.15.148:0/1366842994 conn(0x7f89640b6210 msgr2=0x7f89640b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:41.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.017+0000 7f896b242640 1 -- 172.21.15.148:0/1366842994 shutdown_connections 2024-03-29T04:11:41.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.017+0000 7f896b242640 1 -- 172.21.15.148:0/1366842994 wait complete. 2024-03-29T04:11:41.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.018+0000 7f896b242640 1 Processor -- start 2024-03-29T04:11:41.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.018+0000 7f896b242640 1 -- start start 2024-03-29T04:11:41.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.019+0000 7f896b242640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89640ba030 0x7f89640c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:41.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.019+0000 7f896b242640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89640d10d0 0x7f89640c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.019+0000 7f896b242640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f896415e3e0 0x7f896414e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.019+0000 7f896b242640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8964167f50 con 0x7f89640ba030 2024-03-29T04:11:42.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.019+0000 7f896b242640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f89640d34d0 con 0x7f89640d10d0 2024-03-29T04:11:42.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.019+0000 7f896b242640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f89641680d0 con 0x7f896415e3e0 2024-03-29T04:11:42.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.019+0000 7f8968fde640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89640ba030 0x7f89640c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:42.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.019+0000 7f8968fde640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89640ba030 0x7f89640c8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42396/0 (socket says 172.21.15.148:42396) 2024-03-29T04:11:42.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.019+0000 7f8968fde640 1 -- 172.21.15.148:0/1769577463 learned_addr learned my addr 172.21.15.148:0/1769577463 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:42.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.019+0000 7f8963fff640 1 --2- 172.21.15.148:0/1769577463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89640d10d0 0x7f89640c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:42.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.019+0000 7f8968fde640 1 -- 172.21.15.148:0/1769577463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f896415e3e0 msgr2=0x7f896414e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:42.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.019+0000 7f89697df640 1 --2- 172.21.15.148:0/1769577463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f896415e3e0 0x7f896414e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:42.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.019+0000 7f8968fde640 1 --2- 172.21.15.148:0/1769577463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f896415e3e0 0x7f896414e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.019+0000 7f8968fde640 1 -- 172.21.15.148:0/1769577463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89640d10d0 msgr2=0x7f89640c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:42.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.019+0000 7f8968fde640 1 --2- 172.21.15.148:0/1769577463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89640d10d0 0x7f89640c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.019+0000 7f8968fde640 1 -- 172.21.15.148:0/1769577463 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89640c91b0 con 0x7f89640ba030 2024-03-29T04:11:42.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.019+0000 7f8963fff640 1 --2- 172.21.15.148:0/1769577463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89640d10d0 0x7f89640c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:42.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.019+0000 7f8968fde640 1 --2- 172.21.15.148:0/1769577463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89640ba030 0x7f89640c8580 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f89580010e0 tx=0x7f895800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:42.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.019+0000 7f89617fa640 1 -- 172.21.15.148:0/1769577463 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8958017070 con 0x7f89640ba030 2024-03-29T04:11:42.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.019+0000 7f89617fa640 1 -- 172.21.15.148:0/1769577463 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8958005d00 con 0x7f89640ba030 2024-03-29T04:11:42.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.019+0000 7f896b242640 1 -- 172.21.15.148:0/1769577463 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89641609f0 con 0x7f89640ba030 2024-03-29T04:11:42.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.020+0000 7f89617fa640 1 -- 172.21.15.148:0/1769577463 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8958007310 con 0x7f89640ba030 2024-03-29T04:11:42.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.020+0000 7f896b242640 1 -- 172.21.15.148:0/1769577463 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89641614d0 con 0x7f89640ba030 2024-03-29T04:11:42.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.020+0000 7f896b242640 1 -- 172.21.15.148:0/1769577463 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89640687e0 con 0x7f89640ba030 2024-03-29T04:11:42.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.021+0000 7f89617fa640 1 -- 172.21.15.148:0/1769577463 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f895801a070 con 0x7f89640ba030 2024-03-29T04:11:42.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.021+0000 7f89617fa640 1 --2- 172.21.15.148:0/1769577463 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f894806c530 0x7f894806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.021+0000 7f89617fa640 1 -- 172.21.15.148:0/1769577463 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f895808abe0 con 0x7f89640ba030 2024-03-29T04:11:42.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.021+0000 7f8963fff640 1 --2- 172.21.15.148:0/1769577463 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f894806c530 0x7f894806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:42.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.022+0000 7f8963fff640 1 --2- 172.21.15.148:0/1769577463 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f894806c530 0x7f894806e9f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8950036000 tx=0x7f895003b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:42.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.026+0000 7f89617fa640 1 -- 172.21.15.148:0/1769577463 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f89580590e0 con 0x7f89640ba030 2024-03-29T04:11:42.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.239+0000 7f896b242640 1 -- 172.21.15.148:0/1769577463 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f896406d3b0 con 0x7f89640ba030 2024-03-29T04:11:42.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.239+0000 7f89617fa640 1 -- 172.21.15.148:0/1769577463 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5190 (secure 0 0 0) 0x7f89580589e0 con 0x7f89640ba030 2024-03-29T04:11:42.218 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:11:42.218 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2618617956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2618617956},{"type":"v1","addr":"172.21.15.157:6834","nonce":2618617956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5260,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1177855619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1177855619},{"type":"v1","addr":"172.21.15.157:6835","nonce":1177855619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5562,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3610463670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3610463670},{"type":"v1","addr":"172.21.15.148:6835","nonce":3610463670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:11:32.950391+0000","modified":"2024-03-29T04:11:35.013562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5583},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5583":{"gid":5583,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/473448013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":473448013},{"type":"v1","addr":"172.21.15.148:6837","nonce":473448013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5583,"qdb_cluster":[5583]},"id":2}]} 2024-03-29T04:11:42.219 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 17 2024-03-29T04:11:42.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.241+0000 7f896b242640 1 -- 172.21.15.148:0/1769577463 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f894806c530 msgr2=0x7f894806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:42.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.241+0000 7f896b242640 1 --2- 172.21.15.148:0/1769577463 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f894806c530 0x7f894806e9f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8950036000 tx=0x7f895003b040 comp rx=0 tx=0).stop 2024-03-29T04:11:42.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.241+0000 7f896b242640 1 -- 172.21.15.148:0/1769577463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89640ba030 msgr2=0x7f89640c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:42.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.241+0000 7f896b242640 1 --2- 172.21.15.148:0/1769577463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89640ba030 0x7f89640c8580 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f89580010e0 tx=0x7f895800c890 comp rx=0 tx=0).stop 2024-03-29T04:11:42.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.241+0000 7f896b242640 1 -- 172.21.15.148:0/1769577463 shutdown_connections 2024-03-29T04:11:42.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.241+0000 7f896b242640 1 --2- 172.21.15.148:0/1769577463 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f894806c530 0x7f894806e9f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.241+0000 7f896b242640 1 --2- 172.21.15.148:0/1769577463 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f896415e3e0 0x7f896414e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.241+0000 7f896b242640 1 --2- 172.21.15.148:0/1769577463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89640d10d0 0x7f89640c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.241+0000 7f896b242640 1 --2- 172.21.15.148:0/1769577463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89640ba030 0x7f89640c8580 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.241+0000 7f896b242640 1 -- 172.21.15.148:0/1769577463 >> 172.21.15.148:0/1769577463 conn(0x7f89640b6210 msgr2=0x7f89640b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:42.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.241+0000 7f896b242640 1 -- 172.21.15.148:0/1769577463 shutdown_connections 2024-03-29T04:11:42.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.241+0000 7f896b242640 1 -- 172.21.15.148:0/1769577463 wait complete. 2024-03-29T04:11:42.633 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T04:11:32.950391+0000', 'modified': '2024-03-29T04:11:35.013562+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5583}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5583': {'gid': 5583, 'name': 'a', 'rank': 0, 'incarnation': 15, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6837/473448013', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 473448013}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 473448013}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5583, 'qdb_cluster': [5583]} 2024-03-29T04:11:42.633 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T04:11:42.634 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-29T04:11:42.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.759+0000 7f12a21a8640 1 Processor -- start 2024-03-29T04:11:42.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.759+0000 7f12a21a8640 1 -- start start 2024-03-29T04:11:42.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.760+0000 7f12a21a8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f129c15c5f0 0x7f129c0cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.760+0000 7f12a21a8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f129c1608b0 0x7f129c15cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.760+0000 7f12a21a8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f129c0d19d0 con 0x7f129c0ceb50 2024-03-29T04:11:42.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.760+0000 7f12a21a8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f129c0cf470 con 0x7f129c15c5f0 2024-03-29T04:11:42.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.760+0000 7f12a21a8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f129c0ccde0 con 0x7f129c1608b0 2024-03-29T04:11:42.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.760+0000 7f129affd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f129c15c5f0 0x7f129c0cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:42.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.760+0000 7f129b7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f129c0ceb50 0x7f129c0cfd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40672/0 (socket says 172.21.15.148:40672) 2024-03-29T04:11:42.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.760+0000 7f129b7fe640 1 -- 172.21.15.148:0/2761387528 learned_addr learned my addr 172.21.15.148:0/2761387528 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:42.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.760+0000 7f129bfff640 1 --2- 172.21.15.148:0/2761387528 >> v2:172.21.15.157:3300/0 conn(0x7f129c1608b0 0x7f129c15cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:42.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.760+0000 7f129affd640 1 -- 172.21.15.148:0/2761387528 >> v2:172.21.15.157:3300/0 conn(0x7f129c1608b0 msgr2=0x7f129c15cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:42.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.760+0000 7f129affd640 1 --2- 172.21.15.148:0/2761387528 >> v2:172.21.15.157:3300/0 conn(0x7f129c1608b0 0x7f129c15cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.760+0000 7f129affd640 1 -- 172.21.15.148:0/2761387528 >> v1:172.21.15.148:6789/0 conn(0x7f129c0ceb50 legacy=0x7f129c0cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:42.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.760+0000 7f129affd640 1 -- 172.21.15.148:0/2761387528 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129c0ccb00 con 0x7f129c15c5f0 2024-03-29T04:11:42.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.760+0000 7f129a7fc640 1 -- 172.21.15.148:0/2761387528 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4078170061 0 0) 0x7f129c0d19d0 con 0x7f129c0ceb50 2024-03-29T04:11:42.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.761+0000 7f129affd640 1 --2- 172.21.15.148:0/2761387528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f129c15c5f0 0x7f129c0cef30 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f129c0cc690 tx=0x7f128c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7ed19321e8749d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:42.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.761+0000 7f129a7fc640 1 -- 172.21.15.148:0/2761387528 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c035020 con 0x7f129c15c5f0 2024-03-29T04:11:42.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.761+0000 7f129a7fc640 1 -- 172.21.15.148:0/2761387528 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f128c031830 con 0x7f129c15c5f0 2024-03-29T04:11:42.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.761+0000 7f129a7fc640 1 -- 172.21.15.148:0/2761387528 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c031c00 con 0x7f129c15c5f0 2024-03-29T04:11:42.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.762+0000 7f12a21a8640 1 -- 172.21.15.148:0/2761387528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f129c15c5f0 msgr2=0x7f129c0cef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:42.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.762+0000 7f12a21a8640 1 --2- 172.21.15.148:0/2761387528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f129c15c5f0 0x7f129c0cef30 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f129c0cc690 tx=0x7f128c02b8d0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.762+0000 7f12a21a8640 1 -- 172.21.15.148:0/2761387528 shutdown_connections 2024-03-29T04:11:42.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.762+0000 7f12a21a8640 1 --2- 172.21.15.148:0/2761387528 >> v2:172.21.15.157:3300/0 conn(0x7f129c1608b0 0x7f129c15cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.762+0000 7f12a21a8640 1 --2- 172.21.15.148:0/2761387528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f129c15c5f0 0x7f129c0cef30 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.762+0000 7f12a21a8640 1 -- 172.21.15.148:0/2761387528 >> 172.21.15.148:0/2761387528 conn(0x7f129c0b3c70 msgr2=0x7f129c0b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:42.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.762+0000 7f12a21a8640 1 -- 172.21.15.148:0/2761387528 shutdown_connections 2024-03-29T04:11:42.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.762+0000 7f12a21a8640 1 -- 172.21.15.148:0/2761387528 wait complete. 2024-03-29T04:11:42.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.763+0000 7f12a21a8640 1 Processor -- start 2024-03-29T04:11:42.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.763+0000 7f12a21a8640 1 -- start start 2024-03-29T04:11:42.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.764+0000 7f12a21a8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f129c0ceb50 0x7f129c15cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.764+0000 7f12a21a8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f129c1608b0 0x7f129c0b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.764+0000 7f12a21a8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f129c0b57e0 0x7f129c15ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.764+0000 7f12a21a8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f129c0ce540 con 0x7f129c0ceb50 2024-03-29T04:11:42.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.764+0000 7f12a21a8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f129c0cc790 con 0x7f129c0b57e0 2024-03-29T04:11:42.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.764+0000 7f12a21a8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f129c0ce6c0 con 0x7f129c1608b0 2024-03-29T04:11:42.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.764+0000 7f129b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f129c0ceb50 0x7f129c15cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:42.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.764+0000 7f129b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f129c0ceb50 0x7f129c15cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42416/0 (socket says 172.21.15.148:42416) 2024-03-29T04:11:42.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.764+0000 7f129b7fe640 1 -- 172.21.15.148:0/475263541 learned_addr learned my addr 172.21.15.148:0/475263541 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:42.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.764+0000 7f129bfff640 1 --2- 172.21.15.148:0/475263541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f129c0b57e0 0x7f129c15ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:42.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.764+0000 7f129affd640 1 --2- 172.21.15.148:0/475263541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f129c1608b0 0x7f129c0b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:42.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.764+0000 7f129b7fe640 1 -- 172.21.15.148:0/475263541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f129c1608b0 msgr2=0x7f129c0b52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:42.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.764+0000 7f129b7fe640 1 --2- 172.21.15.148:0/475263541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f129c1608b0 0x7f129c0b52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.764+0000 7f129b7fe640 1 -- 172.21.15.148:0/475263541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f129c0b57e0 msgr2=0x7f129c15ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:42.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.764+0000 7f129b7fe640 1 --2- 172.21.15.148:0/475263541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f129c0b57e0 0x7f129c15ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.764+0000 7f129b7fe640 1 -- 172.21.15.148:0/475263541 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129c15d390 con 0x7f129c0ceb50 2024-03-29T04:11:42.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.764+0000 7f129bfff640 1 --2- 172.21.15.148:0/475263541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f129c0b57e0 0x7f129c15ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:42.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.764+0000 7f129b7fe640 1 --2- 172.21.15.148:0/475263541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f129c0ceb50 0x7f129c15cc80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f129000c4a0 tx=0x7f129000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:42.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.764+0000 7f1283fff640 1 -- 172.21.15.148:0/475263541 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290017070 con 0x7f129c0ceb50 2024-03-29T04:11:42.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.765+0000 7f1283fff640 1 -- 172.21.15.148:0/475263541 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1290005ce0 con 0x7f129c0ceb50 2024-03-29T04:11:42.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.765+0000 7f12a21a8640 1 -- 172.21.15.148:0/475263541 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f129c15d650 con 0x7f129c0ceb50 2024-03-29T04:11:42.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.765+0000 7f1283fff640 1 -- 172.21.15.148:0/475263541 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12900072b0 con 0x7f129c0ceb50 2024-03-29T04:11:42.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.765+0000 7f12a21a8640 1 -- 172.21.15.148:0/475263541 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f129c15dc10 con 0x7f129c0ceb50 2024-03-29T04:11:42.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.766+0000 7f1283fff640 1 -- 172.21.15.148:0/475263541 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f129001a070 con 0x7f129c0ceb50 2024-03-29T04:11:42.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.767+0000 7f1283fff640 1 --2- 172.21.15.148:0/475263541 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f128806c530 0x7f128806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.767+0000 7f129affd640 1 --2- 172.21.15.148:0/475263541 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f128806c530 0x7f128806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:42.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.767+0000 7f1283fff640 1 -- 172.21.15.148:0/475263541 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f129008ac50 con 0x7f129c0ceb50 2024-03-29T04:11:42.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.767+0000 7f129affd640 1 --2- 172.21.15.148:0/475263541 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f128806c530 0x7f128806e9f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f128c036000 tx=0x7f128c008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:42.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.767+0000 7f12a4d98740 1 Processor -- start 2024-03-29T04:11:42.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.767+0000 7f12a4d98740 1 -- start start 2024-03-29T04:11:42.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.768+0000 7f12a4d98740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55867ea232e0 0x55867ea20fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.768+0000 7f12a4d98740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55867ea236c0 0x55867e9da910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.768+0000 7f12a4d98740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55867ea02c20 0x55867e9dae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.768+0000 7f12a4d98740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55867e93f020 con 0x55867ea02c20 2024-03-29T04:11:42.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.768+0000 7f12a4d98740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55867e8e75f0 con 0x55867ea232e0 2024-03-29T04:11:42.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.768+0000 7f12a4d98740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55867e93afd0 con 0x55867ea236c0 2024-03-29T04:11:42.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.768+0000 7f129bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55867ea02c20 0x55867e9dae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:42.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.768+0000 7f129bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55867ea02c20 0x55867e9dae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42422/0 (socket says 172.21.15.148:42422) 2024-03-29T04:11:42.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.768+0000 7f129bfff640 1 -- 172.21.15.148:0/125142655 learned_addr learned my addr 172.21.15.148:0/125142655 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:42.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.768+0000 7f129b7fe640 1 --2- 172.21.15.148:0/125142655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55867ea232e0 0x55867ea20fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:42.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.769+0000 7f129affd640 1 --2- 172.21.15.148:0/125142655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55867ea236c0 0x55867e9da910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:42.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.769+0000 7f129bfff640 1 -- 172.21.15.148:0/125142655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55867ea236c0 msgr2=0x55867e9da910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:42.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.769+0000 7f129bfff640 1 --2- 172.21.15.148:0/125142655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55867ea236c0 0x55867e9da910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.769+0000 7f129bfff640 1 -- 172.21.15.148:0/125142655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55867ea232e0 msgr2=0x55867ea20fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:42.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.769+0000 7f129bfff640 1 --2- 172.21.15.148:0/125142655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55867ea232e0 0x55867ea20fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.769+0000 7f129bfff640 1 -- 172.21.15.148:0/125142655 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55867e6532d0 con 0x55867ea02c20 2024-03-29T04:11:42.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.769+0000 7f129bfff640 1 --2- 172.21.15.148:0/125142655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55867ea02c20 0x55867e9dae50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1294007440 tx=0x7f1294007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a7189f77e86f2c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:42.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.769+0000 7f12817fa640 1 -- 172.21.15.148:0/125142655 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12940147e0 con 0x55867ea02c20 2024-03-29T04:11:42.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.769+0000 7f12817fa640 1 -- 172.21.15.148:0/125142655 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f129401a070 con 0x55867ea02c20 2024-03-29T04:11:42.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.769+0000 7f12817fa640 1 -- 172.21.15.148:0/125142655 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1294014bc0 con 0x55867ea02c20 2024-03-29T04:11:42.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.770+0000 7f12a4d98740 1 -- 172.21.15.148:0/125142655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55867ea02c20 msgr2=0x55867e9dae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:42.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.770+0000 7f12a4d98740 1 --2- 172.21.15.148:0/125142655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55867ea02c20 0x55867e9dae50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1294007440 tx=0x7f1294007900 comp rx=0 tx=0).stop 2024-03-29T04:11:42.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.770+0000 7f12a4d98740 1 -- 172.21.15.148:0/125142655 shutdown_connections 2024-03-29T04:11:42.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.770+0000 7f12a4d98740 1 --2- 172.21.15.148:0/125142655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55867ea02c20 0x55867e9dae50 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.770+0000 7f12a4d98740 1 --2- 172.21.15.148:0/125142655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55867ea236c0 0x55867e9da910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.770+0000 7f12a4d98740 1 --2- 172.21.15.148:0/125142655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55867ea232e0 0x55867ea20fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.770+0000 7f12a4d98740 1 -- 172.21.15.148:0/125142655 >> 172.21.15.148:0/125142655 conn(0x55867e9d7a70 msgr2=0x55867ea205f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:42.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.770+0000 7f12a4d98740 1 -- 172.21.15.148:0/125142655 shutdown_connections 2024-03-29T04:11:42.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.770+0000 7f12a4d98740 1 -- 172.21.15.148:0/125142655 wait complete. 2024-03-29T04:11:42.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.770+0000 7f12a4d98740 1 -- start start 2024-03-29T04:11:42.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.770+0000 7f12a4d98740 1 Processor -- start 2024-03-29T04:11:42.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.771+0000 7f12a4d98740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55867ea232e0 0x55867e9f6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.771+0000 7f12a4d98740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55867ea236c0 0x55867e9f60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.771+0000 7f12a4d98740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55867e9f5bd0 0x55867e9f65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.771+0000 7f12a4d98740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55867e9d60a0 con 0x55867ea232e0 2024-03-29T04:11:42.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.771+0000 7f12a4d98740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55867e8fa9c0 con 0x55867ea236c0 2024-03-29T04:11:42.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.771+0000 7f12a4d98740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55867e9d9ea0 con 0x55867e9f5bd0 2024-03-29T04:11:42.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.771+0000 7f12a4d98740 10 client.0 _ll_register_callbacks cb 0x55867e8defc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:11:42.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.771+0000 7f129b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55867ea232e0 0x55867e9f6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:42.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.771+0000 7f129b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55867ea232e0 0x55867e9f6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42434/0 (socket says 172.21.15.148:42434) 2024-03-29T04:11:42.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.771+0000 7f129b7fe640 1 -- 172.21.15.148:0/3426446891 learned_addr learned my addr 172.21.15.148:0/3426446891 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:42.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.771+0000 7f129affd640 1 --2- 172.21.15.148:0/3426446891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55867ea236c0 0x55867e9f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:42.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.771+0000 7f129bfff640 1 --2- 172.21.15.148:0/3426446891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55867e9f5bd0 0x55867e9f65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:42.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.772+0000 7f129b7fe640 1 -- 172.21.15.148:0/3426446891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55867e9f5bd0 msgr2=0x55867e9f65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:42.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.772+0000 7f129b7fe640 1 --2- 172.21.15.148:0/3426446891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55867e9f5bd0 0x55867e9f65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.772+0000 7f129b7fe640 1 -- 172.21.15.148:0/3426446891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55867ea236c0 msgr2=0x55867e9f60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:42.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.772+0000 7f129b7fe640 1 --2- 172.21.15.148:0/3426446891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55867ea236c0 0x55867e9f60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.772+0000 7f129b7fe640 1 -- 172.21.15.148:0/3426446891 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55867e71d6b0 con 0x55867ea232e0 2024-03-29T04:11:42.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.772+0000 7f129b7fe640 1 --2- 172.21.15.148:0/3426446891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55867ea232e0 0x55867e9f6de0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1290088fd0 tx=0x7f1290058b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1ba9adb1d59368 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:42.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.772+0000 7f127a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-03-29T04:11:42.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.772+0000 7f127a7fc640 1 -- 172.21.15.148:0/3426446891 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290058e20 con 0x55867ea232e0 2024-03-29T04:11:42.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.772+0000 7f127a7fc640 1 -- 172.21.15.148:0/3426446891 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f1290061110 con 0x55867ea232e0 2024-03-29T04:11:42.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.772+0000 7f12a4d98740 1 -- 172.21.15.148:0/3426446891 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f129401a070 con 0x55867ea232e0 2024-03-29T04:11:42.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.772+0000 7f127a7fc640 1 -- 172.21.15.148:0/3426446891 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f1290056d80 con 0x55867ea232e0 2024-03-29T04:11:42.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.772+0000 7f12a4d98740 10 client.5772 fetch_fsmap learned FSMap version 17 2024-03-29T04:11:42.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.772+0000 7f12a4d98740 1 -- 172.21.15.148:0/3426446891 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55867ea216f0 con 0x55867ea232e0 2024-03-29T04:11:42.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.772+0000 7f127a7fc640 1 -- 172.21.15.148:0/3426446891 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f129401a070 con 0x55867ea232e0 2024-03-29T04:11:42.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.772+0000 7f127a7fc640 1 -- 172.21.15.148:0/3426446891 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f1290007450 con 0x55867ea232e0 2024-03-29T04:11:42.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.772+0000 7f12a4d98740 10 client.5772 fetch_fsmap finished waiting for FSMap version 17 2024-03-29T04:11:42.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.772+0000 7f12a4d98740 10 client.5772 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-29T04:11:42.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.772+0000 7f12a4d98740 20 client.5772 populate_metadata read hostname 'smithi148' 2024-03-29T04:11:42.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.772+0000 7f12a4d98740 1 --2- 172.21.15.148:0/3426446891 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x55867e9fffa0 0x55867ea22080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.772+0000 7f12a4d98740 4 client.5772 mds_command: new command op to 5583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T04:11:42.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.772+0000 7f12a4d98740 1 -- 172.21.15.148:0/3426446891 --> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55867e9134e0 con 0x55867e9fffa0 2024-03-29T04:11:42.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.772+0000 7f129affd640 1 --2- 172.21.15.148:0/3426446891 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x55867e9fffa0 0x55867ea22080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:42.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.773+0000 7f127a7fc640 10 client.5772 ms_handle_connect on v2:172.21.15.148:6836/473448013 2024-03-29T04:11:42.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.773+0000 7f129affd640 1 --2- 172.21.15.148:0/3426446891 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x55867e9fffa0 0x55867ea22080 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e668bc5332a8a778 server_cookie=dde9f22cdc907c20 in_seq=0 out_seq=0 2024-03-29T04:11:42.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.773+0000 7f127a7fc640 1 -- 172.21.15.148:0/3426446891 <== mds.0 v2:172.21.15.148:6836/473448013 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55867e9134e0 con 0x55867e9fffa0 2024-03-29T04:11:42.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.773+0000 7f127a7fc640 10 client.5772 handle_command_reply: tid=0 2024-03-29T04:11:42.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.773+0000 7f12a4d98740 1 client.5772 shutdown 2024-03-29T04:11:42.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.774+0000 7f12a4d98740 1 -- 172.21.15.148:0/3426446891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55867ea232e0 msgr2=0x55867e9f6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:42.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.774+0000 7f12a4d98740 1 --2- 172.21.15.148:0/3426446891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55867ea232e0 0x55867e9f6de0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1290088fd0 tx=0x7f1290058b00 comp rx=0 tx=0).stop 2024-03-29T04:11:42.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.774+0000 7f12a4d98740 1 -- 172.21.15.148:0/3426446891 shutdown_connections 2024-03-29T04:11:42.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.774+0000 7f12a4d98740 1 --2- 172.21.15.148:0/3426446891 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x55867e9fffa0 0x55867ea22080 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.774+0000 7f12a4d98740 1 --2- 172.21.15.148:0/3426446891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55867e9f5bd0 0x55867e9f65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.774+0000 7f12a4d98740 1 --2- 172.21.15.148:0/3426446891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55867ea236c0 0x55867e9f60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.774+0000 7f127a7fc640 0 client.5772 ms_handle_reset on v2:172.21.15.148:6836/473448013 2024-03-29T04:11:42.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.774+0000 7f12a4d98740 1 --2- 172.21.15.148:0/3426446891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55867ea232e0 0x55867e9f6de0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.774+0000 7f12a4d98740 1 -- 172.21.15.148:0/3426446891 >> 172.21.15.148:0/3426446891 conn(0x55867ea02c20 msgr2=0x55867e9d46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:42.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.774+0000 7f12a4d98740 1 -- 172.21.15.148:0/3426446891 shutdown_connections 2024-03-29T04:11:42.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.774+0000 7f12a4d98740 1 -- 172.21.15.148:0/3426446891 wait complete. 2024-03-29T04:11:42.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.774+0000 7f12a4d98740 20 client.5772 trim_cache size 0 max 16384 2024-03-29T04:11:42.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.792+0000 7f12a4d98740 1 Processor -- start 2024-03-29T04:11:42.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.792+0000 7f12a4d98740 1 -- start start 2024-03-29T04:11:42.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.793+0000 7f12a4d98740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55867e9f5bd0 0x55867ea47c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.793+0000 7f12a4d98740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55867ea232e0 0x55867ea20850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.793+0000 7f12a4d98740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55867ea236c0 0x55867ea20d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.793+0000 7f12a4d98740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f129c06dff0 con 0x55867ea232e0 2024-03-29T04:11:42.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.793+0000 7f12a4d98740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f129c1f42c0 con 0x55867e9f5bd0 2024-03-29T04:11:42.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.793+0000 7f12a4d98740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f129c06d9a0 con 0x55867ea236c0 2024-03-29T04:11:42.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.793+0000 7f129affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55867ea232e0 0x55867ea20850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:42.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.793+0000 7f129affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55867ea232e0 0x55867ea20850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42450/0 (socket says 172.21.15.148:42450) 2024-03-29T04:11:42.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.793+0000 7f129affd640 1 -- 172.21.15.148:0/559945484 learned_addr learned my addr 172.21.15.148:0/559945484 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:42.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.793+0000 7f129affd640 1 -- 172.21.15.148:0/559945484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55867ea236c0 msgr2=0x55867ea20d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:42.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.793+0000 7f129affd640 1 --2- 172.21.15.148:0/559945484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55867ea236c0 0x55867ea20d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.793+0000 7f129affd640 1 -- 172.21.15.148:0/559945484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55867e9f5bd0 msgr2=0x55867ea47c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:42.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.793+0000 7f129affd640 1 --2- 172.21.15.148:0/559945484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55867e9f5bd0 0x55867ea47c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.793+0000 7f129affd640 1 -- 172.21.15.148:0/559945484 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55867e9ff3c0 con 0x55867ea232e0 2024-03-29T04:11:42.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.793+0000 7f129affd640 1 --2- 172.21.15.148:0/559945484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55867ea232e0 0x55867ea20850 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f128c02bee0 tx=0x7f128c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1f24385a89c15b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:42.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.794+0000 7f1280ff9640 1 -- 172.21.15.148:0/559945484 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c05a070 con 0x55867ea232e0 2024-03-29T04:11:42.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.794+0000 7f1280ff9640 1 -- 172.21.15.148:0/559945484 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f128c0550d0 con 0x55867ea232e0 2024-03-29T04:11:42.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.794+0000 7f1280ff9640 1 -- 172.21.15.148:0/559945484 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c035040 con 0x55867ea232e0 2024-03-29T04:11:42.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.794+0000 7f12a4d98740 1 -- 172.21.15.148:0/559945484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55867ea232e0 msgr2=0x55867ea20850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:42.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.794+0000 7f12a4d98740 1 --2- 172.21.15.148:0/559945484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55867ea232e0 0x55867ea20850 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f128c02bee0 tx=0x7f128c030000 comp rx=0 tx=0).stop 2024-03-29T04:11:42.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.794+0000 7f12a4d98740 1 -- 172.21.15.148:0/559945484 shutdown_connections 2024-03-29T04:11:42.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.794+0000 7f12a4d98740 1 --2- 172.21.15.148:0/559945484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55867ea236c0 0x55867ea20d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.794+0000 7f12a4d98740 1 --2- 172.21.15.148:0/559945484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55867ea232e0 0x55867ea20850 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.794+0000 7f12a4d98740 1 --2- 172.21.15.148:0/559945484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55867e9f5bd0 0x55867ea47c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.794+0000 7f12a4d98740 1 -- 172.21.15.148:0/559945484 >> 172.21.15.148:0/559945484 conn(0x55867ea02c20 msgr2=0x55867e9d6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:42.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.794+0000 7f12a4d98740 1 -- 172.21.15.148:0/559945484 shutdown_connections 2024-03-29T04:11:42.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.794+0000 7f12a4d98740 1 -- 172.21.15.148:0/559945484 wait complete. 2024-03-29T04:11:42.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.795+0000 7f12a4d98740 1 -- start start 2024-03-29T04:11:42.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.795+0000 7f12a4d98740 1 Processor -- start 2024-03-29T04:11:42.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.796+0000 7f12a4d98740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55867e9f5bd0 0x55867ea343e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.796+0000 7f12a4d98740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55867ea232e0 0x55867eaa49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.796+0000 7f12a4d98740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55867ea236c0 0x55867e9fb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.796+0000 7f12a4d98740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55867e9ffe20 con 0x55867ea232e0 2024-03-29T04:11:42.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.796+0000 7f12a4d98740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55867ea22c90 con 0x55867e9f5bd0 2024-03-29T04:11:42.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.796+0000 7f12a4d98740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55867ea00410 con 0x55867ea236c0 2024-03-29T04:11:42.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.796+0000 7f12a4d98740 10 client.0 _ll_register_callbacks cb 0x7f1290007f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:11:42.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.796+0000 7f129affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55867ea232e0 0x55867eaa49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:42.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.796+0000 7f129affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55867ea232e0 0x55867eaa49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42452/0 (socket says 172.21.15.148:42452) 2024-03-29T04:11:42.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.796+0000 7f129affd640 1 -- 172.21.15.148:0/678265306 learned_addr learned my addr 172.21.15.148:0/678265306 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:42.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.796+0000 7f129affd640 1 -- 172.21.15.148:0/678265306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55867ea236c0 msgr2=0x55867e9fb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:42.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.796+0000 7f129affd640 1 --2- 172.21.15.148:0/678265306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55867ea236c0 0x55867e9fb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.796+0000 7f129b7fe640 1 --2- 172.21.15.148:0/678265306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55867e9f5bd0 0x55867ea343e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:42.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.796+0000 7f129affd640 1 -- 172.21.15.148:0/678265306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55867e9f5bd0 msgr2=0x55867ea343e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:42.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.796+0000 7f129affd640 1 --2- 172.21.15.148:0/678265306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55867e9f5bd0 0x55867ea343e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.796+0000 7f129affd640 1 -- 172.21.15.148:0/678265306 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55867eaa4ef0 con 0x55867ea232e0 2024-03-29T04:11:42.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.796+0000 7f129affd640 1 --2- 172.21.15.148:0/678265306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55867ea232e0 0x55867eaa49b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f128c02bee0 tx=0x7f128c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e335a9c626f84f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:42.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.796+0000 7f127a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-03-29T04:11:42.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.796+0000 7f127a7fc640 1 -- 172.21.15.148:0/678265306 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c05a040 con 0x55867ea232e0 2024-03-29T04:11:42.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.796+0000 7f127a7fc640 1 -- 172.21.15.148:0/678265306 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f128c005990 con 0x55867ea232e0 2024-03-29T04:11:42.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.796+0000 7f12a4d98740 1 -- 172.21.15.148:0/678265306 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f128c0550d0 con 0x55867ea232e0 2024-03-29T04:11:42.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.796+0000 7f127a7fc640 1 -- 172.21.15.148:0/678265306 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f128c06b050 con 0x55867ea232e0 2024-03-29T04:11:42.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.796+0000 7f12a4d98740 10 client.5778 fetch_fsmap learned FSMap version 17 2024-03-29T04:11:42.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.796+0000 7f12a4d98740 1 -- 172.21.15.148:0/678265306 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55867ea1f340 con 0x55867ea232e0 2024-03-29T04:11:42.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.797+0000 7f127a7fc640 1 -- 172.21.15.148:0/678265306 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f128c0550d0 con 0x55867ea232e0 2024-03-29T04:11:42.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.797+0000 7f127a7fc640 1 -- 172.21.15.148:0/678265306 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f128c06e020 con 0x55867ea232e0 2024-03-29T04:11:42.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.797+0000 7f12a4d98740 10 client.5778 fetch_fsmap finished waiting for FSMap version 17 2024-03-29T04:11:42.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.797+0000 7f12a4d98740 10 client.5778 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-29T04:11:42.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.797+0000 7f12a4d98740 20 client.5778 populate_metadata read hostname 'smithi148' 2024-03-29T04:11:42.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.797+0000 7f12a4d98740 1 --2- 172.21.15.148:0/678265306 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x55867e9f6280 0x55867e9d46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.797+0000 7f12a4d98740 4 client.5778 mds_command: new command op to 5583 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T04:11:42.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.797+0000 7f12a4d98740 1 -- 172.21.15.148:0/678265306 --> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] -- command(tid 0: {"prefix": "status"}) -- 0x55867e94c560 con 0x55867e9f6280 2024-03-29T04:11:42.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.797+0000 7f129b7fe640 1 --2- 172.21.15.148:0/678265306 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x55867e9f6280 0x55867e9d46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:42.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.797+0000 7f127a7fc640 10 client.5778 ms_handle_connect on v2:172.21.15.148:6836/473448013 2024-03-29T04:11:42.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.797+0000 7f129b7fe640 1 --2- 172.21.15.148:0/678265306 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x55867e9f6280 0x55867e9d46f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3325c0099a2f1702 server_cookie=9373ca26dfecc711 in_seq=0 out_seq=0 2024-03-29T04:11:42.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.797+0000 7f127a7fc640 1 -- 172.21.15.148:0/678265306 <== mds.0 v2:172.21.15.148:6836/473448013 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55867e94c560 con 0x55867e9f6280 2024-03-29T04:11:42.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.797+0000 7f127a7fc640 10 client.5778 handle_command_reply: tid=0 2024-03-29T04:11:42.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.797+0000 7f12a4d98740 1 client.5778 shutdown 2024-03-29T04:11:42.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.797+0000 7f12a4d98740 1 -- 172.21.15.148:0/678265306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55867ea232e0 msgr2=0x55867eaa49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:42.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.797+0000 7f12a4d98740 1 --2- 172.21.15.148:0/678265306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55867ea232e0 0x55867eaa49b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f128c02bee0 tx=0x7f128c004730 comp rx=0 tx=0).stop 2024-03-29T04:11:42.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.797+0000 7f12a4d98740 1 -- 172.21.15.148:0/678265306 shutdown_connections 2024-03-29T04:11:42.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.797+0000 7f12a4d98740 1 --2- 172.21.15.148:0/678265306 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x55867e9f6280 0x55867e9d46f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.798+0000 7f12a4d98740 1 --2- 172.21.15.148:0/678265306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55867ea236c0 0x55867e9fb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.798+0000 7f12a4d98740 1 --2- 172.21.15.148:0/678265306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55867ea232e0 0x55867eaa49b0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.798+0000 7f12a4d98740 1 --2- 172.21.15.148:0/678265306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55867e9f5bd0 0x55867ea343e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.798+0000 7f127a7fc640 0 client.5778 ms_handle_reset on v2:172.21.15.148:6836/473448013 2024-03-29T04:11:42.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.798+0000 7f12a4d98740 1 -- 172.21.15.148:0/678265306 >> 172.21.15.148:0/678265306 conn(0x55867ea02c20 msgr2=0x55867e9d6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:42.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.798+0000 7f12a4d98740 1 -- 172.21.15.148:0/678265306 shutdown_connections 2024-03-29T04:11:42.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.798+0000 7f12a4d98740 1 -- 172.21.15.148:0/678265306 wait complete. 2024-03-29T04:11:42.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.798+0000 7f12a4d98740 20 client.5778 trim_cache size 0 max 16384 2024-03-29T04:11:42.784 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-03-29T04:11:42.784 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "56b659bb-6db8-45e8-b612-135349c3fd6b", 2024-03-29T04:11:42.784 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2024-03-29T04:11:42.784 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 5583, 2024-03-29T04:11:42.784 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2024-03-29T04:11:42.785 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2024-03-29T04:11:42.785 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2024-03-29T04:11:42.785 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 9.2107990700000002, 2024-03-29T04:11:42.785 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 17, 2024-03-29T04:11:42.785 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 41, 2024-03-29T04:11:42.785 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 39, 2024-03-29T04:11:42.785 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 16.866653885000002 2024-03-29T04:11:42.785 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-03-29T04:11:42.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.798+0000 7f12a21a8640 1 -- 172.21.15.148:0/475263541 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f128806c530 msgr2=0x7f128806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:42.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.798+0000 7f12a21a8640 1 --2- 172.21.15.148:0/475263541 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f128806c530 0x7f128806e9f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f128c036000 tx=0x7f128c008030 comp rx=0 tx=0).stop 2024-03-29T04:11:42.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.798+0000 7f12a21a8640 1 -- 172.21.15.148:0/475263541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f129c0ceb50 msgr2=0x7f129c15cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:42.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.798+0000 7f12a21a8640 1 --2- 172.21.15.148:0/475263541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f129c0ceb50 0x7f129c15cc80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f129000c4a0 tx=0x7f129000c960 comp rx=0 tx=0).stop 2024-03-29T04:11:42.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.798+0000 7f12a21a8640 1 -- 172.21.15.148:0/475263541 shutdown_connections 2024-03-29T04:11:42.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.798+0000 7f12a21a8640 1 --2- 172.21.15.148:0/475263541 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f128806c530 0x7f128806e9f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.798+0000 7f12a21a8640 1 --2- 172.21.15.148:0/475263541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f129c0b57e0 0x7f129c15ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.798+0000 7f12a21a8640 1 --2- 172.21.15.148:0/475263541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f129c1608b0 0x7f129c0b52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.798+0000 7f12a21a8640 1 --2- 172.21.15.148:0/475263541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f129c0ceb50 0x7f129c15cc80 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.798+0000 7f12a21a8640 1 -- 172.21.15.148:0/475263541 >> 172.21.15.148:0/475263541 conn(0x7f129c0b3c70 msgr2=0x7f129c0d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:42.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.798+0000 7f12a21a8640 1 -- 172.21.15.148:0/475263541 shutdown_connections 2024-03-29T04:11:42.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.798+0000 7f12a21a8640 1 -- 172.21.15.148:0/475263541 wait complete. 2024-03-29T04:11:42.791 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T04:11:42.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.915+0000 7f91475d4640 1 Processor -- start 2024-03-29T04:11:42.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.915+0000 7f91475d4640 1 -- start start 2024-03-29T04:11:42.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.916+0000 7f91475d4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f914015e800 0x7f914006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.916+0000 7f91475d4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f914015ed50 0x7f91400cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.916+0000 7f91475d4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f914006d010 con 0x7f91400d0220 2024-03-29T04:11:42.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.916+0000 7f91475d4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f91400d0690 con 0x7f914015ed50 2024-03-29T04:11:42.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.916+0000 7f91475d4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f914006d2f0 con 0x7f914015e800 2024-03-29T04:11:42.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.916+0000 7f9145b71640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f91400d0220 0x7f9140162240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.148:36946/0 (socket says 172.21.15.148:36946) 2024-03-29T04:11:42.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.916+0000 7f9145b71640 1 -- 172.21.15.148:0/330612097 learned_addr learned my addr 172.21.15.148:0/330612097 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:42.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.916+0000 7f9145370640 1 --2- 172.21.15.148:0/330612097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f914015e800 0x7f914006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:42.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.916+0000 7f9144b6f640 1 --2- 172.21.15.148:0/330612097 >> v2:172.21.15.157:3300/0 conn(0x7f914015ed50 0x7f91400cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:42.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.916+0000 7f9137fff640 1 -- 172.21.15.148:0/330612097 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1493369772 0 0) 0x7f914006d010 con 0x7f91400d0220 2024-03-29T04:11:42.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.916+0000 7f9137fff640 1 -- 172.21.15.148:0/330612097 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f912c003520 con 0x7f91400d0220 2024-03-29T04:11:42.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.916+0000 7f9144b6f640 1 -- 172.21.15.148:0/330612097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f914015e800 msgr2=0x7f914006c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:42.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.916+0000 7f9144b6f640 1 --2- 172.21.15.148:0/330612097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f914015e800 0x7f914006c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.916+0000 7f9144b6f640 1 -- 172.21.15.148:0/330612097 >> v1:172.21.15.157:6789/0 conn(0x7f91400d0220 legacy=0x7f9140162240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:42.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.916+0000 7f9144b6f640 1 -- 172.21.15.148:0/330612097 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f914006d5d0 con 0x7f914015ed50 2024-03-29T04:11:42.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.917+0000 7f9144b6f640 1 --2- 172.21.15.148:0/330612097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f914015ed50 0x7f91400cfce0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f91300097a0 tx=0x7f913002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cf90944afbd1772 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:42.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.917+0000 7f9137fff640 1 -- 172.21.15.148:0/330612097 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91300315a0 con 0x7f914015ed50 2024-03-29T04:11:42.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.917+0000 7f9137fff640 1 -- 172.21.15.148:0/330612097 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9130037070 con 0x7f914015ed50 2024-03-29T04:11:42.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.917+0000 7f9137fff640 1 -- 172.21.15.148:0/330612097 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130031960 con 0x7f914015ed50 2024-03-29T04:11:42.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.918+0000 7f91475d4640 1 -- 172.21.15.148:0/330612097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f914015ed50 msgr2=0x7f91400cfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:42.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.918+0000 7f91475d4640 1 --2- 172.21.15.148:0/330612097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f914015ed50 0x7f91400cfce0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f91300097a0 tx=0x7f913002b840 comp rx=0 tx=0).stop 2024-03-29T04:11:42.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.918+0000 7f91475d4640 1 -- 172.21.15.148:0/330612097 shutdown_connections 2024-03-29T04:11:42.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.918+0000 7f91475d4640 1 --2- 172.21.15.148:0/330612097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f914015ed50 0x7f91400cfce0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.918+0000 7f91475d4640 1 --2- 172.21.15.148:0/330612097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f914015e800 0x7f914006c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.918+0000 7f91475d4640 1 -- 172.21.15.148:0/330612097 >> 172.21.15.148:0/330612097 conn(0x7f91400d2110 msgr2=0x7f91400d24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:42.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.918+0000 7f91475d4640 1 -- 172.21.15.148:0/330612097 shutdown_connections 2024-03-29T04:11:42.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.918+0000 7f91475d4640 1 -- 172.21.15.148:0/330612097 wait complete. 2024-03-29T04:11:42.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.920+0000 7f91475d4640 1 Processor -- start 2024-03-29T04:11:42.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.920+0000 7f91475d4640 1 -- start start 2024-03-29T04:11:42.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.920+0000 7f91475d4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91400d0220 0x7f91400ae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.920+0000 7f91475d4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f914015e800 0x7f914014a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.920+0000 7f91475d4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f914015ed50 0x7f91400b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.920+0000 7f91475d4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f914006cc00 con 0x7f91400d0220 2024-03-29T04:11:42.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.920+0000 7f91475d4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f91400b1b30 con 0x7f914015ed50 2024-03-29T04:11:42.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.920+0000 7f91475d4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f914006cd80 con 0x7f914015e800 2024-03-29T04:11:42.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.920+0000 7f9145370640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91400d0220 0x7f91400ae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:42.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.920+0000 7f9145370640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91400d0220 0x7f91400ae2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42460/0 (socket says 172.21.15.148:42460) 2024-03-29T04:11:42.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.920+0000 7f9145370640 1 -- 172.21.15.148:0/3358771663 learned_addr learned my addr 172.21.15.148:0/3358771663 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:42.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.920+0000 7f9144b6f640 1 --2- 172.21.15.148:0/3358771663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f914015e800 0x7f914014a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:42.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.921+0000 7f9145b71640 1 --2- 172.21.15.148:0/3358771663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f914015ed50 0x7f91400b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:42.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.921+0000 7f9145370640 1 -- 172.21.15.148:0/3358771663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f914015e800 msgr2=0x7f914014a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:42.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.921+0000 7f9145370640 1 --2- 172.21.15.148:0/3358771663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f914015e800 0x7f914014a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.921+0000 7f9145370640 1 -- 172.21.15.148:0/3358771663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f914015ed50 msgr2=0x7f91400b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:42.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.921+0000 7f9145370640 1 --2- 172.21.15.148:0/3358771663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f914015ed50 0x7f91400b0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:42.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.921+0000 7f9145370640 1 -- 172.21.15.148:0/3358771663 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91400afef0 con 0x7f91400d0220 2024-03-29T04:11:42.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.921+0000 7f9145370640 1 --2- 172.21.15.148:0/3358771663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91400d0220 0x7f91400ae2f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f913c00e9b0 tx=0x7f913c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:42.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.921+0000 7f9135ffb640 1 -- 172.21.15.148:0/3358771663 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c017070 con 0x7f91400d0220 2024-03-29T04:11:42.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.921+0000 7f9135ffb640 1 -- 172.21.15.148:0/3358771663 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f913c00c9a0 con 0x7f91400d0220 2024-03-29T04:11:42.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.921+0000 7f91475d4640 1 -- 172.21.15.148:0/3358771663 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91400b0190 con 0x7f91400d0220 2024-03-29T04:11:42.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.921+0000 7f9135ffb640 1 -- 172.21.15.148:0/3358771663 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c00cd90 con 0x7f91400d0220 2024-03-29T04:11:42.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.921+0000 7f91475d4640 1 -- 172.21.15.148:0/3358771663 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91400b0750 con 0x7f91400d0220 2024-03-29T04:11:42.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.922+0000 7f91475d4640 1 -- 172.21.15.148:0/3358771663 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91400687e0 con 0x7f91400d0220 2024-03-29T04:11:42.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.923+0000 7f9135ffb640 1 -- 172.21.15.148:0/3358771663 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f913c01a070 con 0x7f91400d0220 2024-03-29T04:11:42.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.923+0000 7f9135ffb640 1 --2- 172.21.15.148:0/3358771663 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f912406c530 0x7f912406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:42.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.923+0000 7f9144b6f640 1 --2- 172.21.15.148:0/3358771663 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f912406c530 0x7f912406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:42.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.923+0000 7f9135ffb640 1 -- 172.21.15.148:0/3358771663 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f913c08bbc0 con 0x7f91400d0220 2024-03-29T04:11:42.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.924+0000 7f9144b6f640 1 --2- 172.21.15.148:0/3358771663 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f912406c530 0x7f912406e9f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9130036000 tx=0x7f913003d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:42.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:42.928+0000 7f9135ffb640 1 -- 172.21.15.148:0/3358771663 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f913c0599f0 con 0x7f91400d0220 2024-03-29T04:11:43.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:43.150+0000 7f91475d4640 1 -- 172.21.15.148:0/3358771663 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f91400d2ff0 con 0x7f91400d0220 2024-03-29T04:11:43.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:43.151+0000 7f9135ffb640 1 -- 172.21.15.148:0/3358771663 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f913c05dcf0 con 0x7f91400d0220 2024-03-29T04:11:43.130 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T04:11:43.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:43.153+0000 7f91475d4640 1 -- 172.21.15.148:0/3358771663 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f912406c530 msgr2=0x7f912406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:43.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:43.153+0000 7f91475d4640 1 --2- 172.21.15.148:0/3358771663 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f912406c530 0x7f912406e9f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9130036000 tx=0x7f913003d040 comp rx=0 tx=0).stop 2024-03-29T04:11:43.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:43.153+0000 7f91475d4640 1 -- 172.21.15.148:0/3358771663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91400d0220 msgr2=0x7f91400ae2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:43.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:43.153+0000 7f91475d4640 1 --2- 172.21.15.148:0/3358771663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91400d0220 0x7f91400ae2f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f913c00e9b0 tx=0x7f913c00ee70 comp rx=0 tx=0).stop 2024-03-29T04:11:43.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:43.153+0000 7f91475d4640 1 -- 172.21.15.148:0/3358771663 shutdown_connections 2024-03-29T04:11:43.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:43.153+0000 7f91475d4640 1 --2- 172.21.15.148:0/3358771663 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f912406c530 0x7f912406e9f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:43.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:43.153+0000 7f91475d4640 1 --2- 172.21.15.148:0/3358771663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f914015ed50 0x7f91400b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:43.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:43.153+0000 7f91475d4640 1 --2- 172.21.15.148:0/3358771663 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f914015e800 0x7f914014a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:43.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:43.153+0000 7f91475d4640 1 --2- 172.21.15.148:0/3358771663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91400d0220 0x7f91400ae2f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:43.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:43.153+0000 7f91475d4640 1 -- 172.21.15.148:0/3358771663 >> 172.21.15.148:0/3358771663 conn(0x7f91400d2110 msgr2=0x7f914006dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:43.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:43.153+0000 7f91475d4640 1 -- 172.21.15.148:0/3358771663 shutdown_connections 2024-03-29T04:11:43.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:43.153+0000 7f91475d4640 1 -- 172.21.15.148:0/3358771663 wait complete. 2024-03-29T04:11:43.143 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T04:11:43.143 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:11:43.143 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-03-29T04:11:43.143 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T04:11:43.143 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T04:11:43.143 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T04:11:43.143 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T04:11:43.143 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:11:43.143 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T04:11:43.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:43.144 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-03-29T04:11:43.160 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:11:43.160 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:11:43.160 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:11:43.160 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:11:43.160 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-03-29T04:11:43.160 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:11:43.160 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:11:43.160 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:11:43.160 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-03-29T04:11:43.160 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:11:43.161 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:11:43.161 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-03-29T04:11:43.161 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:11:43.161 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:11:43.161 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:11:43.161 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:11:43.161 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T04:11:43.161 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42639sec preferred_lft 42639sec 2024-03-29T04:11:43.161 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link 2024-03-29T04:11:43.161 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:11:43.161 INFO:teuthology.orchestra.run.smithi148.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:11:43.161 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 62:cb:c4:12:20:16 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:11:43.161 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T04:11:43.161 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:11:43.161 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::38f9:90ff:fe15:4429/64 scope link 2024-03-29T04:11:43.162 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:11:43.162 INFO:teuthology.orchestra.run.smithi148.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T04:11:43.162 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 62:cb:c4:12:20:16 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:11:43.162 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::60cb:c4ff:fe12:2016/64 scope link 2024-03-29T04:11:43.162 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:11:43.162 INFO:teuthology.orchestra.run.smithi148.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T04:11:43.162 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether da:b2:53:42:28:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:11:43.162 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::d8b2:53ff:fe42:2852/64 scope link 2024-03-29T04:11:43.162 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:11:43.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:43.163 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:11:43.226 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-03-29T04:11:43.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:43.228 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-03-29T04:11:43.286 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-03-29T04:11:43.286 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:11:43.286 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T04:11:43.287 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T04:11:43.287 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:11:43.287 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:11:43.287 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-29T04:11:43.287 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T04:11:43.287 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T04:11:43.287 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T04:11:43.287 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-29T04:11:43.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:11:43.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:11:43.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:43.423 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-03-29T04:11:43.438 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T04:11:43.438 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T04:11:43.438 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T04:11:43.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:43.438 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2024-03-29T04:11:43.505 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T04:11:43.505 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:11:43.506 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T04:11:43.560 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T04:11:43.560 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:11:43.560 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T04:11:43.615 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2024-03-29T04:11:43.686 DEBUG:teuthology.orchestra.run.smithi148:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:11:43.745 INFO:teuthology.orchestra.run.smithi148.stdout:/proc 2024-03-29T04:11:43.745 INFO:teuthology.orchestra.run.smithi148.stdout:/sys 2024-03-29T04:11:43.745 INFO:teuthology.orchestra.run.smithi148.stdout:/dev 2024-03-29T04:11:43.745 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/security 2024-03-29T04:11:43.746 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/shm 2024-03-29T04:11:43.746 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/pts 2024-03-29T04:11:43.746 INFO:teuthology.orchestra.run.smithi148.stdout:/run 2024-03-29T04:11:43.746 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/cgroup 2024-03-29T04:11:43.746 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/pstore 2024-03-29T04:11:43.746 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/bpf 2024-03-29T04:11:43.746 INFO:teuthology.orchestra.run.smithi148.stdout:/ 2024-03-29T04:11:43.746 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/selinux 2024-03-29T04:11:43.746 INFO:teuthology.orchestra.run.smithi148.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:11:43.746 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/hugepages 2024-03-29T04:11:43.746 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/mqueue 2024-03-29T04:11:43.746 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/debug 2024-03-29T04:11:43.747 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/tracing 2024-03-29T04:11:43.747 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-29T04:11:43.747 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/fuse/connections 2024-03-29T04:11:43.747 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-sysctl.service 2024-03-29T04:11:43.747 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/config 2024-03-29T04:11:43.747 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-29T04:11:43.747 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/nfs/rpc_pipefs 2024-03-29T04:11:43.748 INFO:teuthology.orchestra.run.smithi148.stdout:/run/user/1000 2024-03-29T04:11:43.748 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph 2024-03-29T04:11:43.748 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-0 2024-03-29T04:11:43.748 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-1 2024-03-29T04:11:43.748 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-2 2024-03-29T04:11:43.748 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-3 2024-03-29T04:11:43.748 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns 2024-03-29T04:11:43.748 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:11:43.748 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:11:43.748 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:11:43.748 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:11:43.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:43.749 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2024-03-29T04:11:43.802 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T04:11:43.802 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T04:11:43.844 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2024-03-29T04:11:43.872 DEBUG:teuthology.orchestra.run.smithi148:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:11:43.929 INFO:teuthology.orchestra.run.smithi148.stdout:/proc 2024-03-29T04:11:43.929 INFO:teuthology.orchestra.run.smithi148.stdout:/sys 2024-03-29T04:11:43.929 INFO:teuthology.orchestra.run.smithi148.stdout:/dev 2024-03-29T04:11:43.929 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/security 2024-03-29T04:11:43.929 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/shm 2024-03-29T04:11:43.929 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/pts 2024-03-29T04:11:43.929 INFO:teuthology.orchestra.run.smithi148.stdout:/run 2024-03-29T04:11:43.929 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/cgroup 2024-03-29T04:11:43.929 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/pstore 2024-03-29T04:11:43.929 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/bpf 2024-03-29T04:11:43.930 INFO:teuthology.orchestra.run.smithi148.stdout:/ 2024-03-29T04:11:43.930 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/selinux 2024-03-29T04:11:43.930 INFO:teuthology.orchestra.run.smithi148.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:11:43.930 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/hugepages 2024-03-29T04:11:43.930 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/mqueue 2024-03-29T04:11:43.930 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/debug 2024-03-29T04:11:43.930 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/tracing 2024-03-29T04:11:43.930 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-29T04:11:43.930 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/fuse/connections 2024-03-29T04:11:43.930 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-sysctl.service 2024-03-29T04:11:43.930 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/config 2024-03-29T04:11:43.930 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-29T04:11:43.930 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/nfs/rpc_pipefs 2024-03-29T04:11:43.930 INFO:teuthology.orchestra.run.smithi148.stdout:/run/user/1000 2024-03-29T04:11:43.930 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph 2024-03-29T04:11:43.931 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-0 2024-03-29T04:11:43.931 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-1 2024-03-29T04:11:43.931 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-2 2024-03-29T04:11:43.931 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-3 2024-03-29T04:11:43.931 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns 2024-03-29T04:11:43.931 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:11:43.931 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:11:43.931 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:11:43.931 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:11:43.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:43.931 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2024-03-29T04:11:43.932 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi148.stderr:2024-03-29T04:11:43.953+0000 7fd281387480 -1 init, newargv = 0x55712e7a5f80 newargc=15 2024-03-29T04:11:43.932 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi148.stderr:ceph-fuse[54627]: starting ceph client 2024-03-29T04:11:43.936 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi148.stderr:ceph-fuse[54627]: starting fuse 2024-03-29T04:11:43.948 INFO:teuthology.orchestra.run.smithi148.stdout:42 2024-03-29T04:11:43.948 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-03-29T04:11:43.948 DEBUG:teuthology.orchestra.run.smithi148:> sudo stdin-killer -- python3 -c ' 2024-03-29T04:11:43.948 DEBUG:teuthology.orchestra.run.smithi148:> import glob 2024-03-29T04:11:43.948 DEBUG:teuthology.orchestra.run.smithi148:> import re 2024-03-29T04:11:43.948 DEBUG:teuthology.orchestra.run.smithi148:> import os 2024-03-29T04:11:43.949 DEBUG:teuthology.orchestra.run.smithi148:> import subprocess 2024-03-29T04:11:43.949 DEBUG:teuthology.orchestra.run.smithi148:> 2024-03-29T04:11:43.949 DEBUG:teuthology.orchestra.run.smithi148:> def _find_admin_socket(client_name): 2024-03-29T04:11:43.949 DEBUG:teuthology.orchestra.run.smithi148:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T04:11:43.949 DEBUG:teuthology.orchestra.run.smithi148:> files = glob.glob(asok_path) 2024-03-29T04:11:43.949 DEBUG:teuthology.orchestra.run.smithi148:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T04:11:43.949 DEBUG:teuthology.orchestra.run.smithi148:> 2024-03-29T04:11:43.949 DEBUG:teuthology.orchestra.run.smithi148:> # Given a non-glob path, it better be there 2024-03-29T04:11:43.949 DEBUG:teuthology.orchestra.run.smithi148:> if "*" not in asok_path: 2024-03-29T04:11:43.949 DEBUG:teuthology.orchestra.run.smithi148:> assert(len(files) == 1) 2024-03-29T04:11:43.949 DEBUG:teuthology.orchestra.run.smithi148:> return files[0] 2024-03-29T04:11:43.949 DEBUG:teuthology.orchestra.run.smithi148:> 2024-03-29T04:11:43.949 DEBUG:teuthology.orchestra.run.smithi148:> for f in files: 2024-03-29T04:11:43.949 DEBUG:teuthology.orchestra.run.smithi148:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T04:11:43.949 DEBUG:teuthology.orchestra.run.smithi148:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T04:11:43.949 DEBUG:teuthology.orchestra.run.smithi148:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T04:11:43.949 DEBUG:teuthology.orchestra.run.smithi148:> contents = proc_f.read() 2024-03-29T04:11:43.950 DEBUG:teuthology.orchestra.run.smithi148:> if mountpoint in contents: 2024-03-29T04:11:43.950 DEBUG:teuthology.orchestra.run.smithi148:> return f 2024-03-29T04:11:43.950 DEBUG:teuthology.orchestra.run.smithi148:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T04:11:43.950 DEBUG:teuthology.orchestra.run.smithi148:> 2024-03-29T04:11:43.950 DEBUG:teuthology.orchestra.run.smithi148:> print(_find_admin_socket("client.0")) 2024-03-29T04:11:43.950 DEBUG:teuthology.orchestra.run.smithi148:> ' 2024-03-29T04:11:44.070 INFO:teuthology.orchestra.run.smithi148.stdout:/var/run/ceph/ceph-client.0.54627.asok 2024-03-29T04:11:44.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:11:44.081 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.54627.asok 2024-03-29T04:11:44.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:44.081 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.54627.asok status 2024-03-29T04:11:44.219 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-03-29T04:11:44.220 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": { 2024-03-29T04:11:44.220 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_sha1": "69e2cd73c52284dfd5096c66b60c6e6b26a3c447", 2024-03-29T04:11:44.220 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_version": "ceph version 19.0.0-2520-g69e2cd73 (69e2cd73c52284dfd5096c66b60c6e6b26a3c447) squid (dev)", 2024-03-29T04:11:44.220 INFO:teuthology.orchestra.run.smithi148.stdout: "entity_id": "0", 2024-03-29T04:11:44.220 INFO:teuthology.orchestra.run.smithi148.stdout: "hostname": "smithi148", 2024-03-29T04:11:44.220 INFO:teuthology.orchestra.run.smithi148.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T04:11:44.220 INFO:teuthology.orchestra.run.smithi148.stdout: "pid": "54627", 2024-03-29T04:11:44.220 INFO:teuthology.orchestra.run.smithi148.stdout: "root": "/" 2024-03-29T04:11:44.220 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:44.220 INFO:teuthology.orchestra.run.smithi148.stdout: "dentry_count": 0, 2024-03-29T04:11:44.220 INFO:teuthology.orchestra.run.smithi148.stdout: "dentry_pinned_count": 0, 2024-03-29T04:11:44.220 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 5790, 2024-03-29T04:11:44.220 INFO:teuthology.orchestra.run.smithi148.stdout: "inst": { 2024-03-29T04:11:44.220 INFO:teuthology.orchestra.run.smithi148.stdout: "name": { 2024-03-29T04:11:44.220 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client", 2024-03-29T04:11:44.221 INFO:teuthology.orchestra.run.smithi148.stdout: "num": 5790 2024-03-29T04:11:44.221 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:44.221 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": { 2024-03-29T04:11:44.221 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2024-03-29T04:11:44.221 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "192.168.0.1:0", 2024-03-29T04:11:44.221 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 4186653557 2024-03-29T04:11:44.221 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:44.221 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:44.221 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": { 2024-03-29T04:11:44.221 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2024-03-29T04:11:44.221 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "192.168.0.1:0", 2024-03-29T04:11:44.221 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 4186653557 2024-03-29T04:11:44.221 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:44.221 INFO:teuthology.orchestra.run.smithi148.stdout: "inst_str": "client.5790 192.168.0.1:0/4186653557", 2024-03-29T04:11:44.221 INFO:teuthology.orchestra.run.smithi148.stdout: "addr_str": "192.168.0.1:0/4186653557", 2024-03-29T04:11:44.222 INFO:teuthology.orchestra.run.smithi148.stdout: "inode_count": 1, 2024-03-29T04:11:44.222 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_epoch": 17, 2024-03-29T04:11:44.222 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_epoch": 41, 2024-03-29T04:11:44.222 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_epoch_barrier": 0, 2024-03-29T04:11:44.222 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklisted": false, 2024-03-29T04:11:44.222 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs" 2024-03-29T04:11:44.222 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-03-29T04:11:44.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:44.228 DEBUG:teuthology.orchestra.run.smithi148:> stat --file-system '--printf=%T 2024-03-29T04:11:44.228 DEBUG:teuthology.orchestra.run.smithi148:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T04:11:44.245 INFO:teuthology.orchestra.run.smithi148.stdout:fuseblk 2024-03-29T04:11:44.245 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T04:11:44.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:44.245 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T04:11:44.315 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T04:11:44.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.479+0000 7f713aa1b640 1 Processor -- start 2024-03-29T04:11:44.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.479+0000 7f713aa1b640 1 -- start start 2024-03-29T04:11:44.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.480+0000 7f713aa1b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f713415e3e0 0x7f71340d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:44.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.480+0000 7f713aa1b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71340ba030 0x7f71340d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:44.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.480+0000 7f713aa1b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f713415e7c0 con 0x7f71340d10d0 2024-03-29T04:11:44.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.480+0000 7f713aa1b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f71340d3840 con 0x7f713415e3e0 2024-03-29T04:11:44.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.480+0000 7f713aa1b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f71340d2c30 con 0x7f71340ba030 2024-03-29T04:11:44.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.480+0000 7f7138fb8640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f71340d10d0 0x7f71340b72d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40682/0 (socket says 172.21.15.148:40682) 2024-03-29T04:11:44.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.480+0000 7f7138fb8640 1 -- 172.21.15.148:0/3991808009 learned_addr learned my addr 172.21.15.148:0/3991808009 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:44.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.480+0000 7f7133fff640 1 --2- 172.21.15.148:0/3991808009 >> v2:172.21.15.157:3300/0 conn(0x7f713415e3e0 0x7f71340d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:44.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.480+0000 7f71337fe640 1 --2- 172.21.15.148:0/3991808009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71340ba030 0x7f71340d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:44.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.480+0000 7f7132ffd640 1 -- 172.21.15.148:0/3991808009 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1878830145 0 0) 0x7f713415e7c0 con 0x7f71340d10d0 2024-03-29T04:11:44.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.480+0000 7f7132ffd640 1 -- 172.21.15.148:0/3991808009 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f711c003520 con 0x7f71340d10d0 2024-03-29T04:11:44.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.480+0000 7f7132ffd640 1 -- 172.21.15.148:0/3991808009 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4210327985 0 0) 0x7f711c003520 con 0x7f71340d10d0 2024-03-29T04:11:44.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.480+0000 7f7132ffd640 1 -- 172.21.15.148:0/3991808009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71340ba030 msgr2=0x7f71340d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:44.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.480+0000 7f7132ffd640 1 --2- 172.21.15.148:0/3991808009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71340ba030 0x7f71340d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:44.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.480+0000 7f7132ffd640 1 -- 172.21.15.148:0/3991808009 >> v2:172.21.15.157:3300/0 conn(0x7f713415e3e0 msgr2=0x7f71340d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:44.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.480+0000 7f7132ffd640 1 --2- 172.21.15.148:0/3991808009 >> v2:172.21.15.157:3300/0 conn(0x7f713415e3e0 0x7f71340d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:44.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.480+0000 7f7132ffd640 1 -- 172.21.15.148:0/3991808009 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71340d2f10 con 0x7f71340d10d0 2024-03-29T04:11:44.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.480+0000 7f7132ffd640 1 -- 172.21.15.148:0/3991808009 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f712c0019f0 con 0x7f71340d10d0 2024-03-29T04:11:44.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.480+0000 7f7132ffd640 1 -- 172.21.15.148:0/3991808009 >> v1:172.21.15.148:6789/0 conn(0x7f71340d10d0 legacy=0x7f71340b72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:44.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.480+0000 7f7132ffd640 1 --2- 172.21.15.148:0/3991808009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f711c0039b0 0x7f711c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:44.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.480+0000 7f7132ffd640 1 --2- 172.21.15.148:0/3991808009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71340ba030 0x7f711c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:44.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.480+0000 7f7132ffd640 1 --2- 172.21.15.148:0/3991808009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f711c004d40 0x7f711c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:44.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.480+0000 7f7132ffd640 1 -- 172.21.15.148:0/3991808009 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f71340d3840 con 0x7f711c0039b0 2024-03-29T04:11:44.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.480+0000 7f7132ffd640 1 -- 172.21.15.148:0/3991808009 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f71340d2c30 con 0x7f711c004d40 2024-03-29T04:11:44.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.480+0000 7f7132ffd640 1 -- 172.21.15.148:0/3991808009 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f711c00b9c0 con 0x7f71340ba030 2024-03-29T04:11:44.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.481+0000 7f7133fff640 1 --2- 172.21.15.148:0/3991808009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f711c0039b0 0x7f711c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:44.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.481+0000 7f71337fe640 1 --2- 172.21.15.148:0/3991808009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71340ba030 0x7f711c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:44.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.481+0000 7f7138fb8640 1 --2- 172.21.15.148:0/3991808009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f711c004d40 0x7f711c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:44.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.481+0000 7f7133fff640 1 -- 172.21.15.148:0/3991808009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71340ba030 msgr2=0x7f711c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:44.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.481+0000 7f7133fff640 1 --2- 172.21.15.148:0/3991808009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71340ba030 0x7f711c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:44.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.481+0000 7f7133fff640 1 -- 172.21.15.148:0/3991808009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f711c004d40 msgr2=0x7f711c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:44.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.481+0000 7f7133fff640 1 --2- 172.21.15.148:0/3991808009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f711c004d40 0x7f711c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:44.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.481+0000 7f7133fff640 1 -- 172.21.15.148:0/3991808009 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711c003520 con 0x7f711c0039b0 2024-03-29T04:11:44.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.481+0000 7f7138fb8640 1 --2- 172.21.15.148:0/3991808009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f711c004d40 0x7f711c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:44.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.482+0000 7f7133fff640 1 --2- 172.21.15.148:0/3991808009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f711c0039b0 0x7f711c003dd0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7128009ac0 tx=0x7f711c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2818e4ecd457de4a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:44.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.482+0000 7f7132ffd640 1 -- 172.21.15.148:0/3991808009 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128039070 con 0x7f711c0039b0 2024-03-29T04:11:44.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.482+0000 7f7132ffd640 1 -- 172.21.15.148:0/3991808009 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f712802c710 con 0x7f711c0039b0 2024-03-29T04:11:44.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.482+0000 7f7132ffd640 1 -- 172.21.15.148:0/3991808009 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128009690 con 0x7f711c0039b0 2024-03-29T04:11:44.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.483+0000 7f713aa1b640 1 -- 172.21.15.148:0/3991808009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f711c0039b0 msgr2=0x7f711c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:44.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.483+0000 7f713aa1b640 1 --2- 172.21.15.148:0/3991808009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f711c0039b0 0x7f711c003dd0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7128009ac0 tx=0x7f711c005830 comp rx=0 tx=0).stop 2024-03-29T04:11:44.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.483+0000 7f7138fb8640 1 -- 172.21.15.148:0/3991808009 reap_dead start 2024-03-29T04:11:44.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.483+0000 7f713aa1b640 1 -- 172.21.15.148:0/3991808009 shutdown_connections 2024-03-29T04:11:44.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.483+0000 7f713aa1b640 1 -- 172.21.15.148:0/3991808009 >> 172.21.15.148:0/3991808009 conn(0x7f71340b6210 msgr2=0x7f71340b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:44.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.483+0000 7f713aa1b640 1 -- 172.21.15.148:0/3991808009 shutdown_connections 2024-03-29T04:11:44.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.483+0000 7f713aa1b640 1 -- 172.21.15.148:0/3991808009 wait complete. 2024-03-29T04:11:44.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.484+0000 7f713aa1b640 1 Processor -- start 2024-03-29T04:11:44.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.484+0000 7f713aa1b640 1 -- start start 2024-03-29T04:11:44.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.485+0000 7f713aa1b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f711c0039b0 0x7f7134156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:44.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.485+0000 7f713aa1b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71341553d0 0x7f7134153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:44.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.485+0000 7f713aa1b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71340c7c90 0x7f71340c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:44.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.485+0000 7f713aa1b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f71340d1580 con 0x7f71340c7c90 2024-03-29T04:11:44.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.485+0000 7f713aa1b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f71340d34d0 con 0x7f71341553d0 2024-03-29T04:11:44.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.485+0000 7f713aa1b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f71340d1700 con 0x7f711c0039b0 2024-03-29T04:11:44.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.485+0000 7f7138fb8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71340c7c90 0x7f71340c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:44.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.485+0000 7f7138fb8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71340c7c90 0x7f71340c8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42482/0 (socket says 172.21.15.148:42482) 2024-03-29T04:11:44.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.485+0000 7f7138fb8640 1 -- 172.21.15.148:0/3025718668 learned_addr learned my addr 172.21.15.148:0/3025718668 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:44.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.485+0000 7f71337fe640 1 --2- 172.21.15.148:0/3025718668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71341553d0 0x7f7134153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:44.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.485+0000 7f7133fff640 1 --2- 172.21.15.148:0/3025718668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f711c0039b0 0x7f7134156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:44.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.485+0000 7f7138fb8640 1 -- 172.21.15.148:0/3025718668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f711c0039b0 msgr2=0x7f7134156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:44.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.485+0000 7f7138fb8640 1 --2- 172.21.15.148:0/3025718668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f711c0039b0 0x7f7134156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:44.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.485+0000 7f7138fb8640 1 -- 172.21.15.148:0/3025718668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71341553d0 msgr2=0x7f7134153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:44.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.485+0000 7f7138fb8640 1 --2- 172.21.15.148:0/3025718668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71341553d0 0x7f7134153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:44.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.485+0000 7f7138fb8640 1 -- 172.21.15.148:0/3025718668 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71341558b0 con 0x7f71340c7c90 2024-03-29T04:11:44.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.485+0000 7f7133fff640 1 --2- 172.21.15.148:0/3025718668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f711c0039b0 0x7f7134156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:44.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.485+0000 7f7138fb8640 1 --2- 172.21.15.148:0/3025718668 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71340c7c90 0x7f71340c8140 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f71340c7c60 tx=0x7f712c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:44.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.485+0000 7f7130ff9640 1 -- 172.21.15.148:0/3025718668 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c007a80 con 0x7f71340c7c90 2024-03-29T04:11:44.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.486+0000 7f7130ff9640 1 -- 172.21.15.148:0/3025718668 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f712c00ece0 con 0x7f71340c7c90 2024-03-29T04:11:44.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.486+0000 7f7130ff9640 1 -- 172.21.15.148:0/3025718668 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c014670 con 0x7f71340c7c90 2024-03-29T04:11:44.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.486+0000 7f713aa1b640 1 -- 172.21.15.148:0/3025718668 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7134155b90 con 0x7f71340c7c90 2024-03-29T04:11:44.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.486+0000 7f713aa1b640 1 -- 172.21.15.148:0/3025718668 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7134156150 con 0x7f71340c7c90 2024-03-29T04:11:44.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.487+0000 7f713aa1b640 1 -- 172.21.15.148:0/3025718668 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71340687e0 con 0x7f71340c7c90 2024-03-29T04:11:44.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.487+0000 7f7130ff9640 1 -- 172.21.15.148:0/3025718668 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f712c014810 con 0x7f71340c7c90 2024-03-29T04:11:44.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.487+0000 7f7130ff9640 1 --2- 172.21.15.148:0/3025718668 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f712006c530 0x7f712006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:44.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.487+0000 7f7130ff9640 1 -- 172.21.15.148:0/3025718668 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f712c08b6a0 con 0x7f71340c7c90 2024-03-29T04:11:44.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.488+0000 7f7133fff640 1 --2- 172.21.15.148:0/3025718668 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f712006c530 0x7f712006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:44.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.488+0000 7f7133fff640 1 --2- 172.21.15.148:0/3025718668 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f712006c530 0x7f712006e9f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7128036000 tx=0x7f7128030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:44.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.492+0000 7f7130ff9640 1 -- 172.21.15.148:0/3025718668 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f712c059b10 con 0x7f71340c7c90 2024-03-29T04:11:44.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.714+0000 7f713aa1b640 1 -- 172.21.15.148:0/3025718668 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f71340b5c00 con 0x7f71340c7c90 2024-03-29T04:11:44.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.714+0000 7f7130ff9640 1 -- 172.21.15.148:0/3025718668 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f712c059410 con 0x7f71340c7c90 2024-03-29T04:11:44.693 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T04:11:44.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.716+0000 7f713aa1b640 1 -- 172.21.15.148:0/3025718668 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f712006c530 msgr2=0x7f712006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:44.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.716+0000 7f713aa1b640 1 --2- 172.21.15.148:0/3025718668 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f712006c530 0x7f712006e9f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7128036000 tx=0x7f7128030000 comp rx=0 tx=0).stop 2024-03-29T04:11:44.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.716+0000 7f713aa1b640 1 -- 172.21.15.148:0/3025718668 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71340c7c90 msgr2=0x7f71340c8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:44.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.716+0000 7f713aa1b640 1 --2- 172.21.15.148:0/3025718668 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71340c7c90 0x7f71340c8140 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f71340c7c60 tx=0x7f712c00e530 comp rx=0 tx=0).stop 2024-03-29T04:11:44.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.716+0000 7f713aa1b640 1 -- 172.21.15.148:0/3025718668 shutdown_connections 2024-03-29T04:11:44.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.716+0000 7f713aa1b640 1 --2- 172.21.15.148:0/3025718668 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f712006c530 0x7f712006e9f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:44.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.716+0000 7f713aa1b640 1 --2- 172.21.15.148:0/3025718668 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71340c7c90 0x7f71340c8140 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:44.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.716+0000 7f713aa1b640 1 --2- 172.21.15.148:0/3025718668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71341553d0 0x7f7134153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:44.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.716+0000 7f713aa1b640 1 --2- 172.21.15.148:0/3025718668 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f711c0039b0 0x7f7134156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:44.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.716+0000 7f713aa1b640 1 -- 172.21.15.148:0/3025718668 >> 172.21.15.148:0/3025718668 conn(0x7f71340b6210 msgr2=0x7f71340b84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:44.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.717+0000 7f713aa1b640 1 -- 172.21.15.148:0/3025718668 shutdown_connections 2024-03-29T04:11:44.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:44.717+0000 7f713aa1b640 1 -- 172.21.15.148:0/3025718668 wait complete. 2024-03-29T04:11:44.706 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T04:11:44.707 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:11:44.707 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-03-29T04:11:44.707 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-29T04:11:44.707 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-29T04:11:44.707 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T04:11:44.707 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T04:11:44.707 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:11:44.707 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-29T04:11:44.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:44.707 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-03-29T04:11:44.727 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:11:44.727 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:11:44.728 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:11:44.728 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:11:44.728 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-03-29T04:11:44.728 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:11:44.728 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:11:44.728 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:11:44.729 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-03-29T04:11:44.729 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:11:44.729 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:11:44.729 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-03-29T04:11:44.729 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:11:44.729 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:11:44.729 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:11:44.729 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:11:44.729 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T04:11:44.729 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42656sec preferred_lft 42656sec 2024-03-29T04:11:44.729 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-03-29T04:11:44.730 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:11:44.730 INFO:teuthology.orchestra.run.smithi157.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:11:44.730 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether ca:23:27:e5:09:83 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:11:44.730 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T04:11:44.730 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:11:44.730 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4fd:14ff:fef9:9691/64 scope link 2024-03-29T04:11:44.730 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:11:44.730 INFO:teuthology.orchestra.run.smithi157.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T04:11:44.730 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether d6:7c:73:cd:6f:d3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:11:44.730 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::d47c:73ff:fecd:6fd3/64 scope link 2024-03-29T04:11:44.730 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:11:44.730 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T04:11:44.730 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether ca:23:27:e5:09:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:11:44.730 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c823:27ff:fee5:983/64 scope link 2024-03-29T04:11:44.731 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:11:44.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:44.731 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:11:44.800 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-03-29T04:11:44.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:44.802 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-03-29T04:11:44.862 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-03-29T04:11:44.862 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:11:44.863 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T04:11:44.863 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T04:11:44.863 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:11:44.863 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:11:44.864 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-03-29T04:11:44.864 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T04:11:44.864 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T04:11:44.864 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T04:11:44.864 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-03-29T04:11:44.958 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:11:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:11:45.003 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:11:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:11:45.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:45.010 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-03-29T04:11:45.026 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-29T04:11:45.026 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-29T04:11:45.026 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-29T04:11:45.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:45.026 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-03-29T04:11:45.097 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-03-29T04:11:45.097 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:11:45.097 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-29T04:11:45.152 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-29T04:11:45.153 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:11:45.153 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-29T04:11:45.208 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-03-29T04:11:45.276 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:11:45.332 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-03-29T04:11:45.332 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-03-29T04:11:45.332 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-03-29T04:11:45.332 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-03-29T04:11:45.332 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-03-29T04:11:45.332 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-03-29T04:11:45.332 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-03-29T04:11:45.332 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-03-29T04:11:45.332 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-03-29T04:11:45.332 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-03-29T04:11:45.332 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-03-29T04:11:45.332 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/selinux 2024-03-29T04:11:45.332 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:11:45.333 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-03-29T04:11:45.333 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-03-29T04:11:45.333 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-03-29T04:11:45.333 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-03-29T04:11:45.333 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-03-29T04:11:45.333 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-03-29T04:11:45.333 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysctl.service 2024-03-29T04:11:45.333 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-29T04:11:45.333 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-29T04:11:45.333 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/nfs/rpc_pipefs 2024-03-29T04:11:45.333 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-03-29T04:11:45.333 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-03-29T04:11:45.333 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-03-29T04:11:45.333 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-03-29T04:11:45.334 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-03-29T04:11:45.334 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-03-29T04:11:45.334 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-03-29T04:11:45.334 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:11:45.334 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:11:45.334 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:11:45.334 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:11:45.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:45.334 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-03-29T04:11:45.387 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T04:11:45.387 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-03-29T04:11:45.431 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-03-29T04:11:45.457 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:11:45.514 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-03-29T04:11:45.514 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-03-29T04:11:45.514 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-03-29T04:11:45.514 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-03-29T04:11:45.515 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-03-29T04:11:45.515 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-03-29T04:11:45.515 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-03-29T04:11:45.515 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-03-29T04:11:45.515 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-03-29T04:11:45.515 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-03-29T04:11:45.515 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-03-29T04:11:45.515 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/selinux 2024-03-29T04:11:45.515 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:11:45.515 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-03-29T04:11:45.515 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-03-29T04:11:45.515 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-03-29T04:11:45.515 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-03-29T04:11:45.515 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-03-29T04:11:45.515 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-03-29T04:11:45.516 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysctl.service 2024-03-29T04:11:45.516 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-29T04:11:45.516 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-29T04:11:45.516 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/nfs/rpc_pipefs 2024-03-29T04:11:45.516 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-03-29T04:11:45.516 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-03-29T04:11:45.516 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-03-29T04:11:45.516 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-03-29T04:11:45.516 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-03-29T04:11:45.516 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-03-29T04:11:45.516 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-03-29T04:11:45.516 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:11:45.516 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:11:45.516 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:11:45.516 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:11:45.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:45.517 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-03-29T04:11:45.518 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:2024-03-29T04:11:45.537+0000 7f6171cfa480 -1 init, newargv = 0x562772f386f0 newargc=15 2024-03-29T04:11:45.518 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[44482]: starting ceph client 2024-03-29T04:11:45.520 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[44482]: starting fuse 2024-03-29T04:11:45.533 INFO:teuthology.orchestra.run.smithi157.stdout:41 2024-03-29T04:11:45.533 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-03-29T04:11:45.533 DEBUG:teuthology.orchestra.run.smithi157:> sudo stdin-killer -- python3 -c ' 2024-03-29T04:11:45.533 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2024-03-29T04:11:45.533 DEBUG:teuthology.orchestra.run.smithi157:> import re 2024-03-29T04:11:45.533 DEBUG:teuthology.orchestra.run.smithi157:> import os 2024-03-29T04:11:45.533 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2024-03-29T04:11:45.534 DEBUG:teuthology.orchestra.run.smithi157:> 2024-03-29T04:11:45.534 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2024-03-29T04:11:45.534 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-03-29T04:11:45.534 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2024-03-29T04:11:45.534 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-03-29T04:11:45.534 DEBUG:teuthology.orchestra.run.smithi157:> 2024-03-29T04:11:45.534 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2024-03-29T04:11:45.534 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2024-03-29T04:11:45.534 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2024-03-29T04:11:45.534 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2024-03-29T04:11:45.534 DEBUG:teuthology.orchestra.run.smithi157:> 2024-03-29T04:11:45.534 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2024-03-29T04:11:45.534 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T04:11:45.534 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T04:11:45.534 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T04:11:45.535 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2024-03-29T04:11:45.535 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2024-03-29T04:11:45.535 DEBUG:teuthology.orchestra.run.smithi157:> return f 2024-03-29T04:11:45.535 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T04:11:45.535 DEBUG:teuthology.orchestra.run.smithi157:> 2024-03-29T04:11:45.535 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.1")) 2024-03-29T04:11:45.535 DEBUG:teuthology.orchestra.run.smithi157:> ' 2024-03-29T04:11:45.655 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.1.44482.asok 2024-03-29T04:11:45.658 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-29T04:11:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:11:45.665 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.44482.asok 2024-03-29T04:11:45.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:45.665 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.44482.asok status 2024-03-29T04:11:45.801 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-03-29T04:11:45.801 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2024-03-29T04:11:45.801 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "69e2cd73c52284dfd5096c66b60c6e6b26a3c447", 2024-03-29T04:11:45.801 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 19.0.0-2520-g69e2cd73 (69e2cd73c52284dfd5096c66b60c6e6b26a3c447) squid (dev)", 2024-03-29T04:11:45.801 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "1", 2024-03-29T04:11:45.801 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2024-03-29T04:11:45.801 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-03-29T04:11:45.802 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "44482", 2024-03-29T04:11:45.802 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2024-03-29T04:11:45.802 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-29T04:11:45.802 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2024-03-29T04:11:45.802 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2024-03-29T04:11:45.802 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 5066, 2024-03-29T04:11:45.802 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2024-03-29T04:11:45.802 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2024-03-29T04:11:45.802 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2024-03-29T04:11:45.802 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 5066 2024-03-29T04:11:45.802 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-29T04:11:45.802 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-03-29T04:11:45.802 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-03-29T04:11:45.803 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-03-29T04:11:45.803 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 787027294 2024-03-29T04:11:45.803 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-29T04:11:45.803 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-29T04:11:45.803 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-03-29T04:11:45.803 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-03-29T04:11:45.803 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-03-29T04:11:45.803 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 787027294 2024-03-29T04:11:45.803 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-29T04:11:45.803 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.5066 192.168.0.1:0/787027294", 2024-03-29T04:11:45.803 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/787027294", 2024-03-29T04:11:45.803 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2024-03-29T04:11:45.803 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 17, 2024-03-29T04:11:45.803 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 41, 2024-03-29T04:11:45.803 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2024-03-29T04:11:45.804 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2024-03-29T04:11:45.804 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2024-03-29T04:11:45.804 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-03-29T04:11:45.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:45.809 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2024-03-29T04:11:45.810 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1 2024-03-29T04:11:45.825 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2024-03-29T04:11:45.825 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-03-29T04:11:45.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:45.825 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-29T04:11:45.893 DEBUG:tasks.cephfs.test_uninlining:setting inline_data:1 2024-03-29T04:11:45.893 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs inline_data 1 --yes-i-really-really-mean-it 2024-03-29T04:11:45.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.020+0000 7fd591d59640 1 Processor -- start 2024-03-29T04:11:45.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.020+0000 7fd591d59640 1 -- start start 2024-03-29T04:11:46.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.020+0000 7fd591d59640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd58c0d3840 0x7fd58c0d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:46.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.020+0000 7fd591d59640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd58c0b36e0 0x7fd58c1615d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:46.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.021+0000 7fd591d59640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd58c0d27b0 con 0x7fd58c0d3c20 2024-03-29T04:11:46.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.021+0000 7fd591d59640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fd58c0d2a50 con 0x7fd58c0b36e0 2024-03-29T04:11:46.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.021+0000 7fd591d59640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd58c0d2d30 con 0x7fd58c0d3840 2024-03-29T04:11:46.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.021+0000 7fd583fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd58c0b36e0 0x7fd58c1615d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:46.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.021+0000 7fd583fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd58c0b36e0 0x7fd58c1615d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39076/0 (socket says 172.21.15.148:39076) 2024-03-29T04:11:46.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.021+0000 7fd583fff640 1 -- 172.21.15.148:0/3866483603 learned_addr learned my addr 172.21.15.148:0/3866483603 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:46.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.021+0000 7fd58b7fe640 1 --2- 172.21.15.148:0/3866483603 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd58c0d3840 0x7fd58c0d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:46.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.021+0000 7fd583fff640 1 -- 172.21.15.148:0/3866483603 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd58c0d3840 msgr2=0x7fd58c0d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:46.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.021+0000 7fd583fff640 1 --2- 172.21.15.148:0/3866483603 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd58c0d3840 0x7fd58c0d2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:46.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.021+0000 7fd583fff640 1 -- 172.21.15.148:0/3866483603 >> v1:172.21.15.157:6789/0 conn(0x7fd58c0d3c20 legacy=0x7fd58c1621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:46.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.021+0000 7fd583fff640 1 -- 172.21.15.148:0/3866483603 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58c0d3010 con 0x7fd58c0b36e0 2024-03-29T04:11:46.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.021+0000 7fd58b7fe640 1 --2- 172.21.15.148:0/3866483603 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd58c0d3840 0x7fd58c0d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:46.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.022+0000 7fd583fff640 1 --2- 172.21.15.148:0/3866483603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd58c0b36e0 0x7fd58c1615d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd574009870 tx=0x7fd57402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac99d9729ec130d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:46.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.022+0000 7fd58affd640 1 -- 172.21.15.148:0/3866483603 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd574034070 con 0x7fd58c0b36e0 2024-03-29T04:11:46.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.022+0000 7fd58affd640 1 -- 172.21.15.148:0/3866483603 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd574030840 con 0x7fd58c0b36e0 2024-03-29T04:11:46.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.022+0000 7fd58affd640 1 -- 172.21.15.148:0/3866483603 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd574030c10 con 0x7fd58c0b36e0 2024-03-29T04:11:46.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.022+0000 7fd591d59640 1 -- 172.21.15.148:0/3866483603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd58c0b36e0 msgr2=0x7fd58c1615d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:46.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.022+0000 7fd591d59640 1 --2- 172.21.15.148:0/3866483603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd58c0b36e0 0x7fd58c1615d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd574009870 tx=0x7fd57402b980 comp rx=0 tx=0).stop 2024-03-29T04:11:46.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.023+0000 7fd591d59640 1 -- 172.21.15.148:0/3866483603 shutdown_connections 2024-03-29T04:11:46.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.023+0000 7fd591d59640 1 --2- 172.21.15.148:0/3866483603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd58c0b36e0 0x7fd58c1615d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:46.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.023+0000 7fd591d59640 1 --2- 172.21.15.148:0/3866483603 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd58c0d3840 0x7fd58c0d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:46.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.023+0000 7fd591d59640 1 -- 172.21.15.148:0/3866483603 >> 172.21.15.148:0/3866483603 conn(0x7fd58c0d42d0 msgr2=0x7fd58c0d46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:46.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.023+0000 7fd591d59640 1 -- 172.21.15.148:0/3866483603 shutdown_connections 2024-03-29T04:11:46.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.023+0000 7fd591d59640 1 -- 172.21.15.148:0/3866483603 wait complete. 2024-03-29T04:11:46.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.024+0000 7fd591d59640 1 Processor -- start 2024-03-29T04:11:46.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.024+0000 7fd591d59640 1 -- start start 2024-03-29T04:11:46.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.024+0000 7fd591d59640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd58c0b36e0 0x7fd58c0ad330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:46.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.024+0000 7fd591d59640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd58c0d3840 0x7fd58c1f7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:46.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.024+0000 7fd591d59640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd58c0d3c20 0x7fd58c0b2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:46.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.024+0000 7fd591d59640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd58c06ce70 con 0x7fd58c0d3840 2024-03-29T04:11:46.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.024+0000 7fd591d59640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd58c06ccf0 con 0x7fd58c0d3c20 2024-03-29T04:11:46.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.024+0000 7fd591d59640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd58c06cff0 con 0x7fd58c0b36e0 2024-03-29T04:11:46.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.024+0000 7fd583fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd58c0d3840 0x7fd58c1f7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:46.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.025+0000 7fd583fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd58c0d3840 0x7fd58c1f7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39090/0 (socket says 172.21.15.148:39090) 2024-03-29T04:11:46.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.025+0000 7fd583fff640 1 -- 172.21.15.148:0/3353908535 learned_addr learned my addr 172.21.15.148:0/3353908535 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:46.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.025+0000 7fd58b7fe640 1 --2- 172.21.15.148:0/3353908535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd58c0b36e0 0x7fd58c0ad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:46.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.025+0000 7fd58bfff640 1 --2- 172.21.15.148:0/3353908535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd58c0d3c20 0x7fd58c0b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:46.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.025+0000 7fd583fff640 1 -- 172.21.15.148:0/3353908535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd58c0b36e0 msgr2=0x7fd58c0ad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:46.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.025+0000 7fd583fff640 1 --2- 172.21.15.148:0/3353908535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd58c0b36e0 0x7fd58c0ad330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:46.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.025+0000 7fd583fff640 1 -- 172.21.15.148:0/3353908535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd58c0d3c20 msgr2=0x7fd58c0b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:46.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.025+0000 7fd583fff640 1 --2- 172.21.15.148:0/3353908535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd58c0d3c20 0x7fd58c0b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:46.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.025+0000 7fd583fff640 1 -- 172.21.15.148:0/3353908535 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58c1f8150 con 0x7fd58c0d3840 2024-03-29T04:11:46.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.025+0000 7fd58bfff640 1 --2- 172.21.15.148:0/3353908535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd58c0d3c20 0x7fd58c0b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:46.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.025+0000 7fd58b7fe640 1 --2- 172.21.15.148:0/3353908535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd58c0b36e0 0x7fd58c0ad330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:46.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.025+0000 7fd583fff640 1 --2- 172.21.15.148:0/3353908535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd58c0d3840 0x7fd58c1f7900 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd57402b9c0 tx=0x7fd574004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:46.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.025+0000 7fd588ff9640 1 -- 172.21.15.148:0/3353908535 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd574048070 con 0x7fd58c0d3840 2024-03-29T04:11:46.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.025+0000 7fd588ff9640 1 -- 172.21.15.148:0/3353908535 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd574030640 con 0x7fd58c0d3840 2024-03-29T04:11:46.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.025+0000 7fd591d59640 1 -- 172.21.15.148:0/3353908535 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd58c1f8430 con 0x7fd58c0d3840 2024-03-29T04:11:46.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.025+0000 7fd588ff9640 1 -- 172.21.15.148:0/3353908535 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd574034040 con 0x7fd58c0d3840 2024-03-29T04:11:46.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.025+0000 7fd591d59640 1 -- 172.21.15.148:0/3353908535 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd58c1f4620 con 0x7fd58c0d3840 2024-03-29T04:11:46.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.026+0000 7fd591d59640 1 -- 172.21.15.148:0/3353908535 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd58c068740 con 0x7fd58c0d3840 2024-03-29T04:11:46.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.027+0000 7fd588ff9640 1 -- 172.21.15.148:0/3353908535 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd57404b070 con 0x7fd58c0d3840 2024-03-29T04:11:46.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.027+0000 7fd588ff9640 1 --2- 172.21.15.148:0/3353908535 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd56c06c530 0x7fd56c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:46.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.027+0000 7fd58b7fe640 1 --2- 172.21.15.148:0/3353908535 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd56c06c530 0x7fd56c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:46.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.027+0000 7fd588ff9640 1 -- 172.21.15.148:0/3353908535 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7fd5740ad960 con 0x7fd58c0d3840 2024-03-29T04:11:46.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.028+0000 7fd58b7fe640 1 --2- 172.21.15.148:0/3353908535 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd56c06c530 0x7fd56c06e9f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd58c0b4d80 tx=0x7fd57c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:46.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.032+0000 7fd588ff9640 1 -- 172.21.15.148:0/3353908535 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd57407bdd0 con 0x7fd58c0d3840 2024-03-29T04:11:46.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.245+0000 7fd591d59640 1 -- 172.21.15.148:0/3353908535 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "1", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd58c0cff80 con 0x7fd58c0d3840 2024-03-29T04:11:46.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.645+0000 7fd588ff9640 1 -- 172.21.15.148:0/3353908535 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "1", "yes_i_really_really_mean_it": true}]=0 inline data enabled v18) ==== 165+0+0 (secure 0 0 0) 0x7fd57407b6d0 con 0x7fd58c0d3840 2024-03-29T04:11:46.625 INFO:teuthology.orchestra.run.smithi148.stderr:inline data enabled 2024-03-29T04:11:46.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.647+0000 7fd591d59640 1 -- 172.21.15.148:0/3353908535 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd56c06c530 msgr2=0x7fd56c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:46.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.647+0000 7fd591d59640 1 --2- 172.21.15.148:0/3353908535 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd56c06c530 0x7fd56c06e9f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd58c0b4d80 tx=0x7fd57c008040 comp rx=0 tx=0).stop 2024-03-29T04:11:46.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.647+0000 7fd591d59640 1 -- 172.21.15.148:0/3353908535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd58c0d3840 msgr2=0x7fd58c1f7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:46.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.647+0000 7fd591d59640 1 --2- 172.21.15.148:0/3353908535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd58c0d3840 0x7fd58c1f7900 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd57402b9c0 tx=0x7fd574004730 comp rx=0 tx=0).stop 2024-03-29T04:11:46.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.647+0000 7fd591d59640 1 -- 172.21.15.148:0/3353908535 shutdown_connections 2024-03-29T04:11:46.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.647+0000 7fd591d59640 1 --2- 172.21.15.148:0/3353908535 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd56c06c530 0x7fd56c06e9f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:46.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.647+0000 7fd591d59640 1 --2- 172.21.15.148:0/3353908535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd58c0d3c20 0x7fd58c0b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:46.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.647+0000 7fd591d59640 1 --2- 172.21.15.148:0/3353908535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd58c0d3840 0x7fd58c1f7900 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:46.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.647+0000 7fd591d59640 1 --2- 172.21.15.148:0/3353908535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd58c0b36e0 0x7fd58c0ad330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:46.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.647+0000 7fd591d59640 1 -- 172.21.15.148:0/3353908535 >> 172.21.15.148:0/3353908535 conn(0x7fd58c0d42d0 msgr2=0x7fd58c0b2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:46.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.648+0000 7fd591d59640 1 -- 172.21.15.148:0/3353908535 shutdown_connections 2024-03-29T04:11:46.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:46.648+0000 7fd591d59640 1 -- 172.21.15.148:0/3353908535 wait complete. 2024-03-29T04:11:46.641 DEBUG:tasks.cephfs.test_uninlining:fetching tarball 2024-03-29T04:11:46.641 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/mnt.0 && wget http://download.ceph.com/qa/linux-5.4.tar.gz 2024-03-29T04:11:46.678 INFO:teuthology.orchestra.run.smithi148.stderr:--2024-03-29 04:11:46-- http://download.ceph.com/qa/linux-5.4.tar.gz 2024-03-29T04:11:46.679 INFO:teuthology.orchestra.run.smithi148.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-03-29T04:11:46.699 INFO:teuthology.orchestra.run.smithi148.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-03-29T04:11:46.720 INFO:teuthology.orchestra.run.smithi148.stderr:HTTP request sent, awaiting response... 200 OK 2024-03-29T04:11:46.720 INFO:teuthology.orchestra.run.smithi148.stderr:Length: 172616875 (165M) [application/octet-stream] 2024-03-29T04:11:46.724 INFO:teuthology.orchestra.run.smithi148.stderr:Saving to: ‘linux-5.4.tar.gz’ 2024-03-29T04:11:46.727 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-03-29T04:11:46.762 INFO:teuthology.orchestra.run.smithi148.stderr: 0K .......... .......... .......... .......... .......... 0% 1.31M 2m6s 2024-03-29T04:11:46.782 INFO:teuthology.orchestra.run.smithi148.stderr: 50K .......... .......... .......... .......... .......... 0% 2.32M 98s 2024-03-29T04:11:46.803 INFO:teuthology.orchestra.run.smithi148.stderr: 100K .......... .......... .......... .......... .......... 0% 2.46M 88s 2024-03-29T04:11:46.804 INFO:teuthology.orchestra.run.smithi148.stderr: 150K .......... .......... .......... .......... .......... 0% 32.6M 67s 2024-03-29T04:11:46.805 INFO:teuthology.orchestra.run.smithi148.stderr: 200K .......... .......... .......... .......... .......... 0% 39.3M 55s 2024-03-29T04:11:46.824 INFO:teuthology.orchestra.run.smithi148.stderr: 250K .......... .......... .......... .......... .......... 0% 2.63M 56s 2024-03-29T04:11:46.825 INFO:teuthology.orchestra.run.smithi148.stderr: 300K .......... .......... .......... .......... .......... 0% 33.2M 49s 2024-03-29T04:11:46.827 INFO:teuthology.orchestra.run.smithi148.stderr: 350K .......... .......... .......... .......... .......... 0% 37.5M 43s 2024-03-29T04:11:46.828 INFO:teuthology.orchestra.run.smithi148.stderr: 400K .......... .......... .......... .......... .......... 0% 36.1M 39s 2024-03-29T04:11:46.829 INFO:teuthology.orchestra.run.smithi148.stderr: 450K .......... .......... .......... .......... .......... 0% 34.3M 35s 2024-03-29T04:11:46.844 INFO:teuthology.orchestra.run.smithi148.stderr: 500K .......... .......... .......... .......... .......... 0% 3.24M 37s 2024-03-29T04:11:46.846 INFO:teuthology.orchestra.run.smithi148.stderr: 550K .......... .......... .......... .......... .......... 0% 33.9M 34s 2024-03-29T04:11:46.847 INFO:teuthology.orchestra.run.smithi148.stderr: 600K .......... .......... .......... .......... .......... 0% 29.7M 32s 2024-03-29T04:11:46.849 INFO:teuthology.orchestra.run.smithi148.stderr: 650K .......... .......... .......... .......... .......... 0% 33.1M 30s 2024-03-29T04:11:46.850 INFO:teuthology.orchestra.run.smithi148.stderr: 700K .......... .......... .......... .......... .......... 0% 33.8M 28s 2024-03-29T04:11:46.852 INFO:teuthology.orchestra.run.smithi148.stderr: 750K .......... .......... .......... .......... .......... 0% 33.4M 27s 2024-03-29T04:11:46.853 INFO:teuthology.orchestra.run.smithi148.stderr: 800K .......... .......... .......... .......... .......... 0% 29.4M 26s 2024-03-29T04:11:46.855 INFO:teuthology.orchestra.run.smithi148.stderr: 850K .......... .......... .......... .......... .......... 0% 33.7M 24s 2024-03-29T04:11:46.865 INFO:teuthology.orchestra.run.smithi148.stderr: 900K .......... .......... .......... .......... .......... 0% 4.94M 25s 2024-03-29T04:11:46.866 INFO:teuthology.orchestra.run.smithi148.stderr: 950K .......... .......... .......... .......... .......... 0% 32.3M 24s 2024-03-29T04:11:46.868 INFO:teuthology.orchestra.run.smithi148.stderr: 1000K .......... .......... .......... .......... .......... 0% 32.8M 23s 2024-03-29T04:11:46.869 INFO:teuthology.orchestra.run.smithi148.stderr: 1050K .......... .......... .......... .......... .......... 0% 32.7M 22s 2024-03-29T04:11:46.871 INFO:teuthology.orchestra.run.smithi148.stderr: 1100K .......... .......... .......... .......... .......... 0% 27.7M 21s 2024-03-29T04:11:46.873 INFO:teuthology.orchestra.run.smithi148.stderr: 1150K .......... .......... .......... .......... .......... 0% 30.7M 21s 2024-03-29T04:11:46.875 INFO:teuthology.orchestra.run.smithi148.stderr: 1200K .......... .......... .......... .......... .......... 0% 32.5M 20s 2024-03-29T04:11:46.876 INFO:teuthology.orchestra.run.smithi148.stderr: 1250K .......... .......... .......... .......... .......... 0% 31.9M 19s 2024-03-29T04:11:46.877 INFO:teuthology.orchestra.run.smithi148.stderr: 1300K .......... .......... .......... .......... .......... 0% 27.6M 19s 2024-03-29T04:11:46.879 INFO:teuthology.orchestra.run.smithi148.stderr: 1350K .......... .......... .......... .......... .......... 0% 31.8M 18s 2024-03-29T04:11:46.880 INFO:teuthology.orchestra.run.smithi148.stderr: 1400K .......... .......... .......... .......... .......... 0% 31.8M 18s 2024-03-29T04:11:46.882 INFO:teuthology.orchestra.run.smithi148.stderr: 1450K .......... .......... .......... .......... .......... 0% 32.1M 18s 2024-03-29T04:11:46.885 INFO:teuthology.orchestra.run.smithi148.stderr: 1500K .......... .......... .......... .......... .......... 0% 27.2M 17s 2024-03-29T04:11:46.885 INFO:teuthology.orchestra.run.smithi148.stderr: 1550K .......... .......... .......... .......... .......... 0% 30.3M 17s 2024-03-29T04:11:46.887 INFO:teuthology.orchestra.run.smithi148.stderr: 1600K .......... .......... .......... .......... .......... 0% 30.6M 16s 2024-03-29T04:11:46.888 INFO:teuthology.orchestra.run.smithi148.stderr: 1650K .......... .......... .......... .......... .......... 1% 32.0M 16s 2024-03-29T04:11:46.890 INFO:teuthology.orchestra.run.smithi148.stderr: 1700K .......... .......... .......... .......... .......... 1% 30.8M 16s 2024-03-29T04:11:46.891 INFO:teuthology.orchestra.run.smithi148.stderr: 1750K .......... .......... .......... .......... .......... 1% 35.8M 16s 2024-03-29T04:11:46.893 INFO:teuthology.orchestra.run.smithi148.stderr: 1800K .......... .......... .......... .......... .......... 1% 36.8M 15s 2024-03-29T04:11:46.895 INFO:teuthology.orchestra.run.smithi148.stderr: 1850K .......... .......... .......... .......... .......... 1% 35.0M 15s 2024-03-29T04:11:46.896 INFO:teuthology.orchestra.run.smithi148.stderr: 1900K .......... .......... .......... .......... .......... 1% 30.4M 15s 2024-03-29T04:11:46.897 INFO:teuthology.orchestra.run.smithi148.stderr: 1950K .......... .......... .......... .......... .......... 1% 35.7M 14s 2024-03-29T04:11:46.898 INFO:teuthology.orchestra.run.smithi148.stderr: 2000K .......... .......... .......... .......... .......... 1% 35.4M 14s 2024-03-29T04:11:46.900 INFO:teuthology.orchestra.run.smithi148.stderr: 2050K .......... .......... .......... .......... .......... 1% 27.7M 14s 2024-03-29T04:11:46.902 INFO:teuthology.orchestra.run.smithi148.stderr: 2100K .......... .......... .......... .......... .......... 1% 24.2M 14s 2024-03-29T04:11:46.904 INFO:teuthology.orchestra.run.smithi148.stderr: 2150K .......... .......... .......... .......... .......... 1% 34.7M 14s 2024-03-29T04:11:46.905 INFO:teuthology.orchestra.run.smithi148.stderr: 2200K .......... .......... .......... .......... .......... 1% 34.5M 13s 2024-03-29T04:11:46.906 INFO:teuthology.orchestra.run.smithi148.stderr: 2250K .......... .......... .......... .......... .......... 1% 33.5M 13s 2024-03-29T04:11:46.914 INFO:teuthology.orchestra.run.smithi148.stderr: 2300K .......... .......... .......... .......... .......... 1% 29.8M 13s 2024-03-29T04:11:46.914 INFO:teuthology.orchestra.run.smithi148.stderr: 2350K .......... .......... .......... .......... .......... 1% 34.1M 13s 2024-03-29T04:11:46.914 INFO:teuthology.orchestra.run.smithi148.stderr: 2400K .......... .......... .......... .......... .......... 1% 34.9M 13s 2024-03-29T04:11:46.914 INFO:teuthology.orchestra.run.smithi148.stderr: 2450K .......... .......... .......... .......... .......... 1% 32.9M 13s 2024-03-29T04:11:46.916 INFO:teuthology.orchestra.run.smithi148.stderr: 2500K .......... .......... .......... .......... .......... 1% 28.8M 12s 2024-03-29T04:11:46.916 INFO:teuthology.orchestra.run.smithi148.stderr: 2550K .......... .......... .......... .......... .......... 1% 33.9M 12s 2024-03-29T04:11:46.917 INFO:teuthology.orchestra.run.smithi148.stderr: 2600K .......... .......... .......... .......... .......... 1% 34.0M 12s 2024-03-29T04:11:46.919 INFO:teuthology.orchestra.run.smithi148.stderr: 2650K .......... .......... .......... .......... .......... 1% 32.6M 12s 2024-03-29T04:11:46.920 INFO:teuthology.orchestra.run.smithi148.stderr: 2700K .......... .......... .......... .......... .......... 1% 26.2M 12s 2024-03-29T04:11:46.922 INFO:teuthology.orchestra.run.smithi148.stderr: 2750K .......... .......... .......... .......... .......... 1% 29.7M 12s 2024-03-29T04:11:46.924 INFO:teuthology.orchestra.run.smithi148.stderr: 2800K .......... .......... .......... .......... .......... 1% 28.8M 12s 2024-03-29T04:11:46.925 INFO:teuthology.orchestra.run.smithi148.stderr: 2850K .......... .......... .......... .......... .......... 1% 30.2M 11s 2024-03-29T04:11:46.927 INFO:teuthology.orchestra.run.smithi148.stderr: 2900K .......... .......... .......... .......... .......... 1% 25.4M 11s 2024-03-29T04:11:46.929 INFO:teuthology.orchestra.run.smithi148.stderr: 2950K .......... .......... .......... .......... .......... 1% 27.1M 11s 2024-03-29T04:11:46.931 INFO:teuthology.orchestra.run.smithi148.stderr: 3000K .......... .......... .......... .......... .......... 1% 27.9M 11s 2024-03-29T04:11:46.933 INFO:teuthology.orchestra.run.smithi148.stderr: 3050K .......... .......... .......... .......... .......... 1% 26.5M 11s 2024-03-29T04:11:46.935 INFO:teuthology.orchestra.run.smithi148.stderr: 3100K .......... .......... .......... .......... .......... 1% 23.3M 11s 2024-03-29T04:11:46.936 INFO:teuthology.orchestra.run.smithi148.stderr: 3150K .......... .......... .......... .......... .......... 1% 29.4M 11s 2024-03-29T04:11:46.938 INFO:teuthology.orchestra.run.smithi148.stderr: 3200K .......... .......... .......... .......... .......... 1% 30.2M 11s 2024-03-29T04:11:46.940 INFO:teuthology.orchestra.run.smithi148.stderr: 3250K .......... .......... .......... .......... .......... 1% 30.4M 11s 2024-03-29T04:11:46.942 INFO:teuthology.orchestra.run.smithi148.stderr: 3300K .......... .......... .......... .......... .......... 1% 21.6M 11s 2024-03-29T04:11:46.944 INFO:teuthology.orchestra.run.smithi148.stderr: 3350K .......... .......... .......... .......... .......... 2% 25.6M 11s 2024-03-29T04:11:46.946 INFO:teuthology.orchestra.run.smithi148.stderr: 3400K .......... .......... .......... .......... .......... 2% 25.3M 11s 2024-03-29T04:11:46.948 INFO:teuthology.orchestra.run.smithi148.stderr: 3450K .......... .......... .......... .......... .......... 2% 25.2M 11s 2024-03-29T04:11:46.950 INFO:teuthology.orchestra.run.smithi148.stderr: 3500K .......... .......... .......... .......... .......... 2% 21.8M 10s 2024-03-29T04:11:46.952 INFO:teuthology.orchestra.run.smithi148.stderr: 3550K .......... .......... .......... .......... .......... 2% 24.1M 10s 2024-03-29T04:11:46.954 INFO:teuthology.orchestra.run.smithi148.stderr: 3600K .......... .......... .......... .......... .......... 2% 24.3M 10s 2024-03-29T04:11:46.956 INFO:teuthology.orchestra.run.smithi148.stderr: 3650K .......... .......... .......... .......... .......... 2% 23.5M 10s 2024-03-29T04:11:46.958 INFO:teuthology.orchestra.run.smithi148.stderr: 3700K .......... .......... .......... .......... .......... 2% 21.3M 10s 2024-03-29T04:11:46.960 INFO:teuthology.orchestra.run.smithi148.stderr: 3750K .......... .......... .......... .......... .......... 2% 24.3M 10s 2024-03-29T04:11:46.962 INFO:teuthology.orchestra.run.smithi148.stderr: 3800K .......... .......... .......... .......... .......... 2% 23.3M 10s 2024-03-29T04:11:46.964 INFO:teuthology.orchestra.run.smithi148.stderr: 3850K .......... .......... .......... .......... .......... 2% 25.1M 10s 2024-03-29T04:11:46.967 INFO:teuthology.orchestra.run.smithi148.stderr: 3900K .......... .......... .......... .......... .......... 2% 21.9M 10s 2024-03-29T04:11:46.969 INFO:teuthology.orchestra.run.smithi148.stderr: 3950K .......... .......... .......... .......... .......... 2% 23.8M 10s 2024-03-29T04:11:46.971 INFO:teuthology.orchestra.run.smithi148.stderr: 4000K .......... .......... .......... .......... .......... 2% 25.1M 10s 2024-03-29T04:11:46.973 INFO:teuthology.orchestra.run.smithi148.stderr: 4050K .......... .......... .......... .......... .......... 2% 25.1M 10s 2024-03-29T04:11:46.974 INFO:teuthology.orchestra.run.smithi148.stderr: 4100K .......... .......... .......... .......... .......... 2% 27.9M 10s 2024-03-29T04:11:46.976 INFO:teuthology.orchestra.run.smithi148.stderr: 4150K .......... .......... .......... .......... .......... 2% 33.1M 10s 2024-03-29T04:11:46.977 INFO:teuthology.orchestra.run.smithi148.stderr: 4200K .......... .......... .......... .......... .......... 2% 33.2M 10s 2024-03-29T04:11:46.979 INFO:teuthology.orchestra.run.smithi148.stderr: 4250K .......... .......... .......... .......... .......... 2% 30.6M 10s 2024-03-29T04:11:46.981 INFO:teuthology.orchestra.run.smithi148.stderr: 4300K .......... .......... .......... .......... .......... 2% 26.9M 10s 2024-03-29T04:11:46.982 INFO:teuthology.orchestra.run.smithi148.stderr: 4350K .......... .......... .......... .......... .......... 2% 31.5M 10s 2024-03-29T04:11:46.984 INFO:teuthology.orchestra.run.smithi148.stderr: 4400K .......... .......... .......... .......... .......... 2% 30.8M 10s 2024-03-29T04:11:46.985 INFO:teuthology.orchestra.run.smithi148.stderr: 4450K .......... .......... .......... .......... .......... 2% 32.3M 10s 2024-03-29T04:11:46.987 INFO:teuthology.orchestra.run.smithi148.stderr: 4500K .......... .......... .......... .......... .......... 2% 27.1M 9s 2024-03-29T04:11:46.989 INFO:teuthology.orchestra.run.smithi148.stderr: 4550K .......... .......... .......... .......... .......... 2% 32.1M 9s 2024-03-29T04:11:46.990 INFO:teuthology.orchestra.run.smithi148.stderr: 4600K .......... .......... .......... .......... .......... 2% 32.3M 9s 2024-03-29T04:11:46.992 INFO:teuthology.orchestra.run.smithi148.stderr: 4650K .......... .......... .......... .......... .......... 2% 31.7M 9s 2024-03-29T04:11:46.993 INFO:teuthology.orchestra.run.smithi148.stderr: 4700K .......... .......... .......... .......... .......... 2% 26.8M 9s 2024-03-29T04:11:46.995 INFO:teuthology.orchestra.run.smithi148.stderr: 4750K .......... .......... .......... .......... .......... 2% 31.5M 9s 2024-03-29T04:11:46.997 INFO:teuthology.orchestra.run.smithi148.stderr: 4800K .......... .......... .......... .......... .......... 2% 31.5M 9s 2024-03-29T04:11:46.998 INFO:teuthology.orchestra.run.smithi148.stderr: 4850K .......... .......... .......... .......... .......... 2% 31.4M 9s 2024-03-29T04:11:47.000 INFO:teuthology.orchestra.run.smithi148.stderr: 4900K .......... .......... .......... .......... .......... 2% 26.4M 9s 2024-03-29T04:11:47.002 INFO:teuthology.orchestra.run.smithi148.stderr: 4950K .......... .......... .......... .......... .......... 2% 30.8M 9s 2024-03-29T04:11:47.003 INFO:teuthology.orchestra.run.smithi148.stderr: 5000K .......... .......... .......... .......... .......... 2% 31.1M 9s 2024-03-29T04:11:47.005 INFO:teuthology.orchestra.run.smithi148.stderr: 5050K .......... .......... .......... .......... .......... 3% 31.2M 9s 2024-03-29T04:11:47.007 INFO:teuthology.orchestra.run.smithi148.stderr: 5100K .......... .......... .......... .......... .......... 3% 26.6M 9s 2024-03-29T04:11:47.008 INFO:teuthology.orchestra.run.smithi148.stderr: 5150K .......... .......... .......... .......... .......... 3% 28.8M 9s 2024-03-29T04:11:47.010 INFO:teuthology.orchestra.run.smithi148.stderr: 5200K .......... .......... .......... .......... .......... 3% 31.2M 9s 2024-03-29T04:11:47.011 INFO:teuthology.orchestra.run.smithi148.stderr: 5250K .......... .......... .......... .......... .......... 3% 33.6M 9s 2024-03-29T04:11:47.013 INFO:teuthology.orchestra.run.smithi148.stderr: 5300K .......... .......... .......... .......... .......... 3% 28.6M 9s 2024-03-29T04:11:47.015 INFO:teuthology.orchestra.run.smithi148.stderr: 5350K .......... .......... .......... .......... .......... 3% 31.4M 9s 2024-03-29T04:11:47.016 INFO:teuthology.orchestra.run.smithi148.stderr: 5400K .......... .......... .......... .......... .......... 3% 34.0M 9s 2024-03-29T04:11:47.018 INFO:teuthology.orchestra.run.smithi148.stderr: 5450K .......... .......... .......... .......... .......... 3% 34.1M 9s 2024-03-29T04:11:47.019 INFO:teuthology.orchestra.run.smithi148.stderr: 5500K .......... .......... .......... .......... .......... 3% 29.7M 9s 2024-03-29T04:11:47.021 INFO:teuthology.orchestra.run.smithi148.stderr: 5550K .......... .......... .......... .......... .......... 3% 33.7M 9s 2024-03-29T04:11:47.022 INFO:teuthology.orchestra.run.smithi148.stderr: 5600K .......... .......... .......... .......... .......... 3% 34.7M 9s 2024-03-29T04:11:47.023 INFO:teuthology.orchestra.run.smithi148.stderr: 5650K .......... .......... .......... .......... .......... 3% 37.2M 9s 2024-03-29T04:11:47.025 INFO:teuthology.orchestra.run.smithi148.stderr: 5700K .......... .......... .......... .......... .......... 3% 32.3M 9s 2024-03-29T04:11:47.026 INFO:teuthology.orchestra.run.smithi148.stderr: 5750K .......... .......... .......... .......... .......... 3% 37.9M 8s 2024-03-29T04:11:47.027 INFO:teuthology.orchestra.run.smithi148.stderr: 5800K .......... .......... .......... .......... .......... 3% 37.6M 8s 2024-03-29T04:11:47.029 INFO:teuthology.orchestra.run.smithi148.stderr: 5850K .......... .......... .......... .......... .......... 3% 36.8M 8s 2024-03-29T04:11:47.030 INFO:teuthology.orchestra.run.smithi148.stderr: 5900K .......... .......... .......... .......... .......... 3% 31.9M 8s 2024-03-29T04:11:47.032 INFO:teuthology.orchestra.run.smithi148.stderr: 5950K .......... .......... .......... .......... .......... 3% 36.7M 8s 2024-03-29T04:11:47.033 INFO:teuthology.orchestra.run.smithi148.stderr: 6000K .......... .......... .......... .......... .......... 3% 36.4M 8s 2024-03-29T04:11:47.036 INFO:teuthology.orchestra.run.smithi148.stderr: 6050K .......... .......... .......... .......... .......... 3% 35.1M 8s 2024-03-29T04:11:47.036 INFO:teuthology.orchestra.run.smithi148.stderr: 6100K .......... .......... .......... .......... .......... 3% 31.3M 8s 2024-03-29T04:11:47.037 INFO:teuthology.orchestra.run.smithi148.stderr: 6150K .......... .......... .......... .......... .......... 3% 36.3M 8s 2024-03-29T04:11:47.038 INFO:teuthology.orchestra.run.smithi148.stderr: 6200K .......... .......... .......... .......... .......... 3% 36.0M 8s 2024-03-29T04:11:47.040 INFO:teuthology.orchestra.run.smithi148.stderr: 6250K .......... .......... .......... .......... .......... 3% 36.0M 8s 2024-03-29T04:11:47.041 INFO:teuthology.orchestra.run.smithi148.stderr: 6300K .......... .......... .......... .......... .......... 3% 29.9M 8s 2024-03-29T04:11:47.043 INFO:teuthology.orchestra.run.smithi148.stderr: 6350K .......... .......... .......... .......... .......... 3% 34.3M 8s 2024-03-29T04:11:47.044 INFO:teuthology.orchestra.run.smithi148.stderr: 6400K .......... .......... .......... .......... .......... 3% 35.8M 8s 2024-03-29T04:11:47.046 INFO:teuthology.orchestra.run.smithi148.stderr: 6450K .......... .......... .......... .......... .......... 3% 34.1M 8s 2024-03-29T04:11:47.047 INFO:teuthology.orchestra.run.smithi148.stderr: 6500K .......... .......... .......... .......... .......... 3% 30.0M 8s 2024-03-29T04:11:47.049 INFO:teuthology.orchestra.run.smithi148.stderr: 6550K .......... .......... .......... .......... .......... 3% 31.1M 8s 2024-03-29T04:11:47.050 INFO:teuthology.orchestra.run.smithi148.stderr: 6600K .......... .......... .......... .......... .......... 3% 30.7M 8s 2024-03-29T04:11:47.052 INFO:teuthology.orchestra.run.smithi148.stderr: 6650K .......... .......... .......... .......... .......... 3% 30.8M 8s 2024-03-29T04:11:47.054 INFO:teuthology.orchestra.run.smithi148.stderr: 6700K .......... .......... .......... .......... .......... 4% 29.1M 8s 2024-03-29T04:11:47.055 INFO:teuthology.orchestra.run.smithi148.stderr: 6750K .......... .......... .......... .......... .......... 4% 32.8M 8s 2024-03-29T04:11:47.057 INFO:teuthology.orchestra.run.smithi148.stderr: 6800K .......... .......... .......... .......... .......... 4% 33.0M 8s 2024-03-29T04:11:47.058 INFO:teuthology.orchestra.run.smithi148.stderr: 6850K .......... .......... .......... .......... .......... 4% 33.1M 8s 2024-03-29T04:11:47.060 INFO:teuthology.orchestra.run.smithi148.stderr: 6900K .......... .......... .......... .......... .......... 4% 28.5M 8s 2024-03-29T04:11:47.061 INFO:teuthology.orchestra.run.smithi148.stderr: 6950K .......... .......... .......... .......... .......... 4% 32.4M 8s 2024-03-29T04:11:47.063 INFO:teuthology.orchestra.run.smithi148.stderr: 7000K .......... .......... .......... .......... .......... 4% 32.9M 8s 2024-03-29T04:11:47.064 INFO:teuthology.orchestra.run.smithi148.stderr: 7050K .......... .......... .......... .......... .......... 4% 31.9M 8s 2024-03-29T04:11:47.067 INFO:teuthology.orchestra.run.smithi148.stderr: 7100K .......... .......... .......... .......... .......... 4% 20.3M 8s 2024-03-29T04:11:47.068 INFO:teuthology.orchestra.run.smithi148.stderr: 7150K .......... .......... .......... .......... .......... 4% 30.3M 8s 2024-03-29T04:11:47.070 INFO:teuthology.orchestra.run.smithi148.stderr: 7200K .......... .......... .......... .......... .......... 4% 30.9M 8s 2024-03-29T04:11:47.072 INFO:teuthology.orchestra.run.smithi148.stderr: 7250K .......... .......... .......... .......... .......... 4% 30.7M 8s 2024-03-29T04:11:47.073 INFO:teuthology.orchestra.run.smithi148.stderr: 7300K .......... .......... .......... .......... .......... 4% 26.6M 8s 2024-03-29T04:11:47.075 INFO:teuthology.orchestra.run.smithi148.stderr: 7350K .......... .......... .......... .......... .......... 4% 31.1M 8s 2024-03-29T04:11:47.076 INFO:teuthology.orchestra.run.smithi148.stderr: 7400K .......... .......... .......... .......... .......... 4% 30.7M 8s 2024-03-29T04:11:47.078 INFO:teuthology.orchestra.run.smithi148.stderr: 7450K .......... .......... .......... .......... .......... 4% 30.5M 8s 2024-03-29T04:11:47.080 INFO:teuthology.orchestra.run.smithi148.stderr: 7500K .......... .......... .......... .......... .......... 4% 26.1M 8s 2024-03-29T04:11:47.082 INFO:teuthology.orchestra.run.smithi148.stderr: 7550K .......... .......... .......... .......... .......... 4% 30.4M 8s 2024-03-29T04:11:47.083 INFO:teuthology.orchestra.run.smithi148.stderr: 7600K .......... .......... .......... .......... .......... 4% 30.6M 8s 2024-03-29T04:11:47.085 INFO:teuthology.orchestra.run.smithi148.stderr: 7650K .......... .......... .......... .......... .......... 4% 30.4M 8s 2024-03-29T04:11:47.087 INFO:teuthology.orchestra.run.smithi148.stderr: 7700K .......... .......... .......... .......... .......... 4% 25.7M 8s 2024-03-29T04:11:47.089 INFO:teuthology.orchestra.run.smithi148.stderr: 7750K .......... .......... .......... .......... .......... 4% 30.0M 8s 2024-03-29T04:11:47.090 INFO:teuthology.orchestra.run.smithi148.stderr: 7800K .......... .......... .......... .......... .......... 4% 29.8M 7s 2024-03-29T04:11:47.092 INFO:teuthology.orchestra.run.smithi148.stderr: 7850K .......... .......... .......... .......... .......... 4% 28.7M 7s 2024-03-29T04:11:47.094 INFO:teuthology.orchestra.run.smithi148.stderr: 7900K .......... .......... .......... .......... .......... 4% 21.4M 7s 2024-03-29T04:11:47.096 INFO:teuthology.orchestra.run.smithi148.stderr: 7950K .......... .......... .......... .......... .......... 4% 27.1M 7s 2024-03-29T04:11:47.097 INFO:teuthology.orchestra.run.smithi148.stderr: 8000K .......... .......... .......... .......... .......... 4% 27.5M 7s 2024-03-29T04:11:47.099 INFO:teuthology.orchestra.run.smithi148.stderr: 8050K .......... .......... .......... .......... .......... 4% 28.9M 7s 2024-03-29T04:11:47.101 INFO:teuthology.orchestra.run.smithi148.stderr: 8100K .......... .......... .......... .......... .......... 4% 24.7M 7s 2024-03-29T04:11:47.103 INFO:teuthology.orchestra.run.smithi148.stderr: 8150K .......... .......... .......... .......... .......... 4% 30.3M 7s 2024-03-29T04:11:47.105 INFO:teuthology.orchestra.run.smithi148.stderr: 8200K .......... .......... .......... .......... .......... 4% 25.6M 7s 2024-03-29T04:11:47.107 INFO:teuthology.orchestra.run.smithi148.stderr: 8250K .......... .......... .......... .......... .......... 4% 22.0M 7s 2024-03-29T04:11:47.110 INFO:teuthology.orchestra.run.smithi148.stderr: 8300K .......... .......... .......... .......... .......... 4% 20.1M 7s 2024-03-29T04:11:47.113 INFO:teuthology.orchestra.run.smithi148.stderr: 8350K .......... .......... .......... .......... .......... 4% 41.8M 7s 2024-03-29T04:11:47.116 INFO:teuthology.orchestra.run.smithi148.stderr: 8400K .......... .......... .......... .......... .......... 5% 41.5M 7s 2024-03-29T04:11:47.116 INFO:teuthology.orchestra.run.smithi148.stderr: 8450K .......... .......... .......... .......... .......... 5% 40.7M 7s 2024-03-29T04:11:47.116 INFO:teuthology.orchestra.run.smithi148.stderr: 8500K .......... .......... .......... .......... .......... 5% 34.6M 7s 2024-03-29T04:11:47.116 INFO:teuthology.orchestra.run.smithi148.stderr: 8550K .......... .......... .......... .......... .......... 5% 39.5M 7s 2024-03-29T04:11:47.117 INFO:teuthology.orchestra.run.smithi148.stderr: 8600K .......... .......... .......... .......... .......... 5% 39.8M 7s 2024-03-29T04:11:47.118 INFO:teuthology.orchestra.run.smithi148.stderr: 8650K .......... .......... .......... .......... .......... 5% 39.8M 7s 2024-03-29T04:11:47.120 INFO:teuthology.orchestra.run.smithi148.stderr: 8700K .......... .......... .......... .......... .......... 5% 26.3M 7s 2024-03-29T04:11:47.122 INFO:teuthology.orchestra.run.smithi148.stderr: 8750K .......... .......... .......... .......... .......... 5% 26.7M 7s 2024-03-29T04:11:47.123 INFO:teuthology.orchestra.run.smithi148.stderr: 8800K .......... .......... .......... .......... .......... 5% 36.6M 7s 2024-03-29T04:11:47.124 INFO:teuthology.orchestra.run.smithi148.stderr: 8850K .......... .......... .......... .......... .......... 5% 40.8M 7s 2024-03-29T04:11:47.125 INFO:teuthology.orchestra.run.smithi148.stderr: 8900K .......... .......... .......... .......... .......... 5% 34.7M 7s 2024-03-29T04:11:47.127 INFO:teuthology.orchestra.run.smithi148.stderr: 8950K .......... .......... .......... .......... .......... 5% 40.1M 7s 2024-03-29T04:11:47.128 INFO:teuthology.orchestra.run.smithi148.stderr: 9000K .......... .......... .......... .......... .......... 5% 38.6M 7s 2024-03-29T04:11:47.129 INFO:teuthology.orchestra.run.smithi148.stderr: 9050K .......... .......... .......... .......... .......... 5% 37.3M 7s 2024-03-29T04:11:47.131 INFO:teuthology.orchestra.run.smithi148.stderr: 9100K .......... .......... .......... .......... .......... 5% 27.2M 7s 2024-03-29T04:11:47.133 INFO:teuthology.orchestra.run.smithi148.stderr: 9150K .......... .......... .......... .......... .......... 5% 30.1M 7s 2024-03-29T04:11:47.135 INFO:teuthology.orchestra.run.smithi148.stderr: 9200K .......... .......... .......... .......... .......... 5% 26.6M 7s 2024-03-29T04:11:47.136 INFO:teuthology.orchestra.run.smithi148.stderr: 9250K .......... .......... .......... .......... .......... 5% 37.5M 7s 2024-03-29T04:11:47.138 INFO:teuthology.orchestra.run.smithi148.stderr: 9300K .......... .......... .......... .......... .......... 5% 26.0M 7s 2024-03-29T04:11:47.140 INFO:teuthology.orchestra.run.smithi148.stderr: 9350K .......... .......... .......... .......... .......... 5% 26.4M 7s 2024-03-29T04:11:47.141 INFO:teuthology.orchestra.run.smithi148.stderr: 9400K .......... .......... .......... .......... .......... 5% 35.7M 7s 2024-03-29T04:11:47.143 INFO:teuthology.orchestra.run.smithi148.stderr: 9450K .......... .......... .......... .......... .......... 5% 25.6M 7s 2024-03-29T04:11:47.145 INFO:teuthology.orchestra.run.smithi148.stderr: 9500K .......... .......... .......... .......... .......... 5% 33.3M 7s 2024-03-29T04:11:47.146 INFO:teuthology.orchestra.run.smithi148.stderr: 9550K .......... .......... .......... .......... .......... 5% 28.4M 7s 2024-03-29T04:11:47.148 INFO:teuthology.orchestra.run.smithi148.stderr: 9600K .......... .......... .......... .......... .......... 5% 26.4M 7s 2024-03-29T04:11:47.149 INFO:teuthology.orchestra.run.smithi148.stderr: 9650K .......... .......... .......... .......... .......... 5% 36.3M 7s 2024-03-29T04:11:47.151 INFO:teuthology.orchestra.run.smithi148.stderr: 9700K .......... .......... .......... .......... .......... 5% 27.4M 7s 2024-03-29T04:11:47.153 INFO:teuthology.orchestra.run.smithi148.stderr: 9750K .......... .......... .......... .......... .......... 5% 29.5M 7s 2024-03-29T04:11:47.155 INFO:teuthology.orchestra.run.smithi148.stderr: 9800K .......... .......... .......... .......... .......... 5% 25.0M 7s 2024-03-29T04:11:47.157 INFO:teuthology.orchestra.run.smithi148.stderr: 9850K .......... .......... .......... .......... .......... 5% 28.6M 7s 2024-03-29T04:11:47.158 INFO:teuthology.orchestra.run.smithi148.stderr: 9900K .......... .......... .......... .......... .......... 5% 42.1M 7s 2024-03-29T04:11:47.159 INFO:teuthology.orchestra.run.smithi148.stderr: 9950K .......... .......... .......... .......... .......... 5% 26.6M 7s 2024-03-29T04:11:47.161 INFO:teuthology.orchestra.run.smithi148.stderr: 10000K .......... .......... .......... .......... .......... 5% 28.4M 7s 2024-03-29T04:11:47.163 INFO:teuthology.orchestra.run.smithi148.stderr: 10050K .......... .......... .......... .......... .......... 5% 25.3M 7s 2024-03-29T04:11:47.164 INFO:teuthology.orchestra.run.smithi148.stderr: 10100K .......... .......... .......... .......... .......... 6% 34.5M 7s 2024-03-29T04:11:47.167 INFO:teuthology.orchestra.run.smithi148.stderr: 10150K .......... .......... .......... .......... .......... 6% 25.2M 7s 2024-03-29T04:11:47.168 INFO:teuthology.orchestra.run.smithi148.stderr: 10200K .......... .......... .......... .......... .......... 6% 27.1M 7s 2024-03-29T04:11:47.170 INFO:teuthology.orchestra.run.smithi148.stderr: 10250K .......... .......... .......... .......... .......... 6% 31.6M 7s 2024-03-29T04:11:47.171 INFO:teuthology.orchestra.run.smithi148.stderr: 10300K .......... .......... .......... .......... .......... 6% 30.3M 7s 2024-03-29T04:11:47.173 INFO:teuthology.orchestra.run.smithi148.stderr: 10350K .......... .......... .......... .......... .......... 6% 26.6M 7s 2024-03-29T04:11:47.175 INFO:teuthology.orchestra.run.smithi148.stderr: 10400K .......... .......... .......... .......... .......... 6% 26.4M 7s 2024-03-29T04:11:47.177 INFO:teuthology.orchestra.run.smithi148.stderr: 10450K .......... .......... .......... .......... .......... 6% 30.3M 7s 2024-03-29T04:11:47.178 INFO:teuthology.orchestra.run.smithi148.stderr: 10500K .......... .......... .......... .......... .......... 6% 30.8M 7s 2024-03-29T04:11:47.180 INFO:teuthology.orchestra.run.smithi148.stderr: 10550K .......... .......... .......... .......... .......... 6% 25.9M 7s 2024-03-29T04:11:47.182 INFO:teuthology.orchestra.run.smithi148.stderr: 10600K .......... .......... .......... .......... .......... 6% 29.9M 7s 2024-03-29T04:11:47.184 INFO:teuthology.orchestra.run.smithi148.stderr: 10650K .......... .......... .......... .......... .......... 6% 30.6M 7s 2024-03-29T04:11:47.185 INFO:teuthology.orchestra.run.smithi148.stderr: 10700K .......... .......... .......... .......... .......... 6% 26.1M 7s 2024-03-29T04:11:47.187 INFO:teuthology.orchestra.run.smithi148.stderr: 10750K .......... .......... .......... .......... .......... 6% 29.4M 7s 2024-03-29T04:11:47.189 INFO:teuthology.orchestra.run.smithi148.stderr: 10800K .......... .......... .......... .......... .......... 6% 29.6M 7s 2024-03-29T04:11:47.191 INFO:teuthology.orchestra.run.smithi148.stderr: 10850K .......... .......... .......... .......... .......... 6% 25.2M 7s 2024-03-29T04:11:47.192 INFO:teuthology.orchestra.run.smithi148.stderr: 10900K .......... .......... .......... .......... .......... 6% 29.2M 7s 2024-03-29T04:11:47.194 INFO:teuthology.orchestra.run.smithi148.stderr: 10950K .......... .......... .......... .......... .......... 6% 31.5M 7s 2024-03-29T04:11:47.195 INFO:teuthology.orchestra.run.smithi148.stderr: 11000K .......... .......... .......... .......... .......... 6% 31.8M 7s 2024-03-29T04:11:47.197 INFO:teuthology.orchestra.run.smithi148.stderr: 11050K .......... .......... .......... .......... .......... 6% 28.9M 7s 2024-03-29T04:11:47.199 INFO:teuthology.orchestra.run.smithi148.stderr: 11100K .......... .......... .......... .......... .......... 6% 33.4M 7s 2024-03-29T04:11:47.200 INFO:teuthology.orchestra.run.smithi148.stderr: 11150K .......... .......... .......... .......... .......... 6% 28.1M 7s 2024-03-29T04:11:47.202 INFO:teuthology.orchestra.run.smithi148.stderr: 11200K .......... .......... .......... .......... .......... 6% 26.8M 7s 2024-03-29T04:11:47.204 INFO:teuthology.orchestra.run.smithi148.stderr: 11250K .......... .......... .......... .......... .......... 6% 32.1M 7s 2024-03-29T04:11:47.206 INFO:teuthology.orchestra.run.smithi148.stderr: 11300K .......... .......... .......... .......... .......... 6% 24.2M 7s 2024-03-29T04:11:47.207 INFO:teuthology.orchestra.run.smithi148.stderr: 11350K .......... .......... .......... .......... .......... 6% 31.5M 7s 2024-03-29T04:11:47.209 INFO:teuthology.orchestra.run.smithi148.stderr: 11400K .......... .......... .......... .......... .......... 6% 31.6M 7s 2024-03-29T04:11:47.210 INFO:teuthology.orchestra.run.smithi148.stderr: 11450K .......... .......... .......... .......... .......... 6% 31.6M 7s 2024-03-29T04:11:47.212 INFO:teuthology.orchestra.run.smithi148.stderr: 11500K .......... .......... .......... .......... .......... 6% 24.8M 7s 2024-03-29T04:11:47.214 INFO:teuthology.orchestra.run.smithi148.stderr: 11550K .......... .......... .......... .......... .......... 6% 28.1M 7s 2024-03-29T04:11:47.216 INFO:teuthology.orchestra.run.smithi148.stderr: 11600K .......... .......... .......... .......... .......... 6% 30.5M 7s 2024-03-29T04:11:47.217 INFO:teuthology.orchestra.run.smithi148.stderr: 11650K .......... .......... .......... .......... .......... 6% 26.9M 7s 2024-03-29T04:11:47.219 INFO:teuthology.orchestra.run.smithi148.stderr: 11700K .......... .......... .......... .......... .......... 6% 31.0M 7s 2024-03-29T04:11:47.221 INFO:teuthology.orchestra.run.smithi148.stderr: 11750K .......... .......... .......... .......... .......... 7% 26.4M 7s 2024-03-29T04:11:47.222 INFO:teuthology.orchestra.run.smithi148.stderr: 11800K .......... .......... .......... .......... .......... 7% 32.6M 7s 2024-03-29T04:11:47.224 INFO:teuthology.orchestra.run.smithi148.stderr: 11850K .......... .......... .......... .......... .......... 7% 33.4M 7s 2024-03-29T04:11:47.226 INFO:teuthology.orchestra.run.smithi148.stderr: 11900K .......... .......... .......... .......... .......... 7% 29.0M 7s 2024-03-29T04:11:47.227 INFO:teuthology.orchestra.run.smithi148.stderr: 11950K .......... .......... .......... .......... .......... 7% 29.0M 7s 2024-03-29T04:11:47.229 INFO:teuthology.orchestra.run.smithi148.stderr: 12000K .......... .......... .......... .......... .......... 7% 26.8M 7s 2024-03-29T04:11:47.231 INFO:teuthology.orchestra.run.smithi148.stderr: 12050K .......... .......... .......... .......... .......... 7% 30.6M 7s 2024-03-29T04:11:47.232 INFO:teuthology.orchestra.run.smithi148.stderr: 12100K .......... .......... .......... .......... .......... 7% 26.1M 7s 2024-03-29T04:11:47.234 INFO:teuthology.orchestra.run.smithi148.stderr: 12150K .......... .......... .......... .......... .......... 7% 30.6M 7s 2024-03-29T04:11:47.236 INFO:teuthology.orchestra.run.smithi148.stderr: 12200K .......... .......... .......... .......... .......... 7% 26.4M 7s 2024-03-29T04:11:47.237 INFO:teuthology.orchestra.run.smithi148.stderr: 12250K .......... .......... .......... .......... .......... 7% 26.6M 7s 2024-03-29T04:11:47.239 INFO:teuthology.orchestra.run.smithi148.stderr: 12300K .......... .......... .......... .......... .......... 7% 35.8M 7s 2024-03-29T04:11:47.241 INFO:teuthology.orchestra.run.smithi148.stderr: 12350K .......... .......... .......... .......... .......... 7% 33.9M 7s 2024-03-29T04:11:47.243 INFO:teuthology.orchestra.run.smithi148.stderr: 12400K .......... .......... .......... .......... .......... 7% 20.4M 7s 2024-03-29T04:11:47.245 INFO:teuthology.orchestra.run.smithi148.stderr: 12450K .......... .......... .......... .......... .......... 7% 27.3M 7s 2024-03-29T04:11:47.246 INFO:teuthology.orchestra.run.smithi148.stderr: 12500K .......... .......... .......... .......... .......... 7% 34.4M 6s 2024-03-29T04:11:47.248 INFO:teuthology.orchestra.run.smithi148.stderr: 12550K .......... .......... .......... .......... .......... 7% 28.1M 6s 2024-03-29T04:11:47.249 INFO:teuthology.orchestra.run.smithi148.stderr: 12600K .......... .......... .......... .......... .......... 7% 35.1M 6s 2024-03-29T04:11:47.251 INFO:teuthology.orchestra.run.smithi148.stderr: 12650K .......... .......... .......... .......... .......... 7% 30.3M 6s 2024-03-29T04:11:47.253 INFO:teuthology.orchestra.run.smithi148.stderr: 12700K .......... .......... .......... .......... .......... 7% 29.0M 6s 2024-03-29T04:11:47.254 INFO:teuthology.orchestra.run.smithi148.stderr: 12750K .......... .......... .......... .......... .......... 7% 28.8M 6s 2024-03-29T04:11:47.256 INFO:teuthology.orchestra.run.smithi148.stderr: 12800K .......... .......... .......... .......... .......... 7% 26.3M 6s 2024-03-29T04:11:47.257 INFO:teuthology.orchestra.run.smithi148.stderr: 12850K .......... .......... .......... .......... .......... 7% 37.2M 6s 2024-03-29T04:11:47.259 INFO:teuthology.orchestra.run.smithi148.stderr: 12900K .......... .......... .......... .......... .......... 7% 27.6M 6s 2024-03-29T04:11:47.261 INFO:teuthology.orchestra.run.smithi148.stderr: 12950K .......... .......... .......... .......... .......... 7% 27.5M 6s 2024-03-29T04:11:47.263 INFO:teuthology.orchestra.run.smithi148.stderr: 13000K .......... .......... .......... .......... .......... 7% 23.9M 6s 2024-03-29T04:11:47.265 INFO:teuthology.orchestra.run.smithi148.stderr: 13050K .......... .......... .......... .......... .......... 7% 39.8M 6s 2024-03-29T04:11:47.266 INFO:teuthology.orchestra.run.smithi148.stderr: 13100K .......... .......... .......... .......... .......... 7% 27.9M 6s 2024-03-29T04:11:47.267 INFO:teuthology.orchestra.run.smithi148.stderr: 13150K .......... .......... .......... .......... .......... 7% 40.0M 6s 2024-03-29T04:11:47.269 INFO:teuthology.orchestra.run.smithi148.stderr: 13200K .......... .......... .......... .......... .......... 7% 24.9M 6s 2024-03-29T04:11:47.271 INFO:teuthology.orchestra.run.smithi148.stderr: 13250K .......... .......... .......... .......... .......... 7% 27.6M 6s 2024-03-29T04:11:47.273 INFO:teuthology.orchestra.run.smithi148.stderr: 13300K .......... .......... .......... .......... .......... 7% 28.5M 6s 2024-03-29T04:11:47.274 INFO:teuthology.orchestra.run.smithi148.stderr: 13350K .......... .......... .......... .......... .......... 7% 28.4M 6s 2024-03-29T04:11:47.276 INFO:teuthology.orchestra.run.smithi148.stderr: 13400K .......... .......... .......... .......... .......... 7% 31.9M 6s 2024-03-29T04:11:47.278 INFO:teuthology.orchestra.run.smithi148.stderr: 13450K .......... .......... .......... .......... .......... 8% 28.5M 6s 2024-03-29T04:11:47.279 INFO:teuthology.orchestra.run.smithi148.stderr: 13500K .......... .......... .......... .......... .......... 8% 27.6M 6s 2024-03-29T04:11:47.281 INFO:teuthology.orchestra.run.smithi148.stderr: 13550K .......... .......... .......... .......... .......... 8% 27.7M 6s 2024-03-29T04:11:47.284 INFO:teuthology.orchestra.run.smithi148.stderr: 13600K .......... .......... .......... .......... .......... 8% 24.8M 6s 2024-03-29T04:11:47.285 INFO:teuthology.orchestra.run.smithi148.stderr: 13650K .......... .......... .......... .......... .......... 8% 36.3M 6s 2024-03-29T04:11:47.286 INFO:teuthology.orchestra.run.smithi148.stderr: 13700K .......... .......... .......... .......... .......... 8% 28.2M 6s 2024-03-29T04:11:47.288 INFO:teuthology.orchestra.run.smithi148.stderr: 13750K .......... .......... .......... .......... .......... 8% 26.7M 6s 2024-03-29T04:11:47.290 INFO:teuthology.orchestra.run.smithi148.stderr: 13800K .......... .......... .......... .......... .......... 8% 29.9M 6s 2024-03-29T04:11:47.291 INFO:teuthology.orchestra.run.smithi148.stderr: 13850K .......... .......... .......... .......... .......... 8% 33.8M 6s 2024-03-29T04:11:47.294 INFO:teuthology.orchestra.run.smithi148.stderr: 13900K .......... .......... .......... .......... .......... 8% 24.7M 6s 2024-03-29T04:11:47.295 INFO:teuthology.orchestra.run.smithi148.stderr: 13950K .......... .......... .......... .......... .......... 8% 32.5M 6s 2024-03-29T04:11:47.296 INFO:teuthology.orchestra.run.smithi148.stderr: 14000K .......... .......... .......... .......... .......... 8% 28.7M 6s 2024-03-29T04:11:47.300 INFO:teuthology.orchestra.run.smithi148.stderr: 14050K .......... .......... .......... .......... .......... 8% 12.4M 6s 2024-03-29T04:11:47.302 INFO:teuthology.orchestra.run.smithi148.stderr: 14100K .......... .......... .......... .......... .......... 8% 32.9M 6s 2024-03-29T04:11:47.303 INFO:teuthology.orchestra.run.smithi148.stderr: 14150K .......... .......... .......... .......... .......... 8% 33.2M 6s 2024-03-29T04:11:47.305 INFO:teuthology.orchestra.run.smithi148.stderr: 14200K .......... .......... .......... .......... .......... 8% 28.4M 6s 2024-03-29T04:11:47.306 INFO:teuthology.orchestra.run.smithi148.stderr: 14250K .......... .......... .......... .......... .......... 8% 33.4M 6s 2024-03-29T04:11:47.308 INFO:teuthology.orchestra.run.smithi148.stderr: 14300K .......... .......... .......... .......... .......... 8% 32.8M 6s 2024-03-29T04:11:47.309 INFO:teuthology.orchestra.run.smithi148.stderr: 14350K .......... .......... .......... .......... .......... 8% 32.7M 6s 2024-03-29T04:11:47.311 INFO:teuthology.orchestra.run.smithi148.stderr: 14400K .......... .......... .......... .......... .......... 8% 27.9M 6s 2024-03-29T04:11:47.313 INFO:teuthology.orchestra.run.smithi148.stderr: 14450K .......... .......... .......... .......... .......... 8% 32.4M 6s 2024-03-29T04:11:47.314 INFO:teuthology.orchestra.run.smithi148.stderr: 14500K .......... .......... .......... .......... .......... 8% 32.0M 6s 2024-03-29T04:11:47.318 INFO:teuthology.orchestra.run.smithi148.stderr: 14550K .......... .......... .......... .......... .......... 8% 28.8M 6s 2024-03-29T04:11:47.319 INFO:teuthology.orchestra.run.smithi148.stderr: 14600K .......... .......... .......... .......... .......... 8% 19.4M 6s 2024-03-29T04:11:47.320 INFO:teuthology.orchestra.run.smithi148.stderr: 14650K .......... .......... .......... .......... .......... 8% 32.3M 6s 2024-03-29T04:11:47.321 INFO:teuthology.orchestra.run.smithi148.stderr: 14700K .......... .......... .......... .......... .......... 8% 35.0M 6s 2024-03-29T04:11:47.323 INFO:teuthology.orchestra.run.smithi148.stderr: 14750K .......... .......... .......... .......... .......... 8% 34.9M 6s 2024-03-29T04:11:47.324 INFO:teuthology.orchestra.run.smithi148.stderr: 14800K .......... .......... .......... .......... .......... 8% 30.0M 6s 2024-03-29T04:11:47.326 INFO:teuthology.orchestra.run.smithi148.stderr: 14850K .......... .......... .......... .......... .......... 8% 34.2M 6s 2024-03-29T04:11:47.327 INFO:teuthology.orchestra.run.smithi148.stderr: 14900K .......... .......... .......... .......... .......... 8% 34.6M 6s 2024-03-29T04:11:47.329 INFO:teuthology.orchestra.run.smithi148.stderr: 14950K .......... .......... .......... .......... .......... 8% 34.0M 6s 2024-03-29T04:11:47.331 INFO:teuthology.orchestra.run.smithi148.stderr: 15000K .......... .......... .......... .......... .......... 8% 24.9M 6s 2024-03-29T04:11:47.332 INFO:teuthology.orchestra.run.smithi148.stderr: 15050K .......... .......... .......... .......... .......... 8% 29.2M 6s 2024-03-29T04:11:47.334 INFO:teuthology.orchestra.run.smithi148.stderr: 15100K .......... .......... .......... .......... .......... 8% 30.6M 6s 2024-03-29T04:11:47.335 INFO:teuthology.orchestra.run.smithi148.stderr: 15150K .......... .......... .......... .......... .......... 9% 31.2M 6s 2024-03-29T04:11:47.337 INFO:teuthology.orchestra.run.smithi148.stderr: 15200K .......... .......... .......... .......... .......... 9% 25.7M 6s 2024-03-29T04:11:47.339 INFO:teuthology.orchestra.run.smithi148.stderr: 15250K .......... .......... .......... .......... .......... 9% 30.5M 6s 2024-03-29T04:11:47.340 INFO:teuthology.orchestra.run.smithi148.stderr: 15300K .......... .......... .......... .......... .......... 9% 30.3M 6s 2024-03-29T04:11:47.342 INFO:teuthology.orchestra.run.smithi148.stderr: 15350K .......... .......... .......... .......... .......... 9% 30.3M 6s 2024-03-29T04:11:47.344 INFO:teuthology.orchestra.run.smithi148.stderr: 15400K .......... .......... .......... .......... .......... 9% 26.1M 6s 2024-03-29T04:11:47.346 INFO:teuthology.orchestra.run.smithi148.stderr: 15450K .......... .......... .......... .......... .......... 9% 30.3M 6s 2024-03-29T04:11:47.347 INFO:teuthology.orchestra.run.smithi148.stderr: 15500K .......... .......... .......... .......... .......... 9% 30.0M 6s 2024-03-29T04:11:47.349 INFO:teuthology.orchestra.run.smithi148.stderr: 15550K .......... .......... .......... .......... .......... 9% 31.5M 6s 2024-03-29T04:11:47.351 INFO:teuthology.orchestra.run.smithi148.stderr: 15600K .......... .......... .......... .......... .......... 9% 27.0M 6s 2024-03-29T04:11:47.352 INFO:teuthology.orchestra.run.smithi148.stderr: 15650K .......... .......... .......... .......... .......... 9% 31.4M 6s 2024-03-29T04:11:47.354 INFO:teuthology.orchestra.run.smithi148.stderr: 15700K .......... .......... .......... .......... .......... 9% 28.6M 6s 2024-03-29T04:11:47.356 INFO:teuthology.orchestra.run.smithi148.stderr: 15750K .......... .......... .......... .......... .......... 9% 30.0M 6s 2024-03-29T04:11:47.357 INFO:teuthology.orchestra.run.smithi148.stderr: 15800K .......... .......... .......... .......... .......... 9% 25.1M 6s 2024-03-29T04:11:47.359 INFO:teuthology.orchestra.run.smithi148.stderr: 15850K .......... .......... .......... .......... .......... 9% 29.1M 6s 2024-03-29T04:11:47.361 INFO:teuthology.orchestra.run.smithi148.stderr: 15900K .......... .......... .......... .......... .......... 9% 28.8M 6s 2024-03-29T04:11:47.362 INFO:teuthology.orchestra.run.smithi148.stderr: 15950K .......... .......... .......... .......... .......... 9% 28.9M 6s 2024-03-29T04:11:47.364 INFO:teuthology.orchestra.run.smithi148.stderr: 16000K .......... .......... .......... .......... .......... 9% 24.9M 6s 2024-03-29T04:11:47.366 INFO:teuthology.orchestra.run.smithi148.stderr: 16050K .......... .......... .......... .......... .......... 9% 26.4M 6s 2024-03-29T04:11:47.368 INFO:teuthology.orchestra.run.smithi148.stderr: 16100K .......... .......... .......... .......... .......... 9% 26.3M 6s 2024-03-29T04:11:47.370 INFO:teuthology.orchestra.run.smithi148.stderr: 16150K .......... .......... .......... .......... .......... 9% 27.3M 6s 2024-03-29T04:11:47.372 INFO:teuthology.orchestra.run.smithi148.stderr: 16200K .......... .......... .......... .......... .......... 9% 22.5M 6s 2024-03-29T04:11:47.374 INFO:teuthology.orchestra.run.smithi148.stderr: 16250K .......... .......... .......... .......... .......... 9% 29.4M 6s 2024-03-29T04:11:47.375 INFO:teuthology.orchestra.run.smithi148.stderr: 16300K .......... .......... .......... .......... .......... 9% 31.0M 6s 2024-03-29T04:11:47.377 INFO:teuthology.orchestra.run.smithi148.stderr: 16350K .......... .......... .......... .......... .......... 9% 27.5M 6s 2024-03-29T04:11:47.379 INFO:teuthology.orchestra.run.smithi148.stderr: 16400K .......... .......... .......... .......... .......... 9% 27.4M 6s 2024-03-29T04:11:47.381 INFO:teuthology.orchestra.run.smithi148.stderr: 16450K .......... .......... .......... .......... .......... 9% 24.8M 6s 2024-03-29T04:11:47.382 INFO:teuthology.orchestra.run.smithi148.stderr: 16500K .......... .......... .......... .......... .......... 9% 27.2M 6s 2024-03-29T04:11:47.384 INFO:teuthology.orchestra.run.smithi148.stderr: 16550K .......... .......... .......... .......... .......... 9% 33.2M 6s 2024-03-29T04:11:47.386 INFO:teuthology.orchestra.run.smithi148.stderr: 16600K .......... .......... .......... .......... .......... 9% 28.9M 6s 2024-03-29T04:11:47.387 INFO:teuthology.orchestra.run.smithi148.stderr: 16650K .......... .......... .......... .......... .......... 9% 32.1M 6s 2024-03-29T04:11:47.389 INFO:teuthology.orchestra.run.smithi148.stderr: 16700K .......... .......... .......... .......... .......... 9% 31.9M 6s 2024-03-29T04:11:47.390 INFO:teuthology.orchestra.run.smithi148.stderr: 16750K .......... .......... .......... .......... .......... 9% 32.0M 6s 2024-03-29T04:11:47.392 INFO:teuthology.orchestra.run.smithi148.stderr: 16800K .......... .......... .......... .......... .......... 9% 27.2M 6s 2024-03-29T04:11:47.394 INFO:teuthology.orchestra.run.smithi148.stderr: 16850K .......... .......... .......... .......... .......... 10% 30.9M 6s 2024-03-29T04:11:47.395 INFO:teuthology.orchestra.run.smithi148.stderr: 16900K .......... .......... .......... .......... .......... 10% 32.8M 6s 2024-03-29T04:11:47.397 INFO:teuthology.orchestra.run.smithi148.stderr: 16950K .......... .......... .......... .......... .......... 10% 31.6M 6s 2024-03-29T04:11:47.398 INFO:teuthology.orchestra.run.smithi148.stderr: 17000K .......... .......... .......... .......... .......... 10% 26.9M 6s 2024-03-29T04:11:47.400 INFO:teuthology.orchestra.run.smithi148.stderr: 17050K .......... .......... .......... .......... .......... 10% 30.5M 6s 2024-03-29T04:11:47.402 INFO:teuthology.orchestra.run.smithi148.stderr: 17100K .......... .......... .......... .......... .......... 10% 30.6M 6s 2024-03-29T04:11:47.403 INFO:teuthology.orchestra.run.smithi148.stderr: 17150K .......... .......... .......... .......... .......... 10% 30.8M 6s 2024-03-29T04:11:47.405 INFO:teuthology.orchestra.run.smithi148.stderr: 17200K .......... .......... .......... .......... .......... 10% 26.0M 6s 2024-03-29T04:11:47.407 INFO:teuthology.orchestra.run.smithi148.stderr: 17250K .......... .......... .......... .......... .......... 10% 30.8M 6s 2024-03-29T04:11:47.408 INFO:teuthology.orchestra.run.smithi148.stderr: 17300K .......... .......... .......... .......... .......... 10% 32.8M 6s 2024-03-29T04:11:47.410 INFO:teuthology.orchestra.run.smithi148.stderr: 17350K .......... .......... .......... .......... .......... 10% 32.0M 6s 2024-03-29T04:11:47.412 INFO:teuthology.orchestra.run.smithi148.stderr: 17400K .......... .......... .......... .......... .......... 10% 27.9M 6s 2024-03-29T04:11:47.413 INFO:teuthology.orchestra.run.smithi148.stderr: 17450K .......... .......... .......... .......... .......... 10% 32.2M 6s 2024-03-29T04:11:47.415 INFO:teuthology.orchestra.run.smithi148.stderr: 17500K .......... .......... .......... .......... .......... 10% 32.6M 6s 2024-03-29T04:11:47.416 INFO:teuthology.orchestra.run.smithi148.stderr: 17550K .......... .......... .......... .......... .......... 10% 32.2M 6s 2024-03-29T04:11:47.418 INFO:teuthology.orchestra.run.smithi148.stderr: 17600K .......... .......... .......... .......... .......... 10% 27.5M 6s 2024-03-29T04:11:47.419 INFO:teuthology.orchestra.run.smithi148.stderr: 17650K .......... .......... .......... .......... .......... 10% 31.9M 6s 2024-03-29T04:11:47.421 INFO:teuthology.orchestra.run.smithi148.stderr: 17700K .......... .......... .......... .......... .......... 10% 31.4M 6s 2024-03-29T04:11:47.423 INFO:teuthology.orchestra.run.smithi148.stderr: 17750K .......... .......... .......... .......... .......... 10% 31.5M 6s 2024-03-29T04:11:47.424 INFO:teuthology.orchestra.run.smithi148.stderr: 17800K .......... .......... .......... .......... .......... 10% 31.3M 6s 2024-03-29T04:11:47.426 INFO:teuthology.orchestra.run.smithi148.stderr: 17850K .......... .......... .......... .......... .......... 10% 30.4M 6s 2024-03-29T04:11:47.428 INFO:teuthology.orchestra.run.smithi148.stderr: 17900K .......... .......... .......... .......... .......... 10% 27.6M 6s 2024-03-29T04:11:47.429 INFO:teuthology.orchestra.run.smithi148.stderr: 17950K .......... .......... .......... .......... .......... 10% 35.9M 6s 2024-03-29T04:11:47.431 INFO:teuthology.orchestra.run.smithi148.stderr: 18000K .......... .......... .......... .......... .......... 10% 26.3M 6s 2024-03-29T04:11:47.433 INFO:teuthology.orchestra.run.smithi148.stderr: 18050K .......... .......... .......... .......... .......... 10% 20.4M 6s 2024-03-29T04:11:47.434 INFO:teuthology.orchestra.run.smithi148.stderr: 18100K .......... .......... .......... .......... .......... 10% 44.4M 6s 2024-03-29T04:11:47.435 INFO:teuthology.orchestra.run.smithi148.stderr: 18150K .......... .......... .......... .......... .......... 10% 44.2M 6s 2024-03-29T04:11:47.437 INFO:teuthology.orchestra.run.smithi148.stderr: 18200K .......... .......... .......... .......... .......... 10% 27.6M 6s 2024-03-29T04:11:47.439 INFO:teuthology.orchestra.run.smithi148.stderr: 18250K .......... .......... .......... .......... .......... 10% 29.4M 6s 2024-03-29T04:11:47.440 INFO:teuthology.orchestra.run.smithi148.stderr: 18300K .......... .......... .......... .......... .......... 10% 27.8M 6s 2024-03-29T04:11:47.442 INFO:teuthology.orchestra.run.smithi148.stderr: 18350K .......... .......... .......... .......... .......... 10% 29.5M 6s 2024-03-29T04:11:47.444 INFO:teuthology.orchestra.run.smithi148.stderr: 18400K .......... .......... .......... .......... .......... 10% 28.5M 6s 2024-03-29T04:11:47.445 INFO:teuthology.orchestra.run.smithi148.stderr: 18450K .......... .......... .......... .......... .......... 10% 32.5M 6s 2024-03-29T04:11:47.447 INFO:teuthology.orchestra.run.smithi148.stderr: 18500K .......... .......... .......... .......... .......... 11% 26.5M 6s 2024-03-29T04:11:47.449 INFO:teuthology.orchestra.run.smithi148.stderr: 18550K .......... .......... .......... .......... .......... 11% 27.4M 6s 2024-03-29T04:11:47.451 INFO:teuthology.orchestra.run.smithi148.stderr: 18600K .......... .......... .......... .......... .......... 11% 26.6M 6s 2024-03-29T04:11:47.452 INFO:teuthology.orchestra.run.smithi148.stderr: 18650K .......... .......... .......... .......... .......... 11% 28.7M 6s 2024-03-29T04:11:47.453 INFO:teuthology.orchestra.run.smithi148.stderr: 18700K .......... .......... .......... .......... .......... 11% 39.9M 6s 2024-03-29T04:11:47.456 INFO:teuthology.orchestra.run.smithi148.stderr: 18750K .......... .......... .......... .......... .......... 11% 25.8M 6s 2024-03-29T04:11:47.458 INFO:teuthology.orchestra.run.smithi148.stderr: 18800K .......... .......... .......... .......... .......... 11% 27.1M 6s 2024-03-29T04:11:47.459 INFO:teuthology.orchestra.run.smithi148.stderr: 18850K .......... .......... .......... .......... .......... 11% 26.5M 6s 2024-03-29T04:11:47.461 INFO:teuthology.orchestra.run.smithi148.stderr: 18900K .......... .......... .......... .......... .......... 11% 33.0M 6s 2024-03-29T04:11:47.463 INFO:teuthology.orchestra.run.smithi148.stderr: 18950K .......... .......... .......... .......... .......... 11% 28.5M 6s 2024-03-29T04:11:47.464 INFO:teuthology.orchestra.run.smithi148.stderr: 19000K .......... .......... .......... .......... .......... 11% 29.6M 6s 2024-03-29T04:11:47.466 INFO:teuthology.orchestra.run.smithi148.stderr: 19050K .......... .......... .......... .......... .......... 11% 27.1M 6s 2024-03-29T04:11:47.467 INFO:teuthology.orchestra.run.smithi148.stderr: 19100K .......... .......... .......... .......... .......... 11% 28.3M 6s 2024-03-29T04:11:47.469 INFO:teuthology.orchestra.run.smithi148.stderr: 19150K .......... .......... .......... .......... .......... 11% 29.0M 6s 2024-03-29T04:11:47.471 INFO:teuthology.orchestra.run.smithi148.stderr: 19200K .......... .......... .......... .......... .......... 11% 33.4M 6s 2024-03-29T04:11:47.472 INFO:teuthology.orchestra.run.smithi148.stderr: 19250K .......... .......... .......... .......... .......... 11% 28.2M 6s 2024-03-29T04:11:47.474 INFO:teuthology.orchestra.run.smithi148.stderr: 19300K .......... .......... .......... .......... .......... 11% 27.4M 6s 2024-03-29T04:11:47.476 INFO:teuthology.orchestra.run.smithi148.stderr: 19350K .......... .......... .......... .......... .......... 11% 28.1M 6s 2024-03-29T04:11:47.478 INFO:teuthology.orchestra.run.smithi148.stderr: 19400K .......... .......... .......... .......... .......... 11% 27.2M 6s 2024-03-29T04:11:47.479 INFO:teuthology.orchestra.run.smithi148.stderr: 19450K .......... .......... .......... .......... .......... 11% 36.3M 6s 2024-03-29T04:11:47.481 INFO:teuthology.orchestra.run.smithi148.stderr: 19500K .......... .......... .......... .......... .......... 11% 26.4M 6s 2024-03-29T04:11:47.483 INFO:teuthology.orchestra.run.smithi148.stderr: 19550K .......... .......... .......... .......... .......... 11% 26.8M 6s 2024-03-29T04:11:47.485 INFO:teuthology.orchestra.run.smithi148.stderr: 19600K .......... .......... .......... .......... .......... 11% 27.8M 6s 2024-03-29T04:11:47.486 INFO:teuthology.orchestra.run.smithi148.stderr: 19650K .......... .......... .......... .......... .......... 11% 32.5M 6s 2024-03-29T04:11:47.488 INFO:teuthology.orchestra.run.smithi148.stderr: 19700K .......... .......... .......... .......... .......... 11% 28.5M 6s 2024-03-29T04:11:47.489 INFO:teuthology.orchestra.run.smithi148.stderr: 19750K .......... .......... .......... .......... .......... 11% 30.8M 6s 2024-03-29T04:11:47.491 INFO:teuthology.orchestra.run.smithi148.stderr: 19800K .......... .......... .......... .......... .......... 11% 27.6M 6s 2024-03-29T04:11:47.493 INFO:teuthology.orchestra.run.smithi148.stderr: 19850K .......... .......... .......... .......... .......... 11% 30.5M 6s 2024-03-29T04:11:47.495 INFO:teuthology.orchestra.run.smithi148.stderr: 19900K .......... .......... .......... .......... .......... 11% 25.8M 6s 2024-03-29T04:11:47.496 INFO:teuthology.orchestra.run.smithi148.stderr: 19950K .......... .......... .......... .......... .......... 11% 29.8M 6s 2024-03-29T04:11:47.498 INFO:teuthology.orchestra.run.smithi148.stderr: 20000K .......... .......... .......... .......... .......... 11% 35.0M 6s 2024-03-29T04:11:47.499 INFO:teuthology.orchestra.run.smithi148.stderr: 20050K .......... .......... .......... .......... .......... 11% 26.6M 6s 2024-03-29T04:11:47.501 INFO:teuthology.orchestra.run.smithi148.stderr: 20100K .......... .......... .......... .......... .......... 11% 26.6M 6s 2024-03-29T04:11:47.503 INFO:teuthology.orchestra.run.smithi148.stderr: 20150K .......... .......... .......... .......... .......... 11% 29.0M 6s 2024-03-29T04:11:47.504 INFO:teuthology.orchestra.run.smithi148.stderr: 20200K .......... .......... .......... .......... .......... 12% 34.6M 6s 2024-03-29T04:11:47.507 INFO:teuthology.orchestra.run.smithi148.stderr: 20250K .......... .......... .......... .......... .......... 12% 27.4M 6s 2024-03-29T04:11:47.508 INFO:teuthology.orchestra.run.smithi148.stderr: 20300K .......... .......... .......... .......... .......... 12% 29.7M 6s 2024-03-29T04:11:47.510 INFO:teuthology.orchestra.run.smithi148.stderr: 20350K .......... .......... .......... .......... .......... 12% 28.1M 6s 2024-03-29T04:11:47.511 INFO:teuthology.orchestra.run.smithi148.stderr: 20400K .......... .......... .......... .......... .......... 12% 29.1M 6s 2024-03-29T04:11:47.513 INFO:teuthology.orchestra.run.smithi148.stderr: 20450K .......... .......... .......... .......... .......... 12% 24.2M 6s 2024-03-29T04:11:47.515 INFO:teuthology.orchestra.run.smithi148.stderr: 20500K .......... .......... .......... .......... .......... 12% 31.1M 6s 2024-03-29T04:11:47.517 INFO:teuthology.orchestra.run.smithi148.stderr: 20550K .......... .......... .......... .......... .......... 12% 28.1M 6s 2024-03-29T04:11:47.520 INFO:teuthology.orchestra.run.smithi148.stderr: 20600K .......... .......... .......... .......... .......... 12% 23.6M 6s 2024-03-29T04:11:47.523 INFO:teuthology.orchestra.run.smithi148.stderr: 20650K .......... .......... .......... .......... .......... 12% 14.9M 6s 2024-03-29T04:11:47.604 INFO:teuthology.orchestra.run.smithi148.stderr: 20700K .......... .......... .......... .......... .......... 12% 609K 6s 2024-03-29T04:11:47.606 INFO:teuthology.orchestra.run.smithi148.stderr: 20750K .......... .......... .......... .......... .......... 12% 32.7M 6s 2024-03-29T04:11:47.607 INFO:teuthology.orchestra.run.smithi148.stderr: 20800K .......... .......... .......... .......... .......... 12% 31.2M 6s 2024-03-29T04:11:47.609 INFO:teuthology.orchestra.run.smithi148.stderr: 20850K .......... .......... .......... .......... .......... 12% 26.3M 6s 2024-03-29T04:11:47.611 INFO:teuthology.orchestra.run.smithi148.stderr: 20900K .......... .......... .......... .......... .......... 12% 31.0M 6s 2024-03-29T04:11:47.612 INFO:teuthology.orchestra.run.smithi148.stderr: 20950K .......... .......... .......... .......... .......... 12% 30.6M 6s 2024-03-29T04:11:47.614 INFO:teuthology.orchestra.run.smithi148.stderr: 21000K .......... .......... .......... .......... .......... 12% 31.6M 6s 2024-03-29T04:11:47.615 INFO:teuthology.orchestra.run.smithi148.stderr: 21050K .......... .......... .......... .......... .......... 12% 26.7M 6s 2024-03-29T04:11:47.617 INFO:teuthology.orchestra.run.smithi148.stderr: 21100K .......... .......... .......... .......... .......... 12% 30.6M 6s 2024-03-29T04:11:47.619 INFO:teuthology.orchestra.run.smithi148.stderr: 21150K .......... .......... .......... .......... .......... 12% 31.0M 6s 2024-03-29T04:11:47.620 INFO:teuthology.orchestra.run.smithi148.stderr: 21200K .......... .......... .......... .......... .......... 12% 31.0M 6s 2024-03-29T04:11:47.622 INFO:teuthology.orchestra.run.smithi148.stderr: 21250K .......... .......... .......... .......... .......... 12% 26.7M 6s 2024-03-29T04:11:47.623 INFO:teuthology.orchestra.run.smithi148.stderr: 21300K .......... .......... .......... .......... .......... 12% 30.9M 6s 2024-03-29T04:11:47.625 INFO:teuthology.orchestra.run.smithi148.stderr: 21350K .......... .......... .......... .......... .......... 12% 27.0M 6s 2024-03-29T04:11:47.628 INFO:teuthology.orchestra.run.smithi148.stderr: 21400K .......... .......... .......... .......... .......... 12% 24.3M 6s 2024-03-29T04:11:47.629 INFO:teuthology.orchestra.run.smithi148.stderr: 21450K .......... .......... .......... .......... .......... 12% 25.6M 6s 2024-03-29T04:11:47.631 INFO:teuthology.orchestra.run.smithi148.stderr: 21500K .......... .......... .......... .......... .......... 12% 33.8M 6s 2024-03-29T04:11:47.632 INFO:teuthology.orchestra.run.smithi148.stderr: 21550K .......... .......... .......... .......... .......... 12% 33.4M 6s 2024-03-29T04:11:47.634 INFO:teuthology.orchestra.run.smithi148.stderr: 21600K .......... .......... .......... .......... .......... 12% 32.6M 6s 2024-03-29T04:11:47.635 INFO:teuthology.orchestra.run.smithi148.stderr: 21650K .......... .......... .......... .......... .......... 12% 28.7M 6s 2024-03-29T04:11:47.637 INFO:teuthology.orchestra.run.smithi148.stderr: 21700K .......... .......... .......... .......... .......... 12% 33.5M 6s 2024-03-29T04:11:47.638 INFO:teuthology.orchestra.run.smithi148.stderr: 21750K .......... .......... .......... .......... .......... 12% 33.6M 6s 2024-03-29T04:11:47.640 INFO:teuthology.orchestra.run.smithi148.stderr: 21800K .......... .......... .......... .......... .......... 12% 31.7M 6s 2024-03-29T04:11:47.642 INFO:teuthology.orchestra.run.smithi148.stderr: 21850K .......... .......... .......... .......... .......... 12% 27.9M 6s 2024-03-29T04:11:47.643 INFO:teuthology.orchestra.run.smithi148.stderr: 21900K .......... .......... .......... .......... .......... 13% 32.8M 6s 2024-03-29T04:11:47.645 INFO:teuthology.orchestra.run.smithi148.stderr: 21950K .......... .......... .......... .......... .......... 13% 33.0M 6s 2024-03-29T04:11:47.646 INFO:teuthology.orchestra.run.smithi148.stderr: 22000K .......... .......... .......... .......... .......... 13% 33.2M 6s 2024-03-29T04:11:47.648 INFO:teuthology.orchestra.run.smithi148.stderr: 22050K .......... .......... .......... .......... .......... 13% 28.2M 6s 2024-03-29T04:11:47.650 INFO:teuthology.orchestra.run.smithi148.stderr: 22100K .......... .......... .......... .......... .......... 13% 25.7M 6s 2024-03-29T04:11:47.652 INFO:teuthology.orchestra.run.smithi148.stderr: 22150K .......... .......... .......... .......... .......... 13% 24.9M 6s 2024-03-29T04:11:47.654 INFO:teuthology.orchestra.run.smithi148.stderr: 22200K .......... .......... .......... .......... .......... 13% 26.1M 6s 2024-03-29T04:11:47.656 INFO:teuthology.orchestra.run.smithi148.stderr: 22250K .......... .......... .......... .......... .......... 13% 21.4M 6s 2024-03-29T04:11:47.658 INFO:teuthology.orchestra.run.smithi148.stderr: 22300K .......... .......... .......... .......... .......... 13% 29.3M 6s 2024-03-29T04:11:47.659 INFO:teuthology.orchestra.run.smithi148.stderr: 22350K .......... .......... .......... .......... .......... 13% 29.8M 6s 2024-03-29T04:11:47.661 INFO:teuthology.orchestra.run.smithi148.stderr: 22400K .......... .......... .......... .......... .......... 13% 29.8M 6s 2024-03-29T04:11:47.663 INFO:teuthology.orchestra.run.smithi148.stderr: 22450K .......... .......... .......... .......... .......... 13% 24.9M 6s 2024-03-29T04:11:47.664 INFO:teuthology.orchestra.run.smithi148.stderr: 22500K .......... .......... .......... .......... .......... 13% 30.0M 6s 2024-03-29T04:11:47.666 INFO:teuthology.orchestra.run.smithi148.stderr: 22550K .......... .......... .......... .......... .......... 13% 28.9M 6s 2024-03-29T04:11:47.668 INFO:teuthology.orchestra.run.smithi148.stderr: 22600K .......... .......... .......... .......... .......... 13% 28.5M 6s 2024-03-29T04:11:47.670 INFO:teuthology.orchestra.run.smithi148.stderr: 22650K .......... .......... .......... .......... .......... 13% 24.9M 6s 2024-03-29T04:11:47.672 INFO:teuthology.orchestra.run.smithi148.stderr: 22700K .......... .......... .......... .......... .......... 13% 29.8M 6s 2024-03-29T04:11:47.673 INFO:teuthology.orchestra.run.smithi148.stderr: 22750K .......... .......... .......... .......... .......... 13% 27.7M 6s 2024-03-29T04:11:47.675 INFO:teuthology.orchestra.run.smithi148.stderr: 22800K .......... .......... .......... .......... .......... 13% 26.1M 6s 2024-03-29T04:11:47.677 INFO:teuthology.orchestra.run.smithi148.stderr: 22850K .......... .......... .......... .......... .......... 13% 21.9M 6s 2024-03-29T04:11:47.679 INFO:teuthology.orchestra.run.smithi148.stderr: 22900K .......... .......... .......... .......... .......... 13% 25.8M 6s 2024-03-29T04:11:47.681 INFO:teuthology.orchestra.run.smithi148.stderr: 22950K .......... .......... .......... .......... .......... 13% 25.0M 6s 2024-03-29T04:11:47.683 INFO:teuthology.orchestra.run.smithi148.stderr: 23000K .......... .......... .......... .......... .......... 13% 26.3M 6s 2024-03-29T04:11:47.685 INFO:teuthology.orchestra.run.smithi148.stderr: 23050K .......... .......... .......... .......... .......... 13% 22.1M 6s 2024-03-29T04:11:47.687 INFO:teuthology.orchestra.run.smithi148.stderr: 23100K .......... .......... .......... .......... .......... 13% 25.9M 6s 2024-03-29T04:11:47.689 INFO:teuthology.orchestra.run.smithi148.stderr: 23150K .......... .......... .......... .......... .......... 13% 25.7M 6s 2024-03-29T04:11:47.691 INFO:teuthology.orchestra.run.smithi148.stderr: 23200K .......... .......... .......... .......... .......... 13% 25.7M 6s 2024-03-29T04:11:47.693 INFO:teuthology.orchestra.run.smithi148.stderr: 23250K .......... .......... .......... .......... .......... 13% 21.3M 6s 2024-03-29T04:11:47.695 INFO:teuthology.orchestra.run.smithi148.stderr: 23300K .......... .......... .......... .......... .......... 13% 26.3M 6s 2024-03-29T04:11:47.697 INFO:teuthology.orchestra.run.smithi148.stderr: 23350K .......... .......... .......... .......... .......... 13% 25.9M 6s 2024-03-29T04:11:47.699 INFO:teuthology.orchestra.run.smithi148.stderr: 23400K .......... .......... .......... .......... .......... 13% 26.2M 6s 2024-03-29T04:11:47.701 INFO:teuthology.orchestra.run.smithi148.stderr: 23450K .......... .......... .......... .......... .......... 13% 22.1M 6s 2024-03-29T04:11:47.703 INFO:teuthology.orchestra.run.smithi148.stderr: 23500K .......... .......... .......... .......... .......... 13% 25.6M 6s 2024-03-29T04:11:47.705 INFO:teuthology.orchestra.run.smithi148.stderr: 23550K .......... .......... .......... .......... .......... 14% 25.9M 6s 2024-03-29T04:11:47.707 INFO:teuthology.orchestra.run.smithi148.stderr: 23600K .......... .......... .......... .......... .......... 14% 25.6M 6s 2024-03-29T04:11:47.709 INFO:teuthology.orchestra.run.smithi148.stderr: 23650K .......... .......... .......... .......... .......... 14% 20.3M 6s 2024-03-29T04:11:47.711 INFO:teuthology.orchestra.run.smithi148.stderr: 23700K .......... .......... .......... .......... .......... 14% 21.5M 6s 2024-03-29T04:11:47.713 INFO:teuthology.orchestra.run.smithi148.stderr: 23750K .......... .......... .......... .......... .......... 14% 23.0M 6s 2024-03-29T04:11:47.716 INFO:teuthology.orchestra.run.smithi148.stderr: 23800K .......... .......... .......... .......... .......... 14% 22.0M 6s 2024-03-29T04:11:47.718 INFO:teuthology.orchestra.run.smithi148.stderr: 23850K .......... .......... .......... .......... .......... 14% 19.2M 6s 2024-03-29T04:11:47.720 INFO:teuthology.orchestra.run.smithi148.stderr: 23900K .......... .......... .......... .......... .......... 14% 22.9M 6s 2024-03-29T04:11:47.722 INFO:teuthology.orchestra.run.smithi148.stderr: 23950K .......... .......... .......... .......... .......... 14% 22.4M 6s 2024-03-29T04:11:47.724 INFO:teuthology.orchestra.run.smithi148.stderr: 24000K .......... .......... .......... .......... .......... 14% 29.0M 6s 2024-03-29T04:11:47.726 INFO:teuthology.orchestra.run.smithi148.stderr: 24050K .......... .......... .......... .......... .......... 14% 24.3M 6s 2024-03-29T04:11:47.731 INFO:teuthology.orchestra.run.smithi148.stderr: 24100K .......... .......... .......... .......... .......... 14% 28.7M 6s 2024-03-29T04:11:47.731 INFO:teuthology.orchestra.run.smithi148.stderr: 24150K .......... .......... .......... .......... .......... 14% 28.2M 6s 2024-03-29T04:11:47.732 INFO:teuthology.orchestra.run.smithi148.stderr: 24200K .......... .......... .......... .......... .......... 14% 22.5M 6s 2024-03-29T04:11:47.734 INFO:teuthology.orchestra.run.smithi148.stderr: 24250K .......... .......... .......... .......... .......... 14% 18.8M 6s 2024-03-29T04:11:47.737 INFO:teuthology.orchestra.run.smithi148.stderr: 24300K .......... .......... .......... .......... .......... 14% 22.1M 6s 2024-03-29T04:11:47.739 INFO:teuthology.orchestra.run.smithi148.stderr: 24350K .......... .......... .......... .......... .......... 14% 22.3M 6s 2024-03-29T04:11:47.741 INFO:teuthology.orchestra.run.smithi148.stderr: 24400K .......... .......... .......... .......... .......... 14% 21.8M 6s 2024-03-29T04:11:47.744 INFO:teuthology.orchestra.run.smithi148.stderr: 24450K .......... .......... .......... .......... .......... 14% 18.3M 6s 2024-03-29T04:11:47.746 INFO:teuthology.orchestra.run.smithi148.stderr: 24500K .......... .......... .......... .......... .......... 14% 21.9M 6s 2024-03-29T04:11:47.748 INFO:teuthology.orchestra.run.smithi148.stderr: 24550K .......... .......... .......... .......... .......... 14% 21.8M 6s 2024-03-29T04:11:47.750 INFO:teuthology.orchestra.run.smithi148.stderr: 24600K .......... .......... .......... .......... .......... 14% 28.2M 6s 2024-03-29T04:11:47.752 INFO:teuthology.orchestra.run.smithi148.stderr: 24650K .......... .......... .......... .......... .......... 14% 20.7M 6s 2024-03-29T04:11:47.754 INFO:teuthology.orchestra.run.smithi148.stderr: 24700K .......... .......... .......... .......... .......... 14% 32.4M 6s 2024-03-29T04:11:47.755 INFO:teuthology.orchestra.run.smithi148.stderr: 24750K .......... .......... .......... .......... .......... 14% 34.2M 6s 2024-03-29T04:11:47.757 INFO:teuthology.orchestra.run.smithi148.stderr: 24800K .......... .......... .......... .......... .......... 14% 33.0M 6s 2024-03-29T04:11:47.758 INFO:teuthology.orchestra.run.smithi148.stderr: 24850K .......... .......... .......... .......... .......... 14% 29.6M 6s 2024-03-29T04:11:47.760 INFO:teuthology.orchestra.run.smithi148.stderr: 24900K .......... .......... .......... .......... .......... 14% 33.4M 6s 2024-03-29T04:11:47.761 INFO:teuthology.orchestra.run.smithi148.stderr: 24950K .......... .......... .......... .......... .......... 14% 33.7M 6s 2024-03-29T04:11:47.763 INFO:teuthology.orchestra.run.smithi148.stderr: 25000K .......... .......... .......... .......... .......... 14% 32.5M 6s 2024-03-29T04:11:47.764 INFO:teuthology.orchestra.run.smithi148.stderr: 25050K .......... .......... .......... .......... .......... 14% 28.7M 6s 2024-03-29T04:11:47.766 INFO:teuthology.orchestra.run.smithi148.stderr: 25100K .......... .......... .......... .......... .......... 14% 32.0M 6s 2024-03-29T04:11:47.768 INFO:teuthology.orchestra.run.smithi148.stderr: 25150K .......... .......... .......... .......... .......... 14% 31.2M 6s 2024-03-29T04:11:47.769 INFO:teuthology.orchestra.run.smithi148.stderr: 25200K .......... .......... .......... .......... .......... 14% 32.6M 6s 2024-03-29T04:11:47.771 INFO:teuthology.orchestra.run.smithi148.stderr: 25250K .......... .......... .......... .......... .......... 15% 27.7M 6s 2024-03-29T04:11:47.772 INFO:teuthology.orchestra.run.smithi148.stderr: 25300K .......... .......... .......... .......... .......... 15% 31.4M 6s 2024-03-29T04:11:47.774 INFO:teuthology.orchestra.run.smithi148.stderr: 25350K .......... .......... .......... .......... .......... 15% 31.7M 6s 2024-03-29T04:11:47.776 INFO:teuthology.orchestra.run.smithi148.stderr: 25400K .......... .......... .......... .......... .......... 15% 31.7M 6s 2024-03-29T04:11:47.777 INFO:teuthology.orchestra.run.smithi148.stderr: 25450K .......... .......... .......... .......... .......... 15% 27.6M 6s 2024-03-29T04:11:47.779 INFO:teuthology.orchestra.run.smithi148.stderr: 25500K .......... .......... .......... .......... .......... 15% 32.1M 6s 2024-03-29T04:11:47.780 INFO:teuthology.orchestra.run.smithi148.stderr: 25550K .......... .......... .......... .......... .......... 15% 30.8M 6s 2024-03-29T04:11:47.782 INFO:teuthology.orchestra.run.smithi148.stderr: 25600K .......... .......... .......... .......... .......... 15% 33.9M 6s 2024-03-29T04:11:47.783 INFO:teuthology.orchestra.run.smithi148.stderr: 25650K .......... .......... .......... .......... .......... 15% 31.3M 6s 2024-03-29T04:11:47.785 INFO:teuthology.orchestra.run.smithi148.stderr: 25700K .......... .......... .......... .......... .......... 15% 37.2M 6s 2024-03-29T04:11:47.786 INFO:teuthology.orchestra.run.smithi148.stderr: 25750K .......... .......... .......... .......... .......... 15% 37.2M 6s 2024-03-29T04:11:47.787 INFO:teuthology.orchestra.run.smithi148.stderr: 25800K .......... .......... .......... .......... .......... 15% 37.1M 6s 2024-03-29T04:11:47.789 INFO:teuthology.orchestra.run.smithi148.stderr: 25850K .......... .......... .......... .......... .......... 15% 32.0M 6s 2024-03-29T04:11:47.790 INFO:teuthology.orchestra.run.smithi148.stderr: 25900K .......... .......... .......... .......... .......... 15% 35.7M 6s 2024-03-29T04:11:47.792 INFO:teuthology.orchestra.run.smithi148.stderr: 25950K .......... .......... .......... .......... .......... 15% 36.8M 6s 2024-03-29T04:11:47.793 INFO:teuthology.orchestra.run.smithi148.stderr: 26000K .......... .......... .......... .......... .......... 15% 36.0M 6s 2024-03-29T04:11:47.795 INFO:teuthology.orchestra.run.smithi148.stderr: 26050K .......... .......... .......... .......... .......... 15% 24.9M 6s 2024-03-29T04:11:47.797 INFO:teuthology.orchestra.run.smithi148.stderr: 26100K .......... .......... .......... .......... .......... 15% 26.7M 6s 2024-03-29T04:11:47.798 INFO:teuthology.orchestra.run.smithi148.stderr: 26150K .......... .......... .......... .......... .......... 15% 26.6M 6s 2024-03-29T04:11:47.801 INFO:teuthology.orchestra.run.smithi148.stderr: 26200K .......... .......... .......... .......... .......... 15% 26.2M 6s 2024-03-29T04:11:47.803 INFO:teuthology.orchestra.run.smithi148.stderr: 26250K .......... .......... .......... .......... .......... 15% 21.2M 6s 2024-03-29T04:11:47.805 INFO:teuthology.orchestra.run.smithi148.stderr: 26300K .......... .......... .......... .......... .......... 15% 26.1M 6s 2024-03-29T04:11:47.806 INFO:teuthology.orchestra.run.smithi148.stderr: 26350K .......... .......... .......... .......... .......... 15% 25.0M 6s 2024-03-29T04:11:47.809 INFO:teuthology.orchestra.run.smithi148.stderr: 26400K .......... .......... .......... .......... .......... 15% 25.1M 6s 2024-03-29T04:11:47.811 INFO:teuthology.orchestra.run.smithi148.stderr: 26450K .......... .......... .......... .......... .......... 15% 21.7M 6s 2024-03-29T04:11:47.813 INFO:teuthology.orchestra.run.smithi148.stderr: 26500K .......... .......... .......... .......... .......... 15% 25.2M 6s 2024-03-29T04:11:47.815 INFO:teuthology.orchestra.run.smithi148.stderr: 26550K .......... .......... .......... .......... .......... 15% 25.3M 6s 2024-03-29T04:11:47.817 INFO:teuthology.orchestra.run.smithi148.stderr: 26600K .......... .......... .......... .......... .......... 15% 24.7M 6s 2024-03-29T04:11:47.819 INFO:teuthology.orchestra.run.smithi148.stderr: 26650K .......... .......... .......... .......... .......... 15% 21.7M 6s 2024-03-29T04:11:47.821 INFO:teuthology.orchestra.run.smithi148.stderr: 26700K .......... .......... .......... .......... .......... 15% 24.4M 6s 2024-03-29T04:11:47.823 INFO:teuthology.orchestra.run.smithi148.stderr: 26750K .......... .......... .......... .......... .......... 15% 24.6M 6s 2024-03-29T04:11:47.825 INFO:teuthology.orchestra.run.smithi148.stderr: 26800K .......... .......... .......... .......... .......... 15% 23.2M 6s 2024-03-29T04:11:47.827 INFO:teuthology.orchestra.run.smithi148.stderr: 26850K .......... .......... .......... .......... .......... 15% 20.7M 6s 2024-03-29T04:11:47.829 INFO:teuthology.orchestra.run.smithi148.stderr: 26900K .......... .......... .......... .......... .......... 15% 24.9M 6s 2024-03-29T04:11:47.831 INFO:teuthology.orchestra.run.smithi148.stderr: 26950K .......... .......... .......... .......... .......... 16% 24.8M 6s 2024-03-29T04:11:47.833 INFO:teuthology.orchestra.run.smithi148.stderr: 27000K .......... .......... .......... .......... .......... 16% 27.9M 6s 2024-03-29T04:11:47.835 INFO:teuthology.orchestra.run.smithi148.stderr: 27050K .......... .......... .......... .......... .......... 16% 27.6M 6s 2024-03-29T04:11:47.836 INFO:teuthology.orchestra.run.smithi148.stderr: 27100K .......... .......... .......... .......... .......... 16% 31.7M 6s 2024-03-29T04:11:47.838 INFO:teuthology.orchestra.run.smithi148.stderr: 27150K .......... .......... .......... .......... .......... 16% 32.1M 6s 2024-03-29T04:11:47.839 INFO:teuthology.orchestra.run.smithi148.stderr: 27200K .......... .......... .......... .......... .......... 16% 32.1M 6s 2024-03-29T04:11:47.841 INFO:teuthology.orchestra.run.smithi148.stderr: 27250K .......... .......... .......... .......... .......... 16% 27.0M 6s 2024-03-29T04:11:47.843 INFO:teuthology.orchestra.run.smithi148.stderr: 27300K .......... .......... .......... .......... .......... 16% 30.7M 6s 2024-03-29T04:11:47.844 INFO:teuthology.orchestra.run.smithi148.stderr: 27350K .......... .......... .......... .......... .......... 16% 31.8M 6s 2024-03-29T04:11:47.846 INFO:teuthology.orchestra.run.smithi148.stderr: 27400K .......... .......... .......... .......... .......... 16% 32.3M 6s 2024-03-29T04:11:47.847 INFO:teuthology.orchestra.run.smithi148.stderr: 27450K .......... .......... .......... .......... .......... 16% 27.3M 6s 2024-03-29T04:11:47.849 INFO:teuthology.orchestra.run.smithi148.stderr: 27500K .......... .......... .......... .......... .......... 16% 31.4M 6s 2024-03-29T04:11:47.851 INFO:teuthology.orchestra.run.smithi148.stderr: 27550K .......... .......... .......... .......... .......... 16% 31.2M 6s 2024-03-29T04:11:47.852 INFO:teuthology.orchestra.run.smithi148.stderr: 27600K .......... .......... .......... .......... .......... 16% 32.3M 6s 2024-03-29T04:11:47.854 INFO:teuthology.orchestra.run.smithi148.stderr: 27650K .......... .......... .......... .......... .......... 16% 27.0M 6s 2024-03-29T04:11:47.855 INFO:teuthology.orchestra.run.smithi148.stderr: 27700K .......... .......... .......... .......... .......... 16% 30.9M 6s 2024-03-29T04:11:47.857 INFO:teuthology.orchestra.run.smithi148.stderr: 27750K .......... .......... .......... .......... .......... 16% 30.8M 6s 2024-03-29T04:11:47.859 INFO:teuthology.orchestra.run.smithi148.stderr: 27800K .......... .......... .......... .......... .......... 16% 30.2M 6s 2024-03-29T04:11:47.860 INFO:teuthology.orchestra.run.smithi148.stderr: 27850K .......... .......... .......... .......... .......... 16% 26.7M 6s 2024-03-29T04:11:47.862 INFO:teuthology.orchestra.run.smithi148.stderr: 27900K .......... .......... .......... .......... .......... 16% 31.5M 6s 2024-03-29T04:11:47.864 INFO:teuthology.orchestra.run.smithi148.stderr: 27950K .......... .......... .......... .......... .......... 16% 30.8M 6s 2024-03-29T04:11:47.865 INFO:teuthology.orchestra.run.smithi148.stderr: 28000K .......... .......... .......... .......... .......... 16% 31.6M 6s 2024-03-29T04:11:47.867 INFO:teuthology.orchestra.run.smithi148.stderr: 28050K .......... .......... .......... .......... .......... 16% 26.4M 6s 2024-03-29T04:11:47.869 INFO:teuthology.orchestra.run.smithi148.stderr: 28100K .......... .......... .......... .......... .......... 16% 30.5M 6s 2024-03-29T04:11:47.870 INFO:teuthology.orchestra.run.smithi148.stderr: 28150K .......... .......... .......... .......... .......... 16% 30.1M 6s 2024-03-29T04:11:47.872 INFO:teuthology.orchestra.run.smithi148.stderr: 28200K .......... .......... .......... .......... .......... 16% 29.3M 6s 2024-03-29T04:11:47.874 INFO:teuthology.orchestra.run.smithi148.stderr: 28250K .......... .......... .......... .......... .......... 16% 25.7M 6s 2024-03-29T04:11:47.876 INFO:teuthology.orchestra.run.smithi148.stderr: 28300K .......... .......... .......... .......... .......... 16% 28.7M 6s 2024-03-29T04:11:47.879 INFO:teuthology.orchestra.run.smithi148.stderr: 28350K .......... .......... .......... .......... .......... 16% 29.2M 6s 2024-03-29T04:11:47.880 INFO:teuthology.orchestra.run.smithi148.stderr: 28400K .......... .......... .......... .......... .......... 16% 28.7M 6s 2024-03-29T04:11:47.881 INFO:teuthology.orchestra.run.smithi148.stderr: 28450K .......... .......... .......... .......... .......... 16% 23.5M 6s 2024-03-29T04:11:47.883 INFO:teuthology.orchestra.run.smithi148.stderr: 28500K .......... .......... .......... .......... .......... 16% 24.8M 6s 2024-03-29T04:11:47.886 INFO:teuthology.orchestra.run.smithi148.stderr: 28550K .......... .......... .......... .......... .......... 16% 24.4M 6s 2024-03-29T04:11:47.887 INFO:teuthology.orchestra.run.smithi148.stderr: 28600K .......... .......... .......... .......... .......... 16% 25.0M 6s 2024-03-29T04:11:47.890 INFO:teuthology.orchestra.run.smithi148.stderr: 28650K .......... .......... .......... .......... .......... 17% 22.2M 6s 2024-03-29T04:11:47.891 INFO:teuthology.orchestra.run.smithi148.stderr: 28700K .......... .......... .......... .......... .......... 17% 30.1M 6s 2024-03-29T04:11:47.893 INFO:teuthology.orchestra.run.smithi148.stderr: 28750K .......... .......... .......... .......... .......... 17% 24.8M 6s 2024-03-29T04:11:47.895 INFO:teuthology.orchestra.run.smithi148.stderr: 28800K .......... .......... .......... .......... .......... 17% 24.4M 6s 2024-03-29T04:11:47.896 INFO:teuthology.orchestra.run.smithi148.stderr: 28850K .......... .......... .......... .......... .......... 17% 26.6M 6s 2024-03-29T04:11:47.898 INFO:teuthology.orchestra.run.smithi148.stderr: 28900K .......... .......... .......... .......... .......... 17% 40.8M 6s 2024-03-29T04:11:47.899 INFO:teuthology.orchestra.run.smithi148.stderr: 28950K .......... .......... .......... .......... .......... 17% 37.6M 6s 2024-03-29T04:11:47.900 INFO:teuthology.orchestra.run.smithi148.stderr: 29000K .......... .......... .......... .......... .......... 17% 38.5M 6s 2024-03-29T04:11:47.902 INFO:teuthology.orchestra.run.smithi148.stderr: 29050K .......... .......... .......... .......... .......... 17% 31.4M 6s 2024-03-29T04:11:47.904 INFO:teuthology.orchestra.run.smithi148.stderr: 29100K .......... .......... .......... .......... .......... 17% 28.8M 6s 2024-03-29T04:11:47.905 INFO:teuthology.orchestra.run.smithi148.stderr: 29150K .......... .......... .......... .......... .......... 17% 37.4M 6s 2024-03-29T04:11:47.907 INFO:teuthology.orchestra.run.smithi148.stderr: 29200K .......... .......... .......... .......... .......... 17% 37.5M 6s 2024-03-29T04:11:47.908 INFO:teuthology.orchestra.run.smithi148.stderr: 29250K .......... .......... .......... .......... .......... 17% 31.7M 6s 2024-03-29T04:11:47.909 INFO:teuthology.orchestra.run.smithi148.stderr: 29300K .......... .......... .......... .......... .......... 17% 37.5M 6s 2024-03-29T04:11:47.910 INFO:teuthology.orchestra.run.smithi148.stderr: 29350K .......... .......... .......... .......... .......... 17% 36.6M 6s 2024-03-29T04:11:47.912 INFO:teuthology.orchestra.run.smithi148.stderr: 29400K .......... .......... .......... .......... .......... 17% 35.8M 6s 2024-03-29T04:11:47.913 INFO:teuthology.orchestra.run.smithi148.stderr: 29450K .......... .......... .......... .......... .......... 17% 31.4M 6s 2024-03-29T04:11:47.915 INFO:teuthology.orchestra.run.smithi148.stderr: 29500K .......... .......... .......... .......... .......... 17% 36.2M 6s 2024-03-29T04:11:47.916 INFO:teuthology.orchestra.run.smithi148.stderr: 29550K .......... .......... .......... .......... .......... 17% 37.1M 6s 2024-03-29T04:11:47.917 INFO:teuthology.orchestra.run.smithi148.stderr: 29600K .......... .......... .......... .......... .......... 17% 36.4M 6s 2024-03-29T04:11:47.919 INFO:teuthology.orchestra.run.smithi148.stderr: 29650K .......... .......... .......... .......... .......... 17% 30.8M 6s 2024-03-29T04:11:47.920 INFO:teuthology.orchestra.run.smithi148.stderr: 29700K .......... .......... .......... .......... .......... 17% 36.6M 6s 2024-03-29T04:11:47.921 INFO:teuthology.orchestra.run.smithi148.stderr: 29750K .......... .......... .......... .......... .......... 17% 36.0M 6s 2024-03-29T04:11:47.923 INFO:teuthology.orchestra.run.smithi148.stderr: 29800K .......... .......... .......... .......... .......... 17% 35.5M 6s 2024-03-29T04:11:47.925 INFO:teuthology.orchestra.run.smithi148.stderr: 29850K .......... .......... .......... .......... .......... 17% 31.0M 6s 2024-03-29T04:11:47.926 INFO:teuthology.orchestra.run.smithi148.stderr: 29900K .......... .......... .......... .......... .......... 17% 35.7M 6s 2024-03-29T04:11:47.927 INFO:teuthology.orchestra.run.smithi148.stderr: 29950K .......... .......... .......... .......... .......... 17% 35.1M 6s 2024-03-29T04:11:47.931 INFO:teuthology.orchestra.run.smithi148.stderr: 30000K .......... .......... .......... .......... .......... 17% 35.1M 6s 2024-03-29T04:11:47.931 INFO:teuthology.orchestra.run.smithi148.stderr: 30050K .......... .......... .......... .......... .......... 17% 30.5M 6s 2024-03-29T04:11:47.932 INFO:teuthology.orchestra.run.smithi148.stderr: 30100K .......... .......... .......... .......... .......... 17% 35.1M 6s 2024-03-29T04:11:47.933 INFO:teuthology.orchestra.run.smithi148.stderr: 30150K .......... .......... .......... .......... .......... 17% 34.6M 6s 2024-03-29T04:11:47.935 INFO:teuthology.orchestra.run.smithi148.stderr: 30200K .......... .......... .......... .......... .......... 17% 28.4M 6s 2024-03-29T04:11:47.937 INFO:teuthology.orchestra.run.smithi148.stderr: 30250K .......... .......... .......... .......... .......... 17% 26.7M 6s 2024-03-29T04:11:47.938 INFO:teuthology.orchestra.run.smithi148.stderr: 30300K .......... .......... .......... .......... .......... 18% 33.9M 6s 2024-03-29T04:11:47.940 INFO:teuthology.orchestra.run.smithi148.stderr: 30350K .......... .......... .......... .......... .......... 18% 28.3M 6s 2024-03-29T04:11:47.942 INFO:teuthology.orchestra.run.smithi148.stderr: 30400K .......... .......... .......... .......... .......... 18% 24.1M 6s 2024-03-29T04:11:47.944 INFO:teuthology.orchestra.run.smithi148.stderr: 30450K .......... .......... .......... .......... .......... 18% 21.3M 6s 2024-03-29T04:11:47.946 INFO:teuthology.orchestra.run.smithi148.stderr: 30500K .......... .......... .......... .......... .......... 18% 25.5M 6s 2024-03-29T04:11:47.948 INFO:teuthology.orchestra.run.smithi148.stderr: 30550K .......... .......... .......... .......... .......... 18% 25.4M 6s 2024-03-29T04:11:47.949 INFO:teuthology.orchestra.run.smithi148.stderr: 30600K .......... .......... .......... .......... .......... 18% 33.1M 6s 2024-03-29T04:11:47.951 INFO:teuthology.orchestra.run.smithi148.stderr: 30650K .......... .......... .......... .......... .......... 18% 30.9M 6s 2024-03-29T04:11:47.952 INFO:teuthology.orchestra.run.smithi148.stderr: 30700K .......... .......... .......... .......... .......... 18% 36.1M 6s 2024-03-29T04:11:47.954 INFO:teuthology.orchestra.run.smithi148.stderr: 30750K .......... .......... .......... .......... .......... 18% 35.6M 6s 2024-03-29T04:11:47.955 INFO:teuthology.orchestra.run.smithi148.stderr: 30800K .......... .......... .......... .......... .......... 18% 35.1M 5s 2024-03-29T04:11:47.957 INFO:teuthology.orchestra.run.smithi148.stderr: 30850K .......... .......... .......... .......... .......... 18% 30.8M 5s 2024-03-29T04:11:47.958 INFO:teuthology.orchestra.run.smithi148.stderr: 30900K .......... .......... .......... .......... .......... 18% 35.8M 5s 2024-03-29T04:11:47.959 INFO:teuthology.orchestra.run.smithi148.stderr: 30950K .......... .......... .......... .......... .......... 18% 35.2M 5s 2024-03-29T04:11:47.961 INFO:teuthology.orchestra.run.smithi148.stderr: 31000K .......... .......... .......... .......... .......... 18% 35.1M 5s 2024-03-29T04:11:47.962 INFO:teuthology.orchestra.run.smithi148.stderr: 31050K .......... .......... .......... .......... .......... 18% 29.5M 5s 2024-03-29T04:11:47.964 INFO:teuthology.orchestra.run.smithi148.stderr: 31100K .......... .......... .......... .......... .......... 18% 33.3M 5s 2024-03-29T04:11:47.966 INFO:teuthology.orchestra.run.smithi148.stderr: 31150K .......... .......... .......... .......... .......... 18% 34.1M 5s 2024-03-29T04:11:47.968 INFO:teuthology.orchestra.run.smithi148.stderr: 31200K .......... .......... .......... .......... .......... 18% 34.3M 5s 2024-03-29T04:11:47.969 INFO:teuthology.orchestra.run.smithi148.stderr: 31250K .......... .......... .......... .......... .......... 18% 29.2M 5s 2024-03-29T04:11:47.971 INFO:teuthology.orchestra.run.smithi148.stderr: 31300K .......... .......... .......... .......... .......... 18% 33.0M 5s 2024-03-29T04:11:47.972 INFO:teuthology.orchestra.run.smithi148.stderr: 31350K .......... .......... .......... .......... .......... 18% 32.9M 5s 2024-03-29T04:11:47.973 INFO:teuthology.orchestra.run.smithi148.stderr: 31400K .......... .......... .......... .......... .......... 18% 30.9M 5s 2024-03-29T04:11:47.976 INFO:teuthology.orchestra.run.smithi148.stderr: 31450K .......... .......... .......... .......... .......... 18% 21.6M 5s 2024-03-29T04:11:47.978 INFO:teuthology.orchestra.run.smithi148.stderr: 31500K .......... .......... .......... .......... .......... 18% 25.1M 5s 2024-03-29T04:11:47.980 INFO:teuthology.orchestra.run.smithi148.stderr: 31550K .......... .......... .......... .......... .......... 18% 24.7M 5s 2024-03-29T04:11:47.981 INFO:teuthology.orchestra.run.smithi148.stderr: 31600K .......... .......... .......... .......... .......... 18% 25.3M 5s 2024-03-29T04:11:47.983 INFO:teuthology.orchestra.run.smithi148.stderr: 31650K .......... .......... .......... .......... .......... 18% 21.2M 5s 2024-03-29T04:11:47.986 INFO:teuthology.orchestra.run.smithi148.stderr: 31700K .......... .......... .......... .......... .......... 18% 25.2M 5s 2024-03-29T04:11:47.987 INFO:teuthology.orchestra.run.smithi148.stderr: 31750K .......... .......... .......... .......... .......... 18% 24.5M 5s 2024-03-29T04:11:47.990 INFO:teuthology.orchestra.run.smithi148.stderr: 31800K .......... .......... .......... .......... .......... 18% 25.4M 5s 2024-03-29T04:11:47.992 INFO:teuthology.orchestra.run.smithi148.stderr: 31850K .......... .......... .......... .......... .......... 18% 20.7M 5s 2024-03-29T04:11:47.994 INFO:teuthology.orchestra.run.smithi148.stderr: 31900K .......... .......... .......... .......... .......... 18% 23.9M 5s 2024-03-29T04:11:47.996 INFO:teuthology.orchestra.run.smithi148.stderr: 31950K .......... .......... .......... .......... .......... 18% 21.2M 5s 2024-03-29T04:11:47.999 INFO:teuthology.orchestra.run.smithi148.stderr: 32000K .......... .......... .......... .......... .......... 19% 21.6M 5s 2024-03-29T04:11:48.001 INFO:teuthology.orchestra.run.smithi148.stderr: 32050K .......... .......... .......... .......... .......... 19% 18.1M 5s 2024-03-29T04:11:48.003 INFO:teuthology.orchestra.run.smithi148.stderr: 32100K .......... .......... .......... .......... .......... 19% 21.8M 5s 2024-03-29T04:11:48.006 INFO:teuthology.orchestra.run.smithi148.stderr: 32150K .......... .......... .......... .......... .......... 19% 21.6M 5s 2024-03-29T04:11:48.008 INFO:teuthology.orchestra.run.smithi148.stderr: 32200K .......... .......... .......... .......... .......... 19% 21.4M 5s 2024-03-29T04:11:48.011 INFO:teuthology.orchestra.run.smithi148.stderr: 32250K .......... .......... .......... .......... .......... 19% 18.3M 5s 2024-03-29T04:11:48.013 INFO:teuthology.orchestra.run.smithi148.stderr: 32300K .......... .......... .......... .......... .......... 19% 21.7M 5s 2024-03-29T04:11:48.015 INFO:teuthology.orchestra.run.smithi148.stderr: 32350K .......... .......... .......... .......... .......... 19% 21.5M 5s 2024-03-29T04:11:48.018 INFO:teuthology.orchestra.run.smithi148.stderr: 32400K .......... .......... .......... .......... .......... 19% 21.1M 5s 2024-03-29T04:11:48.020 INFO:teuthology.orchestra.run.smithi148.stderr: 32450K .......... .......... .......... .......... .......... 19% 18.2M 5s 2024-03-29T04:11:48.022 INFO:teuthology.orchestra.run.smithi148.stderr: 32500K .......... .......... .......... .......... .......... 19% 21.3M 5s 2024-03-29T04:11:48.024 INFO:teuthology.orchestra.run.smithi148.stderr: 32550K .......... .......... .......... .......... .......... 19% 22.3M 5s 2024-03-29T04:11:48.026 INFO:teuthology.orchestra.run.smithi148.stderr: 32600K .......... .......... .......... .......... .......... 19% 25.2M 5s 2024-03-29T04:11:48.028 INFO:teuthology.orchestra.run.smithi148.stderr: 32650K .......... .......... .......... .......... .......... 19% 23.9M 5s 2024-03-29T04:11:48.031 INFO:teuthology.orchestra.run.smithi148.stderr: 32700K .......... .......... .......... .......... .......... 19% 28.5M 5s 2024-03-29T04:11:48.037 INFO:teuthology.orchestra.run.smithi148.stderr: 32750K .......... .......... .......... .......... .......... 19% 25.9M 5s 2024-03-29T04:11:48.037 INFO:teuthology.orchestra.run.smithi148.stderr: 32800K .......... .......... .......... .......... .......... 19% 39.5M 5s 2024-03-29T04:11:48.037 INFO:teuthology.orchestra.run.smithi148.stderr: 32850K .......... .......... .......... .......... .......... 19% 35.2M 5s 2024-03-29T04:11:48.038 INFO:teuthology.orchestra.run.smithi148.stderr: 32900K .......... .......... .......... .......... .......... 19% 40.6M 5s 2024-03-29T04:11:48.038 INFO:teuthology.orchestra.run.smithi148.stderr: 32950K .......... .......... .......... .......... .......... 19% 38.3M 5s 2024-03-29T04:11:48.041 INFO:teuthology.orchestra.run.smithi148.stderr: 33000K .......... .......... .......... .......... .......... 19% 34.4M 5s 2024-03-29T04:11:48.041 INFO:teuthology.orchestra.run.smithi148.stderr: 33050K .......... .......... .......... .......... .......... 19% 29.1M 5s 2024-03-29T04:11:48.041 INFO:teuthology.orchestra.run.smithi148.stderr: 33100K .......... .......... .......... .......... .......... 19% 33.4M 5s 2024-03-29T04:11:48.043 INFO:teuthology.orchestra.run.smithi148.stderr: 33150K .......... .......... .......... .......... .......... 19% 31.2M 5s 2024-03-29T04:11:48.045 INFO:teuthology.orchestra.run.smithi148.stderr: 33200K .......... .......... .......... .......... .......... 19% 34.4M 5s 2024-03-29T04:11:48.046 INFO:teuthology.orchestra.run.smithi148.stderr: 33250K .......... .......... .......... .......... .......... 19% 30.0M 5s 2024-03-29T04:11:48.048 INFO:teuthology.orchestra.run.smithi148.stderr: 33300K .......... .......... .......... .......... .......... 19% 36.5M 5s 2024-03-29T04:11:48.050 INFO:teuthology.orchestra.run.smithi148.stderr: 33350K .......... .......... .......... .......... .......... 19% 34.7M 5s 2024-03-29T04:11:48.051 INFO:teuthology.orchestra.run.smithi148.stderr: 33400K .......... .......... .......... .......... .......... 19% 38.2M 5s 2024-03-29T04:11:48.052 INFO:teuthology.orchestra.run.smithi148.stderr: 33450K .......... .......... .......... .......... .......... 19% 29.6M 5s 2024-03-29T04:11:48.053 INFO:teuthology.orchestra.run.smithi148.stderr: 33500K .......... .......... .......... .......... .......... 19% 39.6M 5s 2024-03-29T04:11:48.055 INFO:teuthology.orchestra.run.smithi148.stderr: 33550K .......... .......... .......... .......... .......... 19% 38.9M 5s 2024-03-29T04:11:48.056 INFO:teuthology.orchestra.run.smithi148.stderr: 33600K .......... .......... .......... .......... .......... 19% 38.3M 5s 2024-03-29T04:11:48.057 INFO:teuthology.orchestra.run.smithi148.stderr: 33650K .......... .......... .......... .......... .......... 19% 33.1M 5s 2024-03-29T04:11:48.058 INFO:teuthology.orchestra.run.smithi148.stderr: 33700K .......... .......... .......... .......... .......... 20% 37.3M 5s 2024-03-29T04:11:48.061 INFO:teuthology.orchestra.run.smithi148.stderr: 33750K .......... .......... .......... .......... .......... 20% 37.6M 5s 2024-03-29T04:11:48.061 INFO:teuthology.orchestra.run.smithi148.stderr: 33800K .......... .......... .......... .......... .......... 20% 36.9M 5s 2024-03-29T04:11:48.063 INFO:teuthology.orchestra.run.smithi148.stderr: 33850K .......... .......... .......... .......... .......... 20% 32.0M 5s 2024-03-29T04:11:48.064 INFO:teuthology.orchestra.run.smithi148.stderr: 33900K .......... .......... .......... .......... .......... 20% 38.1M 5s 2024-03-29T04:11:48.071 INFO:teuthology.orchestra.run.smithi148.stderr: 33950K .......... .......... .......... .......... .......... 20% 37.4M 5s 2024-03-29T04:11:48.072 INFO:teuthology.orchestra.run.smithi148.stderr: 34000K .......... .......... .......... .......... .......... 20% 30.3M 5s 2024-03-29T04:11:48.072 INFO:teuthology.orchestra.run.smithi148.stderr: 34050K .......... .......... .......... .......... .......... 20% 32.2M 5s 2024-03-29T04:11:48.072 INFO:teuthology.orchestra.run.smithi148.stderr: 34100K .......... .......... .......... .......... .......... 20% 37.4M 5s 2024-03-29T04:11:48.072 INFO:teuthology.orchestra.run.smithi148.stderr: 34150K .......... .......... .......... .......... .......... 20% 37.7M 5s 2024-03-29T04:11:48.072 INFO:teuthology.orchestra.run.smithi148.stderr: 34200K .......... .......... .......... .......... .......... 20% 37.6M 5s 2024-03-29T04:11:48.074 INFO:teuthology.orchestra.run.smithi148.stderr: 34250K .......... .......... .......... .......... .......... 20% 31.0M 5s 2024-03-29T04:11:48.075 INFO:teuthology.orchestra.run.smithi148.stderr: 34300K .......... .......... .......... .......... .......... 20% 36.7M 5s 2024-03-29T04:11:48.076 INFO:teuthology.orchestra.run.smithi148.stderr: 34350K .......... .......... .......... .......... .......... 20% 36.6M 5s 2024-03-29T04:11:48.078 INFO:teuthology.orchestra.run.smithi148.stderr: 34400K .......... .......... .......... .......... .......... 20% 36.6M 5s 2024-03-29T04:11:48.079 INFO:teuthology.orchestra.run.smithi148.stderr: 34450K .......... .......... .......... .......... .......... 20% 31.9M 5s 2024-03-29T04:11:48.080 INFO:teuthology.orchestra.run.smithi148.stderr: 34500K .......... .......... .......... .......... .......... 20% 36.5M 5s 2024-03-29T04:11:48.082 INFO:teuthology.orchestra.run.smithi148.stderr: 34550K .......... .......... .......... .......... .......... 20% 36.5M 5s 2024-03-29T04:11:48.083 INFO:teuthology.orchestra.run.smithi148.stderr: 34600K .......... .......... .......... .......... .......... 20% 35.9M 5s 2024-03-29T04:11:48.085 INFO:teuthology.orchestra.run.smithi148.stderr: 34650K .......... .......... .......... .......... .......... 20% 30.9M 5s 2024-03-29T04:11:48.086 INFO:teuthology.orchestra.run.smithi148.stderr: 34700K .......... .......... .......... .......... .......... 20% 35.3M 5s 2024-03-29T04:11:48.088 INFO:teuthology.orchestra.run.smithi148.stderr: 34750K .......... .......... .......... .......... .......... 20% 34.9M 5s 2024-03-29T04:11:48.089 INFO:teuthology.orchestra.run.smithi148.stderr: 34800K .......... .......... .......... .......... .......... 20% 35.4M 5s 2024-03-29T04:11:48.091 INFO:teuthology.orchestra.run.smithi148.stderr: 34850K .......... .......... .......... .......... .......... 20% 29.9M 5s 2024-03-29T04:11:48.092 INFO:teuthology.orchestra.run.smithi148.stderr: 34900K .......... .......... .......... .......... .......... 20% 33.4M 5s 2024-03-29T04:11:48.094 INFO:teuthology.orchestra.run.smithi148.stderr: 34950K .......... .......... .......... .......... .......... 20% 29.1M 5s 2024-03-29T04:11:48.096 INFO:teuthology.orchestra.run.smithi148.stderr: 35000K .......... .......... .......... .......... .......... 20% 27.6M 5s 2024-03-29T04:11:48.098 INFO:teuthology.orchestra.run.smithi148.stderr: 35050K .......... .......... .......... .......... .......... 20% 25.2M 5s 2024-03-29T04:11:48.099 INFO:teuthology.orchestra.run.smithi148.stderr: 35100K .......... .......... .......... .......... .......... 20% 29.4M 5s 2024-03-29T04:11:48.101 INFO:teuthology.orchestra.run.smithi148.stderr: 35150K .......... .......... .......... .......... .......... 20% 29.5M 5s 2024-03-29T04:11:48.103 INFO:teuthology.orchestra.run.smithi148.stderr: 35200K .......... .......... .......... .......... .......... 20% 29.7M 5s 2024-03-29T04:11:48.104 INFO:teuthology.orchestra.run.smithi148.stderr: 35250K .......... .......... .......... .......... .......... 20% 25.8M 5s 2024-03-29T04:11:48.106 INFO:teuthology.orchestra.run.smithi148.stderr: 35300K .......... .......... .......... .......... .......... 20% 30.3M 5s 2024-03-29T04:11:48.108 INFO:teuthology.orchestra.run.smithi148.stderr: 35350K .......... .......... .......... .......... .......... 21% 28.7M 5s 2024-03-29T04:11:48.110 INFO:teuthology.orchestra.run.smithi148.stderr: 35400K .......... .......... .......... .......... .......... 21% 28.6M 5s 2024-03-29T04:11:48.111 INFO:teuthology.orchestra.run.smithi148.stderr: 35450K .......... .......... .......... .......... .......... 21% 24.3M 5s 2024-03-29T04:11:48.113 INFO:teuthology.orchestra.run.smithi148.stderr: 35500K .......... .......... .......... .......... .......... 21% 26.6M 5s 2024-03-29T04:11:48.115 INFO:teuthology.orchestra.run.smithi148.stderr: 35550K .......... .......... .......... .......... .......... 21% 26.3M 5s 2024-03-29T04:11:48.118 INFO:teuthology.orchestra.run.smithi148.stderr: 35600K .......... .......... .......... .......... .......... 21% 27.0M 5s 2024-03-29T04:11:48.119 INFO:teuthology.orchestra.run.smithi148.stderr: 35650K .......... .......... .......... .......... .......... 21% 22.4M 5s 2024-03-29T04:11:48.121 INFO:teuthology.orchestra.run.smithi148.stderr: 35700K .......... .......... .......... .......... .......... 21% 26.0M 5s 2024-03-29T04:11:48.123 INFO:teuthology.orchestra.run.smithi148.stderr: 35750K .......... .......... .......... .......... .......... 21% 26.2M 5s 2024-03-29T04:11:48.125 INFO:teuthology.orchestra.run.smithi148.stderr: 35800K .......... .......... .......... .......... .......... 21% 25.3M 5s 2024-03-29T04:11:48.127 INFO:teuthology.orchestra.run.smithi148.stderr: 35850K .......... .......... .......... .......... .......... 21% 22.1M 5s 2024-03-29T04:11:48.129 INFO:teuthology.orchestra.run.smithi148.stderr: 35900K .......... .......... .......... .......... .......... 21% 25.6M 5s 2024-03-29T04:11:48.131 INFO:teuthology.orchestra.run.smithi148.stderr: 35950K .......... .......... .......... .......... .......... 21% 25.7M 5s 2024-03-29T04:11:48.133 INFO:teuthology.orchestra.run.smithi148.stderr: 36000K .......... .......... .......... .......... .......... 21% 24.6M 5s 2024-03-29T04:11:48.136 INFO:teuthology.orchestra.run.smithi148.stderr: 36050K .......... .......... .......... .......... .......... 21% 21.8M 5s 2024-03-29T04:11:48.138 INFO:teuthology.orchestra.run.smithi148.stderr: 36100K .......... .......... .......... .......... .......... 21% 25.3M 5s 2024-03-29T04:11:48.140 INFO:teuthology.orchestra.run.smithi148.stderr: 36150K .......... .......... .......... .......... .......... 21% 24.2M 5s 2024-03-29T04:11:48.142 INFO:teuthology.orchestra.run.smithi148.stderr: 36200K .......... .......... .......... .......... .......... 21% 24.3M 5s 2024-03-29T04:11:48.144 INFO:teuthology.orchestra.run.smithi148.stderr: 36250K .......... .......... .......... .......... .......... 21% 21.7M 5s 2024-03-29T04:11:48.146 INFO:teuthology.orchestra.run.smithi148.stderr: 36300K .......... .......... .......... .......... .......... 21% 24.3M 5s 2024-03-29T04:11:48.148 INFO:teuthology.orchestra.run.smithi148.stderr: 36350K .......... .......... .......... .......... .......... 21% 25.1M 5s 2024-03-29T04:11:48.150 INFO:teuthology.orchestra.run.smithi148.stderr: 36400K .......... .......... .......... .......... .......... 21% 22.9M 5s 2024-03-29T04:11:48.153 INFO:teuthology.orchestra.run.smithi148.stderr: 36450K .......... .......... .......... .......... .......... 21% 18.8M 5s 2024-03-29T04:11:48.155 INFO:teuthology.orchestra.run.smithi148.stderr: 36500K .......... .......... .......... .......... .......... 21% 23.3M 5s 2024-03-29T04:11:48.157 INFO:teuthology.orchestra.run.smithi148.stderr: 36550K .......... .......... .......... .......... .......... 21% 23.7M 5s 2024-03-29T04:11:48.159 INFO:teuthology.orchestra.run.smithi148.stderr: 36600K .......... .......... .......... .......... .......... 21% 23.7M 5s 2024-03-29T04:11:48.161 INFO:teuthology.orchestra.run.smithi148.stderr: 36650K .......... .......... .......... .......... .......... 21% 19.9M 5s 2024-03-29T04:11:48.163 INFO:teuthology.orchestra.run.smithi148.stderr: 36700K .......... .......... .......... .......... .......... 21% 23.5M 5s 2024-03-29T04:11:48.165 INFO:teuthology.orchestra.run.smithi148.stderr: 36750K .......... .......... .......... .......... .......... 21% 22.9M 5s 2024-03-29T04:11:48.167 INFO:teuthology.orchestra.run.smithi148.stderr: 36800K .......... .......... .......... .......... .......... 21% 22.8M 5s 2024-03-29T04:11:48.169 INFO:teuthology.orchestra.run.smithi148.stderr: 36850K .......... .......... .......... .......... .......... 21% 20.6M 5s 2024-03-29T04:11:48.170 INFO:teuthology.orchestra.run.smithi148.stderr: 36900K .......... .......... .......... .......... .......... 21% 40.1M 5s 2024-03-29T04:11:48.172 INFO:teuthology.orchestra.run.smithi148.stderr: 36950K .......... .......... .......... .......... .......... 21% 31.7M 5s 2024-03-29T04:11:48.174 INFO:teuthology.orchestra.run.smithi148.stderr: 37000K .......... .......... .......... .......... .......... 21% 26.9M 5s 2024-03-29T04:11:48.176 INFO:teuthology.orchestra.run.smithi148.stderr: 37050K .......... .......... .......... .......... .......... 22% 22.3M 5s 2024-03-29T04:11:48.178 INFO:teuthology.orchestra.run.smithi148.stderr: 37100K .......... .......... .......... .......... .......... 22% 27.9M 5s 2024-03-29T04:11:48.179 INFO:teuthology.orchestra.run.smithi148.stderr: 37150K .......... .......... .......... .......... .......... 22% 32.5M 5s 2024-03-29T04:11:48.181 INFO:teuthology.orchestra.run.smithi148.stderr: 37200K .......... .......... .......... .......... .......... 22% 33.6M 5s 2024-03-29T04:11:48.182 INFO:teuthology.orchestra.run.smithi148.stderr: 37250K .......... .......... .......... .......... .......... 22% 33.5M 5s 2024-03-29T04:11:48.183 INFO:teuthology.orchestra.run.smithi148.stderr: 37300K .......... .......... .......... .......... .......... 22% 39.1M 5s 2024-03-29T04:11:48.185 INFO:teuthology.orchestra.run.smithi148.stderr: 37350K .......... .......... .......... .......... .......... 22% 28.8M 5s 2024-03-29T04:11:48.187 INFO:teuthology.orchestra.run.smithi148.stderr: 37400K .......... .......... .......... .......... .......... 22% 33.7M 5s 2024-03-29T04:11:48.188 INFO:teuthology.orchestra.run.smithi148.stderr: 37450K .......... .......... .......... .......... .......... 22% 30.1M 5s 2024-03-29T04:11:48.191 INFO:teuthology.orchestra.run.smithi148.stderr: 37500K .......... .......... .......... .......... .......... 22% 33.5M 5s 2024-03-29T04:11:48.191 INFO:teuthology.orchestra.run.smithi148.stderr: 37550K .......... .......... .......... .......... .......... 22% 33.6M 5s 2024-03-29T04:11:48.193 INFO:teuthology.orchestra.run.smithi148.stderr: 37600K .......... .......... .......... .......... .......... 22% 34.9M 5s 2024-03-29T04:11:48.194 INFO:teuthology.orchestra.run.smithi148.stderr: 37650K .......... .......... .......... .......... .......... 22% 29.9M 5s 2024-03-29T04:11:48.196 INFO:teuthology.orchestra.run.smithi148.stderr: 37700K .......... .......... .......... .......... .......... 22% 34.2M 5s 2024-03-29T04:11:48.197 INFO:teuthology.orchestra.run.smithi148.stderr: 37750K .......... .......... .......... .......... .......... 22% 35.3M 5s 2024-03-29T04:11:48.198 INFO:teuthology.orchestra.run.smithi148.stderr: 37800K .......... .......... .......... .......... .......... 22% 33.9M 5s 2024-03-29T04:11:48.200 INFO:teuthology.orchestra.run.smithi148.stderr: 37850K .......... .......... .......... .......... .......... 22% 29.6M 5s 2024-03-29T04:11:48.202 INFO:teuthology.orchestra.run.smithi148.stderr: 37900K .......... .......... .......... .......... .......... 22% 34.0M 5s 2024-03-29T04:11:48.203 INFO:teuthology.orchestra.run.smithi148.stderr: 37950K .......... .......... .......... .......... .......... 22% 34.3M 5s 2024-03-29T04:11:48.204 INFO:teuthology.orchestra.run.smithi148.stderr: 38000K .......... .......... .......... .......... .......... 22% 33.3M 5s 2024-03-29T04:11:48.206 INFO:teuthology.orchestra.run.smithi148.stderr: 38050K .......... .......... .......... .......... .......... 22% 28.9M 5s 2024-03-29T04:11:48.208 INFO:teuthology.orchestra.run.smithi148.stderr: 38100K .......... .......... .......... .......... .......... 22% 32.9M 5s 2024-03-29T04:11:48.209 INFO:teuthology.orchestra.run.smithi148.stderr: 38150K .......... .......... .......... .......... .......... 22% 33.4M 5s 2024-03-29T04:11:48.211 INFO:teuthology.orchestra.run.smithi148.stderr: 38200K .......... .......... .......... .......... .......... 22% 33.6M 5s 2024-03-29T04:11:48.212 INFO:teuthology.orchestra.run.smithi148.stderr: 38250K .......... .......... .......... .......... .......... 22% 28.9M 5s 2024-03-29T04:11:48.214 INFO:teuthology.orchestra.run.smithi148.stderr: 38300K .......... .......... .......... .......... .......... 22% 34.3M 5s 2024-03-29T04:11:48.215 INFO:teuthology.orchestra.run.smithi148.stderr: 38350K .......... .......... .......... .......... .......... 22% 33.8M 5s 2024-03-29T04:11:48.217 INFO:teuthology.orchestra.run.smithi148.stderr: 38400K .......... .......... .......... .......... .......... 22% 32.5M 5s 2024-03-29T04:11:48.218 INFO:teuthology.orchestra.run.smithi148.stderr: 38450K .......... .......... .......... .......... .......... 22% 28.4M 5s 2024-03-29T04:11:48.220 INFO:teuthology.orchestra.run.smithi148.stderr: 38500K .......... .......... .......... .......... .......... 22% 32.5M 5s 2024-03-29T04:11:48.221 INFO:teuthology.orchestra.run.smithi148.stderr: 38550K .......... .......... .......... .......... .......... 22% 32.7M 5s 2024-03-29T04:11:48.223 INFO:teuthology.orchestra.run.smithi148.stderr: 38600K .......... .......... .......... .......... .......... 22% 31.9M 5s 2024-03-29T04:11:48.225 INFO:teuthology.orchestra.run.smithi148.stderr: 38650K .......... .......... .......... .......... .......... 22% 25.2M 5s 2024-03-29T04:11:48.226 INFO:teuthology.orchestra.run.smithi148.stderr: 38700K .......... .......... .......... .......... .......... 22% 32.4M 5s 2024-03-29T04:11:48.228 INFO:teuthology.orchestra.run.smithi148.stderr: 38750K .......... .......... .......... .......... .......... 23% 31.5M 5s 2024-03-29T04:11:48.229 INFO:teuthology.orchestra.run.smithi148.stderr: 38800K .......... .......... .......... .......... .......... 23% 31.5M 5s 2024-03-29T04:11:48.231 INFO:teuthology.orchestra.run.smithi148.stderr: 38850K .......... .......... .......... .......... .......... 23% 27.1M 5s 2024-03-29T04:11:48.233 INFO:teuthology.orchestra.run.smithi148.stderr: 38900K .......... .......... .......... .......... .......... 23% 32.2M 5s 2024-03-29T04:11:48.234 INFO:teuthology.orchestra.run.smithi148.stderr: 38950K .......... .......... .......... .......... .......... 23% 31.9M 5s 2024-03-29T04:11:48.236 INFO:teuthology.orchestra.run.smithi148.stderr: 39000K .......... .......... .......... .......... .......... 23% 31.4M 5s 2024-03-29T04:11:48.238 INFO:teuthology.orchestra.run.smithi148.stderr: 39050K .......... .......... .......... .......... .......... 23% 27.0M 5s 2024-03-29T04:11:48.239 INFO:teuthology.orchestra.run.smithi148.stderr: 39100K .......... .......... .......... .......... .......... 23% 31.8M 5s 2024-03-29T04:11:48.241 INFO:teuthology.orchestra.run.smithi148.stderr: 39150K .......... .......... .......... .......... .......... 23% 31.7M 5s 2024-03-29T04:11:48.242 INFO:teuthology.orchestra.run.smithi148.stderr: 39200K .......... .......... .......... .......... .......... 23% 31.6M 5s 2024-03-29T04:11:48.244 INFO:teuthology.orchestra.run.smithi148.stderr: 39250K .......... .......... .......... .......... .......... 23% 26.5M 5s 2024-03-29T04:11:48.246 INFO:teuthology.orchestra.run.smithi148.stderr: 39300K .......... .......... .......... .......... .......... 23% 31.2M 5s 2024-03-29T04:11:48.247 INFO:teuthology.orchestra.run.smithi148.stderr: 39350K .......... .......... .......... .......... .......... 23% 31.4M 5s 2024-03-29T04:11:48.249 INFO:teuthology.orchestra.run.smithi148.stderr: 39400K .......... .......... .......... .......... .......... 23% 31.0M 5s 2024-03-29T04:11:48.251 INFO:teuthology.orchestra.run.smithi148.stderr: 39450K .......... .......... .......... .......... .......... 23% 26.4M 5s 2024-03-29T04:11:48.252 INFO:teuthology.orchestra.run.smithi148.stderr: 39500K .......... .......... .......... .......... .......... 23% 31.1M 5s 2024-03-29T04:11:48.254 INFO:teuthology.orchestra.run.smithi148.stderr: 39550K .......... .......... .......... .......... .......... 23% 29.7M 5s 2024-03-29T04:11:48.256 INFO:teuthology.orchestra.run.smithi148.stderr: 39600K .......... .......... .......... .......... .......... 23% 30.4M 5s 2024-03-29T04:11:48.257 INFO:teuthology.orchestra.run.smithi148.stderr: 39650K .......... .......... .......... .......... .......... 23% 26.1M 5s 2024-03-29T04:11:48.259 INFO:teuthology.orchestra.run.smithi148.stderr: 39700K .......... .......... .......... .......... .......... 23% 30.1M 5s 2024-03-29T04:11:48.261 INFO:teuthology.orchestra.run.smithi148.stderr: 39750K .......... .......... .......... .......... .......... 23% 30.3M 5s 2024-03-29T04:11:48.262 INFO:teuthology.orchestra.run.smithi148.stderr: 39800K .......... .......... .......... .......... .......... 23% 30.4M 5s 2024-03-29T04:11:48.264 INFO:teuthology.orchestra.run.smithi148.stderr: 39850K .......... .......... .......... .......... .......... 23% 26.0M 5s 2024-03-29T04:11:48.266 INFO:teuthology.orchestra.run.smithi148.stderr: 39900K .......... .......... .......... .......... .......... 23% 29.9M 5s 2024-03-29T04:11:48.267 INFO:teuthology.orchestra.run.smithi148.stderr: 39950K .......... .......... .......... .......... .......... 23% 30.1M 5s 2024-03-29T04:11:48.269 INFO:teuthology.orchestra.run.smithi148.stderr: 40000K .......... .......... .......... .......... .......... 23% 29.5M 5s 2024-03-29T04:11:48.271 INFO:teuthology.orchestra.run.smithi148.stderr: 40050K .......... .......... .......... .......... .......... 23% 25.0M 5s 2024-03-29T04:11:48.272 INFO:teuthology.orchestra.run.smithi148.stderr: 40100K .......... .......... .......... .......... .......... 23% 30.2M 5s 2024-03-29T04:11:48.274 INFO:teuthology.orchestra.run.smithi148.stderr: 40150K .......... .......... .......... .......... .......... 23% 29.2M 5s 2024-03-29T04:11:48.278 INFO:teuthology.orchestra.run.smithi148.stderr: 40200K .......... .......... .......... .......... .......... 23% 28.0M 5s 2024-03-29T04:11:48.278 INFO:teuthology.orchestra.run.smithi148.stderr: 40250K .......... .......... .......... .......... .......... 23% 24.7M 5s 2024-03-29T04:11:48.279 INFO:teuthology.orchestra.run.smithi148.stderr: 40300K .......... .......... .......... .......... .......... 23% 28.9M 5s 2024-03-29T04:11:48.281 INFO:teuthology.orchestra.run.smithi148.stderr: 40350K .......... .......... .......... .......... .......... 23% 28.5M 5s 2024-03-29T04:11:48.283 INFO:teuthology.orchestra.run.smithi148.stderr: 40400K .......... .......... .......... .......... .......... 23% 28.4M 5s 2024-03-29T04:11:48.285 INFO:teuthology.orchestra.run.smithi148.stderr: 40450K .......... .......... .......... .......... .......... 24% 21.1M 5s 2024-03-29T04:11:48.287 INFO:teuthology.orchestra.run.smithi148.stderr: 40500K .......... .......... .......... .......... .......... 24% 25.2M 5s 2024-03-29T04:11:48.289 INFO:teuthology.orchestra.run.smithi148.stderr: 40550K .......... .......... .......... .......... .......... 24% 23.9M 5s 2024-03-29T04:11:48.292 INFO:teuthology.orchestra.run.smithi148.stderr: 40600K .......... .......... .......... .......... .......... 24% 24.5M 5s 2024-03-29T04:11:48.294 INFO:teuthology.orchestra.run.smithi148.stderr: 40650K .......... .......... .......... .......... .......... 24% 20.2M 5s 2024-03-29T04:11:48.296 INFO:teuthology.orchestra.run.smithi148.stderr: 40700K .......... .......... .......... .......... .......... 24% 24.3M 5s 2024-03-29T04:11:48.298 INFO:teuthology.orchestra.run.smithi148.stderr: 40750K .......... .......... .......... .......... .......... 24% 25.2M 5s 2024-03-29T04:11:48.300 INFO:teuthology.orchestra.run.smithi148.stderr: 40800K .......... .......... .......... .......... .......... 24% 25.4M 5s 2024-03-29T04:11:48.302 INFO:teuthology.orchestra.run.smithi148.stderr: 40850K .......... .......... .......... .......... .......... 24% 23.8M 5s 2024-03-29T04:11:48.303 INFO:teuthology.orchestra.run.smithi148.stderr: 40900K .......... .......... .......... .......... .......... 24% 27.7M 5s 2024-03-29T04:11:48.305 INFO:teuthology.orchestra.run.smithi148.stderr: 40950K .......... .......... .......... .......... .......... 24% 30.5M 5s 2024-03-29T04:11:48.307 INFO:teuthology.orchestra.run.smithi148.stderr: 41000K .......... .......... .......... .......... .......... 24% 41.7M 5s 2024-03-29T04:11:48.308 INFO:teuthology.orchestra.run.smithi148.stderr: 41050K .......... .......... .......... .......... .......... 24% 35.1M 5s 2024-03-29T04:11:48.309 INFO:teuthology.orchestra.run.smithi148.stderr: 41100K .......... .......... .......... .......... .......... 24% 37.5M 5s 2024-03-29T04:11:48.311 INFO:teuthology.orchestra.run.smithi148.stderr: 41150K .......... .......... .......... .......... .......... 24% 39.3M 5s 2024-03-29T04:11:48.311 INFO:teuthology.orchestra.run.smithi148.stderr: 41200K .......... .......... .......... .......... .......... 24% 40.5M 5s 2024-03-29T04:11:48.319 INFO:teuthology.orchestra.run.smithi148.stderr: 41250K .......... .......... .......... .......... .......... 24% 35.2M 5s 2024-03-29T04:11:48.319 INFO:teuthology.orchestra.run.smithi148.stderr: 41300K .......... .......... .......... .......... .......... 24% 40.5M 5s 2024-03-29T04:11:48.319 INFO:teuthology.orchestra.run.smithi148.stderr: 41350K .......... .......... .......... .......... .......... 24% 40.5M 5s 2024-03-29T04:11:48.319 INFO:teuthology.orchestra.run.smithi148.stderr: 41400K .......... .......... .......... .......... .......... 24% 40.6M 5s 2024-03-29T04:11:48.319 INFO:teuthology.orchestra.run.smithi148.stderr: 41450K .......... .......... .......... .......... .......... 24% 34.1M 5s 2024-03-29T04:11:48.328 INFO:teuthology.orchestra.run.smithi148.stderr: 41500K .......... .......... .......... .......... .......... 24% 37.1M 5s 2024-03-29T04:11:48.329 INFO:teuthology.orchestra.run.smithi148.stderr: 41550K .......... .......... .......... .......... .......... 24% 39.5M 5s 2024-03-29T04:11:48.329 INFO:teuthology.orchestra.run.smithi148.stderr: 41600K .......... .......... .......... .......... .......... 24% 38.8M 5s 2024-03-29T04:11:48.329 INFO:teuthology.orchestra.run.smithi148.stderr: 41650K .......... .......... .......... .......... .......... 24% 33.6M 5s 2024-03-29T04:11:48.329 INFO:teuthology.orchestra.run.smithi148.stderr: 41700K .......... .......... .......... .......... .......... 24% 34.4M 5s 2024-03-29T04:11:48.329 INFO:teuthology.orchestra.run.smithi148.stderr: 41750K .......... .......... .......... .......... .......... 24% 33.8M 5s 2024-03-29T04:11:48.329 INFO:teuthology.orchestra.run.smithi148.stderr: 41800K .......... .......... .......... .......... .......... 24% 34.7M 5s 2024-03-29T04:11:48.336 INFO:teuthology.orchestra.run.smithi148.stderr: 41850K .......... .......... .......... .......... .......... 24% 28.7M 5s 2024-03-29T04:11:48.336 INFO:teuthology.orchestra.run.smithi148.stderr: 41900K .......... .......... .......... .......... .......... 24% 34.0M 5s 2024-03-29T04:11:48.336 INFO:teuthology.orchestra.run.smithi148.stderr: 41950K .......... .......... .......... .......... .......... 24% 34.0M 5s 2024-03-29T04:11:48.336 INFO:teuthology.orchestra.run.smithi148.stderr: 42000K .......... .......... .......... .......... .......... 24% 33.5M 5s 2024-03-29T04:11:48.336 INFO:teuthology.orchestra.run.smithi148.stderr: 42050K .......... .......... .......... .......... .......... 24% 28.2M 5s 2024-03-29T04:11:48.341 INFO:teuthology.orchestra.run.smithi148.stderr: 42100K .......... .......... .......... .......... .......... 25% 32.6M 5s 2024-03-29T04:11:48.341 INFO:teuthology.orchestra.run.smithi148.stderr: 42150K .......... .......... .......... .......... .......... 25% 33.0M 5s 2024-03-29T04:11:48.341 INFO:teuthology.orchestra.run.smithi148.stderr: 42200K .......... .......... .......... .......... .......... 25% 33.2M 5s 2024-03-29T04:11:48.343 INFO:teuthology.orchestra.run.smithi148.stderr: 42250K .......... .......... .......... .......... .......... 25% 27.7M 5s 2024-03-29T04:11:48.343 INFO:teuthology.orchestra.run.smithi148.stderr: 42300K .......... .......... .......... .......... .......... 25% 33.1M 5s 2024-03-29T04:11:48.346 INFO:teuthology.orchestra.run.smithi148.stderr: 42350K .......... .......... .......... .......... .......... 25% 32.4M 5s 2024-03-29T04:11:48.347 INFO:teuthology.orchestra.run.smithi148.stderr: 42400K .......... .......... .......... .......... .......... 25% 33.2M 5s 2024-03-29T04:11:48.350 INFO:teuthology.orchestra.run.smithi148.stderr: 42450K .......... .......... .......... .......... .......... 25% 26.8M 5s 2024-03-29T04:11:48.350 INFO:teuthology.orchestra.run.smithi148.stderr: 42500K .......... .......... .......... .......... .......... 25% 32.1M 5s 2024-03-29T04:11:48.351 INFO:teuthology.orchestra.run.smithi148.stderr: 42550K .......... .......... .......... .......... .......... 25% 31.0M 5s 2024-03-29T04:11:48.352 INFO:teuthology.orchestra.run.smithi148.stderr: 42600K .......... .......... .......... .......... .......... 25% 32.0M 5s 2024-03-29T04:11:48.354 INFO:teuthology.orchestra.run.smithi148.stderr: 42650K .......... .......... .......... .......... .......... 25% 27.0M 5s 2024-03-29T04:11:48.356 INFO:teuthology.orchestra.run.smithi148.stderr: 42700K .......... .......... .......... .......... .......... 25% 27.5M 5s 2024-03-29T04:11:48.358 INFO:teuthology.orchestra.run.smithi148.stderr: 42750K .......... .......... .......... .......... .......... 25% 27.1M 5s 2024-03-29T04:11:48.360 INFO:teuthology.orchestra.run.smithi148.stderr: 42800K .......... .......... .......... .......... .......... 25% 26.9M 5s 2024-03-29T04:11:48.361 INFO:teuthology.orchestra.run.smithi148.stderr: 42850K .......... .......... .......... .......... .......... 25% 23.9M 5s 2024-03-29T04:11:48.363 INFO:teuthology.orchestra.run.smithi148.stderr: 42900K .......... .......... .......... .......... .......... 25% 26.7M 5s 2024-03-29T04:11:48.365 INFO:teuthology.orchestra.run.smithi148.stderr: 42950K .......... .......... .......... .......... .......... 25% 26.7M 5s 2024-03-29T04:11:48.367 INFO:teuthology.orchestra.run.smithi148.stderr: 43000K .......... .......... .......... .......... .......... 25% 25.9M 5s 2024-03-29T04:11:48.369 INFO:teuthology.orchestra.run.smithi148.stderr: 43050K .......... .......... .......... .......... .......... 25% 23.0M 5s 2024-03-29T04:11:48.371 INFO:teuthology.orchestra.run.smithi148.stderr: 43100K .......... .......... .......... .......... .......... 25% 26.5M 5s 2024-03-29T04:11:48.373 INFO:teuthology.orchestra.run.smithi148.stderr: 43150K .......... .......... .......... .......... .......... 25% 26.0M 5s 2024-03-29T04:11:48.375 INFO:teuthology.orchestra.run.smithi148.stderr: 43200K .......... .......... .......... .......... .......... 25% 27.4M 5s 2024-03-29T04:11:48.376 INFO:teuthology.orchestra.run.smithi148.stderr: 43250K .......... .......... .......... .......... .......... 25% 23.4M 5s 2024-03-29T04:11:48.378 INFO:teuthology.orchestra.run.smithi148.stderr: 43300K .......... .......... .......... .......... .......... 25% 27.0M 5s 2024-03-29T04:11:48.380 INFO:teuthology.orchestra.run.smithi148.stderr: 43350K .......... .......... .......... .......... .......... 25% 26.0M 5s 2024-03-29T04:11:48.382 INFO:teuthology.orchestra.run.smithi148.stderr: 43400K .......... .......... .......... .......... .......... 25% 25.9M 5s 2024-03-29T04:11:48.384 INFO:teuthology.orchestra.run.smithi148.stderr: 43450K .......... .......... .......... .......... .......... 25% 22.2M 5s 2024-03-29T04:11:48.386 INFO:teuthology.orchestra.run.smithi148.stderr: 43500K .......... .......... .......... .......... .......... 25% 25.6M 5s 2024-03-29T04:11:48.388 INFO:teuthology.orchestra.run.smithi148.stderr: 43550K .......... .......... .......... .......... .......... 25% 25.2M 5s 2024-03-29T04:11:48.390 INFO:teuthology.orchestra.run.smithi148.stderr: 43600K .......... .......... .......... .......... .......... 25% 25.0M 5s 2024-03-29T04:11:48.392 INFO:teuthology.orchestra.run.smithi148.stderr: 43650K .......... .......... .......... .......... .......... 25% 21.7M 5s 2024-03-29T04:11:48.394 INFO:teuthology.orchestra.run.smithi148.stderr: 43700K .......... .......... .......... .......... .......... 25% 26.9M 5s 2024-03-29T04:11:48.396 INFO:teuthology.orchestra.run.smithi148.stderr: 43750K .......... .......... .......... .......... .......... 25% 28.3M 5s 2024-03-29T04:11:48.398 INFO:teuthology.orchestra.run.smithi148.stderr: 43800K .......... .......... .......... .......... .......... 26% 29.7M 5s 2024-03-29T04:11:48.400 INFO:teuthology.orchestra.run.smithi148.stderr: 43850K .......... .......... .......... .......... .......... 26% 25.7M 5s 2024-03-29T04:11:48.401 INFO:teuthology.orchestra.run.smithi148.stderr: 43900K .......... .......... .......... .......... .......... 26% 29.7M 5s 2024-03-29T04:11:48.403 INFO:teuthology.orchestra.run.smithi148.stderr: 43950K .......... .......... .......... .......... .......... 26% 28.9M 5s 2024-03-29T04:11:48.405 INFO:teuthology.orchestra.run.smithi148.stderr: 44000K .......... .......... .......... .......... .......... 26% 29.1M 5s 2024-03-29T04:11:48.406 INFO:teuthology.orchestra.run.smithi148.stderr: 44050K .......... .......... .......... .......... .......... 26% 25.1M 5s 2024-03-29T04:11:48.408 INFO:teuthology.orchestra.run.smithi148.stderr: 44100K .......... .......... .......... .......... .......... 26% 28.6M 5s 2024-03-29T04:11:48.410 INFO:teuthology.orchestra.run.smithi148.stderr: 44150K .......... .......... .......... .......... .......... 26% 29.1M 5s 2024-03-29T04:11:48.412 INFO:teuthology.orchestra.run.smithi148.stderr: 44200K .......... .......... .......... .......... .......... 26% 27.2M 5s 2024-03-29T04:11:48.414 INFO:teuthology.orchestra.run.smithi148.stderr: 44250K .......... .......... .......... .......... .......... 26% 23.0M 5s 2024-03-29T04:11:48.416 INFO:teuthology.orchestra.run.smithi148.stderr: 44300K .......... .......... .......... .......... .......... 26% 26.9M 5s 2024-03-29T04:11:48.417 INFO:teuthology.orchestra.run.smithi148.stderr: 44350K .......... .......... .......... .......... .......... 26% 25.9M 5s 2024-03-29T04:11:48.419 INFO:teuthology.orchestra.run.smithi148.stderr: 44400K .......... .......... .......... .......... .......... 26% 27.0M 5s 2024-03-29T04:11:48.422 INFO:teuthology.orchestra.run.smithi148.stderr: 44450K .......... .......... .......... .......... .......... 26% 21.4M 5s 2024-03-29T04:11:48.423 INFO:teuthology.orchestra.run.smithi148.stderr: 44500K .......... .......... .......... .......... .......... 26% 26.4M 5s 2024-03-29T04:11:48.425 INFO:teuthology.orchestra.run.smithi148.stderr: 44550K .......... .......... .......... .......... .......... 26% 25.2M 5s 2024-03-29T04:11:48.427 INFO:teuthology.orchestra.run.smithi148.stderr: 44600K .......... .......... .......... .......... .......... 26% 29.7M 5s 2024-03-29T04:11:48.429 INFO:teuthology.orchestra.run.smithi148.stderr: 44650K .......... .......... .......... .......... .......... 26% 25.3M 5s 2024-03-29T04:11:48.430 INFO:teuthology.orchestra.run.smithi148.stderr: 44700K .......... .......... .......... .......... .......... 26% 28.9M 5s 2024-03-29T04:11:48.432 INFO:teuthology.orchestra.run.smithi148.stderr: 44750K .......... .......... .......... .......... .......... 26% 29.2M 5s 2024-03-29T04:11:48.434 INFO:teuthology.orchestra.run.smithi148.stderr: 44800K .......... .......... .......... .......... .......... 26% 28.7M 5s 2024-03-29T04:11:48.436 INFO:teuthology.orchestra.run.smithi148.stderr: 44850K .......... .......... .......... .......... .......... 26% 25.0M 5s 2024-03-29T04:11:48.438 INFO:teuthology.orchestra.run.smithi148.stderr: 44900K .......... .......... .......... .......... .......... 26% 28.5M 5s 2024-03-29T04:11:48.439 INFO:teuthology.orchestra.run.smithi148.stderr: 44950K .......... .......... .......... .......... .......... 26% 29.4M 5s 2024-03-29T04:11:48.441 INFO:teuthology.orchestra.run.smithi148.stderr: 45000K .......... .......... .......... .......... .......... 26% 29.7M 5s 2024-03-29T04:11:48.443 INFO:teuthology.orchestra.run.smithi148.stderr: 45050K .......... .......... .......... .......... .......... 26% 25.1M 5s 2024-03-29T04:11:48.445 INFO:teuthology.orchestra.run.smithi148.stderr: 45100K .......... .......... .......... .......... .......... 26% 33.8M 5s 2024-03-29T04:11:48.447 INFO:teuthology.orchestra.run.smithi148.stderr: 45150K .......... .......... .......... .......... .......... 26% 29.5M 5s 2024-03-29T04:11:48.447 INFO:teuthology.orchestra.run.smithi148.stderr: 45200K .......... .......... .......... .......... .......... 26% 36.6M 5s 2024-03-29T04:11:48.449 INFO:teuthology.orchestra.run.smithi148.stderr: 45250K .......... .......... .......... .......... .......... 26% 27.3M 5s 2024-03-29T04:11:48.451 INFO:teuthology.orchestra.run.smithi148.stderr: 45300K .......... .......... .......... .......... .......... 26% 23.8M 5s 2024-03-29T04:11:48.453 INFO:teuthology.orchestra.run.smithi148.stderr: 45350K .......... .......... .......... .......... .......... 26% 24.2M 5s 2024-03-29T04:11:48.455 INFO:teuthology.orchestra.run.smithi148.stderr: 45400K .......... .......... .......... .......... .......... 26% 25.6M 5s 2024-03-29T04:11:48.457 INFO:teuthology.orchestra.run.smithi148.stderr: 45450K .......... .......... .......... .......... .......... 26% 25.0M 5s 2024-03-29T04:11:48.459 INFO:teuthology.orchestra.run.smithi148.stderr: 45500K .......... .......... .......... .......... .......... 27% 27.2M 5s 2024-03-29T04:11:48.461 INFO:teuthology.orchestra.run.smithi148.stderr: 45550K .......... .......... .......... .......... .......... 27% 31.9M 5s 2024-03-29T04:11:48.462 INFO:teuthology.orchestra.run.smithi148.stderr: 45600K .......... .......... .......... .......... .......... 27% 39.4M 5s 2024-03-29T04:11:48.463 INFO:teuthology.orchestra.run.smithi148.stderr: 45650K .......... .......... .......... .......... .......... 27% 34.1M 5s 2024-03-29T04:11:48.464 INFO:teuthology.orchestra.run.smithi148.stderr: 45700K .......... .......... .......... .......... .......... 27% 40.5M 5s 2024-03-29T04:11:48.465 INFO:teuthology.orchestra.run.smithi148.stderr: 45750K .......... .......... .......... .......... .......... 27% 39.7M 5s 2024-03-29T04:11:48.467 INFO:teuthology.orchestra.run.smithi148.stderr: 45800K .......... .......... .......... .......... .......... 27% 40.2M 5s 2024-03-29T04:11:48.468 INFO:teuthology.orchestra.run.smithi148.stderr: 45850K .......... .......... .......... .......... .......... 27% 34.8M 5s 2024-03-29T04:11:48.469 INFO:teuthology.orchestra.run.smithi148.stderr: 45900K .......... .......... .......... .......... .......... 27% 39.5M 5s 2024-03-29T04:11:48.471 INFO:teuthology.orchestra.run.smithi148.stderr: 45950K .......... .......... .......... .......... .......... 27% 40.3M 5s 2024-03-29T04:11:48.472 INFO:teuthology.orchestra.run.smithi148.stderr: 46000K .......... .......... .......... .......... .......... 27% 40.2M 5s 2024-03-29T04:11:48.473 INFO:teuthology.orchestra.run.smithi148.stderr: 46050K .......... .......... .......... .......... .......... 27% 34.1M 5s 2024-03-29T04:11:48.474 INFO:teuthology.orchestra.run.smithi148.stderr: 46100K .......... .......... .......... .......... .......... 27% 38.0M 5s 2024-03-29T04:11:48.476 INFO:teuthology.orchestra.run.smithi148.stderr: 46150K .......... .......... .......... .......... .......... 27% 36.4M 5s 2024-03-29T04:11:48.477 INFO:teuthology.orchestra.run.smithi148.stderr: 46200K .......... .......... .......... .......... .......... 27% 38.5M 5s 2024-03-29T04:11:48.479 INFO:teuthology.orchestra.run.smithi148.stderr: 46250K .......... .......... .......... .......... .......... 27% 32.3M 5s 2024-03-29T04:11:48.480 INFO:teuthology.orchestra.run.smithi148.stderr: 46300K .......... .......... .......... .......... .......... 27% 37.5M 5s 2024-03-29T04:11:48.481 INFO:teuthology.orchestra.run.smithi148.stderr: 46350K .......... .......... .......... .......... .......... 27% 37.5M 5s 2024-03-29T04:11:48.483 INFO:teuthology.orchestra.run.smithi148.stderr: 46400K .......... .......... .......... .......... .......... 27% 36.0M 5s 2024-03-29T04:11:48.484 INFO:teuthology.orchestra.run.smithi148.stderr: 46450K .......... .......... .......... .......... .......... 27% 31.6M 5s 2024-03-29T04:11:48.485 INFO:teuthology.orchestra.run.smithi148.stderr: 46500K .......... .......... .......... .......... .......... 27% 36.8M 5s 2024-03-29T04:11:48.487 INFO:teuthology.orchestra.run.smithi148.stderr: 46550K .......... .......... .......... .......... .......... 27% 36.4M 5s 2024-03-29T04:11:48.488 INFO:teuthology.orchestra.run.smithi148.stderr: 46600K .......... .......... .......... .......... .......... 27% 36.3M 5s 2024-03-29T04:11:48.490 INFO:teuthology.orchestra.run.smithi148.stderr: 46650K .......... .......... .......... .......... .......... 27% 30.6M 5s 2024-03-29T04:11:48.491 INFO:teuthology.orchestra.run.smithi148.stderr: 46700K .......... .......... .......... .......... .......... 27% 36.4M 5s 2024-03-29T04:11:48.492 INFO:teuthology.orchestra.run.smithi148.stderr: 46750K .......... .......... .......... .......... .......... 27% 35.8M 5s 2024-03-29T04:11:48.494 INFO:teuthology.orchestra.run.smithi148.stderr: 46800K .......... .......... .......... .......... .......... 27% 35.7M 5s 2024-03-29T04:11:48.495 INFO:teuthology.orchestra.run.smithi148.stderr: 46850K .......... .......... .......... .......... .......... 27% 29.7M 5s 2024-03-29T04:11:48.497 INFO:teuthology.orchestra.run.smithi148.stderr: 46900K .......... .......... .......... .......... .......... 27% 35.1M 5s 2024-03-29T04:11:48.498 INFO:teuthology.orchestra.run.smithi148.stderr: 46950K .......... .......... .......... .......... .......... 27% 34.9M 5s 2024-03-29T04:11:48.500 INFO:teuthology.orchestra.run.smithi148.stderr: 47000K .......... .......... .......... .......... .......... 27% 34.2M 5s 2024-03-29T04:11:48.501 INFO:teuthology.orchestra.run.smithi148.stderr: 47050K .......... .......... .......... .......... .......... 27% 29.9M 5s 2024-03-29T04:11:48.503 INFO:teuthology.orchestra.run.smithi148.stderr: 47100K .......... .......... .......... .......... .......... 27% 33.7M 5s 2024-03-29T04:11:48.504 INFO:teuthology.orchestra.run.smithi148.stderr: 47150K .......... .......... .......... .......... .......... 28% 34.3M 5s 2024-03-29T04:11:48.506 INFO:teuthology.orchestra.run.smithi148.stderr: 47200K .......... .......... .......... .......... .......... 28% 33.6M 5s 2024-03-29T04:11:48.507 INFO:teuthology.orchestra.run.smithi148.stderr: 47250K .......... .......... .......... .......... .......... 28% 28.9M 5s 2024-03-29T04:11:48.509 INFO:teuthology.orchestra.run.smithi148.stderr: 47300K .......... .......... .......... .......... .......... 28% 33.2M 5s 2024-03-29T04:11:48.510 INFO:teuthology.orchestra.run.smithi148.stderr: 47350K .......... .......... .......... .......... .......... 28% 33.2M 5s 2024-03-29T04:11:48.512 INFO:teuthology.orchestra.run.smithi148.stderr: 47400K .......... .......... .......... .......... .......... 28% 33.1M 5s 2024-03-29T04:11:48.514 INFO:teuthology.orchestra.run.smithi148.stderr: 47450K .......... .......... .......... .......... .......... 28% 28.4M 5s 2024-03-29T04:11:48.515 INFO:teuthology.orchestra.run.smithi148.stderr: 47500K .......... .......... .......... .......... .......... 28% 32.7M 5s 2024-03-29T04:11:48.516 INFO:teuthology.orchestra.run.smithi148.stderr: 47550K .......... .......... .......... .......... .......... 28% 32.3M 5s 2024-03-29T04:11:48.518 INFO:teuthology.orchestra.run.smithi148.stderr: 47600K .......... .......... .......... .......... .......... 28% 28.3M 5s 2024-03-29T04:11:48.521 INFO:teuthology.orchestra.run.smithi148.stderr: 47650K .......... .......... .......... .......... .......... 28% 21.1M 5s 2024-03-29T04:11:48.523 INFO:teuthology.orchestra.run.smithi148.stderr: 47700K .......... .......... .......... .......... .......... 28% 23.9M 5s 2024-03-29T04:11:48.525 INFO:teuthology.orchestra.run.smithi148.stderr: 47750K .......... .......... .......... .......... .......... 28% 24.4M 5s 2024-03-29T04:11:48.527 INFO:teuthology.orchestra.run.smithi148.stderr: 47800K .......... .......... .......... .......... .......... 28% 24.0M 5s 2024-03-29T04:11:48.529 INFO:teuthology.orchestra.run.smithi148.stderr: 47850K .......... .......... .......... .......... .......... 28% 20.8M 5s 2024-03-29T04:11:48.531 INFO:teuthology.orchestra.run.smithi148.stderr: 47900K .......... .......... .......... .......... .......... 28% 24.0M 5s 2024-03-29T04:11:48.533 INFO:teuthology.orchestra.run.smithi148.stderr: 47950K .......... .......... .......... .......... .......... 28% 23.6M 5s 2024-03-29T04:11:48.535 INFO:teuthology.orchestra.run.smithi148.stderr: 48000K .......... .......... .......... .......... .......... 28% 24.2M 5s 2024-03-29T04:11:48.537 INFO:teuthology.orchestra.run.smithi148.stderr: 48050K .......... .......... .......... .......... .......... 28% 20.6M 5s 2024-03-29T04:11:48.542 INFO:teuthology.orchestra.run.smithi148.stderr: 48100K .......... .......... .......... .......... .......... 28% 23.0M 5s 2024-03-29T04:11:48.542 INFO:teuthology.orchestra.run.smithi148.stderr: 48150K .......... .......... .......... .......... .......... 28% 25.1M 5s 2024-03-29T04:11:48.544 INFO:teuthology.orchestra.run.smithi148.stderr: 48200K .......... .......... .......... .......... .......... 28% 25.7M 5s 2024-03-29T04:11:48.545 INFO:teuthology.orchestra.run.smithi148.stderr: 48250K .......... .......... .......... .......... .......... 28% 22.6M 5s 2024-03-29T04:11:48.547 INFO:teuthology.orchestra.run.smithi148.stderr: 48300K .......... .......... .......... .......... .......... 28% 25.5M 5s 2024-03-29T04:11:48.549 INFO:teuthology.orchestra.run.smithi148.stderr: 48350K .......... .......... .......... .......... .......... 28% 26.0M 5s 2024-03-29T04:11:48.551 INFO:teuthology.orchestra.run.smithi148.stderr: 48400K .......... .......... .......... .......... .......... 28% 25.4M 5s 2024-03-29T04:11:48.554 INFO:teuthology.orchestra.run.smithi148.stderr: 48450K .......... .......... .......... .......... .......... 28% 21.8M 5s 2024-03-29T04:11:48.555 INFO:teuthology.orchestra.run.smithi148.stderr: 48500K .......... .......... .......... .......... .......... 28% 25.7M 5s 2024-03-29T04:11:48.558 INFO:teuthology.orchestra.run.smithi148.stderr: 48550K .......... .......... .......... .......... .......... 28% 26.3M 5s 2024-03-29T04:11:48.559 INFO:teuthology.orchestra.run.smithi148.stderr: 48600K .......... .......... .......... .......... .......... 28% 25.9M 5s 2024-03-29T04:11:48.561 INFO:teuthology.orchestra.run.smithi148.stderr: 48650K .......... .......... .......... .......... .......... 28% 21.9M 5s 2024-03-29T04:11:48.563 INFO:teuthology.orchestra.run.smithi148.stderr: 48700K .......... .......... .......... .......... .......... 28% 24.7M 5s 2024-03-29T04:11:48.565 INFO:teuthology.orchestra.run.smithi148.stderr: 48750K .......... .......... .......... .......... .......... 28% 25.2M 5s 2024-03-29T04:11:48.567 INFO:teuthology.orchestra.run.smithi148.stderr: 48800K .......... .......... .......... .......... .......... 28% 25.4M 5s 2024-03-29T04:11:48.569 INFO:teuthology.orchestra.run.smithi148.stderr: 48850K .......... .......... .......... .......... .......... 29% 24.7M 5s 2024-03-29T04:11:48.571 INFO:teuthology.orchestra.run.smithi148.stderr: 48900K .......... .......... .......... .......... .......... 29% 28.7M 5s 2024-03-29T04:11:48.573 INFO:teuthology.orchestra.run.smithi148.stderr: 48950K .......... .......... .......... .......... .......... 29% 26.7M 5s 2024-03-29T04:11:48.575 INFO:teuthology.orchestra.run.smithi148.stderr: 49000K .......... .......... .......... .......... .......... 29% 28.0M 5s 2024-03-29T04:11:48.577 INFO:teuthology.orchestra.run.smithi148.stderr: 49050K .......... .......... .......... .......... .......... 29% 17.1M 5s 2024-03-29T04:11:48.580 INFO:teuthology.orchestra.run.smithi148.stderr: 49100K .......... .......... .......... .......... .......... 29% 21.7M 5s 2024-03-29T04:11:48.582 INFO:teuthology.orchestra.run.smithi148.stderr: 49150K .......... .......... .......... .......... .......... 29% 22.1M 5s 2024-03-29T04:11:48.584 INFO:teuthology.orchestra.run.smithi148.stderr: 49200K .......... .......... .......... .......... .......... 29% 25.6M 5s 2024-03-29T04:11:48.585 INFO:teuthology.orchestra.run.smithi148.stderr: 49250K .......... .......... .......... .......... .......... 29% 32.8M 5s 2024-03-29T04:11:48.586 INFO:teuthology.orchestra.run.smithi148.stderr: 49300K .......... .......... .......... .......... .......... 29% 36.4M 4s 2024-03-29T04:11:48.588 INFO:teuthology.orchestra.run.smithi148.stderr: 49350K .......... .......... .......... .......... .......... 29% 36.7M 4s 2024-03-29T04:11:48.590 INFO:teuthology.orchestra.run.smithi148.stderr: 49400K .......... .......... .......... .......... .......... 29% 25.8M 4s 2024-03-29T04:11:48.592 INFO:teuthology.orchestra.run.smithi148.stderr: 49450K .......... .......... .......... .......... .......... 29% 23.2M 4s 2024-03-29T04:11:48.594 INFO:teuthology.orchestra.run.smithi148.stderr: 49500K .......... .......... .......... .......... .......... 29% 29.1M 4s 2024-03-29T04:11:48.595 INFO:teuthology.orchestra.run.smithi148.stderr: 49550K .......... .......... .......... .......... .......... 29% 33.7M 4s 2024-03-29T04:11:48.596 INFO:teuthology.orchestra.run.smithi148.stderr: 49600K .......... .......... .......... .......... .......... 29% 38.6M 4s 2024-03-29T04:11:48.598 INFO:teuthology.orchestra.run.smithi148.stderr: 49650K .......... .......... .......... .......... .......... 29% 32.4M 4s 2024-03-29T04:11:48.599 INFO:teuthology.orchestra.run.smithi148.stderr: 49700K .......... .......... .......... .......... .......... 29% 38.0M 4s 2024-03-29T04:11:48.601 INFO:teuthology.orchestra.run.smithi148.stderr: 49750K .......... .......... .......... .......... .......... 29% 31.8M 4s 2024-03-29T04:11:48.602 INFO:teuthology.orchestra.run.smithi148.stderr: 49800K .......... .......... .......... .......... .......... 29% 33.5M 4s 2024-03-29T04:11:48.604 INFO:teuthology.orchestra.run.smithi148.stderr: 49850K .......... .......... .......... .......... .......... 29% 29.4M 4s 2024-03-29T04:11:48.605 INFO:teuthology.orchestra.run.smithi148.stderr: 49900K .......... .......... .......... .......... .......... 29% 37.9M 4s 2024-03-29T04:11:48.606 INFO:teuthology.orchestra.run.smithi148.stderr: 49950K .......... .......... .......... .......... .......... 29% 37.7M 4s 2024-03-29T04:11:48.608 INFO:teuthology.orchestra.run.smithi148.stderr: 50000K .......... .......... .......... .......... .......... 29% 37.5M 4s 2024-03-29T04:11:48.609 INFO:teuthology.orchestra.run.smithi148.stderr: 50050K .......... .......... .......... .......... .......... 29% 32.6M 4s 2024-03-29T04:11:48.610 INFO:teuthology.orchestra.run.smithi148.stderr: 50100K .......... .......... .......... .......... .......... 29% 37.1M 4s 2024-03-29T04:11:48.612 INFO:teuthology.orchestra.run.smithi148.stderr: 50150K .......... .......... .......... .......... .......... 29% 37.0M 4s 2024-03-29T04:11:48.613 INFO:teuthology.orchestra.run.smithi148.stderr: 50200K .......... .......... .......... .......... .......... 29% 36.9M 4s 2024-03-29T04:11:48.615 INFO:teuthology.orchestra.run.smithi148.stderr: 50250K .......... .......... .......... .......... .......... 29% 26.0M 4s 2024-03-29T04:11:48.617 INFO:teuthology.orchestra.run.smithi148.stderr: 50300K .......... .......... .......... .......... .......... 29% 28.5M 4s 2024-03-29T04:11:48.618 INFO:teuthology.orchestra.run.smithi148.stderr: 50350K .......... .......... .......... .......... .......... 29% 28.0M 4s 2024-03-29T04:11:48.620 INFO:teuthology.orchestra.run.smithi148.stderr: 50400K .......... .......... .......... .......... .......... 29% 28.8M 4s 2024-03-29T04:11:48.622 INFO:teuthology.orchestra.run.smithi148.stderr: 50450K .......... .......... .......... .......... .......... 29% 24.2M 4s 2024-03-29T04:11:48.624 INFO:teuthology.orchestra.run.smithi148.stderr: 50500K .......... .......... .......... .......... .......... 29% 29.1M 4s 2024-03-29T04:11:48.625 INFO:teuthology.orchestra.run.smithi148.stderr: 50550K .......... .......... .......... .......... .......... 30% 28.2M 4s 2024-03-29T04:11:48.627 INFO:teuthology.orchestra.run.smithi148.stderr: 50600K .......... .......... .......... .......... .......... 30% 26.9M 4s 2024-03-29T04:11:48.629 INFO:teuthology.orchestra.run.smithi148.stderr: 50650K .......... .......... .......... .......... .......... 30% 23.8M 4s 2024-03-29T04:11:48.631 INFO:teuthology.orchestra.run.smithi148.stderr: 50700K .......... .......... .......... .......... .......... 30% 31.3M 4s 2024-03-29T04:11:48.632 INFO:teuthology.orchestra.run.smithi148.stderr: 50750K .......... .......... .......... .......... .......... 30% 31.4M 4s 2024-03-29T04:11:48.634 INFO:teuthology.orchestra.run.smithi148.stderr: 50800K .......... .......... .......... .......... .......... 30% 31.8M 4s 2024-03-29T04:11:48.636 INFO:teuthology.orchestra.run.smithi148.stderr: 50850K .......... .......... .......... .......... .......... 30% 23.3M 4s 2024-03-29T04:11:48.638 INFO:teuthology.orchestra.run.smithi148.stderr: 50900K .......... .......... .......... .......... .......... 30% 31.5M 4s 2024-03-29T04:11:48.639 INFO:teuthology.orchestra.run.smithi148.stderr: 50950K .......... .......... .......... .......... .......... 30% 32.5M 4s 2024-03-29T04:11:48.641 INFO:teuthology.orchestra.run.smithi148.stderr: 51000K .......... .......... .......... .......... .......... 30% 31.8M 4s 2024-03-29T04:11:48.642 INFO:teuthology.orchestra.run.smithi148.stderr: 51050K .......... .......... .......... .......... .......... 30% 27.5M 4s 2024-03-29T04:11:48.644 INFO:teuthology.orchestra.run.smithi148.stderr: 51100K .......... .......... .......... .......... .......... 30% 30.5M 4s 2024-03-29T04:11:48.646 INFO:teuthology.orchestra.run.smithi148.stderr: 51150K .......... .......... .......... .......... .......... 30% 30.5M 4s 2024-03-29T04:11:48.647 INFO:teuthology.orchestra.run.smithi148.stderr: 51200K .......... .......... .......... .......... .......... 30% 31.3M 4s 2024-03-29T04:11:48.649 INFO:teuthology.orchestra.run.smithi148.stderr: 51250K .......... .......... .......... .......... .......... 30% 27.4M 4s 2024-03-29T04:11:48.651 INFO:teuthology.orchestra.run.smithi148.stderr: 51300K .......... .......... .......... .......... .......... 30% 31.4M 4s 2024-03-29T04:11:48.652 INFO:teuthology.orchestra.run.smithi148.stderr: 51350K .......... .......... .......... .......... .......... 30% 31.5M 4s 2024-03-29T04:11:48.654 INFO:teuthology.orchestra.run.smithi148.stderr: 51400K .......... .......... .......... .......... .......... 30% 30.7M 4s 2024-03-29T04:11:48.656 INFO:teuthology.orchestra.run.smithi148.stderr: 51450K .......... .......... .......... .......... .......... 30% 24.4M 4s 2024-03-29T04:11:48.657 INFO:teuthology.orchestra.run.smithi148.stderr: 51500K .......... .......... .......... .......... .......... 30% 29.9M 4s 2024-03-29T04:11:48.659 INFO:teuthology.orchestra.run.smithi148.stderr: 51550K .......... .......... .......... .......... .......... 30% 29.5M 4s 2024-03-29T04:11:48.661 INFO:teuthology.orchestra.run.smithi148.stderr: 51600K .......... .......... .......... .......... .......... 30% 29.3M 4s 2024-03-29T04:11:48.663 INFO:teuthology.orchestra.run.smithi148.stderr: 51650K .......... .......... .......... .......... .......... 30% 25.2M 4s 2024-03-29T04:11:48.664 INFO:teuthology.orchestra.run.smithi148.stderr: 51700K .......... .......... .......... .......... .......... 30% 29.3M 4s 2024-03-29T04:11:48.666 INFO:teuthology.orchestra.run.smithi148.stderr: 51750K .......... .......... .......... .......... .......... 30% 29.1M 4s 2024-03-29T04:11:48.668 INFO:teuthology.orchestra.run.smithi148.stderr: 51800K .......... .......... .......... .......... .......... 30% 27.9M 4s 2024-03-29T04:11:48.670 INFO:teuthology.orchestra.run.smithi148.stderr: 51850K .......... .......... .......... .......... .......... 30% 24.0M 4s 2024-03-29T04:11:48.671 INFO:teuthology.orchestra.run.smithi148.stderr: 51900K .......... .......... .......... .......... .......... 30% 27.3M 4s 2024-03-29T04:11:48.673 INFO:teuthology.orchestra.run.smithi148.stderr: 51950K .......... .......... .......... .......... .......... 30% 28.9M 4s 2024-03-29T04:11:48.675 INFO:teuthology.orchestra.run.smithi148.stderr: 52000K .......... .......... .......... .......... .......... 30% 28.2M 4s 2024-03-29T04:11:48.677 INFO:teuthology.orchestra.run.smithi148.stderr: 52050K .......... .......... .......... .......... .......... 30% 25.2M 4s 2024-03-29T04:11:48.679 INFO:teuthology.orchestra.run.smithi148.stderr: 52100K .......... .......... .......... .......... .......... 30% 26.0M 4s 2024-03-29T04:11:48.681 INFO:teuthology.orchestra.run.smithi148.stderr: 52150K .......... .......... .......... .......... .......... 30% 26.4M 4s 2024-03-29T04:11:48.683 INFO:teuthology.orchestra.run.smithi148.stderr: 52200K .......... .......... .......... .......... .......... 30% 26.0M 4s 2024-03-29T04:11:48.684 INFO:teuthology.orchestra.run.smithi148.stderr: 52250K .......... .......... .......... .......... .......... 31% 22.7M 4s 2024-03-29T04:11:48.686 INFO:teuthology.orchestra.run.smithi148.stderr: 52300K .......... .......... .......... .......... .......... 31% 26.9M 4s 2024-03-29T04:11:48.688 INFO:teuthology.orchestra.run.smithi148.stderr: 52350K .......... .......... .......... .......... .......... 31% 26.9M 4s 2024-03-29T04:11:48.690 INFO:teuthology.orchestra.run.smithi148.stderr: 52400K .......... .......... .......... .......... .......... 31% 26.0M 4s 2024-03-29T04:11:48.692 INFO:teuthology.orchestra.run.smithi148.stderr: 52450K .......... .......... .......... .......... .......... 31% 22.9M 4s 2024-03-29T04:11:48.694 INFO:teuthology.orchestra.run.smithi148.stderr: 52500K .......... .......... .......... .......... .......... 31% 26.5M 4s 2024-03-29T04:11:48.696 INFO:teuthology.orchestra.run.smithi148.stderr: 52550K .......... .......... .......... .......... .......... 31% 26.2M 4s 2024-03-29T04:11:48.698 INFO:teuthology.orchestra.run.smithi148.stderr: 52600K .......... .......... .......... .......... .......... 31% 26.6M 4s 2024-03-29T04:11:48.700 INFO:teuthology.orchestra.run.smithi148.stderr: 52650K .......... .......... .......... .......... .......... 31% 22.5M 4s 2024-03-29T04:11:48.702 INFO:teuthology.orchestra.run.smithi148.stderr: 52700K .......... .......... .......... .......... .......... 31% 27.3M 4s 2024-03-29T04:11:48.704 INFO:teuthology.orchestra.run.smithi148.stderr: 52750K .......... .......... .......... .......... .......... 31% 26.0M 4s 2024-03-29T04:11:48.706 INFO:teuthology.orchestra.run.smithi148.stderr: 52800K .......... .......... .......... .......... .......... 31% 28.0M 4s 2024-03-29T04:11:48.708 INFO:teuthology.orchestra.run.smithi148.stderr: 52850K .......... .......... .......... .......... .......... 31% 21.9M 4s 2024-03-29T04:11:48.709 INFO:teuthology.orchestra.run.smithi148.stderr: 52900K .......... .......... .......... .......... .......... 31% 25.1M 4s 2024-03-29T04:11:48.712 INFO:teuthology.orchestra.run.smithi148.stderr: 52950K .......... .......... .......... .......... .......... 31% 26.3M 4s 2024-03-29T04:11:48.714 INFO:teuthology.orchestra.run.smithi148.stderr: 53000K .......... .......... .......... .......... .......... 31% 24.1M 4s 2024-03-29T04:11:48.716 INFO:teuthology.orchestra.run.smithi148.stderr: 53050K .......... .......... .......... .......... .......... 31% 21.2M 4s 2024-03-29T04:11:48.718 INFO:teuthology.orchestra.run.smithi148.stderr: 53100K .......... .......... .......... .......... .......... 31% 25.7M 4s 2024-03-29T04:11:48.720 INFO:teuthology.orchestra.run.smithi148.stderr: 53150K .......... .......... .......... .......... .......... 31% 24.1M 4s 2024-03-29T04:11:48.722 INFO:teuthology.orchestra.run.smithi148.stderr: 53200K .......... .......... .......... .......... .......... 31% 25.7M 4s 2024-03-29T04:11:48.724 INFO:teuthology.orchestra.run.smithi148.stderr: 53250K .......... .......... .......... .......... .......... 31% 18.2M 4s 2024-03-29T04:11:48.726 INFO:teuthology.orchestra.run.smithi148.stderr: 53300K .......... .......... .......... .......... .......... 31% 25.5M 4s 2024-03-29T04:11:48.728 INFO:teuthology.orchestra.run.smithi148.stderr: 53350K .......... .......... .......... .......... .......... 31% 32.4M 4s 2024-03-29T04:11:48.729 INFO:teuthology.orchestra.run.smithi148.stderr: 53400K .......... .......... .......... .......... .......... 31% 40.1M 4s 2024-03-29T04:11:48.730 INFO:teuthology.orchestra.run.smithi148.stderr: 53450K .......... .......... .......... .......... .......... 31% 33.8M 4s 2024-03-29T04:11:48.732 INFO:teuthology.orchestra.run.smithi148.stderr: 53500K .......... .......... .......... .......... .......... 31% 39.3M 4s 2024-03-29T04:11:48.733 INFO:teuthology.orchestra.run.smithi148.stderr: 53550K .......... .......... .......... .......... .......... 31% 39.3M 4s 2024-03-29T04:11:48.734 INFO:teuthology.orchestra.run.smithi148.stderr: 53600K .......... .......... .......... .......... .......... 31% 40.6M 4s 2024-03-29T04:11:48.735 INFO:teuthology.orchestra.run.smithi148.stderr: 53650K .......... .......... .......... .......... .......... 31% 34.4M 4s 2024-03-29T04:11:48.737 INFO:teuthology.orchestra.run.smithi148.stderr: 53700K .......... .......... .......... .......... .......... 31% 40.6M 4s 2024-03-29T04:11:48.738 INFO:teuthology.orchestra.run.smithi148.stderr: 53750K .......... .......... .......... .......... .......... 31% 37.8M 4s 2024-03-29T04:11:48.739 INFO:teuthology.orchestra.run.smithi148.stderr: 53800K .......... .......... .......... .......... .......... 31% 41.1M 4s 2024-03-29T04:11:48.741 INFO:teuthology.orchestra.run.smithi148.stderr: 53850K .......... .......... .......... .......... .......... 31% 36.0M 4s 2024-03-29T04:11:48.742 INFO:teuthology.orchestra.run.smithi148.stderr: 53900K .......... .......... .......... .......... .......... 32% 41.8M 4s 2024-03-29T04:11:48.745 INFO:teuthology.orchestra.run.smithi148.stderr: 53950K .......... .......... .......... .......... .......... 32% 41.7M 4s 2024-03-29T04:11:48.748 INFO:teuthology.orchestra.run.smithi148.stderr: 54000K .......... .......... .......... .......... .......... 32% 41.9M 4s 2024-03-29T04:11:48.748 INFO:teuthology.orchestra.run.smithi148.stderr: 54050K .......... .......... .......... .......... .......... 32% 35.7M 4s 2024-03-29T04:11:48.748 INFO:teuthology.orchestra.run.smithi148.stderr: 54100K .......... .......... .......... .......... .......... 32% 40.9M 4s 2024-03-29T04:11:48.748 INFO:teuthology.orchestra.run.smithi148.stderr: 54150K .......... .......... .......... .......... .......... 32% 41.0M 4s 2024-03-29T04:11:48.749 INFO:teuthology.orchestra.run.smithi148.stderr: 54200K .......... .......... .......... .......... .......... 32% 40.4M 4s 2024-03-29T04:11:48.750 INFO:teuthology.orchestra.run.smithi148.stderr: 54250K .......... .......... .......... .......... .......... 32% 34.2M 4s 2024-03-29T04:11:48.752 INFO:teuthology.orchestra.run.smithi148.stderr: 54300K .......... .......... .......... .......... .......... 32% 39.8M 4s 2024-03-29T04:11:48.753 INFO:teuthology.orchestra.run.smithi148.stderr: 54350K .......... .......... .......... .......... .......... 32% 40.5M 4s 2024-03-29T04:11:48.754 INFO:teuthology.orchestra.run.smithi148.stderr: 54400K .......... .......... .......... .......... .......... 32% 40.0M 4s 2024-03-29T04:11:48.755 INFO:teuthology.orchestra.run.smithi148.stderr: 54450K .......... .......... .......... .......... .......... 32% 34.1M 4s 2024-03-29T04:11:48.757 INFO:teuthology.orchestra.run.smithi148.stderr: 54500K .......... .......... .......... .......... .......... 32% 35.5M 4s 2024-03-29T04:11:48.758 INFO:teuthology.orchestra.run.smithi148.stderr: 54550K .......... .......... .......... .......... .......... 32% 36.5M 4s 2024-03-29T04:11:48.760 INFO:teuthology.orchestra.run.smithi148.stderr: 54600K .......... .......... .......... .......... .......... 32% 33.4M 4s 2024-03-29T04:11:48.761 INFO:teuthology.orchestra.run.smithi148.stderr: 54650K .......... .......... .......... .......... .......... 32% 29.7M 4s 2024-03-29T04:11:48.763 INFO:teuthology.orchestra.run.smithi148.stderr: 54700K .......... .......... .......... .......... .......... 32% 34.0M 4s 2024-03-29T04:11:48.764 INFO:teuthology.orchestra.run.smithi148.stderr: 54750K .......... .......... .......... .......... .......... 32% 35.1M 4s 2024-03-29T04:11:48.766 INFO:teuthology.orchestra.run.smithi148.stderr: 54800K .......... .......... .......... .......... .......... 32% 33.7M 4s 2024-03-29T04:11:48.767 INFO:teuthology.orchestra.run.smithi148.stderr: 54850K .......... .......... .......... .......... .......... 32% 28.4M 4s 2024-03-29T04:11:48.769 INFO:teuthology.orchestra.run.smithi148.stderr: 54900K .......... .......... .......... .......... .......... 32% 31.3M 4s 2024-03-29T04:11:48.770 INFO:teuthology.orchestra.run.smithi148.stderr: 54950K .......... .......... .......... .......... .......... 32% 33.6M 4s 2024-03-29T04:11:48.772 INFO:teuthology.orchestra.run.smithi148.stderr: 55000K .......... .......... .......... .......... .......... 32% 32.3M 4s 2024-03-29T04:11:48.774 INFO:teuthology.orchestra.run.smithi148.stderr: 55050K .......... .......... .......... .......... .......... 32% 23.2M 4s 2024-03-29T04:11:48.776 INFO:teuthology.orchestra.run.smithi148.stderr: 55100K .......... .......... .......... .......... .......... 32% 23.7M 4s 2024-03-29T04:11:48.777 INFO:teuthology.orchestra.run.smithi148.stderr: 55150K .......... .......... .......... .......... .......... 32% 34.7M 4s 2024-03-29T04:11:48.779 INFO:teuthology.orchestra.run.smithi148.stderr: 55200K .......... .......... .......... .......... .......... 32% 35.2M 4s 2024-03-29T04:11:48.780 INFO:teuthology.orchestra.run.smithi148.stderr: 55250K .......... .......... .......... .......... .......... 32% 29.7M 4s 2024-03-29T04:11:48.782 INFO:teuthology.orchestra.run.smithi148.stderr: 55300K .......... .......... .......... .......... .......... 32% 34.7M 4s 2024-03-29T04:11:48.783 INFO:teuthology.orchestra.run.smithi148.stderr: 55350K .......... .......... .......... .......... .......... 32% 33.6M 4s 2024-03-29T04:11:48.785 INFO:teuthology.orchestra.run.smithi148.stderr: 55400K .......... .......... .......... .......... .......... 32% 33.3M 4s 2024-03-29T04:11:48.786 INFO:teuthology.orchestra.run.smithi148.stderr: 55450K .......... .......... .......... .......... .......... 32% 29.0M 4s 2024-03-29T04:11:48.788 INFO:teuthology.orchestra.run.smithi148.stderr: 55500K .......... .......... .......... .......... .......... 32% 33.7M 4s 2024-03-29T04:11:48.789 INFO:teuthology.orchestra.run.smithi148.stderr: 55550K .......... .......... .......... .......... .......... 32% 33.7M 4s 2024-03-29T04:11:48.791 INFO:teuthology.orchestra.run.smithi148.stderr: 55600K .......... .......... .......... .......... .......... 33% 33.6M 4s 2024-03-29T04:11:48.792 INFO:teuthology.orchestra.run.smithi148.stderr: 55650K .......... .......... .......... .......... .......... 33% 28.7M 4s 2024-03-29T04:11:48.794 INFO:teuthology.orchestra.run.smithi148.stderr: 55700K .......... .......... .......... .......... .......... 33% 33.7M 4s 2024-03-29T04:11:48.795 INFO:teuthology.orchestra.run.smithi148.stderr: 55750K .......... .......... .......... .......... .......... 33% 33.3M 4s 2024-03-29T04:11:48.797 INFO:teuthology.orchestra.run.smithi148.stderr: 55800K .......... .......... .......... .......... .......... 33% 33.3M 4s 2024-03-29T04:11:48.799 INFO:teuthology.orchestra.run.smithi148.stderr: 55850K .......... .......... .......... .......... .......... 33% 28.1M 4s 2024-03-29T04:11:48.800 INFO:teuthology.orchestra.run.smithi148.stderr: 55900K .......... .......... .......... .......... .......... 33% 33.0M 4s 2024-03-29T04:11:48.802 INFO:teuthology.orchestra.run.smithi148.stderr: 55950K .......... .......... .......... .......... .......... 33% 32.4M 4s 2024-03-29T04:11:48.803 INFO:teuthology.orchestra.run.smithi148.stderr: 56000K .......... .......... .......... .......... .......... 33% 32.4M 4s 2024-03-29T04:11:48.805 INFO:teuthology.orchestra.run.smithi148.stderr: 56050K .......... .......... .......... .......... .......... 33% 27.3M 4s 2024-03-29T04:11:48.806 INFO:teuthology.orchestra.run.smithi148.stderr: 56100K .......... .......... .......... .......... .......... 33% 32.2M 4s 2024-03-29T04:11:48.808 INFO:teuthology.orchestra.run.smithi148.stderr: 56150K .......... .......... .......... .......... .......... 33% 32.1M 4s 2024-03-29T04:11:48.809 INFO:teuthology.orchestra.run.smithi148.stderr: 56200K .......... .......... .......... .......... .......... 33% 32.1M 4s 2024-03-29T04:11:48.811 INFO:teuthology.orchestra.run.smithi148.stderr: 56250K .......... .......... .......... .......... .......... 33% 27.4M 4s 2024-03-29T04:11:48.813 INFO:teuthology.orchestra.run.smithi148.stderr: 56300K .......... .......... .......... .......... .......... 33% 31.7M 4s 2024-03-29T04:11:48.814 INFO:teuthology.orchestra.run.smithi148.stderr: 56350K .......... .......... .......... .......... .......... 33% 32.1M 4s 2024-03-29T04:11:48.816 INFO:teuthology.orchestra.run.smithi148.stderr: 56400K .......... .......... .......... .......... .......... 33% 31.4M 4s 2024-03-29T04:11:48.818 INFO:teuthology.orchestra.run.smithi148.stderr: 56450K .......... .......... .......... .......... .......... 33% 26.4M 4s 2024-03-29T04:11:48.819 INFO:teuthology.orchestra.run.smithi148.stderr: 56500K .......... .......... .......... .......... .......... 33% 30.8M 4s 2024-03-29T04:11:48.821 INFO:teuthology.orchestra.run.smithi148.stderr: 56550K .......... .......... .......... .......... .......... 33% 31.3M 4s 2024-03-29T04:11:48.822 INFO:teuthology.orchestra.run.smithi148.stderr: 56600K .......... .......... .......... .......... .......... 33% 30.7M 4s 2024-03-29T04:11:48.824 INFO:teuthology.orchestra.run.smithi148.stderr: 56650K .......... .......... .......... .......... .......... 33% 26.4M 4s 2024-03-29T04:11:48.826 INFO:teuthology.orchestra.run.smithi148.stderr: 56700K .......... .......... .......... .......... .......... 33% 26.2M 4s 2024-03-29T04:11:48.828 INFO:teuthology.orchestra.run.smithi148.stderr: 56750K .......... .......... .......... .......... .......... 33% 27.4M 4s 2024-03-29T04:11:48.830 INFO:teuthology.orchestra.run.smithi148.stderr: 56800K .......... .......... .......... .......... .......... 33% 28.0M 4s 2024-03-29T04:11:48.831 INFO:teuthology.orchestra.run.smithi148.stderr: 56850K .......... .......... .......... .......... .......... 33% 26.0M 4s 2024-03-29T04:11:48.833 INFO:teuthology.orchestra.run.smithi148.stderr: 56900K .......... .......... .......... .......... .......... 33% 29.4M 4s 2024-03-29T04:11:48.835 INFO:teuthology.orchestra.run.smithi148.stderr: 56950K .......... .......... .......... .......... .......... 33% 29.6M 4s 2024-03-29T04:11:48.839 INFO:teuthology.orchestra.run.smithi148.stderr: 57000K .......... .......... .......... .......... .......... 33% 29.9M 4s 2024-03-29T04:11:48.851 INFO:teuthology.orchestra.run.smithi148.stderr: 57050K .......... .......... .......... .......... .......... 33% 25.3M 4s 2024-03-29T04:11:48.851 INFO:teuthology.orchestra.run.smithi148.stderr: 57100K .......... .......... .......... .......... .......... 33% 29.1M 4s 2024-03-29T04:11:48.851 INFO:teuthology.orchestra.run.smithi148.stderr: 57150K .......... .......... .......... .......... .......... 33% 29.9M 4s 2024-03-29T04:11:48.851 INFO:teuthology.orchestra.run.smithi148.stderr: 57200K .......... .......... .......... .......... .......... 33% 29.1M 4s 2024-03-29T04:11:48.851 INFO:teuthology.orchestra.run.smithi148.stderr: 57250K .......... .......... .......... .......... .......... 33% 24.6M 4s 2024-03-29T04:11:48.851 INFO:teuthology.orchestra.run.smithi148.stderr: 57300K .......... .......... .......... .......... .......... 34% 29.4M 4s 2024-03-29T04:11:48.851 INFO:teuthology.orchestra.run.smithi148.stderr: 57350K .......... .......... .......... .......... .......... 34% 32.8M 4s 2024-03-29T04:11:48.851 INFO:teuthology.orchestra.run.smithi148.stderr: 57400K .......... .......... .......... .......... .......... 34% 38.8M 4s 2024-03-29T04:11:48.853 INFO:teuthology.orchestra.run.smithi148.stderr: 57450K .......... .......... .......... .......... .......... 34% 31.9M 4s 2024-03-29T04:11:48.853 INFO:teuthology.orchestra.run.smithi148.stderr: 57500K .......... .......... .......... .......... .......... 34% 40.9M 4s 2024-03-29T04:11:48.854 INFO:teuthology.orchestra.run.smithi148.stderr: 57550K .......... .......... .......... .......... .......... 34% 41.2M 4s 2024-03-29T04:11:48.855 INFO:teuthology.orchestra.run.smithi148.stderr: 57600K .......... .......... .......... .......... .......... 34% 41.5M 4s 2024-03-29T04:11:48.856 INFO:teuthology.orchestra.run.smithi148.stderr: 57650K .......... .......... .......... .......... .......... 34% 35.7M 4s 2024-03-29T04:11:48.857 INFO:teuthology.orchestra.run.smithi148.stderr: 57700K .......... .......... .......... .......... .......... 34% 40.6M 4s 2024-03-29T04:11:48.859 INFO:teuthology.orchestra.run.smithi148.stderr: 57750K .......... .......... .......... .......... .......... 34% 40.7M 4s 2024-03-29T04:11:48.860 INFO:teuthology.orchestra.run.smithi148.stderr: 57800K .......... .......... .......... .......... .......... 34% 40.3M 4s 2024-03-29T04:11:48.861 INFO:teuthology.orchestra.run.smithi148.stderr: 57850K .......... .......... .......... .......... .......... 34% 34.3M 4s 2024-03-29T04:11:48.863 INFO:teuthology.orchestra.run.smithi148.stderr: 57900K .......... .......... .......... .......... .......... 34% 35.4M 4s 2024-03-29T04:11:48.864 INFO:teuthology.orchestra.run.smithi148.stderr: 57950K .......... .......... .......... .......... .......... 34% 34.5M 4s 2024-03-29T04:11:48.865 INFO:teuthology.orchestra.run.smithi148.stderr: 58000K .......... .......... .......... .......... .......... 34% 34.1M 4s 2024-03-29T04:11:48.867 INFO:teuthology.orchestra.run.smithi148.stderr: 58050K .......... .......... .......... .......... .......... 34% 30.5M 4s 2024-03-29T04:11:48.869 INFO:teuthology.orchestra.run.smithi148.stderr: 58100K .......... .......... .......... .......... .......... 34% 34.1M 4s 2024-03-29T04:11:48.870 INFO:teuthology.orchestra.run.smithi148.stderr: 58150K .......... .......... .......... .......... .......... 34% 34.8M 4s 2024-03-29T04:11:48.871 INFO:teuthology.orchestra.run.smithi148.stderr: 58200K .......... .......... .......... .......... .......... 34% 34.7M 4s 2024-03-29T04:11:48.873 INFO:teuthology.orchestra.run.smithi148.stderr: 58250K .......... .......... .......... .......... .......... 34% 30.0M 4s 2024-03-29T04:11:48.874 INFO:teuthology.orchestra.run.smithi148.stderr: 58300K .......... .......... .......... .......... .......... 34% 37.2M 4s 2024-03-29T04:11:48.875 INFO:teuthology.orchestra.run.smithi148.stderr: 58350K .......... .......... .......... .......... .......... 34% 37.7M 4s 2024-03-29T04:11:48.877 INFO:teuthology.orchestra.run.smithi148.stderr: 58400K .......... .......... .......... .......... .......... 34% 31.9M 4s 2024-03-29T04:11:48.879 INFO:teuthology.orchestra.run.smithi148.stderr: 58450K .......... .......... .......... .......... .......... 34% 26.8M 4s 2024-03-29T04:11:48.881 INFO:teuthology.orchestra.run.smithi148.stderr: 58500K .......... .......... .......... .......... .......... 34% 31.4M 4s 2024-03-29T04:11:48.882 INFO:teuthology.orchestra.run.smithi148.stderr: 58550K .......... .......... .......... .......... .......... 34% 31.1M 4s 2024-03-29T04:11:48.884 INFO:teuthology.orchestra.run.smithi148.stderr: 58600K .......... .......... .......... .......... .......... 34% 37.5M 4s 2024-03-29T04:11:48.885 INFO:teuthology.orchestra.run.smithi148.stderr: 58650K .......... .......... .......... .......... .......... 34% 33.4M 4s 2024-03-29T04:11:48.886 INFO:teuthology.orchestra.run.smithi148.stderr: 58700K .......... .......... .......... .......... .......... 34% 39.1M 4s 2024-03-29T04:11:48.888 INFO:teuthology.orchestra.run.smithi148.stderr: 58750K .......... .......... .......... .......... .......... 34% 35.1M 4s 2024-03-29T04:11:48.889 INFO:teuthology.orchestra.run.smithi148.stderr: 58800K .......... .......... .......... .......... .......... 34% 34.6M 4s 2024-03-29T04:11:48.890 INFO:teuthology.orchestra.run.smithi148.stderr: 58850K .......... .......... .......... .......... .......... 34% 30.9M 4s 2024-03-29T04:11:48.892 INFO:teuthology.orchestra.run.smithi148.stderr: 58900K .......... .......... .......... .......... .......... 34% 36.3M 4s 2024-03-29T04:11:48.893 INFO:teuthology.orchestra.run.smithi148.stderr: 58950K .......... .......... .......... .......... .......... 35% 36.4M 4s 2024-03-29T04:11:48.895 INFO:teuthology.orchestra.run.smithi148.stderr: 59000K .......... .......... .......... .......... .......... 35% 35.7M 4s 2024-03-29T04:11:48.896 INFO:teuthology.orchestra.run.smithi148.stderr: 59050K .......... .......... .......... .......... .......... 35% 30.6M 4s 2024-03-29T04:11:48.897 INFO:teuthology.orchestra.run.smithi148.stderr: 59100K .......... .......... .......... .......... .......... 35% 35.6M 4s 2024-03-29T04:11:48.899 INFO:teuthology.orchestra.run.smithi148.stderr: 59150K .......... .......... .......... .......... .......... 35% 34.9M 4s 2024-03-29T04:11:48.900 INFO:teuthology.orchestra.run.smithi148.stderr: 59200K .......... .......... .......... .......... .......... 35% 35.7M 4s 2024-03-29T04:11:48.902 INFO:teuthology.orchestra.run.smithi148.stderr: 59250K .......... .......... .......... .......... .......... 35% 30.4M 4s 2024-03-29T04:11:48.903 INFO:teuthology.orchestra.run.smithi148.stderr: 59300K .......... .......... .......... .......... .......... 35% 34.9M 4s 2024-03-29T04:11:48.905 INFO:teuthology.orchestra.run.smithi148.stderr: 59350K .......... .......... .......... .......... .......... 35% 34.7M 4s 2024-03-29T04:11:48.906 INFO:teuthology.orchestra.run.smithi148.stderr: 59400K .......... .......... .......... .......... .......... 35% 34.9M 4s 2024-03-29T04:11:48.908 INFO:teuthology.orchestra.run.smithi148.stderr: 59450K .......... .......... .......... .......... .......... 35% 29.6M 4s 2024-03-29T04:11:48.909 INFO:teuthology.orchestra.run.smithi148.stderr: 59500K .......... .......... .......... .......... .......... 35% 35.5M 4s 2024-03-29T04:11:48.910 INFO:teuthology.orchestra.run.smithi148.stderr: 59550K .......... .......... .......... .......... .......... 35% 34.8M 4s 2024-03-29T04:11:48.912 INFO:teuthology.orchestra.run.smithi148.stderr: 59600K .......... .......... .......... .......... .......... 35% 32.1M 4s 2024-03-29T04:11:48.914 INFO:teuthology.orchestra.run.smithi148.stderr: 59650K .......... .......... .......... .......... .......... 35% 23.1M 4s 2024-03-29T04:11:48.916 INFO:teuthology.orchestra.run.smithi148.stderr: 59700K .......... .......... .......... .......... .......... 35% 27.3M 4s 2024-03-29T04:11:48.918 INFO:teuthology.orchestra.run.smithi148.stderr: 59750K .......... .......... .......... .......... .......... 35% 26.6M 4s 2024-03-29T04:11:48.920 INFO:teuthology.orchestra.run.smithi148.stderr: 59800K .......... .......... .......... .......... .......... 35% 27.2M 4s 2024-03-29T04:11:48.922 INFO:teuthology.orchestra.run.smithi148.stderr: 59850K .......... .......... .......... .......... .......... 35% 23.0M 4s 2024-03-29T04:11:48.923 INFO:teuthology.orchestra.run.smithi148.stderr: 59900K .......... .......... .......... .......... .......... 35% 26.9M 4s 2024-03-29T04:11:48.925 INFO:teuthology.orchestra.run.smithi148.stderr: 59950K .......... .......... .......... .......... .......... 35% 25.2M 4s 2024-03-29T04:11:48.928 INFO:teuthology.orchestra.run.smithi148.stderr: 60000K .......... .......... .......... .......... .......... 35% 24.2M 4s 2024-03-29T04:11:48.930 INFO:teuthology.orchestra.run.smithi148.stderr: 60050K .......... .......... .......... .......... .......... 35% 20.9M 4s 2024-03-29T04:11:48.932 INFO:teuthology.orchestra.run.smithi148.stderr: 60100K .......... .......... .......... .......... .......... 35% 23.4M 4s 2024-03-29T04:11:48.934 INFO:teuthology.orchestra.run.smithi148.stderr: 60150K .......... .......... .......... .......... .......... 35% 23.3M 4s 2024-03-29T04:11:48.936 INFO:teuthology.orchestra.run.smithi148.stderr: 60200K .......... .......... .......... .......... .......... 35% 23.0M 4s 2024-03-29T04:11:48.938 INFO:teuthology.orchestra.run.smithi148.stderr: 60250K .......... .......... .......... .......... .......... 35% 20.2M 4s 2024-03-29T04:11:48.941 INFO:teuthology.orchestra.run.smithi148.stderr: 60300K .......... .......... .......... .......... .......... 35% 23.6M 4s 2024-03-29T04:11:48.943 INFO:teuthology.orchestra.run.smithi148.stderr: 60350K .......... .......... .......... .......... .......... 35% 22.9M 4s 2024-03-29T04:11:48.945 INFO:teuthology.orchestra.run.smithi148.stderr: 60400K .......... .......... .......... .......... .......... 35% 23.1M 4s 2024-03-29T04:11:48.950 INFO:teuthology.orchestra.run.smithi148.stderr: 60450K .......... .......... .......... .......... .......... 35% 20.4M 4s 2024-03-29T04:11:48.951 INFO:teuthology.orchestra.run.smithi148.stderr: 60500K .......... .......... .......... .......... .......... 35% 23.2M 4s 2024-03-29T04:11:48.952 INFO:teuthology.orchestra.run.smithi148.stderr: 60550K .......... .......... .......... .......... .......... 35% 22.7M 4s 2024-03-29T04:11:48.954 INFO:teuthology.orchestra.run.smithi148.stderr: 60600K .......... .......... .......... .......... .......... 35% 23.0M 4s 2024-03-29T04:11:48.956 INFO:teuthology.orchestra.run.smithi148.stderr: 60650K .......... .......... .......... .......... .......... 36% 19.7M 4s 2024-03-29T04:11:48.958 INFO:teuthology.orchestra.run.smithi148.stderr: 60700K .......... .......... .......... .......... .......... 36% 22.9M 4s 2024-03-29T04:11:48.960 INFO:teuthology.orchestra.run.smithi148.stderr: 60750K .......... .......... .......... .......... .......... 36% 22.8M 4s 2024-03-29T04:11:48.963 INFO:teuthology.orchestra.run.smithi148.stderr: 60800K .......... .......... .......... .......... .......... 36% 22.4M 4s 2024-03-29T04:11:48.965 INFO:teuthology.orchestra.run.smithi148.stderr: 60850K .......... .......... .......... .......... .......... 36% 19.1M 4s 2024-03-29T04:11:48.967 INFO:teuthology.orchestra.run.smithi148.stderr: 60900K .......... .......... .......... .......... .......... 36% 22.4M 4s 2024-03-29T04:11:48.969 INFO:teuthology.orchestra.run.smithi148.stderr: 60950K .......... .......... .......... .......... .......... 36% 22.0M 4s 2024-03-29T04:11:48.972 INFO:teuthology.orchestra.run.smithi148.stderr: 61000K .......... .......... .......... .......... .......... 36% 22.2M 4s 2024-03-29T04:11:48.974 INFO:teuthology.orchestra.run.smithi148.stderr: 61050K .......... .......... .......... .......... .......... 36% 19.3M 4s 2024-03-29T04:11:48.976 INFO:teuthology.orchestra.run.smithi148.stderr: 61100K .......... .......... .......... .......... .......... 36% 23.7M 4s 2024-03-29T04:11:48.978 INFO:teuthology.orchestra.run.smithi148.stderr: 61150K .......... .......... .......... .......... .......... 36% 24.6M 4s 2024-03-29T04:11:48.980 INFO:teuthology.orchestra.run.smithi148.stderr: 61200K .......... .......... .......... .......... .......... 36% 25.4M 4s 2024-03-29T04:11:48.982 INFO:teuthology.orchestra.run.smithi148.stderr: 61250K .......... .......... .......... .......... .......... 36% 21.4M 4s 2024-03-29T04:11:48.985 INFO:teuthology.orchestra.run.smithi148.stderr: 61300K .......... .......... .......... .......... .......... 36% 24.3M 4s 2024-03-29T04:11:48.987 INFO:teuthology.orchestra.run.smithi148.stderr: 61350K .......... .......... .......... .......... .......... 36% 24.0M 4s 2024-03-29T04:11:48.989 INFO:teuthology.orchestra.run.smithi148.stderr: 61400K .......... .......... .......... .......... .......... 36% 24.0M 4s 2024-03-29T04:11:48.990 INFO:teuthology.orchestra.run.smithi148.stderr: 61450K .......... .......... .......... .......... .......... 36% 25.9M 4s 2024-03-29T04:11:48.992 INFO:teuthology.orchestra.run.smithi148.stderr: 61500K .......... .......... .......... .......... .......... 36% 35.4M 4s 2024-03-29T04:11:48.993 INFO:teuthology.orchestra.run.smithi148.stderr: 61550K .......... .......... .......... .......... .......... 36% 33.0M 4s 2024-03-29T04:11:48.995 INFO:teuthology.orchestra.run.smithi148.stderr: 61600K .......... .......... .......... .......... .......... 36% 35.2M 4s 2024-03-29T04:11:48.997 INFO:teuthology.orchestra.run.smithi148.stderr: 61650K .......... .......... .......... .......... .......... 36% 23.3M 4s 2024-03-29T04:11:48.998 INFO:teuthology.orchestra.run.smithi148.stderr: 61700K .......... .......... .......... .......... .......... 36% 36.6M 4s 2024-03-29T04:11:48.999 INFO:teuthology.orchestra.run.smithi148.stderr: 61750K .......... .......... .......... .......... .......... 36% 33.8M 4s 2024-03-29T04:11:49.001 INFO:teuthology.orchestra.run.smithi148.stderr: 61800K .......... .......... .......... .......... .......... 36% 31.1M 4s 2024-03-29T04:11:49.003 INFO:teuthology.orchestra.run.smithi148.stderr: 61850K .......... .......... .......... .......... .......... 36% 31.1M 4s 2024-03-29T04:11:49.004 INFO:teuthology.orchestra.run.smithi148.stderr: 61900K .......... .......... .......... .......... .......... 36% 31.3M 4s 2024-03-29T04:11:49.006 INFO:teuthology.orchestra.run.smithi148.stderr: 61950K .......... .......... .......... .......... .......... 36% 33.7M 4s 2024-03-29T04:11:49.007 INFO:teuthology.orchestra.run.smithi148.stderr: 62000K .......... .......... .......... .......... .......... 36% 34.3M 4s 2024-03-29T04:11:49.009 INFO:teuthology.orchestra.run.smithi148.stderr: 62050K .......... .......... .......... .......... .......... 36% 29.1M 4s 2024-03-29T04:11:49.010 INFO:teuthology.orchestra.run.smithi148.stderr: 62100K .......... .......... .......... .......... .......... 36% 34.5M 4s 2024-03-29T04:11:49.012 INFO:teuthology.orchestra.run.smithi148.stderr: 62150K .......... .......... .......... .......... .......... 36% 34.8M 4s 2024-03-29T04:11:49.013 INFO:teuthology.orchestra.run.smithi148.stderr: 62200K .......... .......... .......... .......... .......... 36% 34.0M 4s 2024-03-29T04:11:49.015 INFO:teuthology.orchestra.run.smithi148.stderr: 62250K .......... .......... .......... .......... .......... 36% 29.5M 4s 2024-03-29T04:11:49.016 INFO:teuthology.orchestra.run.smithi148.stderr: 62300K .......... .......... .......... .......... .......... 36% 33.0M 4s 2024-03-29T04:11:49.018 INFO:teuthology.orchestra.run.smithi148.stderr: 62350K .......... .......... .......... .......... .......... 37% 32.8M 4s 2024-03-29T04:11:49.019 INFO:teuthology.orchestra.run.smithi148.stderr: 62400K .......... .......... .......... .......... .......... 37% 31.7M 4s 2024-03-29T04:11:49.021 INFO:teuthology.orchestra.run.smithi148.stderr: 62450K .......... .......... .......... .......... .......... 37% 26.8M 4s 2024-03-29T04:11:49.023 INFO:teuthology.orchestra.run.smithi148.stderr: 62500K .......... .......... .......... .......... .......... 37% 31.3M 4s 2024-03-29T04:11:49.024 INFO:teuthology.orchestra.run.smithi148.stderr: 62550K .......... .......... .......... .......... .......... 37% 33.2M 4s 2024-03-29T04:11:49.026 INFO:teuthology.orchestra.run.smithi148.stderr: 62600K .......... .......... .......... .......... .......... 37% 33.9M 4s 2024-03-29T04:11:49.027 INFO:teuthology.orchestra.run.smithi148.stderr: 62650K .......... .......... .......... .......... .......... 37% 26.8M 4s 2024-03-29T04:11:49.029 INFO:teuthology.orchestra.run.smithi148.stderr: 62700K .......... .......... .......... .......... .......... 37% 32.0M 4s 2024-03-29T04:11:49.030 INFO:teuthology.orchestra.run.smithi148.stderr: 62750K .......... .......... .......... .......... .......... 37% 31.8M 4s 2024-03-29T04:11:49.032 INFO:teuthology.orchestra.run.smithi148.stderr: 62800K .......... .......... .......... .......... .......... 37% 32.2M 4s 2024-03-29T04:11:49.034 INFO:teuthology.orchestra.run.smithi148.stderr: 62850K .......... .......... .......... .......... .......... 37% 27.5M 4s 2024-03-29T04:11:49.036 INFO:teuthology.orchestra.run.smithi148.stderr: 62900K .......... .......... .......... .......... .......... 37% 30.8M 4s 2024-03-29T04:11:49.037 INFO:teuthology.orchestra.run.smithi148.stderr: 62950K .......... .......... .......... .......... .......... 37% 31.1M 4s 2024-03-29T04:11:49.038 INFO:teuthology.orchestra.run.smithi148.stderr: 63000K .......... .......... .......... .......... .......... 37% 31.7M 4s 2024-03-29T04:11:49.040 INFO:teuthology.orchestra.run.smithi148.stderr: 63050K .......... .......... .......... .......... .......... 37% 27.1M 4s 2024-03-29T04:11:49.042 INFO:teuthology.orchestra.run.smithi148.stderr: 63100K .......... .......... .......... .......... .......... 37% 30.7M 4s 2024-03-29T04:11:49.043 INFO:teuthology.orchestra.run.smithi148.stderr: 63150K .......... .......... .......... .......... .......... 37% 30.5M 4s 2024-03-29T04:11:49.045 INFO:teuthology.orchestra.run.smithi148.stderr: 63200K .......... .......... .......... .......... .......... 37% 30.6M 4s 2024-03-29T04:11:49.047 INFO:teuthology.orchestra.run.smithi148.stderr: 63250K .......... .......... .......... .......... .......... 37% 25.8M 4s 2024-03-29T04:11:49.049 INFO:teuthology.orchestra.run.smithi148.stderr: 63300K .......... .......... .......... .......... .......... 37% 29.8M 4s 2024-03-29T04:11:49.050 INFO:teuthology.orchestra.run.smithi148.stderr: 63350K .......... .......... .......... .......... .......... 37% 30.1M 4s 2024-03-29T04:11:49.052 INFO:teuthology.orchestra.run.smithi148.stderr: 63400K .......... .......... .......... .......... .......... 37% 30.3M 4s 2024-03-29T04:11:49.053 INFO:teuthology.orchestra.run.smithi148.stderr: 63450K .......... .......... .......... .......... .......... 37% 25.5M 4s 2024-03-29T04:11:49.055 INFO:teuthology.orchestra.run.smithi148.stderr: 63500K .......... .......... .......... .......... .......... 37% 29.3M 4s 2024-03-29T04:11:49.057 INFO:teuthology.orchestra.run.smithi148.stderr: 63550K .......... .......... .......... .......... .......... 37% 29.7M 4s 2024-03-29T04:11:49.059 INFO:teuthology.orchestra.run.smithi148.stderr: 63600K .......... .......... .......... .......... .......... 37% 28.3M 4s 2024-03-29T04:11:49.061 INFO:teuthology.orchestra.run.smithi148.stderr: 63650K .......... .......... .......... .......... .......... 37% 24.7M 4s 2024-03-29T04:11:49.062 INFO:teuthology.orchestra.run.smithi148.stderr: 63700K .......... .......... .......... .......... .......... 37% 29.5M 4s 2024-03-29T04:11:49.064 INFO:teuthology.orchestra.run.smithi148.stderr: 63750K .......... .......... .......... .......... .......... 37% 28.3M 4s 2024-03-29T04:11:49.066 INFO:teuthology.orchestra.run.smithi148.stderr: 63800K .......... .......... .......... .......... .......... 37% 28.8M 4s 2024-03-29T04:11:49.068 INFO:teuthology.orchestra.run.smithi148.stderr: 63850K .......... .......... .......... .......... .......... 37% 24.7M 4s 2024-03-29T04:11:49.069 INFO:teuthology.orchestra.run.smithi148.stderr: 63900K .......... .......... .......... .......... .......... 37% 28.6M 4s 2024-03-29T04:11:49.071 INFO:teuthology.orchestra.run.smithi148.stderr: 63950K .......... .......... .......... .......... .......... 37% 28.9M 4s 2024-03-29T04:11:49.073 INFO:teuthology.orchestra.run.smithi148.stderr: 64000K .......... .......... .......... .......... .......... 37% 28.1M 4s 2024-03-29T04:11:49.075 INFO:teuthology.orchestra.run.smithi148.stderr: 64050K .......... .......... .......... .......... .......... 38% 23.7M 4s 2024-03-29T04:11:49.077 INFO:teuthology.orchestra.run.smithi148.stderr: 64100K .......... .......... .......... .......... .......... 38% 25.7M 4s 2024-03-29T04:11:49.078 INFO:teuthology.orchestra.run.smithi148.stderr: 64150K .......... .......... .......... .......... .......... 38% 26.6M 4s 2024-03-29T04:11:49.081 INFO:teuthology.orchestra.run.smithi148.stderr: 64200K .......... .......... .......... .......... .......... 38% 26.1M 4s 2024-03-29T04:11:49.083 INFO:teuthology.orchestra.run.smithi148.stderr: 64250K .......... .......... .......... .......... .......... 38% 22.7M 4s 2024-03-29T04:11:49.084 INFO:teuthology.orchestra.run.smithi148.stderr: 64300K .......... .......... .......... .......... .......... 38% 25.8M 4s 2024-03-29T04:11:49.086 INFO:teuthology.orchestra.run.smithi148.stderr: 64350K .......... .......... .......... .......... .......... 38% 26.8M 4s 2024-03-29T04:11:49.088 INFO:teuthology.orchestra.run.smithi148.stderr: 64400K .......... .......... .......... .......... .......... 38% 26.7M 4s 2024-03-29T04:11:49.090 INFO:teuthology.orchestra.run.smithi148.stderr: 64450K .......... .......... .......... .......... .......... 38% 22.2M 4s 2024-03-29T04:11:49.092 INFO:teuthology.orchestra.run.smithi148.stderr: 64500K .......... .......... .......... .......... .......... 38% 26.2M 4s 2024-03-29T04:11:49.094 INFO:teuthology.orchestra.run.smithi148.stderr: 64550K .......... .......... .......... .......... .......... 38% 25.4M 4s 2024-03-29T04:11:49.096 INFO:teuthology.orchestra.run.smithi148.stderr: 64600K .......... .......... .......... .......... .......... 38% 25.7M 4s 2024-03-29T04:11:49.098 INFO:teuthology.orchestra.run.smithi148.stderr: 64650K .......... .......... .......... .......... .......... 38% 22.3M 4s 2024-03-29T04:11:49.100 INFO:teuthology.orchestra.run.smithi148.stderr: 64700K .......... .......... .......... .......... .......... 38% 25.6M 4s 2024-03-29T04:11:49.102 INFO:teuthology.orchestra.run.smithi148.stderr: 64750K .......... .......... .......... .......... .......... 38% 25.5M 4s 2024-03-29T04:11:49.104 INFO:teuthology.orchestra.run.smithi148.stderr: 64800K .......... .......... .......... .......... .......... 38% 26.0M 4s 2024-03-29T04:11:49.106 INFO:teuthology.orchestra.run.smithi148.stderr: 64850K .......... .......... .......... .......... .......... 38% 21.5M 4s 2024-03-29T04:11:49.108 INFO:teuthology.orchestra.run.smithi148.stderr: 64900K .......... .......... .......... .......... .......... 38% 25.8M 4s 2024-03-29T04:11:49.110 INFO:teuthology.orchestra.run.smithi148.stderr: 64950K .......... .......... .......... .......... .......... 38% 24.7M 4s 2024-03-29T04:11:49.112 INFO:teuthology.orchestra.run.smithi148.stderr: 65000K .......... .......... .......... .......... .......... 38% 25.0M 4s 2024-03-29T04:11:49.114 INFO:teuthology.orchestra.run.smithi148.stderr: 65050K .......... .......... .......... .......... .......... 38% 20.8M 4s 2024-03-29T04:11:49.116 INFO:teuthology.orchestra.run.smithi148.stderr: 65100K .......... .......... .......... .......... .......... 38% 24.7M 4s 2024-03-29T04:11:49.118 INFO:teuthology.orchestra.run.smithi148.stderr: 65150K .......... .......... .......... .......... .......... 38% 24.7M 4s 2024-03-29T04:11:49.121 INFO:teuthology.orchestra.run.smithi148.stderr: 65200K .......... .......... .......... .......... .......... 38% 24.4M 4s 2024-03-29T04:11:49.123 INFO:teuthology.orchestra.run.smithi148.stderr: 65250K .......... .......... .......... .......... .......... 38% 21.1M 4s 2024-03-29T04:11:49.125 INFO:teuthology.orchestra.run.smithi148.stderr: 65300K .......... .......... .......... .......... .......... 38% 24.6M 4s 2024-03-29T04:11:49.127 INFO:teuthology.orchestra.run.smithi148.stderr: 65350K .......... .......... .......... .......... .......... 38% 24.2M 4s 2024-03-29T04:11:49.129 INFO:teuthology.orchestra.run.smithi148.stderr: 65400K .......... .......... .......... .......... .......... 38% 24.8M 4s 2024-03-29T04:11:49.131 INFO:teuthology.orchestra.run.smithi148.stderr: 65450K .......... .......... .......... .......... .......... 38% 20.7M 4s 2024-03-29T04:11:49.133 INFO:teuthology.orchestra.run.smithi148.stderr: 65500K .......... .......... .......... .......... .......... 38% 22.8M 4s 2024-03-29T04:11:49.135 INFO:teuthology.orchestra.run.smithi148.stderr: 65550K .......... .......... .......... .......... .......... 38% 36.1M 4s 2024-03-29T04:11:49.136 INFO:teuthology.orchestra.run.smithi148.stderr: 65600K .......... .......... .......... .......... .......... 38% 37.2M 4s 2024-03-29T04:11:49.138 INFO:teuthology.orchestra.run.smithi148.stderr: 65650K .......... .......... .......... .......... .......... 38% 27.2M 4s 2024-03-29T04:11:49.139 INFO:teuthology.orchestra.run.smithi148.stderr: 65700K .......... .......... .......... .......... .......... 39% 31.4M 4s 2024-03-29T04:11:49.141 INFO:teuthology.orchestra.run.smithi148.stderr: 65750K .......... .......... .......... .......... .......... 39% 28.6M 4s 2024-03-29T04:11:49.142 INFO:teuthology.orchestra.run.smithi148.stderr: 65800K .......... .......... .......... .......... .......... 39% 33.2M 4s 2024-03-29T04:11:49.144 INFO:teuthology.orchestra.run.smithi148.stderr: 65850K .......... .......... .......... .......... .......... 39% 32.3M 4s 2024-03-29T04:11:49.146 INFO:teuthology.orchestra.run.smithi148.stderr: 65900K .......... .......... .......... .......... .......... 39% 34.0M 4s 2024-03-29T04:11:49.147 INFO:teuthology.orchestra.run.smithi148.stderr: 65950K .......... .......... .......... .......... .......... 39% 34.6M 4s 2024-03-29T04:11:49.148 INFO:teuthology.orchestra.run.smithi148.stderr: 66000K .......... .......... .......... .......... .......... 39% 36.1M 4s 2024-03-29T04:11:49.150 INFO:teuthology.orchestra.run.smithi148.stderr: 66050K .......... .......... .......... .......... .......... 39% 26.4M 4s 2024-03-29T04:11:49.152 INFO:teuthology.orchestra.run.smithi148.stderr: 66100K .......... .......... .......... .......... .......... 39% 32.3M 4s 2024-03-29T04:11:49.153 INFO:teuthology.orchestra.run.smithi148.stderr: 66150K .......... .......... .......... .......... .......... 39% 33.5M 4s 2024-03-29T04:11:49.154 INFO:teuthology.orchestra.run.smithi148.stderr: 66200K .......... .......... .......... .......... .......... 39% 34.2M 4s 2024-03-29T04:11:49.156 INFO:teuthology.orchestra.run.smithi148.stderr: 66250K .......... .......... .......... .......... .......... 39% 29.2M 4s 2024-03-29T04:11:49.157 INFO:teuthology.orchestra.run.smithi148.stderr: 66300K .......... .......... .......... .......... .......... 39% 33.7M 4s 2024-03-29T04:11:49.159 INFO:teuthology.orchestra.run.smithi148.stderr: 66350K .......... .......... .......... .......... .......... 39% 35.3M 4s 2024-03-29T04:11:49.160 INFO:teuthology.orchestra.run.smithi148.stderr: 66400K .......... .......... .......... .......... .......... 39% 38.0M 4s 2024-03-29T04:11:49.161 INFO:teuthology.orchestra.run.smithi148.stderr: 66450K .......... .......... .......... .......... .......... 39% 32.7M 4s 2024-03-29T04:11:49.163 INFO:teuthology.orchestra.run.smithi148.stderr: 66500K .......... .......... .......... .......... .......... 39% 38.5M 4s 2024-03-29T04:11:49.164 INFO:teuthology.orchestra.run.smithi148.stderr: 66550K .......... .......... .......... .......... .......... 39% 37.5M 4s 2024-03-29T04:11:49.166 INFO:teuthology.orchestra.run.smithi148.stderr: 66600K .......... .......... .......... .......... .......... 39% 34.1M 4s 2024-03-29T04:11:49.167 INFO:teuthology.orchestra.run.smithi148.stderr: 66650K .......... .......... .......... .......... .......... 39% 28.2M 4s 2024-03-29T04:11:49.169 INFO:teuthology.orchestra.run.smithi148.stderr: 66700K .......... .......... .......... .......... .......... 39% 31.5M 4s 2024-03-29T04:11:49.170 INFO:teuthology.orchestra.run.smithi148.stderr: 66750K .......... .......... .......... .......... .......... 39% 32.4M 4s 2024-03-29T04:11:49.172 INFO:teuthology.orchestra.run.smithi148.stderr: 66800K .......... .......... .......... .......... .......... 39% 31.7M 4s 2024-03-29T04:11:49.174 INFO:teuthology.orchestra.run.smithi148.stderr: 66850K .......... .......... .......... .......... .......... 39% 27.2M 4s 2024-03-29T04:11:49.175 INFO:teuthology.orchestra.run.smithi148.stderr: 66900K .......... .......... .......... .......... .......... 39% 31.0M 4s 2024-03-29T04:11:49.177 INFO:teuthology.orchestra.run.smithi148.stderr: 66950K .......... .......... .......... .......... .......... 39% 32.4M 4s 2024-03-29T04:11:49.178 INFO:teuthology.orchestra.run.smithi148.stderr: 67000K .......... .......... .......... .......... .......... 39% 32.1M 4s 2024-03-29T04:11:49.180 INFO:teuthology.orchestra.run.smithi148.stderr: 67050K .......... .......... .......... .......... .......... 39% 29.0M 4s 2024-03-29T04:11:49.181 INFO:teuthology.orchestra.run.smithi148.stderr: 67100K .......... .......... .......... .......... .......... 39% 35.2M 4s 2024-03-29T04:11:49.183 INFO:teuthology.orchestra.run.smithi148.stderr: 67150K .......... .......... .......... .......... .......... 39% 34.4M 4s 2024-03-29T04:11:49.184 INFO:teuthology.orchestra.run.smithi148.stderr: 67200K .......... .......... .......... .......... .......... 39% 33.3M 4s 2024-03-29T04:11:49.186 INFO:teuthology.orchestra.run.smithi148.stderr: 67250K .......... .......... .......... .......... .......... 39% 29.3M 4s 2024-03-29T04:11:49.187 INFO:teuthology.orchestra.run.smithi148.stderr: 67300K .......... .......... .......... .......... .......... 39% 33.7M 4s 2024-03-29T04:11:49.189 INFO:teuthology.orchestra.run.smithi148.stderr: 67350K .......... .......... .......... .......... .......... 39% 32.3M 4s 2024-03-29T04:11:49.191 INFO:teuthology.orchestra.run.smithi148.stderr: 67400K .......... .......... .......... .......... .......... 40% 32.8M 4s 2024-03-29T04:11:49.192 INFO:teuthology.orchestra.run.smithi148.stderr: 67450K .......... .......... .......... .......... .......... 40% 28.0M 4s 2024-03-29T04:11:49.193 INFO:teuthology.orchestra.run.smithi148.stderr: 67500K .......... .......... .......... .......... .......... 40% 33.6M 4s 2024-03-29T04:11:49.195 INFO:teuthology.orchestra.run.smithi148.stderr: 67550K .......... .......... .......... .......... .......... 40% 29.7M 4s 2024-03-29T04:11:49.197 INFO:teuthology.orchestra.run.smithi148.stderr: 67600K .......... .......... .......... .......... .......... 40% 28.3M 4s 2024-03-29T04:11:49.199 INFO:teuthology.orchestra.run.smithi148.stderr: 67650K .......... .......... .......... .......... .......... 40% 24.7M 4s 2024-03-29T04:11:49.201 INFO:teuthology.orchestra.run.smithi148.stderr: 67700K .......... .......... .......... .......... .......... 40% 29.1M 4s 2024-03-29T04:11:49.202 INFO:teuthology.orchestra.run.smithi148.stderr: 67750K .......... .......... .......... .......... .......... 40% 28.0M 4s 2024-03-29T04:11:49.204 INFO:teuthology.orchestra.run.smithi148.stderr: 67800K .......... .......... .......... .......... .......... 40% 28.0M 4s 2024-03-29T04:11:49.206 INFO:teuthology.orchestra.run.smithi148.stderr: 67850K .......... .......... .......... .......... .......... 40% 24.6M 4s 2024-03-29T04:11:49.208 INFO:teuthology.orchestra.run.smithi148.stderr: 67900K .......... .......... .......... .......... .......... 40% 28.0M 4s 2024-03-29T04:11:49.209 INFO:teuthology.orchestra.run.smithi148.stderr: 67950K .......... .......... .......... .......... .......... 40% 27.3M 4s 2024-03-29T04:11:49.211 INFO:teuthology.orchestra.run.smithi148.stderr: 68000K .......... .......... .......... .......... .......... 40% 27.8M 4s 2024-03-29T04:11:49.213 INFO:teuthology.orchestra.run.smithi148.stderr: 68050K .......... .......... .......... .......... .......... 40% 23.6M 4s 2024-03-29T04:11:49.215 INFO:teuthology.orchestra.run.smithi148.stderr: 68100K .......... .......... .......... .......... .......... 40% 27.6M 4s 2024-03-29T04:11:49.217 INFO:teuthology.orchestra.run.smithi148.stderr: 68150K .......... .......... .......... .......... .......... 40% 28.0M 4s 2024-03-29T04:11:49.219 INFO:teuthology.orchestra.run.smithi148.stderr: 68200K .......... .......... .......... .......... .......... 40% 27.5M 4s 2024-03-29T04:11:49.221 INFO:teuthology.orchestra.run.smithi148.stderr: 68250K .......... .......... .......... .......... .......... 40% 23.8M 4s 2024-03-29T04:11:49.223 INFO:teuthology.orchestra.run.smithi148.stderr: 68300K .......... .......... .......... .......... .......... 40% 26.7M 4s 2024-03-29T04:11:49.224 INFO:teuthology.orchestra.run.smithi148.stderr: 68350K .......... .......... .......... .......... .......... 40% 28.4M 4s 2024-03-29T04:11:49.226 INFO:teuthology.orchestra.run.smithi148.stderr: 68400K .......... .......... .......... .......... .......... 40% 26.4M 4s 2024-03-29T04:11:49.228 INFO:teuthology.orchestra.run.smithi148.stderr: 68450K .......... .......... .......... .......... .......... 40% 23.7M 4s 2024-03-29T04:11:49.230 INFO:teuthology.orchestra.run.smithi148.stderr: 68500K .......... .......... .......... .......... .......... 40% 28.0M 4s 2024-03-29T04:11:49.232 INFO:teuthology.orchestra.run.smithi148.stderr: 68550K .......... .......... .......... .......... .......... 40% 27.6M 4s 2024-03-29T04:11:49.234 INFO:teuthology.orchestra.run.smithi148.stderr: 68600K .......... .......... .......... .......... .......... 40% 26.8M 4s 2024-03-29T04:11:49.235 INFO:teuthology.orchestra.run.smithi148.stderr: 68650K .......... .......... .......... .......... .......... 40% 23.6M 4s 2024-03-29T04:11:49.237 INFO:teuthology.orchestra.run.smithi148.stderr: 68700K .......... .......... .......... .......... .......... 40% 26.8M 4s 2024-03-29T04:11:49.239 INFO:teuthology.orchestra.run.smithi148.stderr: 68750K .......... .......... .......... .......... .......... 40% 26.7M 4s 2024-03-29T04:11:49.241 INFO:teuthology.orchestra.run.smithi148.stderr: 68800K .......... .......... .......... .......... .......... 40% 27.4M 4s 2024-03-29T04:11:49.243 INFO:teuthology.orchestra.run.smithi148.stderr: 68850K .......... .......... .......... .......... .......... 40% 23.2M 4s 2024-03-29T04:11:49.245 INFO:teuthology.orchestra.run.smithi148.stderr: 68900K .......... .......... .......... .......... .......... 40% 27.1M 4s 2024-03-29T04:11:49.247 INFO:teuthology.orchestra.run.smithi148.stderr: 68950K .......... .......... .......... .......... .......... 40% 26.9M 4s 2024-03-29T04:11:49.249 INFO:teuthology.orchestra.run.smithi148.stderr: 69000K .......... .......... .......... .......... .......... 40% 27.4M 4s 2024-03-29T04:11:49.251 INFO:teuthology.orchestra.run.smithi148.stderr: 69050K .......... .......... .......... .......... .......... 40% 22.0M 4s 2024-03-29T04:11:49.253 INFO:teuthology.orchestra.run.smithi148.stderr: 69100K .......... .......... .......... .......... .......... 41% 27.0M 4s 2024-03-29T04:11:49.254 INFO:teuthology.orchestra.run.smithi148.stderr: 69150K .......... .......... .......... .......... .......... 41% 26.9M 4s 2024-03-29T04:11:49.256 INFO:teuthology.orchestra.run.smithi148.stderr: 69200K .......... .......... .......... .......... .......... 41% 26.8M 4s 2024-03-29T04:11:49.258 INFO:teuthology.orchestra.run.smithi148.stderr: 69250K .......... .......... .......... .......... .......... 41% 23.2M 4s 2024-03-29T04:11:49.260 INFO:teuthology.orchestra.run.smithi148.stderr: 69300K .......... .......... .......... .......... .......... 41% 26.6M 4s 2024-03-29T04:11:49.262 INFO:teuthology.orchestra.run.smithi148.stderr: 69350K .......... .......... .......... .......... .......... 41% 26.1M 4s 2024-03-29T04:11:49.264 INFO:teuthology.orchestra.run.smithi148.stderr: 69400K .......... .......... .......... .......... .......... 41% 26.6M 4s 2024-03-29T04:11:49.266 INFO:teuthology.orchestra.run.smithi148.stderr: 69450K .......... .......... .......... .......... .......... 41% 22.3M 4s 2024-03-29T04:11:49.268 INFO:teuthology.orchestra.run.smithi148.stderr: 69500K .......... .......... .......... .......... .......... 41% 25.2M 4s 2024-03-29T04:11:49.270 INFO:teuthology.orchestra.run.smithi148.stderr: 69550K .......... .......... .......... .......... .......... 41% 27.6M 4s 2024-03-29T04:11:49.272 INFO:teuthology.orchestra.run.smithi148.stderr: 69600K .......... .......... .......... .......... .......... 41% 24.9M 4s 2024-03-29T04:11:49.273 INFO:teuthology.orchestra.run.smithi148.stderr: 69650K .......... .......... .......... .......... .......... 41% 27.1M 4s 2024-03-29T04:11:49.275 INFO:teuthology.orchestra.run.smithi148.stderr: 69700K .......... .......... .......... .......... .......... 41% 36.6M 4s 2024-03-29T04:11:49.277 INFO:teuthology.orchestra.run.smithi148.stderr: 69750K .......... .......... .......... .......... .......... 41% 31.8M 4s 2024-03-29T04:11:49.278 INFO:teuthology.orchestra.run.smithi148.stderr: 69800K .......... .......... .......... .......... .......... 41% 35.4M 4s 2024-03-29T04:11:49.279 INFO:teuthology.orchestra.run.smithi148.stderr: 69850K .......... .......... .......... .......... .......... 41% 31.3M 4s 2024-03-29T04:11:49.281 INFO:teuthology.orchestra.run.smithi148.stderr: 69900K .......... .......... .......... .......... .......... 41% 35.1M 4s 2024-03-29T04:11:49.282 INFO:teuthology.orchestra.run.smithi148.stderr: 69950K .......... .......... .......... .......... .......... 41% 39.1M 4s 2024-03-29T04:11:49.283 INFO:teuthology.orchestra.run.smithi148.stderr: 70000K .......... .......... .......... .......... .......... 41% 38.8M 4s 2024-03-29T04:11:49.285 INFO:teuthology.orchestra.run.smithi148.stderr: 70050K .......... .......... .......... .......... .......... 41% 31.6M 4s 2024-03-29T04:11:49.286 INFO:teuthology.orchestra.run.smithi148.stderr: 70100K .......... .......... .......... .......... .......... 41% 33.7M 4s 2024-03-29T04:11:49.287 INFO:teuthology.orchestra.run.smithi148.stderr: 70150K .......... .......... .......... .......... .......... 41% 35.5M 4s 2024-03-29T04:11:49.289 INFO:teuthology.orchestra.run.smithi148.stderr: 70200K .......... .......... .......... .......... .......... 41% 35.8M 4s 2024-03-29T04:11:49.290 INFO:teuthology.orchestra.run.smithi148.stderr: 70250K .......... .......... .......... .......... .......... 41% 32.1M 4s 2024-03-29T04:11:49.292 INFO:teuthology.orchestra.run.smithi148.stderr: 70300K .......... .......... .......... .......... .......... 41% 37.8M 4s 2024-03-29T04:11:49.293 INFO:teuthology.orchestra.run.smithi148.stderr: 70350K .......... .......... .......... .......... .......... 41% 37.7M 4s 2024-03-29T04:11:49.294 INFO:teuthology.orchestra.run.smithi148.stderr: 70400K .......... .......... .......... .......... .......... 41% 37.1M 4s 2024-03-29T04:11:49.296 INFO:teuthology.orchestra.run.smithi148.stderr: 70450K .......... .......... .......... .......... .......... 41% 32.0M 4s 2024-03-29T04:11:49.297 INFO:teuthology.orchestra.run.smithi148.stderr: 70500K .......... .......... .......... .......... .......... 41% 37.3M 4s 2024-03-29T04:11:49.298 INFO:teuthology.orchestra.run.smithi148.stderr: 70550K .......... .......... .......... .......... .......... 41% 36.5M 4s 2024-03-29T04:11:49.300 INFO:teuthology.orchestra.run.smithi148.stderr: 70600K .......... .......... .......... .......... .......... 41% 36.8M 4s 2024-03-29T04:11:49.301 INFO:teuthology.orchestra.run.smithi148.stderr: 70650K .......... .......... .......... .......... .......... 41% 29.6M 4s 2024-03-29T04:11:49.303 INFO:teuthology.orchestra.run.smithi148.stderr: 70700K .......... .......... .......... .......... .......... 41% 36.2M 4s 2024-03-29T04:11:49.304 INFO:teuthology.orchestra.run.smithi148.stderr: 70750K .......... .......... .......... .......... .......... 42% 35.8M 4s 2024-03-29T04:11:49.306 INFO:teuthology.orchestra.run.smithi148.stderr: 70800K .......... .......... .......... .......... .......... 42% 35.8M 4s 2024-03-29T04:11:49.307 INFO:teuthology.orchestra.run.smithi148.stderr: 70850K .......... .......... .......... .......... .......... 42% 30.3M 4s 2024-03-29T04:11:49.309 INFO:teuthology.orchestra.run.smithi148.stderr: 70900K .......... .......... .......... .......... .......... 42% 34.6M 4s 2024-03-29T04:11:49.310 INFO:teuthology.orchestra.run.smithi148.stderr: 70950K .......... .......... .......... .......... .......... 42% 35.2M 4s 2024-03-29T04:11:49.311 INFO:teuthology.orchestra.run.smithi148.stderr: 71000K .......... .......... .......... .......... .......... 42% 34.6M 4s 2024-03-29T04:11:49.313 INFO:teuthology.orchestra.run.smithi148.stderr: 71050K .......... .......... .......... .......... .......... 42% 29.7M 4s 2024-03-29T04:11:49.315 INFO:teuthology.orchestra.run.smithi148.stderr: 71100K .......... .......... .......... .......... .......... 42% 34.8M 4s 2024-03-29T04:11:49.316 INFO:teuthology.orchestra.run.smithi148.stderr: 71150K .......... .......... .......... .......... .......... 42% 34.5M 4s 2024-03-29T04:11:49.317 INFO:teuthology.orchestra.run.smithi148.stderr: 71200K .......... .......... .......... .......... .......... 42% 34.2M 4s 2024-03-29T04:11:49.319 INFO:teuthology.orchestra.run.smithi148.stderr: 71250K .......... .......... .......... .......... .......... 42% 26.7M 4s 2024-03-29T04:11:49.321 INFO:teuthology.orchestra.run.smithi148.stderr: 71300K .......... .......... .......... .......... .......... 42% 30.3M 4s 2024-03-29T04:11:49.322 INFO:teuthology.orchestra.run.smithi148.stderr: 71350K .......... .......... .......... .......... .......... 42% 31.1M 4s 2024-03-29T04:11:49.324 INFO:teuthology.orchestra.run.smithi148.stderr: 71400K .......... .......... .......... .......... .......... 42% 29.5M 4s 2024-03-29T04:11:49.326 INFO:teuthology.orchestra.run.smithi148.stderr: 71450K .......... .......... .......... .......... .......... 42% 26.2M 4s 2024-03-29T04:11:49.328 INFO:teuthology.orchestra.run.smithi148.stderr: 71500K .......... .......... .......... .......... .......... 42% 29.9M 4s 2024-03-29T04:11:49.329 INFO:teuthology.orchestra.run.smithi148.stderr: 71550K .......... .......... .......... .......... .......... 42% 29.9M 4s 2024-03-29T04:11:49.331 INFO:teuthology.orchestra.run.smithi148.stderr: 71600K .......... .......... .......... .......... .......... 42% 29.4M 4s 2024-03-29T04:11:49.333 INFO:teuthology.orchestra.run.smithi148.stderr: 71650K .......... .......... .......... .......... .......... 42% 26.1M 4s 2024-03-29T04:11:49.334 INFO:teuthology.orchestra.run.smithi148.stderr: 71700K .......... .......... .......... .......... .......... 42% 27.2M 4s 2024-03-29T04:11:49.336 INFO:teuthology.orchestra.run.smithi148.stderr: 71750K .......... .......... .......... .......... .......... 42% 29.4M 4s 2024-03-29T04:11:49.338 INFO:teuthology.orchestra.run.smithi148.stderr: 71800K .......... .......... .......... .......... .......... 42% 28.9M 4s 2024-03-29T04:11:49.340 INFO:teuthology.orchestra.run.smithi148.stderr: 71850K .......... .......... .......... .......... .......... 42% 24.8M 4s 2024-03-29T04:11:49.342 INFO:teuthology.orchestra.run.smithi148.stderr: 71900K .......... .......... .......... .......... .......... 42% 29.5M 4s 2024-03-29T04:11:49.343 INFO:teuthology.orchestra.run.smithi148.stderr: 71950K .......... .......... .......... .......... .......... 42% 29.0M 4s 2024-03-29T04:11:49.345 INFO:teuthology.orchestra.run.smithi148.stderr: 72000K .......... .......... .......... .......... .......... 42% 29.3M 4s 2024-03-29T04:11:49.347 INFO:teuthology.orchestra.run.smithi148.stderr: 72050K .......... .......... .......... .......... .......... 42% 24.7M 4s 2024-03-29T04:11:49.348 INFO:teuthology.orchestra.run.smithi148.stderr: 72100K .......... .......... .......... .......... .......... 42% 28.7M 4s 2024-03-29T04:11:49.350 INFO:teuthology.orchestra.run.smithi148.stderr: 72150K .......... .......... .......... .......... .......... 42% 30.4M 4s 2024-03-29T04:11:49.352 INFO:teuthology.orchestra.run.smithi148.stderr: 72200K .......... .......... .......... .......... .......... 42% 32.6M 4s 2024-03-29T04:11:49.353 INFO:teuthology.orchestra.run.smithi148.stderr: 72250K .......... .......... .......... .......... .......... 42% 27.5M 4s 2024-03-29T04:11:49.355 INFO:teuthology.orchestra.run.smithi148.stderr: 72300K .......... .......... .......... .......... .......... 42% 32.2M 3s 2024-03-29T04:11:49.358 INFO:teuthology.orchestra.run.smithi148.stderr: 72350K .......... .......... .......... .......... .......... 42% 31.3M 3s 2024-03-29T04:11:49.359 INFO:teuthology.orchestra.run.smithi148.stderr: 72400K .......... .......... .......... .......... .......... 42% 32.2M 3s 2024-03-29T04:11:49.360 INFO:teuthology.orchestra.run.smithi148.stderr: 72450K .......... .......... .......... .......... .......... 43% 27.2M 3s 2024-03-29T04:11:49.361 INFO:teuthology.orchestra.run.smithi148.stderr: 72500K .......... .......... .......... .......... .......... 43% 31.8M 3s 2024-03-29T04:11:49.363 INFO:teuthology.orchestra.run.smithi148.stderr: 72550K .......... .......... .......... .......... .......... 43% 32.1M 3s 2024-03-29T04:11:49.364 INFO:teuthology.orchestra.run.smithi148.stderr: 72600K .......... .......... .......... .......... .......... 43% 32.6M 3s 2024-03-29T04:11:49.366 INFO:teuthology.orchestra.run.smithi148.stderr: 72650K .......... .......... .......... .......... .......... 43% 26.2M 3s 2024-03-29T04:11:49.368 INFO:teuthology.orchestra.run.smithi148.stderr: 72700K .......... .......... .......... .......... .......... 43% 31.0M 3s 2024-03-29T04:11:49.369 INFO:teuthology.orchestra.run.smithi148.stderr: 72750K .......... .......... .......... .......... .......... 43% 30.9M 3s 2024-03-29T04:11:49.371 INFO:teuthology.orchestra.run.smithi148.stderr: 72800K .......... .......... .......... .......... .......... 43% 30.5M 3s 2024-03-29T04:11:49.373 INFO:teuthology.orchestra.run.smithi148.stderr: 72850K .......... .......... .......... .......... .......... 43% 26.1M 3s 2024-03-29T04:11:49.374 INFO:teuthology.orchestra.run.smithi148.stderr: 72900K .......... .......... .......... .......... .......... 43% 30.8M 3s 2024-03-29T04:11:49.376 INFO:teuthology.orchestra.run.smithi148.stderr: 72950K .......... .......... .......... .......... .......... 43% 28.4M 3s 2024-03-29T04:11:49.378 INFO:teuthology.orchestra.run.smithi148.stderr: 73000K .......... .......... .......... .......... .......... 43% 25.4M 3s 2024-03-29T04:11:49.380 INFO:teuthology.orchestra.run.smithi148.stderr: 73050K .......... .......... .......... .......... .......... 43% 22.1M 3s 2024-03-29T04:11:49.382 INFO:teuthology.orchestra.run.smithi148.stderr: 73100K .......... .......... .......... .......... .......... 43% 27.2M 3s 2024-03-29T04:11:49.384 INFO:teuthology.orchestra.run.smithi148.stderr: 73150K .......... .......... .......... .......... .......... 43% 27.1M 3s 2024-03-29T04:11:49.386 INFO:teuthology.orchestra.run.smithi148.stderr: 73200K .......... .......... .......... .......... .......... 43% 29.1M 3s 2024-03-29T04:11:49.388 INFO:teuthology.orchestra.run.smithi148.stderr: 73250K .......... .......... .......... .......... .......... 43% 24.6M 3s 2024-03-29T04:11:49.389 INFO:teuthology.orchestra.run.smithi148.stderr: 73300K .......... .......... .......... .......... .......... 43% 29.9M 3s 2024-03-29T04:11:49.391 INFO:teuthology.orchestra.run.smithi148.stderr: 73350K .......... .......... .......... .......... .......... 43% 28.7M 3s 2024-03-29T04:11:49.393 INFO:teuthology.orchestra.run.smithi148.stderr: 73400K .......... .......... .......... .......... .......... 43% 25.3M 3s 2024-03-29T04:11:49.395 INFO:teuthology.orchestra.run.smithi148.stderr: 73450K .......... .......... .......... .......... .......... 43% 22.1M 3s 2024-03-29T04:11:49.397 INFO:teuthology.orchestra.run.smithi148.stderr: 73500K .......... .......... .......... .......... .......... 43% 25.3M 3s 2024-03-29T04:11:49.399 INFO:teuthology.orchestra.run.smithi148.stderr: 73550K .......... .......... .......... .......... .......... 43% 24.4M 3s 2024-03-29T04:11:49.401 INFO:teuthology.orchestra.run.smithi148.stderr: 73600K .......... .......... .......... .......... .......... 43% 24.8M 3s 2024-03-29T04:11:49.403 INFO:teuthology.orchestra.run.smithi148.stderr: 73650K .......... .......... .......... .......... .......... 43% 22.0M 3s 2024-03-29T04:11:49.405 INFO:teuthology.orchestra.run.smithi148.stderr: 73700K .......... .......... .......... .......... .......... 43% 24.5M 3s 2024-03-29T04:11:49.407 INFO:teuthology.orchestra.run.smithi148.stderr: 73750K .......... .......... .......... .......... .......... 43% 30.5M 3s 2024-03-29T04:11:49.409 INFO:teuthology.orchestra.run.smithi148.stderr: 73800K .......... .......... .......... .......... .......... 43% 31.6M 3s 2024-03-29T04:11:49.410 INFO:teuthology.orchestra.run.smithi148.stderr: 73850K .......... .......... .......... .......... .......... 43% 25.2M 3s 2024-03-29T04:11:49.411 INFO:teuthology.orchestra.run.smithi148.stderr: 73900K .......... .......... .......... .......... .......... 43% 33.6M 3s 2024-03-29T04:11:49.413 INFO:teuthology.orchestra.run.smithi148.stderr: 73950K .......... .......... .......... .......... .......... 43% 33.8M 3s 2024-03-29T04:11:49.415 INFO:teuthology.orchestra.run.smithi148.stderr: 74000K .......... .......... .......... .......... .......... 43% 39.5M 3s 2024-03-29T04:11:49.416 INFO:teuthology.orchestra.run.smithi148.stderr: 74050K .......... .......... .......... .......... .......... 43% 31.5M 3s 2024-03-29T04:11:49.417 INFO:teuthology.orchestra.run.smithi148.stderr: 74100K .......... .......... .......... .......... .......... 43% 40.2M 3s 2024-03-29T04:11:49.418 INFO:teuthology.orchestra.run.smithi148.stderr: 74150K .......... .......... .......... .......... .......... 44% 38.8M 3s 2024-03-29T04:11:49.420 INFO:teuthology.orchestra.run.smithi148.stderr: 74200K .......... .......... .......... .......... .......... 44% 25.8M 3s 2024-03-29T04:11:49.422 INFO:teuthology.orchestra.run.smithi148.stderr: 74250K .......... .......... .......... .......... .......... 44% 24.3M 3s 2024-03-29T04:11:49.424 INFO:teuthology.orchestra.run.smithi148.stderr: 74300K .......... .......... .......... .......... .......... 44% 28.0M 3s 2024-03-29T04:11:49.426 INFO:teuthology.orchestra.run.smithi148.stderr: 74350K .......... .......... .......... .......... .......... 44% 27.2M 3s 2024-03-29T04:11:49.427 INFO:teuthology.orchestra.run.smithi148.stderr: 74400K .......... .......... .......... .......... .......... 44% 32.4M 3s 2024-03-29T04:11:49.429 INFO:teuthology.orchestra.run.smithi148.stderr: 74450K .......... .......... .......... .......... .......... 44% 32.1M 3s 2024-03-29T04:11:49.430 INFO:teuthology.orchestra.run.smithi148.stderr: 74500K .......... .......... .......... .......... .......... 44% 36.9M 3s 2024-03-29T04:11:49.431 INFO:teuthology.orchestra.run.smithi148.stderr: 74550K .......... .......... .......... .......... .......... 44% 36.8M 3s 2024-03-29T04:11:49.433 INFO:teuthology.orchestra.run.smithi148.stderr: 74600K .......... .......... .......... .......... .......... 44% 37.4M 3s 2024-03-29T04:11:49.434 INFO:teuthology.orchestra.run.smithi148.stderr: 74650K .......... .......... .......... .......... .......... 44% 31.6M 3s 2024-03-29T04:11:49.436 INFO:teuthology.orchestra.run.smithi148.stderr: 74700K .......... .......... .......... .......... .......... 44% 36.5M 3s 2024-03-29T04:11:49.437 INFO:teuthology.orchestra.run.smithi148.stderr: 74750K .......... .......... .......... .......... .......... 44% 36.9M 3s 2024-03-29T04:11:49.439 INFO:teuthology.orchestra.run.smithi148.stderr: 74800K .......... .......... .......... .......... .......... 44% 36.3M 3s 2024-03-29T04:11:49.440 INFO:teuthology.orchestra.run.smithi148.stderr: 74850K .......... .......... .......... .......... .......... 44% 29.7M 3s 2024-03-29T04:11:49.441 INFO:teuthology.orchestra.run.smithi148.stderr: 74900K .......... .......... .......... .......... .......... 44% 35.9M 3s 2024-03-29T04:11:49.443 INFO:teuthology.orchestra.run.smithi148.stderr: 74950K .......... .......... .......... .......... .......... 44% 33.2M 3s 2024-03-29T04:11:49.444 INFO:teuthology.orchestra.run.smithi148.stderr: 75000K .......... .......... .......... .......... .......... 44% 37.4M 3s 2024-03-29T04:11:49.446 INFO:teuthology.orchestra.run.smithi148.stderr: 75050K .......... .......... .......... .......... .......... 44% 31.8M 3s 2024-03-29T04:11:49.447 INFO:teuthology.orchestra.run.smithi148.stderr: 75100K .......... .......... .......... .......... .......... 44% 37.5M 3s 2024-03-29T04:11:49.448 INFO:teuthology.orchestra.run.smithi148.stderr: 75150K .......... .......... .......... .......... .......... 44% 37.4M 3s 2024-03-29T04:11:49.449 INFO:teuthology.orchestra.run.smithi148.stderr: 75200K .......... .......... .......... .......... .......... 44% 36.5M 3s 2024-03-29T04:11:49.451 INFO:teuthology.orchestra.run.smithi148.stderr: 75250K .......... .......... .......... .......... .......... 44% 32.0M 3s 2024-03-29T04:11:49.452 INFO:teuthology.orchestra.run.smithi148.stderr: 75300K .......... .......... .......... .......... .......... 44% 36.5M 3s 2024-03-29T04:11:49.454 INFO:teuthology.orchestra.run.smithi148.stderr: 75350K .......... .......... .......... .......... .......... 44% 36.1M 3s 2024-03-29T04:11:49.455 INFO:teuthology.orchestra.run.smithi148.stderr: 75400K .......... .......... .......... .......... .......... 44% 36.1M 3s 2024-03-29T04:11:49.456 INFO:teuthology.orchestra.run.smithi148.stderr: 75450K .......... .......... .......... .......... .......... 44% 30.9M 3s 2024-03-29T04:11:49.458 INFO:teuthology.orchestra.run.smithi148.stderr: 75500K .......... .......... .......... .......... .......... 44% 36.1M 3s 2024-03-29T04:11:49.459 INFO:teuthology.orchestra.run.smithi148.stderr: 75550K .......... .......... .......... .......... .......... 44% 35.3M 3s 2024-03-29T04:11:49.461 INFO:teuthology.orchestra.run.smithi148.stderr: 75600K .......... .......... .......... .......... .......... 44% 35.7M 3s 2024-03-29T04:11:49.462 INFO:teuthology.orchestra.run.smithi148.stderr: 75650K .......... .......... .......... .......... .......... 44% 30.5M 3s 2024-03-29T04:11:49.464 INFO:teuthology.orchestra.run.smithi148.stderr: 75700K .......... .......... .......... .......... .......... 44% 34.6M 3s 2024-03-29T04:11:49.465 INFO:teuthology.orchestra.run.smithi148.stderr: 75750K .......... .......... .......... .......... .......... 44% 34.7M 3s 2024-03-29T04:11:49.467 INFO:teuthology.orchestra.run.smithi148.stderr: 75800K .......... .......... .......... .......... .......... 44% 34.3M 3s 2024-03-29T04:11:49.468 INFO:teuthology.orchestra.run.smithi148.stderr: 75850K .......... .......... .......... .......... .......... 45% 28.9M 3s 2024-03-29T04:11:49.470 INFO:teuthology.orchestra.run.smithi148.stderr: 75900K .......... .......... .......... .......... .......... 45% 32.0M 3s 2024-03-29T04:11:49.471 INFO:teuthology.orchestra.run.smithi148.stderr: 75950K .......... .......... .......... .......... .......... 45% 32.8M 3s 2024-03-29T04:11:49.473 INFO:teuthology.orchestra.run.smithi148.stderr: 76000K .......... .......... .......... .......... .......... 45% 31.5M 3s 2024-03-29T04:11:49.475 INFO:teuthology.orchestra.run.smithi148.stderr: 76050K .......... .......... .......... .......... .......... 45% 27.5M 3s 2024-03-29T04:11:49.476 INFO:teuthology.orchestra.run.smithi148.stderr: 76100K .......... .......... .......... .......... .......... 45% 32.1M 3s 2024-03-29T04:11:49.478 INFO:teuthology.orchestra.run.smithi148.stderr: 76150K .......... .......... .......... .......... .......... 45% 31.4M 3s 2024-03-29T04:11:49.479 INFO:teuthology.orchestra.run.smithi148.stderr: 76200K .......... .......... .......... .......... .......... 45% 31.9M 3s 2024-03-29T04:11:49.481 INFO:teuthology.orchestra.run.smithi148.stderr: 76250K .......... .......... .......... .......... .......... 45% 27.2M 3s 2024-03-29T04:11:49.483 INFO:teuthology.orchestra.run.smithi148.stderr: 76300K .......... .......... .......... .......... .......... 45% 30.6M 3s 2024-03-29T04:11:49.484 INFO:teuthology.orchestra.run.smithi148.stderr: 76350K .......... .......... .......... .......... .......... 45% 28.2M 3s 2024-03-29T04:11:49.486 INFO:teuthology.orchestra.run.smithi148.stderr: 76400K .......... .......... .......... .......... .......... 45% 26.9M 3s 2024-03-29T04:11:49.488 INFO:teuthology.orchestra.run.smithi148.stderr: 76450K .......... .......... .......... .......... .......... 45% 24.1M 3s 2024-03-29T04:11:49.490 INFO:teuthology.orchestra.run.smithi148.stderr: 76500K .......... .......... .......... .......... .......... 45% 31.4M 3s 2024-03-29T04:11:49.491 INFO:teuthology.orchestra.run.smithi148.stderr: 76550K .......... .......... .......... .......... .......... 45% 29.9M 3s 2024-03-29T04:11:49.493 INFO:teuthology.orchestra.run.smithi148.stderr: 76600K .......... .......... .......... .......... .......... 45% 30.0M 3s 2024-03-29T04:11:49.495 INFO:teuthology.orchestra.run.smithi148.stderr: 76650K .......... .......... .......... .......... .......... 45% 26.4M 3s 2024-03-29T04:11:49.496 INFO:teuthology.orchestra.run.smithi148.stderr: 76700K .......... .......... .......... .......... .......... 45% 30.9M 3s 2024-03-29T04:11:49.498 INFO:teuthology.orchestra.run.smithi148.stderr: 76750K .......... .......... .......... .......... .......... 45% 30.7M 3s 2024-03-29T04:11:49.500 INFO:teuthology.orchestra.run.smithi148.stderr: 76800K .......... .......... .......... .......... .......... 45% 24.7M 3s 2024-03-29T04:11:49.502 INFO:teuthology.orchestra.run.smithi148.stderr: 76850K .......... .......... .......... .......... .......... 45% 22.7M 3s 2024-03-29T04:11:49.504 INFO:teuthology.orchestra.run.smithi148.stderr: 76900K .......... .......... .......... .......... .......... 45% 26.1M 3s 2024-03-29T04:11:49.506 INFO:teuthology.orchestra.run.smithi148.stderr: 76950K .......... .......... .......... .......... .......... 45% 25.7M 3s 2024-03-29T04:11:49.508 INFO:teuthology.orchestra.run.smithi148.stderr: 77000K .......... .......... .......... .......... .......... 45% 25.4M 3s 2024-03-29T04:11:49.510 INFO:teuthology.orchestra.run.smithi148.stderr: 77050K .......... .......... .......... .......... .......... 45% 21.6M 3s 2024-03-29T04:11:49.512 INFO:teuthology.orchestra.run.smithi148.stderr: 77100K .......... .......... .......... .......... .......... 45% 24.9M 3s 2024-03-29T04:11:49.514 INFO:teuthology.orchestra.run.smithi148.stderr: 77150K .......... .......... .......... .......... .......... 45% 25.7M 3s 2024-03-29T04:11:49.516 INFO:teuthology.orchestra.run.smithi148.stderr: 77200K .......... .......... .......... .......... .......... 45% 25.1M 3s 2024-03-29T04:11:49.518 INFO:teuthology.orchestra.run.smithi148.stderr: 77250K .......... .......... .......... .......... .......... 45% 20.1M 3s 2024-03-29T04:11:49.520 INFO:teuthology.orchestra.run.smithi148.stderr: 77300K .......... .......... .......... .......... .......... 45% 29.5M 3s 2024-03-29T04:11:49.522 INFO:teuthology.orchestra.run.smithi148.stderr: 77350K .......... .......... .......... .......... .......... 45% 29.7M 3s 2024-03-29T04:11:49.523 INFO:teuthology.orchestra.run.smithi148.stderr: 77400K .......... .......... .......... .......... .......... 45% 29.7M 3s 2024-03-29T04:11:49.525 INFO:teuthology.orchestra.run.smithi148.stderr: 77450K .......... .......... .......... .......... .......... 45% 25.9M 3s 2024-03-29T04:11:49.527 INFO:teuthology.orchestra.run.smithi148.stderr: 77500K .......... .......... .......... .......... .......... 46% 29.1M 3s 2024-03-29T04:11:49.529 INFO:teuthology.orchestra.run.smithi148.stderr: 77550K .......... .......... .......... .......... .......... 46% 29.4M 3s 2024-03-29T04:11:49.530 INFO:teuthology.orchestra.run.smithi148.stderr: 77600K .......... .......... .......... .......... .......... 46% 29.5M 3s 2024-03-29T04:11:49.532 INFO:teuthology.orchestra.run.smithi148.stderr: 77650K .......... .......... .......... .......... .......... 46% 24.8M 3s 2024-03-29T04:11:49.534 INFO:teuthology.orchestra.run.smithi148.stderr: 77700K .......... .......... .......... .......... .......... 46% 28.5M 3s 2024-03-29T04:11:49.536 INFO:teuthology.orchestra.run.smithi148.stderr: 77750K .......... .......... .......... .......... .......... 46% 28.8M 3s 2024-03-29T04:11:49.538 INFO:teuthology.orchestra.run.smithi148.stderr: 77800K .......... .......... .......... .......... .......... 46% 27.7M 3s 2024-03-29T04:11:49.541 INFO:teuthology.orchestra.run.smithi148.stderr: 77850K .......... .......... .......... .......... .......... 46% 35.2M 3s 2024-03-29T04:11:49.541 INFO:teuthology.orchestra.run.smithi148.stderr: 77900K .......... .......... .......... .......... .......... 46% 41.6M 3s 2024-03-29T04:11:49.541 INFO:teuthology.orchestra.run.smithi148.stderr: 77950K .......... .......... .......... .......... .......... 46% 39.8M 3s 2024-03-29T04:11:49.543 INFO:teuthology.orchestra.run.smithi148.stderr: 78000K .......... .......... .......... .......... .......... 46% 40.8M 3s 2024-03-29T04:11:49.544 INFO:teuthology.orchestra.run.smithi148.stderr: 78050K .......... .......... .......... .......... .......... 46% 33.7M 3s 2024-03-29T04:11:49.546 INFO:teuthology.orchestra.run.smithi148.stderr: 78100K .......... .......... .......... .......... .......... 46% 21.1M 3s 2024-03-29T04:11:49.548 INFO:teuthology.orchestra.run.smithi148.stderr: 78150K .......... .......... .......... .......... .......... 46% 27.1M 3s 2024-03-29T04:11:49.550 INFO:teuthology.orchestra.run.smithi148.stderr: 78200K .......... .......... .......... .......... .......... 46% 26.8M 3s 2024-03-29T04:11:49.552 INFO:teuthology.orchestra.run.smithi148.stderr: 78250K .......... .......... .......... .......... .......... 46% 29.0M 3s 2024-03-29T04:11:49.553 INFO:teuthology.orchestra.run.smithi148.stderr: 78300K .......... .......... .......... .......... .......... 46% 30.8M 3s 2024-03-29T04:11:49.554 INFO:teuthology.orchestra.run.smithi148.stderr: 78350K .......... .......... .......... .......... .......... 46% 31.8M 3s 2024-03-29T04:11:49.556 INFO:teuthology.orchestra.run.smithi148.stderr: 78400K .......... .......... .......... .......... .......... 46% 31.8M 3s 2024-03-29T04:11:49.558 INFO:teuthology.orchestra.run.smithi148.stderr: 78450K .......... .......... .......... .......... .......... 46% 28.4M 3s 2024-03-29T04:11:49.559 INFO:teuthology.orchestra.run.smithi148.stderr: 78500K .......... .......... .......... .......... .......... 46% 32.2M 3s 2024-03-29T04:11:49.561 INFO:teuthology.orchestra.run.smithi148.stderr: 78550K .......... .......... .......... .......... .......... 46% 31.8M 3s 2024-03-29T04:11:49.562 INFO:teuthology.orchestra.run.smithi148.stderr: 78600K .......... .......... .......... .......... .......... 46% 32.2M 3s 2024-03-29T04:11:49.568 INFO:teuthology.orchestra.run.smithi148.stderr: 78650K .......... .......... .......... .......... .......... 46% 26.8M 3s 2024-03-29T04:11:49.568 INFO:teuthology.orchestra.run.smithi148.stderr: 78700K .......... .......... .......... .......... .......... 46% 31.9M 3s 2024-03-29T04:11:49.569 INFO:teuthology.orchestra.run.smithi148.stderr: 78750K .......... .......... .......... .......... .......... 46% 34.4M 3s 2024-03-29T04:11:49.569 INFO:teuthology.orchestra.run.smithi148.stderr: 78800K .......... .......... .......... .......... .......... 46% 37.0M 3s 2024-03-29T04:11:49.570 INFO:teuthology.orchestra.run.smithi148.stderr: 78850K .......... .......... .......... .......... .......... 46% 30.8M 3s 2024-03-29T04:11:49.571 INFO:teuthology.orchestra.run.smithi148.stderr: 78900K .......... .......... .......... .......... .......... 46% 37.3M 3s 2024-03-29T04:11:49.573 INFO:teuthology.orchestra.run.smithi148.stderr: 78950K .......... .......... .......... .......... .......... 46% 33.0M 3s 2024-03-29T04:11:49.575 INFO:teuthology.orchestra.run.smithi148.stderr: 79000K .......... .......... .......... .......... .......... 46% 28.3M 3s 2024-03-29T04:11:49.577 INFO:teuthology.orchestra.run.smithi148.stderr: 79050K .......... .......... .......... .......... .......... 46% 24.2M 3s 2024-03-29T04:11:49.578 INFO:teuthology.orchestra.run.smithi148.stderr: 79100K .......... .......... .......... .......... .......... 46% 34.1M 3s 2024-03-29T04:11:49.580 INFO:teuthology.orchestra.run.smithi148.stderr: 79150K .......... .......... .......... .......... .......... 46% 26.7M 3s 2024-03-29T04:11:49.582 INFO:teuthology.orchestra.run.smithi148.stderr: 79200K .......... .......... .......... .......... .......... 47% 26.5M 3s 2024-03-29T04:11:49.584 INFO:teuthology.orchestra.run.smithi148.stderr: 79250K .......... .......... .......... .......... .......... 47% 20.1M 3s 2024-03-29T04:11:49.586 INFO:teuthology.orchestra.run.smithi148.stderr: 79300K .......... .......... .......... .......... .......... 47% 25.0M 3s 2024-03-29T04:11:49.588 INFO:teuthology.orchestra.run.smithi148.stderr: 79350K .......... .......... .......... .......... .......... 47% 28.6M 3s 2024-03-29T04:11:49.590 INFO:teuthology.orchestra.run.smithi148.stderr: 79400K .......... .......... .......... .......... .......... 47% 26.0M 3s 2024-03-29T04:11:49.592 INFO:teuthology.orchestra.run.smithi148.stderr: 79450K .......... .......... .......... .......... .......... 47% 23.8M 3s 2024-03-29T04:11:49.593 INFO:teuthology.orchestra.run.smithi148.stderr: 79500K .......... .......... .......... .......... .......... 47% 33.3M 3s 2024-03-29T04:11:49.595 INFO:teuthology.orchestra.run.smithi148.stderr: 79550K .......... .......... .......... .......... .......... 47% 31.7M 3s 2024-03-29T04:11:49.596 INFO:teuthology.orchestra.run.smithi148.stderr: 79600K .......... .......... .......... .......... .......... 47% 32.8M 3s 2024-03-29T04:11:49.598 INFO:teuthology.orchestra.run.smithi148.stderr: 79650K .......... .......... .......... .......... .......... 47% 27.8M 3s 2024-03-29T04:11:49.599 INFO:teuthology.orchestra.run.smithi148.stderr: 79700K .......... .......... .......... .......... .......... 47% 32.6M 3s 2024-03-29T04:11:49.601 INFO:teuthology.orchestra.run.smithi148.stderr: 79750K .......... .......... .......... .......... .......... 47% 32.8M 3s 2024-03-29T04:11:49.602 INFO:teuthology.orchestra.run.smithi148.stderr: 79800K .......... .......... .......... .......... .......... 47% 31.2M 3s 2024-03-29T04:11:49.604 INFO:teuthology.orchestra.run.smithi148.stderr: 79850K .......... .......... .......... .......... .......... 47% 27.0M 3s 2024-03-29T04:11:49.606 INFO:teuthology.orchestra.run.smithi148.stderr: 79900K .......... .......... .......... .......... .......... 47% 32.1M 3s 2024-03-29T04:11:49.607 INFO:teuthology.orchestra.run.smithi148.stderr: 79950K .......... .......... .......... .......... .......... 47% 32.5M 3s 2024-03-29T04:11:49.609 INFO:teuthology.orchestra.run.smithi148.stderr: 80000K .......... .......... .......... .......... .......... 47% 31.2M 3s 2024-03-29T04:11:49.611 INFO:teuthology.orchestra.run.smithi148.stderr: 80050K .......... .......... .......... .......... .......... 47% 27.0M 3s 2024-03-29T04:11:49.612 INFO:teuthology.orchestra.run.smithi148.stderr: 80100K .......... .......... .......... .......... .......... 47% 31.2M 3s 2024-03-29T04:11:49.614 INFO:teuthology.orchestra.run.smithi148.stderr: 80150K .......... .......... .......... .......... .......... 47% 30.8M 3s 2024-03-29T04:11:49.615 INFO:teuthology.orchestra.run.smithi148.stderr: 80200K .......... .......... .......... .......... .......... 47% 31.5M 3s 2024-03-29T04:11:49.617 INFO:teuthology.orchestra.run.smithi148.stderr: 80250K .......... .......... .......... .......... .......... 47% 25.1M 3s 2024-03-29T04:11:49.619 INFO:teuthology.orchestra.run.smithi148.stderr: 80300K .......... .......... .......... .......... .......... 47% 29.0M 3s 2024-03-29T04:11:49.621 INFO:teuthology.orchestra.run.smithi148.stderr: 80350K .......... .......... .......... .......... .......... 47% 29.6M 3s 2024-03-29T04:11:49.622 INFO:teuthology.orchestra.run.smithi148.stderr: 80400K .......... .......... .......... .......... .......... 47% 29.2M 3s 2024-03-29T04:11:49.624 INFO:teuthology.orchestra.run.smithi148.stderr: 80450K .......... .......... .......... .......... .......... 47% 24.8M 3s 2024-03-29T04:11:49.626 INFO:teuthology.orchestra.run.smithi148.stderr: 80500K .......... .......... .......... .......... .......... 47% 28.6M 3s 2024-03-29T04:11:49.628 INFO:teuthology.orchestra.run.smithi148.stderr: 80550K .......... .......... .......... .......... .......... 47% 23.5M 3s 2024-03-29T04:11:49.630 INFO:teuthology.orchestra.run.smithi148.stderr: 80600K .......... .......... .......... .......... .......... 47% 28.7M 3s 2024-03-29T04:11:49.632 INFO:teuthology.orchestra.run.smithi148.stderr: 80650K .......... .......... .......... .......... .......... 47% 21.2M 3s 2024-03-29T04:11:49.634 INFO:teuthology.orchestra.run.smithi148.stderr: 80700K .......... .......... .......... .......... .......... 47% 25.6M 3s 2024-03-29T04:11:49.636 INFO:teuthology.orchestra.run.smithi148.stderr: 80750K .......... .......... .......... .......... .......... 47% 27.6M 3s 2024-03-29T04:11:49.638 INFO:teuthology.orchestra.run.smithi148.stderr: 80800K .......... .......... .......... .......... .......... 47% 26.0M 3s 2024-03-29T04:11:49.640 INFO:teuthology.orchestra.run.smithi148.stderr: 80850K .......... .......... .......... .......... .......... 47% 23.0M 3s 2024-03-29T04:11:49.642 INFO:teuthology.orchestra.run.smithi148.stderr: 80900K .......... .......... .......... .......... .......... 48% 25.7M 3s 2024-03-29T04:11:49.643 INFO:teuthology.orchestra.run.smithi148.stderr: 80950K .......... .......... .......... .......... .......... 48% 26.2M 3s 2024-03-29T04:11:49.646 INFO:teuthology.orchestra.run.smithi148.stderr: 81000K .......... .......... .......... .......... .......... 48% 26.5M 3s 2024-03-29T04:11:49.647 INFO:teuthology.orchestra.run.smithi148.stderr: 81050K .......... .......... .......... .......... .......... 48% 23.1M 3s 2024-03-29T04:11:49.649 INFO:teuthology.orchestra.run.smithi148.stderr: 81100K .......... .......... .......... .......... .......... 48% 24.6M 3s 2024-03-29T04:11:49.651 INFO:teuthology.orchestra.run.smithi148.stderr: 81150K .......... .......... .......... .......... .......... 48% 25.3M 3s 2024-03-29T04:11:49.654 INFO:teuthology.orchestra.run.smithi148.stderr: 81200K .......... .......... .......... .......... .......... 48% 25.2M 3s 2024-03-29T04:11:49.656 INFO:teuthology.orchestra.run.smithi148.stderr: 81250K .......... .......... .......... .......... .......... 48% 21.1M 3s 2024-03-29T04:11:49.658 INFO:teuthology.orchestra.run.smithi148.stderr: 81300K .......... .......... .......... .......... .......... 48% 24.6M 3s 2024-03-29T04:11:49.660 INFO:teuthology.orchestra.run.smithi148.stderr: 81350K .......... .......... .......... .......... .......... 48% 24.8M 3s 2024-03-29T04:11:49.662 INFO:teuthology.orchestra.run.smithi148.stderr: 81400K .......... .......... .......... .......... .......... 48% 24.8M 3s 2024-03-29T04:11:49.664 INFO:teuthology.orchestra.run.smithi148.stderr: 81450K .......... .......... .......... .......... .......... 48% 21.5M 3s 2024-03-29T04:11:49.666 INFO:teuthology.orchestra.run.smithi148.stderr: 81500K .......... .......... .......... .......... .......... 48% 25.3M 3s 2024-03-29T04:11:49.668 INFO:teuthology.orchestra.run.smithi148.stderr: 81550K .......... .......... .......... .......... .......... 48% 24.6M 3s 2024-03-29T04:11:49.670 INFO:teuthology.orchestra.run.smithi148.stderr: 81600K .......... .......... .......... .......... .......... 48% 24.4M 3s 2024-03-29T04:11:49.672 INFO:teuthology.orchestra.run.smithi148.stderr: 81650K .......... .......... .......... .......... .......... 48% 21.1M 3s 2024-03-29T04:11:49.674 INFO:teuthology.orchestra.run.smithi148.stderr: 81700K .......... .......... .......... .......... .......... 48% 24.5M 3s 2024-03-29T04:11:49.676 INFO:teuthology.orchestra.run.smithi148.stderr: 81750K .......... .......... .......... .......... .......... 48% 24.1M 3s 2024-03-29T04:11:49.678 INFO:teuthology.orchestra.run.smithi148.stderr: 81800K .......... .......... .......... .......... .......... 48% 23.9M 3s 2024-03-29T04:11:49.680 INFO:teuthology.orchestra.run.smithi148.stderr: 81850K .......... .......... .......... .......... .......... 48% 20.6M 3s 2024-03-29T04:11:49.683 INFO:teuthology.orchestra.run.smithi148.stderr: 81900K .......... .......... .......... .......... .......... 48% 24.0M 3s 2024-03-29T04:11:49.684 INFO:teuthology.orchestra.run.smithi148.stderr: 81950K .......... .......... .......... .......... .......... 48% 32.3M 3s 2024-03-29T04:11:49.686 INFO:teuthology.orchestra.run.smithi148.stderr: 82000K .......... .......... .......... .......... .......... 48% 34.6M 3s 2024-03-29T04:11:49.687 INFO:teuthology.orchestra.run.smithi148.stderr: 82050K .......... .......... .......... .......... .......... 48% 28.1M 3s 2024-03-29T04:11:49.689 INFO:teuthology.orchestra.run.smithi148.stderr: 82100K .......... .......... .......... .......... .......... 48% 28.4M 3s 2024-03-29T04:11:49.691 INFO:teuthology.orchestra.run.smithi148.stderr: 82150K .......... .......... .......... .......... .......... 48% 27.2M 3s 2024-03-29T04:11:49.693 INFO:teuthology.orchestra.run.smithi148.stderr: 82200K .......... .......... .......... .......... .......... 48% 30.8M 3s 2024-03-29T04:11:49.694 INFO:teuthology.orchestra.run.smithi148.stderr: 82250K .......... .......... .......... .......... .......... 48% 24.4M 3s 2024-03-29T04:11:49.696 INFO:teuthology.orchestra.run.smithi148.stderr: 82300K .......... .......... .......... .......... .......... 48% 29.6M 3s 2024-03-29T04:11:49.697 INFO:teuthology.orchestra.run.smithi148.stderr: 82350K .......... .......... .......... .......... .......... 48% 37.5M 3s 2024-03-29T04:11:49.699 INFO:teuthology.orchestra.run.smithi148.stderr: 82400K .......... .......... .......... .......... .......... 48% 38.4M 3s 2024-03-29T04:11:49.700 INFO:teuthology.orchestra.run.smithi148.stderr: 82450K .......... .......... .......... .......... .......... 48% 27.1M 3s 2024-03-29T04:11:49.702 INFO:teuthology.orchestra.run.smithi148.stderr: 82500K .......... .......... .......... .......... .......... 48% 35.1M 3s 2024-03-29T04:11:49.703 INFO:teuthology.orchestra.run.smithi148.stderr: 82550K .......... .......... .......... .......... .......... 49% 37.0M 3s 2024-03-29T04:11:49.705 INFO:teuthology.orchestra.run.smithi148.stderr: 82600K .......... .......... .......... .......... .......... 49% 36.2M 3s 2024-03-29T04:11:49.706 INFO:teuthology.orchestra.run.smithi148.stderr: 82650K .......... .......... .......... .......... .......... 49% 31.1M 3s 2024-03-29T04:11:49.708 INFO:teuthology.orchestra.run.smithi148.stderr: 82700K .......... .......... .......... .......... .......... 49% 36.5M 3s 2024-03-29T04:11:49.709 INFO:teuthology.orchestra.run.smithi148.stderr: 82750K .......... .......... .......... .......... .......... 49% 35.8M 3s 2024-03-29T04:11:49.710 INFO:teuthology.orchestra.run.smithi148.stderr: 82800K .......... .......... .......... .......... .......... 49% 33.8M 3s 2024-03-29T04:11:49.712 INFO:teuthology.orchestra.run.smithi148.stderr: 82850K .......... .......... .......... .......... .......... 49% 23.3M 3s 2024-03-29T04:11:49.714 INFO:teuthology.orchestra.run.smithi148.stderr: 82900K .......... .......... .......... .......... .......... 49% 33.5M 3s 2024-03-29T04:11:49.715 INFO:teuthology.orchestra.run.smithi148.stderr: 82950K .......... .......... .......... .......... .......... 49% 35.5M 3s 2024-03-29T04:11:49.716 INFO:teuthology.orchestra.run.smithi148.stderr: 83000K .......... .......... .......... .......... .......... 49% 35.8M 3s 2024-03-29T04:11:49.718 INFO:teuthology.orchestra.run.smithi148.stderr: 83050K .......... .......... .......... .......... .......... 49% 30.1M 3s 2024-03-29T04:11:49.719 INFO:teuthology.orchestra.run.smithi148.stderr: 83100K .......... .......... .......... .......... .......... 49% 35.1M 3s 2024-03-29T04:11:49.721 INFO:teuthology.orchestra.run.smithi148.stderr: 83150K .......... .......... .......... .......... .......... 49% 35.2M 3s 2024-03-29T04:11:49.722 INFO:teuthology.orchestra.run.smithi148.stderr: 83200K .......... .......... .......... .......... .......... 49% 35.0M 3s 2024-03-29T04:11:49.724 INFO:teuthology.orchestra.run.smithi148.stderr: 83250K .......... .......... .......... .......... .......... 49% 28.3M 3s 2024-03-29T04:11:49.726 INFO:teuthology.orchestra.run.smithi148.stderr: 83300K .......... .......... .......... .......... .......... 49% 31.3M 3s 2024-03-29T04:11:49.727 INFO:teuthology.orchestra.run.smithi148.stderr: 83350K .......... .......... .......... .......... .......... 49% 29.9M 3s 2024-03-29T04:11:49.729 INFO:teuthology.orchestra.run.smithi148.stderr: 83400K .......... .......... .......... .......... .......... 49% 33.3M 3s 2024-03-29T04:11:49.730 INFO:teuthology.orchestra.run.smithi148.stderr: 83450K .......... .......... .......... .......... .......... 49% 28.9M 3s 2024-03-29T04:11:49.732 INFO:teuthology.orchestra.run.smithi148.stderr: 83500K .......... .......... .......... .......... .......... 49% 28.3M 3s 2024-03-29T04:11:49.734 INFO:teuthology.orchestra.run.smithi148.stderr: 83550K .......... .......... .......... .......... .......... 49% 28.6M 3s 2024-03-29T04:11:49.735 INFO:teuthology.orchestra.run.smithi148.stderr: 83600K .......... .......... .......... .......... .......... 49% 28.7M 3s 2024-03-29T04:11:49.737 INFO:teuthology.orchestra.run.smithi148.stderr: 83650K .......... .......... .......... .......... .......... 49% 24.9M 3s 2024-03-29T04:11:49.739 INFO:teuthology.orchestra.run.smithi148.stderr: 83700K .......... .......... .......... .......... .......... 49% 29.3M 3s 2024-03-29T04:11:49.741 INFO:teuthology.orchestra.run.smithi148.stderr: 83750K .......... .......... .......... .......... .......... 49% 30.8M 3s 2024-03-29T04:11:49.742 INFO:teuthology.orchestra.run.smithi148.stderr: 83800K .......... .......... .......... .......... .......... 49% 31.3M 3s 2024-03-29T04:11:49.744 INFO:teuthology.orchestra.run.smithi148.stderr: 83850K .......... .......... .......... .......... .......... 49% 26.2M 3s 2024-03-29T04:11:49.746 INFO:teuthology.orchestra.run.smithi148.stderr: 83900K .......... .......... .......... .......... .......... 49% 30.1M 3s 2024-03-29T04:11:49.747 INFO:teuthology.orchestra.run.smithi148.stderr: 83950K .......... .......... .......... .......... .......... 49% 31.1M 3s 2024-03-29T04:11:49.749 INFO:teuthology.orchestra.run.smithi148.stderr: 84000K .......... .......... .......... .......... .......... 49% 30.9M 3s 2024-03-29T04:11:49.751 INFO:teuthology.orchestra.run.smithi148.stderr: 84050K .......... .......... .......... .......... .......... 49% 25.9M 3s 2024-03-29T04:11:49.752 INFO:teuthology.orchestra.run.smithi148.stderr: 84100K .......... .......... .......... .......... .......... 49% 30.7M 3s 2024-03-29T04:11:49.754 INFO:teuthology.orchestra.run.smithi148.stderr: 84150K .......... .......... .......... .......... .......... 49% 30.3M 3s 2024-03-29T04:11:49.756 INFO:teuthology.orchestra.run.smithi148.stderr: 84200K .......... .......... .......... .......... .......... 49% 30.3M 3s 2024-03-29T04:11:49.757 INFO:teuthology.orchestra.run.smithi148.stderr: 84250K .......... .......... .......... .......... .......... 50% 26.0M 3s 2024-03-29T04:11:49.759 INFO:teuthology.orchestra.run.smithi148.stderr: 84300K .......... .......... .......... .......... .......... 50% 29.9M 3s 2024-03-29T04:11:49.761 INFO:teuthology.orchestra.run.smithi148.stderr: 84350K .......... .......... .......... .......... .......... 50% 30.1M 3s 2024-03-29T04:11:49.762 INFO:teuthology.orchestra.run.smithi148.stderr: 84400K .......... .......... .......... .......... .......... 50% 30.3M 3s 2024-03-29T04:11:49.764 INFO:teuthology.orchestra.run.smithi148.stderr: 84450K .......... .......... .......... .......... .......... 50% 25.7M 3s 2024-03-29T04:11:49.766 INFO:teuthology.orchestra.run.smithi148.stderr: 84500K .......... .......... .......... .......... .......... 50% 29.5M 3s 2024-03-29T04:11:49.768 INFO:teuthology.orchestra.run.smithi148.stderr: 84550K .......... .......... .......... .......... .......... 50% 29.3M 3s 2024-03-29T04:11:49.769 INFO:teuthology.orchestra.run.smithi148.stderr: 84600K .......... .......... .......... .......... .......... 50% 28.9M 3s 2024-03-29T04:11:49.775 INFO:teuthology.orchestra.run.smithi148.stderr: 84650K .......... .......... .......... .......... .......... 50% 24.3M 3s 2024-03-29T04:11:49.775 INFO:teuthology.orchestra.run.smithi148.stderr: 84700K .......... .......... .......... .......... .......... 50% 29.0M 3s 2024-03-29T04:11:49.775 INFO:teuthology.orchestra.run.smithi148.stderr: 84750K .......... .......... .......... .......... .......... 50% 28.5M 3s 2024-03-29T04:11:49.776 INFO:teuthology.orchestra.run.smithi148.stderr: 84800K .......... .......... .......... .......... .......... 50% 29.1M 3s 2024-03-29T04:11:49.778 INFO:teuthology.orchestra.run.smithi148.stderr: 84850K .......... .......... .......... .......... .......... 50% 24.0M 3s 2024-03-29T04:11:49.780 INFO:teuthology.orchestra.run.smithi148.stderr: 84900K .......... .......... .......... .......... .......... 50% 28.2M 3s 2024-03-29T04:11:49.782 INFO:teuthology.orchestra.run.smithi148.stderr: 84950K .......... .......... .......... .......... .......... 50% 28.5M 3s 2024-03-29T04:11:49.784 INFO:teuthology.orchestra.run.smithi148.stderr: 85000K .......... .......... .......... .......... .......... 50% 27.9M 3s 2024-03-29T04:11:49.786 INFO:teuthology.orchestra.run.smithi148.stderr: 85050K .......... .......... .......... .......... .......... 50% 23.2M 3s 2024-03-29T04:11:49.787 INFO:teuthology.orchestra.run.smithi148.stderr: 85100K .......... .......... .......... .......... .......... 50% 27.6M 3s 2024-03-29T04:11:49.789 INFO:teuthology.orchestra.run.smithi148.stderr: 85150K .......... .......... .......... .......... .......... 50% 28.3M 3s 2024-03-29T04:11:49.791 INFO:teuthology.orchestra.run.smithi148.stderr: 85200K .......... .......... .......... .......... .......... 50% 27.6M 3s 2024-03-29T04:11:49.793 INFO:teuthology.orchestra.run.smithi148.stderr: 85250K .......... .......... .......... .......... .......... 50% 24.1M 3s 2024-03-29T04:11:49.795 INFO:teuthology.orchestra.run.smithi148.stderr: 85300K .......... .......... .......... .......... .......... 50% 27.3M 3s 2024-03-29T04:11:49.796 INFO:teuthology.orchestra.run.smithi148.stderr: 85350K .......... .......... .......... .......... .......... 50% 29.5M 3s 2024-03-29T04:11:49.798 INFO:teuthology.orchestra.run.smithi148.stderr: 85400K .......... .......... .......... .......... .......... 50% 29.6M 3s 2024-03-29T04:11:49.800 INFO:teuthology.orchestra.run.smithi148.stderr: 85450K .......... .......... .......... .......... .......... 50% 22.6M 3s 2024-03-29T04:11:49.802 INFO:teuthology.orchestra.run.smithi148.stderr: 85500K .......... .......... .......... .......... .......... 50% 26.1M 3s 2024-03-29T04:11:49.804 INFO:teuthology.orchestra.run.smithi148.stderr: 85550K .......... .......... .......... .......... .......... 50% 26.8M 3s 2024-03-29T04:11:49.806 INFO:teuthology.orchestra.run.smithi148.stderr: 85600K .......... .......... .......... .......... .......... 50% 27.4M 3s 2024-03-29T04:11:49.808 INFO:teuthology.orchestra.run.smithi148.stderr: 85650K .......... .......... .......... .......... .......... 50% 23.1M 3s 2024-03-29T04:11:49.810 INFO:teuthology.orchestra.run.smithi148.stderr: 85700K .......... .......... .......... .......... .......... 50% 27.2M 3s 2024-03-29T04:11:49.811 INFO:teuthology.orchestra.run.smithi148.stderr: 85750K .......... .......... .......... .......... .......... 50% 26.8M 3s 2024-03-29T04:11:49.813 INFO:teuthology.orchestra.run.smithi148.stderr: 85800K .......... .......... .......... .......... .......... 50% 25.8M 3s 2024-03-29T04:11:49.815 INFO:teuthology.orchestra.run.smithi148.stderr: 85850K .......... .......... .......... .......... .......... 50% 22.2M 3s 2024-03-29T04:11:49.817 INFO:teuthology.orchestra.run.smithi148.stderr: 85900K .......... .......... .......... .......... .......... 50% 26.4M 3s 2024-03-29T04:11:49.819 INFO:teuthology.orchestra.run.smithi148.stderr: 85950K .......... .......... .......... .......... .......... 51% 25.7M 3s 2024-03-29T04:11:49.821 INFO:teuthology.orchestra.run.smithi148.stderr: 86000K .......... .......... .......... .......... .......... 51% 26.7M 3s 2024-03-29T04:11:49.823 INFO:teuthology.orchestra.run.smithi148.stderr: 86050K .......... .......... .......... .......... .......... 51% 25.3M 3s 2024-03-29T04:11:49.824 INFO:teuthology.orchestra.run.smithi148.stderr: 86100K .......... .......... .......... .......... .......... 51% 30.9M 3s 2024-03-29T04:11:49.826 INFO:teuthology.orchestra.run.smithi148.stderr: 86150K .......... .......... .......... .......... .......... 51% 40.2M 3s 2024-03-29T04:11:49.827 INFO:teuthology.orchestra.run.smithi148.stderr: 86200K .......... .......... .......... .......... .......... 51% 39.7M 3s 2024-03-29T04:11:49.828 INFO:teuthology.orchestra.run.smithi148.stderr: 86250K .......... .......... .......... .......... .......... 51% 31.8M 3s 2024-03-29T04:11:49.830 INFO:teuthology.orchestra.run.smithi148.stderr: 86300K .......... .......... .......... .......... .......... 51% 38.3M 3s 2024-03-29T04:11:49.831 INFO:teuthology.orchestra.run.smithi148.stderr: 86350K .......... .......... .......... .......... .......... 51% 37.8M 3s 2024-03-29T04:11:49.832 INFO:teuthology.orchestra.run.smithi148.stderr: 86400K .......... .......... .......... .......... .......... 51% 37.7M 3s 2024-03-29T04:11:49.834 INFO:teuthology.orchestra.run.smithi148.stderr: 86450K .......... .......... .......... .......... .......... 51% 32.6M 3s 2024-03-29T04:11:49.835 INFO:teuthology.orchestra.run.smithi148.stderr: 86500K .......... .......... .......... .......... .......... 51% 30.3M 3s 2024-03-29T04:11:49.837 INFO:teuthology.orchestra.run.smithi148.stderr: 86550K .......... .......... .......... .......... .......... 51% 33.5M 3s 2024-03-29T04:11:49.839 INFO:teuthology.orchestra.run.smithi148.stderr: 86600K .......... .......... .......... .......... .......... 51% 33.2M 3s 2024-03-29T04:11:49.840 INFO:teuthology.orchestra.run.smithi148.stderr: 86650K .......... .......... .......... .......... .......... 51% 28.1M 3s 2024-03-29T04:11:49.842 INFO:teuthology.orchestra.run.smithi148.stderr: 86700K .......... .......... .......... .......... .......... 51% 33.0M 3s 2024-03-29T04:11:49.843 INFO:teuthology.orchestra.run.smithi148.stderr: 86750K .......... .......... .......... .......... .......... 51% 32.8M 3s 2024-03-29T04:11:49.845 INFO:teuthology.orchestra.run.smithi148.stderr: 86800K .......... .......... .......... .......... .......... 51% 32.0M 3s 2024-03-29T04:11:49.846 INFO:teuthology.orchestra.run.smithi148.stderr: 86850K .......... .......... .......... .......... .......... 51% 27.7M 3s 2024-03-29T04:11:49.848 INFO:teuthology.orchestra.run.smithi148.stderr: 86900K .......... .......... .......... .......... .......... 51% 32.1M 3s 2024-03-29T04:11:49.849 INFO:teuthology.orchestra.run.smithi148.stderr: 86950K .......... .......... .......... .......... .......... 51% 32.0M 3s 2024-03-29T04:11:49.851 INFO:teuthology.orchestra.run.smithi148.stderr: 87000K .......... .......... .......... .......... .......... 51% 32.0M 3s 2024-03-29T04:11:49.853 INFO:teuthology.orchestra.run.smithi148.stderr: 87050K .......... .......... .......... .......... .......... 51% 27.4M 3s 2024-03-29T04:11:49.854 INFO:teuthology.orchestra.run.smithi148.stderr: 87100K .......... .......... .......... .......... .......... 51% 32.1M 3s 2024-03-29T04:11:49.856 INFO:teuthology.orchestra.run.smithi148.stderr: 87150K .......... .......... .......... .......... .......... 51% 31.1M 3s 2024-03-29T04:11:49.857 INFO:teuthology.orchestra.run.smithi148.stderr: 87200K .......... .......... .......... .......... .......... 51% 32.1M 3s 2024-03-29T04:11:49.859 INFO:teuthology.orchestra.run.smithi148.stderr: 87250K .......... .......... .......... .......... .......... 51% 26.4M 3s 2024-03-29T04:11:49.861 INFO:teuthology.orchestra.run.smithi148.stderr: 87300K .......... .......... .......... .......... .......... 51% 30.9M 3s 2024-03-29T04:11:49.862 INFO:teuthology.orchestra.run.smithi148.stderr: 87350K .......... .......... .......... .......... .......... 51% 31.0M 3s 2024-03-29T04:11:49.864 INFO:teuthology.orchestra.run.smithi148.stderr: 87400K .......... .......... .......... .......... .......... 51% 31.0M 3s 2024-03-29T04:11:49.866 INFO:teuthology.orchestra.run.smithi148.stderr: 87450K .......... .......... .......... .......... .......... 51% 26.4M 3s 2024-03-29T04:11:49.867 INFO:teuthology.orchestra.run.smithi148.stderr: 87500K .......... .......... .......... .......... .......... 51% 30.4M 3s 2024-03-29T04:11:49.869 INFO:teuthology.orchestra.run.smithi148.stderr: 87550K .......... .......... .......... .......... .......... 51% 30.4M 3s 2024-03-29T04:11:49.871 INFO:teuthology.orchestra.run.smithi148.stderr: 87600K .......... .......... .......... .......... .......... 51% 30.1M 3s 2024-03-29T04:11:49.872 INFO:teuthology.orchestra.run.smithi148.stderr: 87650K .......... .......... .......... .......... .......... 52% 25.9M 3s 2024-03-29T04:11:49.874 INFO:teuthology.orchestra.run.smithi148.stderr: 87700K .......... .......... .......... .......... .......... 52% 30.2M 3s 2024-03-29T04:11:49.876 INFO:teuthology.orchestra.run.smithi148.stderr: 87750K .......... .......... .......... .......... .......... 52% 29.6M 3s 2024-03-29T04:11:49.878 INFO:teuthology.orchestra.run.smithi148.stderr: 87800K .......... .......... .......... .......... .......... 52% 28.0M 3s 2024-03-29T04:11:49.880 INFO:teuthology.orchestra.run.smithi148.stderr: 87850K .......... .......... .......... .......... .......... 52% 24.1M 3s 2024-03-29T04:11:49.881 INFO:teuthology.orchestra.run.smithi148.stderr: 87900K .......... .......... .......... .......... .......... 52% 27.9M 3s 2024-03-29T04:11:49.883 INFO:teuthology.orchestra.run.smithi148.stderr: 87950K .......... .......... .......... .......... .......... 52% 31.3M 3s 2024-03-29T04:11:49.885 INFO:teuthology.orchestra.run.smithi148.stderr: 88000K .......... .......... .......... .......... .......... 52% 29.3M 3s 2024-03-29T04:11:49.887 INFO:teuthology.orchestra.run.smithi148.stderr: 88050K .......... .......... .......... .......... .......... 52% 27.3M 3s 2024-03-29T04:11:49.888 INFO:teuthology.orchestra.run.smithi148.stderr: 88100K .......... .......... .......... .......... .......... 52% 31.6M 3s 2024-03-29T04:11:49.889 INFO:teuthology.orchestra.run.smithi148.stderr: 88150K .......... .......... .......... .......... .......... 52% 30.5M 3s 2024-03-29T04:11:49.891 INFO:teuthology.orchestra.run.smithi148.stderr: 88200K .......... .......... .......... .......... .......... 52% 31.3M 3s 2024-03-29T04:11:49.893 INFO:teuthology.orchestra.run.smithi148.stderr: 88250K .......... .......... .......... .......... .......... 52% 26.5M 3s 2024-03-29T04:11:49.895 INFO:teuthology.orchestra.run.smithi148.stderr: 88300K .......... .......... .......... .......... .......... 52% 31.1M 3s 2024-03-29T04:11:49.896 INFO:teuthology.orchestra.run.smithi148.stderr: 88350K .......... .......... .......... .......... .......... 52% 31.5M 3s 2024-03-29T04:11:49.898 INFO:teuthology.orchestra.run.smithi148.stderr: 88400K .......... .......... .......... .......... .......... 52% 30.7M 3s 2024-03-29T04:11:49.899 INFO:teuthology.orchestra.run.smithi148.stderr: 88450K .......... .......... .......... .......... .......... 52% 26.4M 3s 2024-03-29T04:11:49.901 INFO:teuthology.orchestra.run.smithi148.stderr: 88500K .......... .......... .......... .......... .......... 52% 30.8M 3s 2024-03-29T04:11:49.902 INFO:teuthology.orchestra.run.smithi148.stderr: 88550K .......... .......... .......... .......... .......... 52% 30.7M 3s 2024-03-29T04:11:49.904 INFO:teuthology.orchestra.run.smithi148.stderr: 88600K .......... .......... .......... .......... .......... 52% 31.3M 3s 2024-03-29T04:11:49.906 INFO:teuthology.orchestra.run.smithi148.stderr: 88650K .......... .......... .......... .......... .......... 52% 25.9M 3s 2024-03-29T04:11:49.908 INFO:teuthology.orchestra.run.smithi148.stderr: 88700K .......... .......... .......... .......... .......... 52% 30.0M 3s 2024-03-29T04:11:49.909 INFO:teuthology.orchestra.run.smithi148.stderr: 88750K .......... .......... .......... .......... .......... 52% 30.3M 3s 2024-03-29T04:11:49.911 INFO:teuthology.orchestra.run.smithi148.stderr: 88800K .......... .......... .......... .......... .......... 52% 30.2M 3s 2024-03-29T04:11:49.913 INFO:teuthology.orchestra.run.smithi148.stderr: 88850K .......... .......... .......... .......... .......... 52% 26.0M 3s 2024-03-29T04:11:49.914 INFO:teuthology.orchestra.run.smithi148.stderr: 88900K .......... .......... .......... .......... .......... 52% 30.2M 3s 2024-03-29T04:11:49.916 INFO:teuthology.orchestra.run.smithi148.stderr: 88950K .......... .......... .......... .......... .......... 52% 30.2M 3s 2024-03-29T04:11:49.918 INFO:teuthology.orchestra.run.smithi148.stderr: 89000K .......... .......... .......... .......... .......... 52% 30.3M 3s 2024-03-29T04:11:49.919 INFO:teuthology.orchestra.run.smithi148.stderr: 89050K .......... .......... .......... .......... .......... 52% 25.6M 3s 2024-03-29T04:11:49.921 INFO:teuthology.orchestra.run.smithi148.stderr: 89100K .......... .......... .......... .......... .......... 52% 30.0M 3s 2024-03-29T04:11:49.923 INFO:teuthology.orchestra.run.smithi148.stderr: 89150K .......... .......... .......... .......... .......... 52% 29.6M 3s 2024-03-29T04:11:49.925 INFO:teuthology.orchestra.run.smithi148.stderr: 89200K .......... .......... .......... .......... .......... 52% 28.8M 3s 2024-03-29T04:11:49.926 INFO:teuthology.orchestra.run.smithi148.stderr: 89250K .......... .......... .......... .......... .......... 52% 26.0M 3s 2024-03-29T04:11:49.928 INFO:teuthology.orchestra.run.smithi148.stderr: 89300K .......... .......... .......... .......... .......... 53% 29.8M 3s 2024-03-29T04:11:49.929 INFO:teuthology.orchestra.run.smithi148.stderr: 89350K .......... .......... .......... .......... .......... 53% 30.6M 3s 2024-03-29T04:11:49.931 INFO:teuthology.orchestra.run.smithi148.stderr: 89400K .......... .......... .......... .......... .......... 53% 30.8M 3s 2024-03-29T04:11:49.933 INFO:teuthology.orchestra.run.smithi148.stderr: 89450K .......... .......... .......... .......... .......... 53% 26.2M 3s 2024-03-29T04:11:49.935 INFO:teuthology.orchestra.run.smithi148.stderr: 89500K .......... .......... .......... .......... .......... 53% 28.1M 3s 2024-03-29T04:11:49.936 INFO:teuthology.orchestra.run.smithi148.stderr: 89550K .......... .......... .......... .......... .......... 53% 29.6M 3s 2024-03-29T04:11:49.938 INFO:teuthology.orchestra.run.smithi148.stderr: 89600K .......... .......... .......... .......... .......... 53% 29.6M 3s 2024-03-29T04:11:49.940 INFO:teuthology.orchestra.run.smithi148.stderr: 89650K .......... .......... .......... .......... .......... 53% 24.6M 3s 2024-03-29T04:11:49.942 INFO:teuthology.orchestra.run.smithi148.stderr: 89700K .......... .......... .......... .......... .......... 53% 28.4M 3s 2024-03-29T04:11:49.943 INFO:teuthology.orchestra.run.smithi148.stderr: 89750K .......... .......... .......... .......... .......... 53% 28.0M 3s 2024-03-29T04:11:49.945 INFO:teuthology.orchestra.run.smithi148.stderr: 89800K .......... .......... .......... .......... .......... 53% 28.2M 3s 2024-03-29T04:11:49.947 INFO:teuthology.orchestra.run.smithi148.stderr: 89850K .......... .......... .......... .......... .......... 53% 24.1M 3s 2024-03-29T04:11:49.949 INFO:teuthology.orchestra.run.smithi148.stderr: 89900K .......... .......... .......... .......... .......... 53% 28.1M 3s 2024-03-29T04:11:49.951 INFO:teuthology.orchestra.run.smithi148.stderr: 89950K .......... .......... .......... .......... .......... 53% 28.0M 3s 2024-03-29T04:11:49.953 INFO:teuthology.orchestra.run.smithi148.stderr: 90000K .......... .......... .......... .......... .......... 53% 28.3M 3s 2024-03-29T04:11:49.954 INFO:teuthology.orchestra.run.smithi148.stderr: 90050K .......... .......... .......... .......... .......... 53% 24.0M 3s 2024-03-29T04:11:49.956 INFO:teuthology.orchestra.run.smithi148.stderr: 90100K .......... .......... .......... .......... .......... 53% 30.6M 3s 2024-03-29T04:11:49.957 INFO:teuthology.orchestra.run.smithi148.stderr: 90150K .......... .......... .......... .......... .......... 53% 41.2M 3s 2024-03-29T04:11:49.959 INFO:teuthology.orchestra.run.smithi148.stderr: 90200K .......... .......... .......... .......... .......... 53% 37.7M 3s 2024-03-29T04:11:49.960 INFO:teuthology.orchestra.run.smithi148.stderr: 90250K .......... .......... .......... .......... .......... 53% 34.1M 3s 2024-03-29T04:11:49.961 INFO:teuthology.orchestra.run.smithi148.stderr: 90300K .......... .......... .......... .......... .......... 53% 38.7M 3s 2024-03-29T04:11:49.963 INFO:teuthology.orchestra.run.smithi148.stderr: 90350K .......... .......... .......... .......... .......... 53% 32.2M 3s 2024-03-29T04:11:49.964 INFO:teuthology.orchestra.run.smithi148.stderr: 90400K .......... .......... .......... .......... .......... 53% 33.5M 3s 2024-03-29T04:11:49.966 INFO:teuthology.orchestra.run.smithi148.stderr: 90450K .......... .......... .......... .......... .......... 53% 30.9M 3s 2024-03-29T04:11:49.967 INFO:teuthology.orchestra.run.smithi148.stderr: 90500K .......... .......... .......... .......... .......... 53% 31.5M 3s 2024-03-29T04:11:49.969 INFO:teuthology.orchestra.run.smithi148.stderr: 90550K .......... .......... .......... .......... .......... 53% 29.1M 3s 2024-03-29T04:11:49.971 INFO:teuthology.orchestra.run.smithi148.stderr: 90600K .......... .......... .......... .......... .......... 53% 29.6M 3s 2024-03-29T04:11:49.972 INFO:teuthology.orchestra.run.smithi148.stderr: 90650K .......... .......... .......... .......... .......... 53% 27.2M 3s 2024-03-29T04:11:49.974 INFO:teuthology.orchestra.run.smithi148.stderr: 90700K .......... .......... .......... .......... .......... 53% 33.1M 3s 2024-03-29T04:11:49.980 INFO:teuthology.orchestra.run.smithi148.stderr: 90750K .......... .......... .......... .......... .......... 53% 37.1M 3s 2024-03-29T04:11:49.980 INFO:teuthology.orchestra.run.smithi148.stderr: 90800K .......... .......... .......... .......... .......... 53% 36.6M 3s 2024-03-29T04:11:49.980 INFO:teuthology.orchestra.run.smithi148.stderr: 90850K .......... .......... .......... .......... .......... 53% 30.8M 3s 2024-03-29T04:11:49.980 INFO:teuthology.orchestra.run.smithi148.stderr: 90900K .......... .......... .......... .......... .......... 53% 36.5M 3s 2024-03-29T04:11:49.981 INFO:teuthology.orchestra.run.smithi148.stderr: 90950K .......... .......... .......... .......... .......... 53% 37.5M 3s 2024-03-29T04:11:49.982 INFO:teuthology.orchestra.run.smithi148.stderr: 91000K .......... .......... .......... .......... .......... 54% 37.3M 3s 2024-03-29T04:11:49.984 INFO:teuthology.orchestra.run.smithi148.stderr: 91050K .......... .......... .......... .......... .......... 54% 30.9M 3s 2024-03-29T04:11:49.985 INFO:teuthology.orchestra.run.smithi148.stderr: 91100K .......... .......... .......... .......... .......... 54% 37.4M 3s 2024-03-29T04:11:49.986 INFO:teuthology.orchestra.run.smithi148.stderr: 91150K .......... .......... .......... .......... .......... 54% 37.4M 3s 2024-03-29T04:11:49.988 INFO:teuthology.orchestra.run.smithi148.stderr: 91200K .......... .......... .......... .......... .......... 54% 36.1M 3s 2024-03-29T04:11:49.989 INFO:teuthology.orchestra.run.smithi148.stderr: 91250K .......... .......... .......... .......... .......... 54% 30.2M 3s 2024-03-29T04:11:49.991 INFO:teuthology.orchestra.run.smithi148.stderr: 91300K .......... .......... .......... .......... .......... 54% 34.6M 3s 2024-03-29T04:11:49.992 INFO:teuthology.orchestra.run.smithi148.stderr: 91350K .......... .......... .......... .......... .......... 54% 35.0M 3s 2024-03-29T04:11:49.993 INFO:teuthology.orchestra.run.smithi148.stderr: 91400K .......... .......... .......... .......... .......... 54% 35.1M 3s 2024-03-29T04:11:49.996 INFO:teuthology.orchestra.run.smithi148.stderr: 91450K .......... .......... .......... .......... .......... 54% 27.1M 3s 2024-03-29T04:11:49.997 INFO:teuthology.orchestra.run.smithi148.stderr: 91500K .......... .......... .......... .......... .......... 54% 27.2M 3s 2024-03-29T04:11:49.999 INFO:teuthology.orchestra.run.smithi148.stderr: 91550K .......... .......... .......... .......... .......... 54% 27.5M 3s 2024-03-29T04:11:50.001 INFO:teuthology.orchestra.run.smithi148.stderr: 91600K .......... .......... .......... .......... .......... 54% 27.9M 3s 2024-03-29T04:11:50.003 INFO:teuthology.orchestra.run.smithi148.stderr: 91650K .......... .......... .......... .......... .......... 54% 24.5M 3s 2024-03-29T04:11:50.004 INFO:teuthology.orchestra.run.smithi148.stderr: 91700K .......... .......... .......... .......... .......... 54% 28.1M 3s 2024-03-29T04:11:50.006 INFO:teuthology.orchestra.run.smithi148.stderr: 91750K .......... .......... .......... .......... .......... 54% 27.8M 3s 2024-03-29T04:11:50.008 INFO:teuthology.orchestra.run.smithi148.stderr: 91800K .......... .......... .......... .......... .......... 54% 27.1M 3s 2024-03-29T04:11:50.010 INFO:teuthology.orchestra.run.smithi148.stderr: 91850K .......... .......... .......... .......... .......... 54% 23.3M 3s 2024-03-29T04:11:50.012 INFO:teuthology.orchestra.run.smithi148.stderr: 91900K .......... .......... .......... .......... .......... 54% 26.7M 3s 2024-03-29T04:11:50.014 INFO:teuthology.orchestra.run.smithi148.stderr: 91950K .......... .......... .......... .......... .......... 54% 27.5M 3s 2024-03-29T04:11:50.016 INFO:teuthology.orchestra.run.smithi148.stderr: 92000K .......... .......... .......... .......... .......... 54% 26.7M 3s 2024-03-29T04:11:50.018 INFO:teuthology.orchestra.run.smithi148.stderr: 92050K .......... .......... .......... .......... .......... 54% 22.8M 3s 2024-03-29T04:11:50.020 INFO:teuthology.orchestra.run.smithi148.stderr: 92100K .......... .......... .......... .......... .......... 54% 27.3M 3s 2024-03-29T04:11:50.021 INFO:teuthology.orchestra.run.smithi148.stderr: 92150K .......... .......... .......... .......... .......... 54% 26.4M 3s 2024-03-29T04:11:50.023 INFO:teuthology.orchestra.run.smithi148.stderr: 92200K .......... .......... .......... .......... .......... 54% 26.4M 3s 2024-03-29T04:11:50.025 INFO:teuthology.orchestra.run.smithi148.stderr: 92250K .......... .......... .......... .......... .......... 54% 22.7M 3s 2024-03-29T04:11:50.027 INFO:teuthology.orchestra.run.smithi148.stderr: 92300K .......... .......... .......... .......... .......... 54% 26.5M 3s 2024-03-29T04:11:50.029 INFO:teuthology.orchestra.run.smithi148.stderr: 92350K .......... .......... .......... .......... .......... 54% 26.2M 3s 2024-03-29T04:11:50.031 INFO:teuthology.orchestra.run.smithi148.stderr: 92400K .......... .......... .......... .......... .......... 54% 26.2M 3s 2024-03-29T04:11:50.033 INFO:teuthology.orchestra.run.smithi148.stderr: 92450K .......... .......... .......... .......... .......... 54% 22.3M 3s 2024-03-29T04:11:50.035 INFO:teuthology.orchestra.run.smithi148.stderr: 92500K .......... .......... .......... .......... .......... 54% 25.9M 3s 2024-03-29T04:11:50.037 INFO:teuthology.orchestra.run.smithi148.stderr: 92550K .......... .......... .......... .......... .......... 54% 26.1M 3s 2024-03-29T04:11:50.039 INFO:teuthology.orchestra.run.smithi148.stderr: 92600K .......... .......... .......... .......... .......... 54% 25.3M 3s 2024-03-29T04:11:50.041 INFO:teuthology.orchestra.run.smithi148.stderr: 92650K .......... .......... .......... .......... .......... 54% 23.4M 3s 2024-03-29T04:11:50.042 INFO:teuthology.orchestra.run.smithi148.stderr: 92700K .......... .......... .......... .......... .......... 55% 31.1M 3s 2024-03-29T04:11:50.044 INFO:teuthology.orchestra.run.smithi148.stderr: 92750K .......... .......... .......... .......... .......... 55% 30.5M 3s 2024-03-29T04:11:50.046 INFO:teuthology.orchestra.run.smithi148.stderr: 92800K .......... .......... .......... .......... .......... 55% 30.3M 3s 2024-03-29T04:11:50.048 INFO:teuthology.orchestra.run.smithi148.stderr: 92850K .......... .......... .......... .......... .......... 55% 26.0M 3s 2024-03-29T04:11:50.049 INFO:teuthology.orchestra.run.smithi148.stderr: 92900K .......... .......... .......... .......... .......... 55% 30.2M 3s 2024-03-29T04:11:50.051 INFO:teuthology.orchestra.run.smithi148.stderr: 92950K .......... .......... .......... .......... .......... 55% 29.9M 3s 2024-03-29T04:11:50.052 INFO:teuthology.orchestra.run.smithi148.stderr: 93000K .......... .......... .......... .......... .......... 55% 30.2M 3s 2024-03-29T04:11:50.054 INFO:teuthology.orchestra.run.smithi148.stderr: 93050K .......... .......... .......... .......... .......... 55% 25.5M 3s 2024-03-29T04:11:50.056 INFO:teuthology.orchestra.run.smithi148.stderr: 93100K .......... .......... .......... .......... .......... 55% 29.3M 3s 2024-03-29T04:11:50.058 INFO:teuthology.orchestra.run.smithi148.stderr: 93150K .......... .......... .......... .......... .......... 55% 29.3M 3s 2024-03-29T04:11:50.059 INFO:teuthology.orchestra.run.smithi148.stderr: 93200K .......... .......... .......... .......... .......... 55% 29.6M 3s 2024-03-29T04:11:50.061 INFO:teuthology.orchestra.run.smithi148.stderr: 93250K .......... .......... .......... .......... .......... 55% 24.8M 3s 2024-03-29T04:11:50.063 INFO:teuthology.orchestra.run.smithi148.stderr: 93300K .......... .......... .......... .......... .......... 55% 29.0M 3s 2024-03-29T04:11:50.064 INFO:teuthology.orchestra.run.smithi148.stderr: 93350K .......... .......... .......... .......... .......... 55% 28.9M 3s 2024-03-29T04:11:50.066 INFO:teuthology.orchestra.run.smithi148.stderr: 93400K .......... .......... .......... .......... .......... 55% 28.9M 3s 2024-03-29T04:11:50.068 INFO:teuthology.orchestra.run.smithi148.stderr: 93450K .......... .......... .......... .......... .......... 55% 24.3M 3s 2024-03-29T04:11:50.070 INFO:teuthology.orchestra.run.smithi148.stderr: 93500K .......... .......... .......... .......... .......... 55% 27.0M 3s 2024-03-29T04:11:50.072 INFO:teuthology.orchestra.run.smithi148.stderr: 93550K .......... .......... .......... .......... .......... 55% 27.3M 3s 2024-03-29T04:11:50.074 INFO:teuthology.orchestra.run.smithi148.stderr: 93600K .......... .......... .......... .......... .......... 55% 27.3M 3s 2024-03-29T04:11:50.076 INFO:teuthology.orchestra.run.smithi148.stderr: 93650K .......... .......... .......... .......... .......... 55% 23.0M 3s 2024-03-29T04:11:50.078 INFO:teuthology.orchestra.run.smithi148.stderr: 93700K .......... .......... .......... .......... .......... 55% 26.5M 3s 2024-03-29T04:11:50.080 INFO:teuthology.orchestra.run.smithi148.stderr: 93750K .......... .......... .......... .......... .......... 55% 25.6M 3s 2024-03-29T04:11:50.082 INFO:teuthology.orchestra.run.smithi148.stderr: 93800K .......... .......... .......... .......... .......... 55% 25.0M 3s 2024-03-29T04:11:50.084 INFO:teuthology.orchestra.run.smithi148.stderr: 93850K .......... .......... .......... .......... .......... 55% 21.6M 3s 2024-03-29T04:11:50.086 INFO:teuthology.orchestra.run.smithi148.stderr: 93900K .......... .......... .......... .......... .......... 55% 24.9M 3s 2024-03-29T04:11:50.088 INFO:teuthology.orchestra.run.smithi148.stderr: 93950K .......... .......... .......... .......... .......... 55% 24.9M 3s 2024-03-29T04:11:50.090 INFO:teuthology.orchestra.run.smithi148.stderr: 94000K .......... .......... .......... .......... .......... 55% 24.8M 3s 2024-03-29T04:11:50.092 INFO:teuthology.orchestra.run.smithi148.stderr: 94050K .......... .......... .......... .......... .......... 55% 20.9M 3s 2024-03-29T04:11:50.094 INFO:teuthology.orchestra.run.smithi148.stderr: 94100K .......... .......... .......... .......... .......... 55% 25.1M 3s 2024-03-29T04:11:50.096 INFO:teuthology.orchestra.run.smithi148.stderr: 94150K .......... .......... .......... .......... .......... 55% 25.4M 3s 2024-03-29T04:11:50.098 INFO:teuthology.orchestra.run.smithi148.stderr: 94200K .......... .......... .......... .......... .......... 55% 25.5M 3s 2024-03-29T04:11:50.100 INFO:teuthology.orchestra.run.smithi148.stderr: 94250K .......... .......... .......... .......... .......... 55% 25.1M 3s 2024-03-29T04:11:50.102 INFO:teuthology.orchestra.run.smithi148.stderr: 94300K .......... .......... .......... .......... .......... 55% 29.2M 3s 2024-03-29T04:11:50.103 INFO:teuthology.orchestra.run.smithi148.stderr: 94350K .......... .......... .......... .......... .......... 56% 34.5M 3s 2024-03-29T04:11:50.104 INFO:teuthology.orchestra.run.smithi148.stderr: 94400K .......... .......... .......... .......... .......... 56% 36.8M 3s 2024-03-29T04:11:50.106 INFO:teuthology.orchestra.run.smithi148.stderr: 94450K .......... .......... .......... .......... .......... 56% 30.8M 3s 2024-03-29T04:11:50.107 INFO:teuthology.orchestra.run.smithi148.stderr: 94500K .......... .......... .......... .......... .......... 56% 36.3M 3s 2024-03-29T04:11:50.108 INFO:teuthology.orchestra.run.smithi148.stderr: 94550K .......... .......... .......... .......... .......... 56% 36.7M 3s 2024-03-29T04:11:50.110 INFO:teuthology.orchestra.run.smithi148.stderr: 94600K .......... .......... .......... .......... .......... 56% 34.2M 3s 2024-03-29T04:11:50.112 INFO:teuthology.orchestra.run.smithi148.stderr: 94650K .......... .......... .......... .......... .......... 56% 28.7M 3s 2024-03-29T04:11:50.113 INFO:teuthology.orchestra.run.smithi148.stderr: 94700K .......... .......... .......... .......... .......... 56% 33.8M 3s 2024-03-29T04:11:50.114 INFO:teuthology.orchestra.run.smithi148.stderr: 94750K .......... .......... .......... .......... .......... 56% 35.5M 3s 2024-03-29T04:11:50.116 INFO:teuthology.orchestra.run.smithi148.stderr: 94800K .......... .......... .......... .......... .......... 56% 34.6M 3s 2024-03-29T04:11:50.117 INFO:teuthology.orchestra.run.smithi148.stderr: 94850K .......... .......... .......... .......... .......... 56% 29.8M 3s 2024-03-29T04:11:50.119 INFO:teuthology.orchestra.run.smithi148.stderr: 94900K .......... .......... .......... .......... .......... 56% 33.9M 3s 2024-03-29T04:11:50.120 INFO:teuthology.orchestra.run.smithi148.stderr: 94950K .......... .......... .......... .......... .......... 56% 33.5M 3s 2024-03-29T04:11:50.122 INFO:teuthology.orchestra.run.smithi148.stderr: 95000K .......... .......... .......... .......... .......... 56% 33.9M 3s 2024-03-29T04:11:50.123 INFO:teuthology.orchestra.run.smithi148.stderr: 95050K .......... .......... .......... .......... .......... 56% 29.2M 3s 2024-03-29T04:11:50.125 INFO:teuthology.orchestra.run.smithi148.stderr: 95100K .......... .......... .......... .......... .......... 56% 34.0M 3s 2024-03-29T04:11:50.126 INFO:teuthology.orchestra.run.smithi148.stderr: 95150K .......... .......... .......... .......... .......... 56% 33.7M 3s 2024-03-29T04:11:50.128 INFO:teuthology.orchestra.run.smithi148.stderr: 95200K .......... .......... .......... .......... .......... 56% 31.0M 3s 2024-03-29T04:11:50.130 INFO:teuthology.orchestra.run.smithi148.stderr: 95250K .......... .......... .......... .......... .......... 56% 27.5M 3s 2024-03-29T04:11:50.131 INFO:teuthology.orchestra.run.smithi148.stderr: 95300K .......... .......... .......... .......... .......... 56% 33.3M 3s 2024-03-29T04:11:50.133 INFO:teuthology.orchestra.run.smithi148.stderr: 95350K .......... .......... .......... .......... .......... 56% 28.1M 3s 2024-03-29T04:11:50.135 INFO:teuthology.orchestra.run.smithi148.stderr: 95400K .......... .......... .......... .......... .......... 56% 31.0M 3s 2024-03-29T04:11:50.136 INFO:teuthology.orchestra.run.smithi148.stderr: 95450K .......... .......... .......... .......... .......... 56% 23.6M 3s 2024-03-29T04:11:50.138 INFO:teuthology.orchestra.run.smithi148.stderr: 95500K .......... .......... .......... .......... .......... 56% 29.1M 3s 2024-03-29T04:11:50.140 INFO:teuthology.orchestra.run.smithi148.stderr: 95550K .......... .......... .......... .......... .......... 56% 28.0M 3s 2024-03-29T04:11:50.142 INFO:teuthology.orchestra.run.smithi148.stderr: 95600K .......... .......... .......... .......... .......... 56% 28.4M 3s 2024-03-29T04:11:50.144 INFO:teuthology.orchestra.run.smithi148.stderr: 95650K .......... .......... .......... .......... .......... 56% 24.6M 3s 2024-03-29T04:11:50.145 INFO:teuthology.orchestra.run.smithi148.stderr: 95700K .......... .......... .......... .......... .......... 56% 28.6M 3s 2024-03-29T04:11:50.147 INFO:teuthology.orchestra.run.smithi148.stderr: 95750K .......... .......... .......... .......... .......... 56% 27.5M 3s 2024-03-29T04:11:50.149 INFO:teuthology.orchestra.run.smithi148.stderr: 95800K .......... .......... .......... .......... .......... 56% 27.9M 3s 2024-03-29T04:11:50.151 INFO:teuthology.orchestra.run.smithi148.stderr: 95850K .......... .......... .......... .......... .......... 56% 23.9M 3s 2024-03-29T04:11:50.153 INFO:teuthology.orchestra.run.smithi148.stderr: 95900K .......... .......... .......... .......... .......... 56% 27.9M 3s 2024-03-29T04:11:50.154 INFO:teuthology.orchestra.run.smithi148.stderr: 95950K .......... .......... .......... .......... .......... 56% 27.0M 3s 2024-03-29T04:11:50.156 INFO:teuthology.orchestra.run.smithi148.stderr: 96000K .......... .......... .......... .......... .......... 56% 28.1M 3s 2024-03-29T04:11:50.158 INFO:teuthology.orchestra.run.smithi148.stderr: 96050K .......... .......... .......... .......... .......... 57% 23.8M 3s 2024-03-29T04:11:50.160 INFO:teuthology.orchestra.run.smithi148.stderr: 96100K .......... .......... .......... .......... .......... 57% 27.0M 3s 2024-03-29T04:11:50.162 INFO:teuthology.orchestra.run.smithi148.stderr: 96150K .......... .......... .......... .......... .......... 57% 27.0M 3s 2024-03-29T04:11:50.164 INFO:teuthology.orchestra.run.smithi148.stderr: 96200K .......... .......... .......... .......... .......... 57% 27.0M 3s 2024-03-29T04:11:50.166 INFO:teuthology.orchestra.run.smithi148.stderr: 96250K .......... .......... .......... .......... .......... 57% 23.0M 3s 2024-03-29T04:11:50.168 INFO:teuthology.orchestra.run.smithi148.stderr: 96300K .......... .......... .......... .......... .......... 57% 26.7M 3s 2024-03-29T04:11:50.169 INFO:teuthology.orchestra.run.smithi148.stderr: 96350K .......... .......... .......... .......... .......... 57% 26.1M 3s 2024-03-29T04:11:50.172 INFO:teuthology.orchestra.run.smithi148.stderr: 96400K .......... .......... .......... .......... .......... 57% 26.4M 3s 2024-03-29T04:11:50.173 INFO:teuthology.orchestra.run.smithi148.stderr: 96450K .......... .......... .......... .......... .......... 57% 22.0M 3s 2024-03-29T04:11:50.175 INFO:teuthology.orchestra.run.smithi148.stderr: 96500K .......... .......... .......... .......... .......... 57% 27.2M 3s 2024-03-29T04:11:50.177 INFO:teuthology.orchestra.run.smithi148.stderr: 96550K .......... .......... .......... .......... .......... 57% 25.6M 3s 2024-03-29T04:11:50.179 INFO:teuthology.orchestra.run.smithi148.stderr: 96600K .......... .......... .......... .......... .......... 57% 25.9M 3s 2024-03-29T04:11:50.181 INFO:teuthology.orchestra.run.smithi148.stderr: 96650K .......... .......... .......... .......... .......... 57% 22.3M 3s 2024-03-29T04:11:50.183 INFO:teuthology.orchestra.run.smithi148.stderr: 96700K .......... .......... .......... .......... .......... 57% 26.0M 3s 2024-03-29T04:11:50.185 INFO:teuthology.orchestra.run.smithi148.stderr: 96750K .......... .......... .......... .......... .......... 57% 26.0M 3s 2024-03-29T04:11:50.187 INFO:teuthology.orchestra.run.smithi148.stderr: 96800K .......... .......... .......... .......... .......... 57% 30.2M 3s 2024-03-29T04:11:50.188 INFO:teuthology.orchestra.run.smithi148.stderr: 96850K .......... .......... .......... .......... .......... 57% 26.7M 3s 2024-03-29T04:11:50.190 INFO:teuthology.orchestra.run.smithi148.stderr: 96900K .......... .......... .......... .......... .......... 57% 30.2M 3s 2024-03-29T04:11:50.192 INFO:teuthology.orchestra.run.smithi148.stderr: 96950K .......... .......... .......... .......... .......... 57% 30.3M 3s 2024-03-29T04:11:50.193 INFO:teuthology.orchestra.run.smithi148.stderr: 97000K .......... .......... .......... .......... .......... 57% 30.2M 3s 2024-03-29T04:11:50.195 INFO:teuthology.orchestra.run.smithi148.stderr: 97050K .......... .......... .......... .......... .......... 57% 25.2M 3s 2024-03-29T04:11:50.197 INFO:teuthology.orchestra.run.smithi148.stderr: 97100K .......... .......... .......... .......... .......... 57% 28.7M 3s 2024-03-29T04:11:50.199 INFO:teuthology.orchestra.run.smithi148.stderr: 97150K .......... .......... .......... .......... .......... 57% 27.8M 3s 2024-03-29T04:11:50.200 INFO:teuthology.orchestra.run.smithi148.stderr: 97200K .......... .......... .......... .......... .......... 57% 31.3M 3s 2024-03-29T04:11:50.202 INFO:teuthology.orchestra.run.smithi148.stderr: 97250K .......... .......... .......... .......... .......... 57% 28.0M 3s 2024-03-29T04:11:50.203 INFO:teuthology.orchestra.run.smithi148.stderr: 97300K .......... .......... .......... .......... .......... 57% 32.3M 3s 2024-03-29T04:11:50.205 INFO:teuthology.orchestra.run.smithi148.stderr: 97350K .......... .......... .......... .......... .......... 57% 31.3M 3s 2024-03-29T04:11:50.207 INFO:teuthology.orchestra.run.smithi148.stderr: 97400K .......... .......... .......... .......... .......... 57% 30.9M 3s 2024-03-29T04:11:50.208 INFO:teuthology.orchestra.run.smithi148.stderr: 97450K .......... .......... .......... .......... .......... 57% 27.4M 3s 2024-03-29T04:11:50.210 INFO:teuthology.orchestra.run.smithi148.stderr: 97500K .......... .......... .......... .......... .......... 57% 32.1M 3s 2024-03-29T04:11:50.212 INFO:teuthology.orchestra.run.smithi148.stderr: 97550K .......... .......... .......... .......... .......... 57% 31.8M 3s 2024-03-29T04:11:50.213 INFO:teuthology.orchestra.run.smithi148.stderr: 97600K .......... .......... .......... .......... .......... 57% 31.7M 3s 2024-03-29T04:11:50.215 INFO:teuthology.orchestra.run.smithi148.stderr: 97650K .......... .......... .......... .......... .......... 57% 27.4M 3s 2024-03-29T04:11:50.216 INFO:teuthology.orchestra.run.smithi148.stderr: 97700K .......... .......... .......... .......... .......... 57% 31.6M 3s 2024-03-29T04:11:50.218 INFO:teuthology.orchestra.run.smithi148.stderr: 97750K .......... .......... .......... .......... .......... 58% 31.3M 3s 2024-03-29T04:11:50.220 INFO:teuthology.orchestra.run.smithi148.stderr: 97800K .......... .......... .......... .......... .......... 58% 31.2M 3s 2024-03-29T04:11:50.221 INFO:teuthology.orchestra.run.smithi148.stderr: 97850K .......... .......... .......... .......... .......... 58% 26.9M 3s 2024-03-29T04:11:50.223 INFO:teuthology.orchestra.run.smithi148.stderr: 97900K .......... .......... .......... .......... .......... 58% 30.9M 3s 2024-03-29T04:11:50.224 INFO:teuthology.orchestra.run.smithi148.stderr: 97950K .......... .......... .......... .......... .......... 58% 31.2M 3s 2024-03-29T04:11:50.226 INFO:teuthology.orchestra.run.smithi148.stderr: 98000K .......... .......... .......... .......... .......... 58% 30.2M 3s 2024-03-29T04:11:50.228 INFO:teuthology.orchestra.run.smithi148.stderr: 98050K .......... .......... .......... .......... .......... 58% 22.1M 3s 2024-03-29T04:11:50.230 INFO:teuthology.orchestra.run.smithi148.stderr: 98100K .......... .......... .......... .......... .......... 58% 24.1M 3s 2024-03-29T04:11:50.232 INFO:teuthology.orchestra.run.smithi148.stderr: 98150K .......... .......... .......... .......... .......... 58% 25.8M 3s 2024-03-29T04:11:50.234 INFO:teuthology.orchestra.run.smithi148.stderr: 98200K .......... .......... .......... .......... .......... 58% 24.0M 3s 2024-03-29T04:11:50.236 INFO:teuthology.orchestra.run.smithi148.stderr: 98250K .......... .......... .......... .......... .......... 58% 21.5M 3s 2024-03-29T04:11:50.238 INFO:teuthology.orchestra.run.smithi148.stderr: 98300K .......... .......... .......... .......... .......... 58% 26.1M 3s 2024-03-29T04:11:50.240 INFO:teuthology.orchestra.run.smithi148.stderr: 98350K .......... .......... .......... .......... .......... 58% 34.9M 3s 2024-03-29T04:11:50.241 INFO:teuthology.orchestra.run.smithi148.stderr: 98400K .......... .......... .......... .......... .......... 58% 33.7M 3s 2024-03-29T04:11:50.243 INFO:teuthology.orchestra.run.smithi148.stderr: 98450K .......... .......... .......... .......... .......... 58% 26.4M 3s 2024-03-29T04:11:50.245 INFO:teuthology.orchestra.run.smithi148.stderr: 98500K .......... .......... .......... .......... .......... 58% 28.1M 3s 2024-03-29T04:11:50.246 INFO:teuthology.orchestra.run.smithi148.stderr: 98550K .......... .......... .......... .......... .......... 58% 32.0M 2s 2024-03-29T04:11:50.248 INFO:teuthology.orchestra.run.smithi148.stderr: 98600K .......... .......... .......... .......... .......... 58% 33.1M 2s 2024-03-29T04:11:50.250 INFO:teuthology.orchestra.run.smithi148.stderr: 98650K .......... .......... .......... .......... .......... 58% 28.1M 2s 2024-03-29T04:11:50.251 INFO:teuthology.orchestra.run.smithi148.stderr: 98700K .......... .......... .......... .......... .......... 58% 30.5M 2s 2024-03-29T04:11:50.253 INFO:teuthology.orchestra.run.smithi148.stderr: 98750K .......... .......... .......... .......... .......... 58% 27.2M 2s 2024-03-29T04:11:50.255 INFO:teuthology.orchestra.run.smithi148.stderr: 98800K .......... .......... .......... .......... .......... 58% 30.4M 2s 2024-03-29T04:11:50.256 INFO:teuthology.orchestra.run.smithi148.stderr: 98850K .......... .......... .......... .......... .......... 58% 25.8M 2s 2024-03-29T04:11:50.258 INFO:teuthology.orchestra.run.smithi148.stderr: 98900K .......... .......... .......... .......... .......... 58% 29.3M 2s 2024-03-29T04:11:50.260 INFO:teuthology.orchestra.run.smithi148.stderr: 98950K .......... .......... .......... .......... .......... 58% 29.5M 2s 2024-03-29T04:11:50.262 INFO:teuthology.orchestra.run.smithi148.stderr: 99000K .......... .......... .......... .......... .......... 58% 29.3M 2s 2024-03-29T04:11:50.263 INFO:teuthology.orchestra.run.smithi148.stderr: 99050K .......... .......... .......... .......... .......... 58% 25.9M 2s 2024-03-29T04:11:50.265 INFO:teuthology.orchestra.run.smithi148.stderr: 99100K .......... .......... .......... .......... .......... 58% 30.6M 2s 2024-03-29T04:11:50.267 INFO:teuthology.orchestra.run.smithi148.stderr: 99150K .......... .......... .......... .......... .......... 58% 29.8M 2s 2024-03-29T04:11:50.268 INFO:teuthology.orchestra.run.smithi148.stderr: 99200K .......... .......... .......... .......... .......... 58% 29.9M 2s 2024-03-29T04:11:50.270 INFO:teuthology.orchestra.run.smithi148.stderr: 99250K .......... .......... .......... .......... .......... 58% 26.8M 2s 2024-03-29T04:11:50.272 INFO:teuthology.orchestra.run.smithi148.stderr: 99300K .......... .......... .......... .......... .......... 58% 31.9M 2s 2024-03-29T04:11:50.273 INFO:teuthology.orchestra.run.smithi148.stderr: 99350K .......... .......... .......... .......... .......... 58% 33.1M 2s 2024-03-29T04:11:50.275 INFO:teuthology.orchestra.run.smithi148.stderr: 99400K .......... .......... .......... .......... .......... 58% 32.1M 2s 2024-03-29T04:11:50.276 INFO:teuthology.orchestra.run.smithi148.stderr: 99450K .......... .......... .......... .......... .......... 59% 27.2M 2s 2024-03-29T04:11:50.278 INFO:teuthology.orchestra.run.smithi148.stderr: 99500K .......... .......... .......... .......... .......... 59% 31.1M 2s 2024-03-29T04:11:50.279 INFO:teuthology.orchestra.run.smithi148.stderr: 99550K .......... .......... .......... .......... .......... 59% 32.8M 2s 2024-03-29T04:11:50.281 INFO:teuthology.orchestra.run.smithi148.stderr: 99600K .......... .......... .......... .......... .......... 59% 31.1M 2s 2024-03-29T04:11:50.283 INFO:teuthology.orchestra.run.smithi148.stderr: 99650K .......... .......... .......... .......... .......... 59% 27.4M 2s 2024-03-29T04:11:50.284 INFO:teuthology.orchestra.run.smithi148.stderr: 99700K .......... .......... .......... .......... .......... 59% 31.7M 2s 2024-03-29T04:11:50.286 INFO:teuthology.orchestra.run.smithi148.stderr: 99750K .......... .......... .......... .......... .......... 59% 30.8M 2s 2024-03-29T04:11:50.288 INFO:teuthology.orchestra.run.smithi148.stderr: 99800K .......... .......... .......... .......... .......... 59% 31.1M 2s 2024-03-29T04:11:50.289 INFO:teuthology.orchestra.run.smithi148.stderr: 99850K .......... .......... .......... .......... .......... 59% 26.1M 2s 2024-03-29T04:11:50.291 INFO:teuthology.orchestra.run.smithi148.stderr: 99900K .......... .......... .......... .......... .......... 59% 31.0M 2s 2024-03-29T04:11:50.293 INFO:teuthology.orchestra.run.smithi148.stderr: 99950K .......... .......... .......... .......... .......... 59% 30.7M 2s 2024-03-29T04:11:50.294 INFO:teuthology.orchestra.run.smithi148.stderr:100000K .......... .......... .......... .......... .......... 59% 30.8M 2s 2024-03-29T04:11:50.296 INFO:teuthology.orchestra.run.smithi148.stderr:100050K .......... .......... .......... .......... .......... 59% 26.2M 2s 2024-03-29T04:11:50.298 INFO:teuthology.orchestra.run.smithi148.stderr:100100K .......... .......... .......... .......... .......... 59% 31.4M 2s 2024-03-29T04:11:50.299 INFO:teuthology.orchestra.run.smithi148.stderr:100150K .......... .......... .......... .......... .......... 59% 29.7M 2s 2024-03-29T04:11:50.301 INFO:teuthology.orchestra.run.smithi148.stderr:100200K .......... .......... .......... .......... .......... 59% 29.9M 2s 2024-03-29T04:11:50.303 INFO:teuthology.orchestra.run.smithi148.stderr:100250K .......... .......... .......... .......... .......... 59% 24.4M 2s 2024-03-29T04:11:50.305 INFO:teuthology.orchestra.run.smithi148.stderr:100300K .......... .......... .......... .......... .......... 59% 28.1M 2s 2024-03-29T04:11:50.306 INFO:teuthology.orchestra.run.smithi148.stderr:100350K .......... .......... .......... .......... .......... 59% 28.2M 2s 2024-03-29T04:11:50.308 INFO:teuthology.orchestra.run.smithi148.stderr:100400K .......... .......... .......... .......... .......... 59% 27.4M 2s 2024-03-29T04:11:50.310 INFO:teuthology.orchestra.run.smithi148.stderr:100450K .......... .......... .......... .......... .......... 59% 23.4M 2s 2024-03-29T04:11:50.312 INFO:teuthology.orchestra.run.smithi148.stderr:100500K .......... .......... .......... .......... .......... 59% 27.6M 2s 2024-03-29T04:11:50.314 INFO:teuthology.orchestra.run.smithi148.stderr:100550K .......... .......... .......... .......... .......... 59% 26.7M 2s 2024-03-29T04:11:50.316 INFO:teuthology.orchestra.run.smithi148.stderr:100600K .......... .......... .......... .......... .......... 59% 27.5M 2s 2024-03-29T04:11:50.318 INFO:teuthology.orchestra.run.smithi148.stderr:100650K .......... .......... .......... .......... .......... 59% 23.4M 2s 2024-03-29T04:11:50.319 INFO:teuthology.orchestra.run.smithi148.stderr:100700K .......... .......... .......... .......... .......... 59% 27.4M 2s 2024-03-29T04:11:50.321 INFO:teuthology.orchestra.run.smithi148.stderr:100750K .......... .......... .......... .......... .......... 59% 26.2M 2s 2024-03-29T04:11:50.323 INFO:teuthology.orchestra.run.smithi148.stderr:100800K .......... .......... .......... .......... .......... 59% 25.4M 2s 2024-03-29T04:11:50.325 INFO:teuthology.orchestra.run.smithi148.stderr:100850K .......... .......... .......... .......... .......... 59% 22.7M 2s 2024-03-29T04:11:50.327 INFO:teuthology.orchestra.run.smithi148.stderr:100900K .......... .......... .......... .......... .......... 59% 26.1M 2s 2024-03-29T04:11:50.329 INFO:teuthology.orchestra.run.smithi148.stderr:100950K .......... .......... .......... .......... .......... 59% 25.6M 2s 2024-03-29T04:11:50.331 INFO:teuthology.orchestra.run.smithi148.stderr:101000K .......... .......... .......... .......... .......... 59% 26.6M 2s 2024-03-29T04:11:50.333 INFO:teuthology.orchestra.run.smithi148.stderr:101050K .......... .......... .......... .......... .......... 59% 21.8M 2s 2024-03-29T04:11:50.335 INFO:teuthology.orchestra.run.smithi148.stderr:101100K .......... .......... .......... .......... .......... 60% 26.3M 2s 2024-03-29T04:11:50.337 INFO:teuthology.orchestra.run.smithi148.stderr:101150K .......... .......... .......... .......... .......... 60% 24.9M 2s 2024-03-29T04:11:50.339 INFO:teuthology.orchestra.run.smithi148.stderr:101200K .......... .......... .......... .......... .......... 60% 25.9M 2s 2024-03-29T04:11:50.341 INFO:teuthology.orchestra.run.smithi148.stderr:101250K .......... .......... .......... .......... .......... 60% 21.9M 2s 2024-03-29T04:11:50.343 INFO:teuthology.orchestra.run.smithi148.stderr:101300K .......... .......... .......... .......... .......... 60% 23.9M 2s 2024-03-29T04:11:50.345 INFO:teuthology.orchestra.run.smithi148.stderr:101350K .......... .......... .......... .......... .......... 60% 23.9M 2s 2024-03-29T04:11:50.347 INFO:teuthology.orchestra.run.smithi148.stderr:101400K .......... .......... .......... .......... .......... 60% 23.7M 2s 2024-03-29T04:11:50.350 INFO:teuthology.orchestra.run.smithi148.stderr:101450K .......... .......... .......... .......... .......... 60% 20.2M 2s 2024-03-29T04:11:50.352 INFO:teuthology.orchestra.run.smithi148.stderr:101500K .......... .......... .......... .......... .......... 60% 24.2M 2s 2024-03-29T04:11:50.354 INFO:teuthology.orchestra.run.smithi148.stderr:101550K .......... .......... .......... .......... .......... 60% 24.1M 2s 2024-03-29T04:11:50.356 INFO:teuthology.orchestra.run.smithi148.stderr:101600K .......... .......... .......... .......... .......... 60% 24.2M 2s 2024-03-29T04:11:50.358 INFO:teuthology.orchestra.run.smithi148.stderr:101650K .......... .......... .......... .......... .......... 60% 20.2M 2s 2024-03-29T04:11:50.360 INFO:teuthology.orchestra.run.smithi148.stderr:101700K .......... .......... .......... .......... .......... 60% 23.6M 2s 2024-03-29T04:11:50.362 INFO:teuthology.orchestra.run.smithi148.stderr:101750K .......... .......... .......... .......... .......... 60% 23.4M 2s 2024-03-29T04:11:50.365 INFO:teuthology.orchestra.run.smithi148.stderr:101800K .......... .......... .......... .......... .......... 60% 22.9M 2s 2024-03-29T04:11:50.367 INFO:teuthology.orchestra.run.smithi148.stderr:101850K .......... .......... .......... .......... .......... 60% 19.8M 2s 2024-03-29T04:11:50.369 INFO:teuthology.orchestra.run.smithi148.stderr:101900K .......... .......... .......... .......... .......... 60% 23.0M 2s 2024-03-29T04:11:50.371 INFO:teuthology.orchestra.run.smithi148.stderr:101950K .......... .......... .......... .......... .......... 60% 23.2M 2s 2024-03-29T04:11:50.373 INFO:teuthology.orchestra.run.smithi148.stderr:102000K .......... .......... .......... .......... .......... 60% 23.1M 2s 2024-03-29T04:11:50.376 INFO:teuthology.orchestra.run.smithi148.stderr:102050K .......... .......... .......... .......... .......... 60% 19.6M 2s 2024-03-29T04:11:50.378 INFO:teuthology.orchestra.run.smithi148.stderr:102100K .......... .......... .......... .......... .......... 60% 23.0M 2s 2024-03-29T04:11:50.380 INFO:teuthology.orchestra.run.smithi148.stderr:102150K .......... .......... .......... .......... .......... 60% 22.6M 2s 2024-03-29T04:11:50.386 INFO:teuthology.orchestra.run.smithi148.stderr:102200K .......... .......... .......... .......... .......... 60% 22.8M 2s 2024-03-29T04:11:50.386 INFO:teuthology.orchestra.run.smithi148.stderr:102250K .......... .......... .......... .......... .......... 60% 19.4M 2s 2024-03-29T04:11:50.387 INFO:teuthology.orchestra.run.smithi148.stderr:102300K .......... .......... .......... .......... .......... 60% 25.9M 2s 2024-03-29T04:11:50.388 INFO:teuthology.orchestra.run.smithi148.stderr:102350K .......... .......... .......... .......... .......... 60% 25.6M 2s 2024-03-29T04:11:50.390 INFO:teuthology.orchestra.run.smithi148.stderr:102400K .......... .......... .......... .......... .......... 60% 27.6M 2s 2024-03-29T04:11:50.392 INFO:teuthology.orchestra.run.smithi148.stderr:102450K .......... .......... .......... .......... .......... 60% 24.1M 2s 2024-03-29T04:11:50.394 INFO:teuthology.orchestra.run.smithi148.stderr:102500K .......... .......... .......... .......... .......... 60% 27.6M 2s 2024-03-29T04:11:50.396 INFO:teuthology.orchestra.run.smithi148.stderr:102550K .......... .......... .......... .......... .......... 60% 29.3M 2s 2024-03-29T04:11:50.398 INFO:teuthology.orchestra.run.smithi148.stderr:102600K .......... .......... .......... .......... .......... 60% 28.8M 2s 2024-03-29T04:11:50.399 INFO:teuthology.orchestra.run.smithi148.stderr:102650K .......... .......... .......... .......... .......... 60% 23.3M 2s 2024-03-29T04:11:50.401 INFO:teuthology.orchestra.run.smithi148.stderr:102700K .......... .......... .......... .......... .......... 60% 27.8M 2s 2024-03-29T04:11:50.403 INFO:teuthology.orchestra.run.smithi148.stderr:102750K .......... .......... .......... .......... .......... 60% 27.7M 2s 2024-03-29T04:11:50.404 INFO:teuthology.orchestra.run.smithi148.stderr:102800K .......... .......... .......... .......... .......... 61% 33.1M 2s 2024-03-29T04:11:50.406 INFO:teuthology.orchestra.run.smithi148.stderr:102850K .......... .......... .......... .......... .......... 61% 32.6M 2s 2024-03-29T04:11:50.407 INFO:teuthology.orchestra.run.smithi148.stderr:102900K .......... .......... .......... .......... .......... 61% 38.9M 2s 2024-03-29T04:11:50.408 INFO:teuthology.orchestra.run.smithi148.stderr:102950K .......... .......... .......... .......... .......... 61% 38.6M 2s 2024-03-29T04:11:50.410 INFO:teuthology.orchestra.run.smithi148.stderr:103000K .......... .......... .......... .......... .......... 61% 38.5M 2s 2024-03-29T04:11:50.411 INFO:teuthology.orchestra.run.smithi148.stderr:103050K .......... .......... .......... .......... .......... 61% 32.9M 2s 2024-03-29T04:11:50.412 INFO:teuthology.orchestra.run.smithi148.stderr:103100K .......... .......... .......... .......... .......... 61% 37.7M 2s 2024-03-29T04:11:50.414 INFO:teuthology.orchestra.run.smithi148.stderr:103150K .......... .......... .......... .......... .......... 61% 37.3M 2s 2024-03-29T04:11:50.415 INFO:teuthology.orchestra.run.smithi148.stderr:103200K .......... .......... .......... .......... .......... 61% 38.0M 2s 2024-03-29T04:11:50.417 INFO:teuthology.orchestra.run.smithi148.stderr:103250K .......... .......... .......... .......... .......... 61% 32.5M 2s 2024-03-29T04:11:50.418 INFO:teuthology.orchestra.run.smithi148.stderr:103300K .......... .......... .......... .......... .......... 61% 37.9M 2s 2024-03-29T04:11:50.419 INFO:teuthology.orchestra.run.smithi148.stderr:103350K .......... .......... .......... .......... .......... 61% 37.6M 2s 2024-03-29T04:11:50.420 INFO:teuthology.orchestra.run.smithi148.stderr:103400K .......... .......... .......... .......... .......... 61% 37.1M 2s 2024-03-29T04:11:50.422 INFO:teuthology.orchestra.run.smithi148.stderr:103450K .......... .......... .......... .......... .......... 61% 30.3M 2s 2024-03-29T04:11:50.423 INFO:teuthology.orchestra.run.smithi148.stderr:103500K .......... .......... .......... .......... .......... 61% 36.7M 2s 2024-03-29T04:11:50.425 INFO:teuthology.orchestra.run.smithi148.stderr:103550K .......... .......... .......... .......... .......... 61% 36.7M 2s 2024-03-29T04:11:50.426 INFO:teuthology.orchestra.run.smithi148.stderr:103600K .......... .......... .......... .......... .......... 61% 31.2M 2s 2024-03-29T04:11:50.428 INFO:teuthology.orchestra.run.smithi148.stderr:103650K .......... .......... .......... .......... .......... 61% 28.9M 2s 2024-03-29T04:11:50.429 INFO:teuthology.orchestra.run.smithi148.stderr:103700K .......... .......... .......... .......... .......... 61% 35.7M 2s 2024-03-29T04:11:50.431 INFO:teuthology.orchestra.run.smithi148.stderr:103750K .......... .......... .......... .......... .......... 61% 36.4M 2s 2024-03-29T04:11:50.432 INFO:teuthology.orchestra.run.smithi148.stderr:103800K .......... .......... .......... .......... .......... 61% 37.3M 2s 2024-03-29T04:11:50.434 INFO:teuthology.orchestra.run.smithi148.stderr:103850K .......... .......... .......... .......... .......... 61% 30.6M 2s 2024-03-29T04:11:50.435 INFO:teuthology.orchestra.run.smithi148.stderr:103900K .......... .......... .......... .......... .......... 61% 34.5M 2s 2024-03-29T04:11:50.437 INFO:teuthology.orchestra.run.smithi148.stderr:103950K .......... .......... .......... .......... .......... 61% 32.4M 2s 2024-03-29T04:11:50.438 INFO:teuthology.orchestra.run.smithi148.stderr:104000K .......... .......... .......... .......... .......... 61% 32.2M 2s 2024-03-29T04:11:50.440 INFO:teuthology.orchestra.run.smithi148.stderr:104050K .......... .......... .......... .......... .......... 61% 27.5M 2s 2024-03-29T04:11:50.441 INFO:teuthology.orchestra.run.smithi148.stderr:104100K .......... .......... .......... .......... .......... 61% 34.0M 2s 2024-03-29T04:11:50.443 INFO:teuthology.orchestra.run.smithi148.stderr:104150K .......... .......... .......... .......... .......... 61% 33.7M 2s 2024-03-29T04:11:50.444 INFO:teuthology.orchestra.run.smithi148.stderr:104200K .......... .......... .......... .......... .......... 61% 34.6M 2s 2024-03-29T04:11:50.446 INFO:teuthology.orchestra.run.smithi148.stderr:104250K .......... .......... .......... .......... .......... 61% 29.3M 2s 2024-03-29T04:11:50.447 INFO:teuthology.orchestra.run.smithi148.stderr:104300K .......... .......... .......... .......... .......... 61% 34.2M 2s 2024-03-29T04:11:50.449 INFO:teuthology.orchestra.run.smithi148.stderr:104350K .......... .......... .......... .......... .......... 61% 33.2M 2s 2024-03-29T04:11:50.450 INFO:teuthology.orchestra.run.smithi148.stderr:104400K .......... .......... .......... .......... .......... 61% 32.9M 2s 2024-03-29T04:11:50.452 INFO:teuthology.orchestra.run.smithi148.stderr:104450K .......... .......... .......... .......... .......... 61% 28.9M 2s 2024-03-29T04:11:50.453 INFO:teuthology.orchestra.run.smithi148.stderr:104500K .......... .......... .......... .......... .......... 62% 33.8M 2s 2024-03-29T04:11:50.455 INFO:teuthology.orchestra.run.smithi148.stderr:104550K .......... .......... .......... .......... .......... 62% 33.6M 2s 2024-03-29T04:11:50.456 INFO:teuthology.orchestra.run.smithi148.stderr:104600K .......... .......... .......... .......... .......... 62% 34.0M 2s 2024-03-29T04:11:50.458 INFO:teuthology.orchestra.run.smithi148.stderr:104650K .......... .......... .......... .......... .......... 62% 28.9M 2s 2024-03-29T04:11:50.459 INFO:teuthology.orchestra.run.smithi148.stderr:104700K .......... .......... .......... .......... .......... 62% 33.1M 2s 2024-03-29T04:11:50.461 INFO:teuthology.orchestra.run.smithi148.stderr:104750K .......... .......... .......... .......... .......... 62% 33.1M 2s 2024-03-29T04:11:50.463 INFO:teuthology.orchestra.run.smithi148.stderr:104800K .......... .......... .......... .......... .......... 62% 28.0M 2s 2024-03-29T04:11:50.465 INFO:teuthology.orchestra.run.smithi148.stderr:104850K .......... .......... .......... .......... .......... 62% 25.7M 2s 2024-03-29T04:11:50.467 INFO:teuthology.orchestra.run.smithi148.stderr:104900K .......... .......... .......... .......... .......... 62% 32.8M 2s 2024-03-29T04:11:50.468 INFO:teuthology.orchestra.run.smithi148.stderr:104950K .......... .......... .......... .......... .......... 62% 32.9M 2s 2024-03-29T04:11:50.469 INFO:teuthology.orchestra.run.smithi148.stderr:105000K .......... .......... .......... .......... .......... 62% 33.0M 2s 2024-03-29T04:11:50.471 INFO:teuthology.orchestra.run.smithi148.stderr:105050K .......... .......... .......... .......... .......... 62% 27.8M 2s 2024-03-29T04:11:50.472 INFO:teuthology.orchestra.run.smithi148.stderr:105100K .......... .......... .......... .......... .......... 62% 32.0M 2s 2024-03-29T04:11:50.474 INFO:teuthology.orchestra.run.smithi148.stderr:105150K .......... .......... .......... .......... .......... 62% 32.1M 2s 2024-03-29T04:11:50.476 INFO:teuthology.orchestra.run.smithi148.stderr:105200K .......... .......... .......... .......... .......... 62% 32.0M 2s 2024-03-29T04:11:50.477 INFO:teuthology.orchestra.run.smithi148.stderr:105250K .......... .......... .......... .......... .......... 62% 27.5M 2s 2024-03-29T04:11:50.479 INFO:teuthology.orchestra.run.smithi148.stderr:105300K .......... .......... .......... .......... .......... 62% 29.3M 2s 2024-03-29T04:11:50.480 INFO:teuthology.orchestra.run.smithi148.stderr:105350K .......... .......... .......... .......... .......... 62% 30.3M 2s 2024-03-29T04:11:50.482 INFO:teuthology.orchestra.run.smithi148.stderr:105400K .......... .......... .......... .......... .......... 62% 30.9M 2s 2024-03-29T04:11:50.484 INFO:teuthology.orchestra.run.smithi148.stderr:105450K .......... .......... .......... .......... .......... 62% 26.4M 2s 2024-03-29T04:11:50.486 INFO:teuthology.orchestra.run.smithi148.stderr:105500K .......... .......... .......... .......... .......... 62% 29.9M 2s 2024-03-29T04:11:50.487 INFO:teuthology.orchestra.run.smithi148.stderr:105550K .......... .......... .......... .......... .......... 62% 29.8M 2s 2024-03-29T04:11:50.489 INFO:teuthology.orchestra.run.smithi148.stderr:105600K .......... .......... .......... .......... .......... 62% 30.0M 2s 2024-03-29T04:11:50.491 INFO:teuthology.orchestra.run.smithi148.stderr:105650K .......... .......... .......... .......... .......... 62% 25.4M 2s 2024-03-29T04:11:50.492 INFO:teuthology.orchestra.run.smithi148.stderr:105700K .......... .......... .......... .......... .......... 62% 30.4M 2s 2024-03-29T04:11:50.494 INFO:teuthology.orchestra.run.smithi148.stderr:105750K .......... .......... .......... .......... .......... 62% 29.6M 2s 2024-03-29T04:11:50.496 INFO:teuthology.orchestra.run.smithi148.stderr:105800K .......... .......... .......... .......... .......... 62% 29.7M 2s 2024-03-29T04:11:50.497 INFO:teuthology.orchestra.run.smithi148.stderr:105850K .......... .......... .......... .......... .......... 62% 25.2M 2s 2024-03-29T04:11:50.499 INFO:teuthology.orchestra.run.smithi148.stderr:105900K .......... .......... .......... .......... .......... 62% 30.6M 2s 2024-03-29T04:11:50.501 INFO:teuthology.orchestra.run.smithi148.stderr:105950K .......... .......... .......... .......... .......... 62% 29.0M 2s 2024-03-29T04:11:50.503 INFO:teuthology.orchestra.run.smithi148.stderr:106000K .......... .......... .......... .......... .......... 62% 28.5M 2s 2024-03-29T04:11:50.505 INFO:teuthology.orchestra.run.smithi148.stderr:106050K .......... .......... .......... .......... .......... 62% 23.8M 2s 2024-03-29T04:11:50.506 INFO:teuthology.orchestra.run.smithi148.stderr:106100K .......... .......... .......... .......... .......... 62% 28.6M 2s 2024-03-29T04:11:50.508 INFO:teuthology.orchestra.run.smithi148.stderr:106150K .......... .......... .......... .......... .......... 63% 29.0M 2s 2024-03-29T04:11:50.510 INFO:teuthology.orchestra.run.smithi148.stderr:106200K .......... .......... .......... .......... .......... 63% 28.4M 2s 2024-03-29T04:11:50.512 INFO:teuthology.orchestra.run.smithi148.stderr:106250K .......... .......... .......... .......... .......... 63% 24.6M 2s 2024-03-29T04:11:50.513 INFO:teuthology.orchestra.run.smithi148.stderr:106300K .......... .......... .......... .......... .......... 63% 28.5M 2s 2024-03-29T04:11:50.515 INFO:teuthology.orchestra.run.smithi148.stderr:106350K .......... .......... .......... .......... .......... 63% 28.2M 2s 2024-03-29T04:11:50.517 INFO:teuthology.orchestra.run.smithi148.stderr:106400K .......... .......... .......... .......... .......... 63% 27.6M 2s 2024-03-29T04:11:50.519 INFO:teuthology.orchestra.run.smithi148.stderr:106450K .......... .......... .......... .......... .......... 63% 23.4M 2s 2024-03-29T04:11:50.520 INFO:teuthology.orchestra.run.smithi148.stderr:106500K .......... .......... .......... .......... .......... 63% 33.9M 2s 2024-03-29T04:11:50.521 INFO:teuthology.orchestra.run.smithi148.stderr:106550K .......... .......... .......... .......... .......... 63% 39.6M 2s 2024-03-29T04:11:50.524 INFO:teuthology.orchestra.run.smithi148.stderr:106600K .......... .......... .......... .......... .......... 63% 28.0M 2s 2024-03-29T04:11:50.526 INFO:teuthology.orchestra.run.smithi148.stderr:106650K .......... .......... .......... .......... .......... 63% 27.0M 2s 2024-03-29T04:11:50.527 INFO:teuthology.orchestra.run.smithi148.stderr:106700K .......... .......... .......... .......... .......... 63% 40.0M 2s 2024-03-29T04:11:50.528 INFO:teuthology.orchestra.run.smithi148.stderr:106750K .......... .......... .......... .......... .......... 63% 35.9M 2s 2024-03-29T04:11:50.529 INFO:teuthology.orchestra.run.smithi148.stderr:106800K .......... .......... .......... .......... .......... 63% 34.0M 2s 2024-03-29T04:11:50.531 INFO:teuthology.orchestra.run.smithi148.stderr:106850K .......... .......... .......... .......... .......... 63% 30.1M 2s 2024-03-29T04:11:50.532 INFO:teuthology.orchestra.run.smithi148.stderr:106900K .......... .......... .......... .......... .......... 63% 36.5M 2s 2024-03-29T04:11:50.534 INFO:teuthology.orchestra.run.smithi148.stderr:106950K .......... .......... .......... .......... .......... 63% 30.2M 2s 2024-03-29T04:11:50.535 INFO:teuthology.orchestra.run.smithi148.stderr:107000K .......... .......... .......... .......... .......... 63% 33.4M 2s 2024-03-29T04:11:50.537 INFO:teuthology.orchestra.run.smithi148.stderr:107050K .......... .......... .......... .......... .......... 63% 28.2M 2s 2024-03-29T04:11:50.539 INFO:teuthology.orchestra.run.smithi148.stderr:107100K .......... .......... .......... .......... .......... 63% 32.9M 2s 2024-03-29T04:11:50.540 INFO:teuthology.orchestra.run.smithi148.stderr:107150K .......... .......... .......... .......... .......... 63% 31.9M 2s 2024-03-29T04:11:50.542 INFO:teuthology.orchestra.run.smithi148.stderr:107200K .......... .......... .......... .......... .......... 63% 32.3M 2s 2024-03-29T04:11:50.544 INFO:teuthology.orchestra.run.smithi148.stderr:107250K .......... .......... .......... .......... .......... 63% 27.8M 2s 2024-03-29T04:11:50.545 INFO:teuthology.orchestra.run.smithi148.stderr:107300K .......... .......... .......... .......... .......... 63% 32.8M 2s 2024-03-29T04:11:50.546 INFO:teuthology.orchestra.run.smithi148.stderr:107350K .......... .......... .......... .......... .......... 63% 32.0M 2s 2024-03-29T04:11:50.548 INFO:teuthology.orchestra.run.smithi148.stderr:107400K .......... .......... .......... .......... .......... 63% 31.5M 2s 2024-03-29T04:11:50.550 INFO:teuthology.orchestra.run.smithi148.stderr:107450K .......... .......... .......... .......... .......... 63% 26.8M 2s 2024-03-29T04:11:50.551 INFO:teuthology.orchestra.run.smithi148.stderr:107500K .......... .......... .......... .......... .......... 63% 30.9M 2s 2024-03-29T04:11:50.553 INFO:teuthology.orchestra.run.smithi148.stderr:107550K .......... .......... .......... .......... .......... 63% 30.7M 2s 2024-03-29T04:11:50.555 INFO:teuthology.orchestra.run.smithi148.stderr:107600K .......... .......... .......... .......... .......... 63% 30.7M 2s 2024-03-29T04:11:50.556 INFO:teuthology.orchestra.run.smithi148.stderr:107650K .......... .......... .......... .......... .......... 63% 26.8M 2s 2024-03-29T04:11:50.558 INFO:teuthology.orchestra.run.smithi148.stderr:107700K .......... .......... .......... .......... .......... 63% 30.3M 2s 2024-03-29T04:11:50.559 INFO:teuthology.orchestra.run.smithi148.stderr:107750K .......... .......... .......... .......... .......... 63% 31.5M 2s 2024-03-29T04:11:50.561 INFO:teuthology.orchestra.run.smithi148.stderr:107800K .......... .......... .......... .......... .......... 63% 30.4M 2s 2024-03-29T04:11:50.563 INFO:teuthology.orchestra.run.smithi148.stderr:107850K .......... .......... .......... .......... .......... 64% 25.9M 2s 2024-03-29T04:11:50.565 INFO:teuthology.orchestra.run.smithi148.stderr:107900K .......... .......... .......... .......... .......... 64% 29.9M 2s 2024-03-29T04:11:50.566 INFO:teuthology.orchestra.run.smithi148.stderr:107950K .......... .......... .......... .......... .......... 64% 29.4M 2s 2024-03-29T04:11:50.568 INFO:teuthology.orchestra.run.smithi148.stderr:108000K .......... .......... .......... .......... .......... 64% 29.2M 2s 2024-03-29T04:11:50.570 INFO:teuthology.orchestra.run.smithi148.stderr:108050K .......... .......... .......... .......... .......... 64% 25.8M 2s 2024-03-29T04:11:50.572 INFO:teuthology.orchestra.run.smithi148.stderr:108100K .......... .......... .......... .......... .......... 64% 29.6M 2s 2024-03-29T04:11:50.573 INFO:teuthology.orchestra.run.smithi148.stderr:108150K .......... .......... .......... .......... .......... 64% 28.4M 2s 2024-03-29T04:11:50.575 INFO:teuthology.orchestra.run.smithi148.stderr:108200K .......... .......... .......... .......... .......... 64% 29.0M 2s 2024-03-29T04:11:50.577 INFO:teuthology.orchestra.run.smithi148.stderr:108250K .......... .......... .......... .......... .......... 64% 24.2M 2s 2024-03-29T04:11:50.578 INFO:teuthology.orchestra.run.smithi148.stderr:108300K .......... .......... .......... .......... .......... 64% 27.1M 2s 2024-03-29T04:11:50.580 INFO:teuthology.orchestra.run.smithi148.stderr:108350K .......... .......... .......... .......... .......... 64% 30.1M 2s 2024-03-29T04:11:50.582 INFO:teuthology.orchestra.run.smithi148.stderr:108400K .......... .......... .......... .......... .......... 64% 30.8M 2s 2024-03-29T04:11:50.583 INFO:teuthology.orchestra.run.smithi148.stderr:108450K .......... .......... .......... .......... .......... 64% 28.9M 2s 2024-03-29T04:11:50.585 INFO:teuthology.orchestra.run.smithi148.stderr:108500K .......... .......... .......... .......... .......... 64% 33.9M 2s 2024-03-29T04:11:50.586 INFO:teuthology.orchestra.run.smithi148.stderr:108550K .......... .......... .......... .......... .......... 64% 33.6M 2s 2024-03-29T04:11:50.588 INFO:teuthology.orchestra.run.smithi148.stderr:108600K .......... .......... .......... .......... .......... 64% 33.9M 2s 2024-03-29T04:11:50.589 INFO:teuthology.orchestra.run.smithi148.stderr:108650K .......... .......... .......... .......... .......... 64% 28.6M 2s 2024-03-29T04:11:50.591 INFO:teuthology.orchestra.run.smithi148.stderr:108700K .......... .......... .......... .......... .......... 64% 32.9M 2s 2024-03-29T04:11:50.595 INFO:teuthology.orchestra.run.smithi148.stderr:108750K .......... .......... .......... .......... .......... 64% 33.5M 2s 2024-03-29T04:11:50.595 INFO:teuthology.orchestra.run.smithi148.stderr:108800K .......... .......... .......... .......... .......... 64% 33.9M 2s 2024-03-29T04:11:50.596 INFO:teuthology.orchestra.run.smithi148.stderr:108850K .......... .......... .......... .......... .......... 64% 28.7M 2s 2024-03-29T04:11:50.597 INFO:teuthology.orchestra.run.smithi148.stderr:108900K .......... .......... .......... .......... .......... 64% 31.6M 2s 2024-03-29T04:11:50.599 INFO:teuthology.orchestra.run.smithi148.stderr:108950K .......... .......... .......... .......... .......... 64% 31.0M 2s 2024-03-29T04:11:50.600 INFO:teuthology.orchestra.run.smithi148.stderr:109000K .......... .......... .......... .......... .......... 64% 31.5M 2s 2024-03-29T04:11:50.602 INFO:teuthology.orchestra.run.smithi148.stderr:109050K .......... .......... .......... .......... .......... 64% 26.4M 2s 2024-03-29T04:11:50.604 INFO:teuthology.orchestra.run.smithi148.stderr:109100K .......... .......... .......... .......... .......... 64% 31.1M 2s 2024-03-29T04:11:50.605 INFO:teuthology.orchestra.run.smithi148.stderr:109150K .......... .......... .......... .......... .......... 64% 31.0M 2s 2024-03-29T04:11:50.607 INFO:teuthology.orchestra.run.smithi148.stderr:109200K .......... .......... .......... .......... .......... 64% 32.1M 2s 2024-03-29T04:11:50.609 INFO:teuthology.orchestra.run.smithi148.stderr:109250K .......... .......... .......... .......... .......... 64% 27.1M 2s 2024-03-29T04:11:50.610 INFO:teuthology.orchestra.run.smithi148.stderr:109300K .......... .......... .......... .......... .......... 64% 31.7M 2s 2024-03-29T04:11:50.612 INFO:teuthology.orchestra.run.smithi148.stderr:109350K .......... .......... .......... .......... .......... 64% 31.4M 2s 2024-03-29T04:11:50.613 INFO:teuthology.orchestra.run.smithi148.stderr:109400K .......... .......... .......... .......... .......... 64% 31.0M 2s 2024-03-29T04:11:50.615 INFO:teuthology.orchestra.run.smithi148.stderr:109450K .......... .......... .......... .......... .......... 64% 26.6M 2s 2024-03-29T04:11:50.617 INFO:teuthology.orchestra.run.smithi148.stderr:109500K .......... .......... .......... .......... .......... 64% 30.7M 2s 2024-03-29T04:11:50.618 INFO:teuthology.orchestra.run.smithi148.stderr:109550K .......... .......... .......... .......... .......... 65% 31.4M 2s 2024-03-29T04:11:50.620 INFO:teuthology.orchestra.run.smithi148.stderr:109600K .......... .......... .......... .......... .......... 65% 30.1M 2s 2024-03-29T04:11:50.622 INFO:teuthology.orchestra.run.smithi148.stderr:109650K .......... .......... .......... .......... .......... 65% 26.0M 2s 2024-03-29T04:11:50.623 INFO:teuthology.orchestra.run.smithi148.stderr:109700K .......... .......... .......... .......... .......... 65% 30.6M 2s 2024-03-29T04:11:50.625 INFO:teuthology.orchestra.run.smithi148.stderr:109750K .......... .......... .......... .......... .......... 65% 30.2M 2s 2024-03-29T04:11:50.627 INFO:teuthology.orchestra.run.smithi148.stderr:109800K .......... .......... .......... .......... .......... 65% 30.2M 2s 2024-03-29T04:11:50.629 INFO:teuthology.orchestra.run.smithi148.stderr:109850K .......... .......... .......... .......... .......... 65% 23.5M 2s 2024-03-29T04:11:50.631 INFO:teuthology.orchestra.run.smithi148.stderr:109900K .......... .......... .......... .......... .......... 65% 21.4M 2s 2024-03-29T04:11:50.633 INFO:teuthology.orchestra.run.smithi148.stderr:109950K .......... .......... .......... .......... .......... 65% 26.7M 2s 2024-03-29T04:11:50.634 INFO:teuthology.orchestra.run.smithi148.stderr:110000K .......... .......... .......... .......... .......... 65% 29.0M 2s 2024-03-29T04:11:50.636 INFO:teuthology.orchestra.run.smithi148.stderr:110050K .......... .......... .......... .......... .......... 65% 24.9M 2s 2024-03-29T04:11:50.638 INFO:teuthology.orchestra.run.smithi148.stderr:110100K .......... .......... .......... .......... .......... 65% 29.2M 2s 2024-03-29T04:11:50.640 INFO:teuthology.orchestra.run.smithi148.stderr:110150K .......... .......... .......... .......... .......... 65% 25.0M 2s 2024-03-29T04:11:50.642 INFO:teuthology.orchestra.run.smithi148.stderr:110200K .......... .......... .......... .......... .......... 65% 24.3M 2s 2024-03-29T04:11:50.644 INFO:teuthology.orchestra.run.smithi148.stderr:110250K .......... .......... .......... .......... .......... 65% 20.8M 2s 2024-03-29T04:11:50.646 INFO:teuthology.orchestra.run.smithi148.stderr:110300K .......... .......... .......... .......... .......... 65% 24.1M 2s 2024-03-29T04:11:50.648 INFO:teuthology.orchestra.run.smithi148.stderr:110350K .......... .......... .......... .......... .......... 65% 23.1M 2s 2024-03-29T04:11:50.651 INFO:teuthology.orchestra.run.smithi148.stderr:110400K .......... .......... .......... .......... .......... 65% 24.1M 2s 2024-03-29T04:11:50.653 INFO:teuthology.orchestra.run.smithi148.stderr:110450K .......... .......... .......... .......... .......... 65% 21.6M 2s 2024-03-29T04:11:50.655 INFO:teuthology.orchestra.run.smithi148.stderr:110500K .......... .......... .......... .......... .......... 65% 24.8M 2s 2024-03-29T04:11:50.657 INFO:teuthology.orchestra.run.smithi148.stderr:110550K .......... .......... .......... .......... .......... 65% 26.0M 2s 2024-03-29T04:11:50.658 INFO:teuthology.orchestra.run.smithi148.stderr:110600K .......... .......... .......... .......... .......... 65% 30.6M 2s 2024-03-29T04:11:50.660 INFO:teuthology.orchestra.run.smithi148.stderr:110650K .......... .......... .......... .......... .......... 65% 27.9M 2s 2024-03-29T04:11:50.661 INFO:teuthology.orchestra.run.smithi148.stderr:110700K .......... .......... .......... .......... .......... 65% 34.0M 2s 2024-03-29T04:11:50.663 INFO:teuthology.orchestra.run.smithi148.stderr:110750K .......... .......... .......... .......... .......... 65% 34.5M 2s 2024-03-29T04:11:50.664 INFO:teuthology.orchestra.run.smithi148.stderr:110800K .......... .......... .......... .......... .......... 65% 33.8M 2s 2024-03-29T04:11:50.666 INFO:teuthology.orchestra.run.smithi148.stderr:110850K .......... .......... .......... .......... .......... 65% 31.2M 2s 2024-03-29T04:11:50.667 INFO:teuthology.orchestra.run.smithi148.stderr:110900K .......... .......... .......... .......... .......... 65% 36.8M 2s 2024-03-29T04:11:50.668 INFO:teuthology.orchestra.run.smithi148.stderr:110950K .......... .......... .......... .......... .......... 65% 36.3M 2s 2024-03-29T04:11:50.670 INFO:teuthology.orchestra.run.smithi148.stderr:111000K .......... .......... .......... .......... .......... 65% 36.4M 2s 2024-03-29T04:11:50.671 INFO:teuthology.orchestra.run.smithi148.stderr:111050K .......... .......... .......... .......... .......... 65% 30.7M 2s 2024-03-29T04:11:50.673 INFO:teuthology.orchestra.run.smithi148.stderr:111100K .......... .......... .......... .......... .......... 65% 35.1M 2s 2024-03-29T04:11:50.674 INFO:teuthology.orchestra.run.smithi148.stderr:111150K .......... .......... .......... .......... .......... 65% 32.9M 2s 2024-03-29T04:11:50.676 INFO:teuthology.orchestra.run.smithi148.stderr:111200K .......... .......... .......... .......... .......... 65% 36.5M 2s 2024-03-29T04:11:50.677 INFO:teuthology.orchestra.run.smithi148.stderr:111250K .......... .......... .......... .......... .......... 66% 33.7M 2s 2024-03-29T04:11:50.678 INFO:teuthology.orchestra.run.smithi148.stderr:111300K .......... .......... .......... .......... .......... 66% 33.3M 2s 2024-03-29T04:11:50.680 INFO:teuthology.orchestra.run.smithi148.stderr:111350K .......... .......... .......... .......... .......... 66% 32.1M 2s 2024-03-29T04:11:50.682 INFO:teuthology.orchestra.run.smithi148.stderr:111400K .......... .......... .......... .......... .......... 66% 31.9M 2s 2024-03-29T04:11:50.683 INFO:teuthology.orchestra.run.smithi148.stderr:111450K .......... .......... .......... .......... .......... 66% 28.6M 2s 2024-03-29T04:11:50.685 INFO:teuthology.orchestra.run.smithi148.stderr:111500K .......... .......... .......... .......... .......... 66% 31.4M 2s 2024-03-29T04:11:50.686 INFO:teuthology.orchestra.run.smithi148.stderr:111550K .......... .......... .......... .......... .......... 66% 30.9M 2s 2024-03-29T04:11:50.688 INFO:teuthology.orchestra.run.smithi148.stderr:111600K .......... .......... .......... .......... .......... 66% 31.0M 2s 2024-03-29T04:11:50.690 INFO:teuthology.orchestra.run.smithi148.stderr:111650K .......... .......... .......... .......... .......... 66% 27.2M 2s 2024-03-29T04:11:50.691 INFO:teuthology.orchestra.run.smithi148.stderr:111700K .......... .......... .......... .......... .......... 66% 30.8M 2s 2024-03-29T04:11:50.693 INFO:teuthology.orchestra.run.smithi148.stderr:111750K .......... .......... .......... .......... .......... 66% 32.0M 2s 2024-03-29T04:11:50.695 INFO:teuthology.orchestra.run.smithi148.stderr:111800K .......... .......... .......... .......... .......... 66% 31.4M 2s 2024-03-29T04:11:50.696 INFO:teuthology.orchestra.run.smithi148.stderr:111850K .......... .......... .......... .......... .......... 66% 26.8M 2s 2024-03-29T04:11:50.698 INFO:teuthology.orchestra.run.smithi148.stderr:111900K .......... .......... .......... .......... .......... 66% 28.2M 2s 2024-03-29T04:11:50.699 INFO:teuthology.orchestra.run.smithi148.stderr:111950K .......... .......... .......... .......... .......... 66% 30.5M 2s 2024-03-29T04:11:50.701 INFO:teuthology.orchestra.run.smithi148.stderr:112000K .......... .......... .......... .......... .......... 66% 29.9M 2s 2024-03-29T04:11:50.703 INFO:teuthology.orchestra.run.smithi148.stderr:112050K .......... .......... .......... .......... .......... 66% 26.4M 2s 2024-03-29T04:11:50.705 INFO:teuthology.orchestra.run.smithi148.stderr:112100K .......... .......... .......... .......... .......... 66% 30.5M 2s 2024-03-29T04:11:50.706 INFO:teuthology.orchestra.run.smithi148.stderr:112150K .......... .......... .......... .......... .......... 66% 29.9M 2s 2024-03-29T04:11:50.708 INFO:teuthology.orchestra.run.smithi148.stderr:112200K .......... .......... .......... .......... .......... 66% 30.6M 2s 2024-03-29T04:11:50.710 INFO:teuthology.orchestra.run.smithi148.stderr:112250K .......... .......... .......... .......... .......... 66% 25.9M 2s 2024-03-29T04:11:50.711 INFO:teuthology.orchestra.run.smithi148.stderr:112300K .......... .......... .......... .......... .......... 66% 30.5M 2s 2024-03-29T04:11:50.713 INFO:teuthology.orchestra.run.smithi148.stderr:112350K .......... .......... .......... .......... .......... 66% 34.9M 2s 2024-03-29T04:11:50.714 INFO:teuthology.orchestra.run.smithi148.stderr:112400K .......... .......... .......... .......... .......... 66% 35.0M 2s 2024-03-29T04:11:50.716 INFO:teuthology.orchestra.run.smithi148.stderr:112450K .......... .......... .......... .......... .......... 66% 29.6M 2s 2024-03-29T04:11:50.717 INFO:teuthology.orchestra.run.smithi148.stderr:112500K .......... .......... .......... .......... .......... 66% 34.3M 2s 2024-03-29T04:11:50.719 INFO:teuthology.orchestra.run.smithi148.stderr:112550K .......... .......... .......... .......... .......... 66% 34.4M 2s 2024-03-29T04:11:50.720 INFO:teuthology.orchestra.run.smithi148.stderr:112600K .......... .......... .......... .......... .......... 66% 34.5M 2s 2024-03-29T04:11:50.722 INFO:teuthology.orchestra.run.smithi148.stderr:112650K .......... .......... .......... .......... .......... 66% 29.0M 2s 2024-03-29T04:11:50.723 INFO:teuthology.orchestra.run.smithi148.stderr:112700K .......... .......... .......... .......... .......... 66% 34.2M 2s 2024-03-29T04:11:50.725 INFO:teuthology.orchestra.run.smithi148.stderr:112750K .......... .......... .......... .......... .......... 66% 33.8M 2s 2024-03-29T04:11:50.726 INFO:teuthology.orchestra.run.smithi148.stderr:112800K .......... .......... .......... .......... .......... 66% 34.1M 2s 2024-03-29T04:11:50.728 INFO:teuthology.orchestra.run.smithi148.stderr:112850K .......... .......... .......... .......... .......... 66% 28.8M 2s 2024-03-29T04:11:50.729 INFO:teuthology.orchestra.run.smithi148.stderr:112900K .......... .......... .......... .......... .......... 67% 31.7M 2s 2024-03-29T04:11:50.731 INFO:teuthology.orchestra.run.smithi148.stderr:112950K .......... .......... .......... .......... .......... 67% 33.0M 2s 2024-03-29T04:11:50.733 INFO:teuthology.orchestra.run.smithi148.stderr:113000K .......... .......... .......... .......... .......... 67% 29.3M 2s 2024-03-29T04:11:50.735 INFO:teuthology.orchestra.run.smithi148.stderr:113050K .......... .......... .......... .......... .......... 67% 23.9M 2s 2024-03-29T04:11:50.736 INFO:teuthology.orchestra.run.smithi148.stderr:113100K .......... .......... .......... .......... .......... 67% 27.5M 2s 2024-03-29T04:11:50.738 INFO:teuthology.orchestra.run.smithi148.stderr:113150K .......... .......... .......... .......... .......... 67% 28.3M 2s 2024-03-29T04:11:50.740 INFO:teuthology.orchestra.run.smithi148.stderr:113200K .......... .......... .......... .......... .......... 67% 27.2M 2s 2024-03-29T04:11:50.742 INFO:teuthology.orchestra.run.smithi148.stderr:113250K .......... .......... .......... .......... .......... 67% 23.7M 2s 2024-03-29T04:11:50.744 INFO:teuthology.orchestra.run.smithi148.stderr:113300K .......... .......... .......... .......... .......... 67% 27.5M 2s 2024-03-29T04:11:50.745 INFO:teuthology.orchestra.run.smithi148.stderr:113350K .......... .......... .......... .......... .......... 67% 27.8M 2s 2024-03-29T04:11:50.747 INFO:teuthology.orchestra.run.smithi148.stderr:113400K .......... .......... .......... .......... .......... 67% 27.4M 2s 2024-03-29T04:11:50.749 INFO:teuthology.orchestra.run.smithi148.stderr:113450K .......... .......... .......... .......... .......... 67% 23.0M 2s 2024-03-29T04:11:50.751 INFO:teuthology.orchestra.run.smithi148.stderr:113500K .......... .......... .......... .......... .......... 67% 27.1M 2s 2024-03-29T04:11:50.753 INFO:teuthology.orchestra.run.smithi148.stderr:113550K .......... .......... .......... .......... .......... 67% 28.3M 2s 2024-03-29T04:11:50.755 INFO:teuthology.orchestra.run.smithi148.stderr:113600K .......... .......... .......... .......... .......... 67% 28.1M 2s 2024-03-29T04:11:50.757 INFO:teuthology.orchestra.run.smithi148.stderr:113650K .......... .......... .......... .......... .......... 67% 24.2M 2s 2024-03-29T04:11:50.758 INFO:teuthology.orchestra.run.smithi148.stderr:113700K .......... .......... .......... .......... .......... 67% 28.0M 2s 2024-03-29T04:11:50.760 INFO:teuthology.orchestra.run.smithi148.stderr:113750K .......... .......... .......... .......... .......... 67% 28.1M 2s 2024-03-29T04:11:50.762 INFO:teuthology.orchestra.run.smithi148.stderr:113800K .......... .......... .......... .......... .......... 67% 27.4M 2s 2024-03-29T04:11:50.764 INFO:teuthology.orchestra.run.smithi148.stderr:113850K .......... .......... .......... .......... .......... 67% 24.0M 2s 2024-03-29T04:11:50.766 INFO:teuthology.orchestra.run.smithi148.stderr:113900K .......... .......... .......... .......... .......... 67% 27.4M 2s 2024-03-29T04:11:50.794 INFO:teuthology.orchestra.run.smithi148.stderr:113950K .......... .......... .......... .......... .......... 67% 1.71M 2s 2024-03-29T04:11:50.799 INFO:teuthology.orchestra.run.smithi148.stderr:114000K .......... .......... .......... .......... .......... 67% 22.8M 2s 2024-03-29T04:11:50.799 INFO:teuthology.orchestra.run.smithi148.stderr:114050K .......... .......... .......... .......... .......... 67% 26.7M 2s 2024-03-29T04:11:50.800 INFO:teuthology.orchestra.run.smithi148.stderr:114100K .......... .......... .......... .......... .......... 67% 26.6M 2s 2024-03-29T04:11:50.802 INFO:teuthology.orchestra.run.smithi148.stderr:114150K .......... .......... .......... .......... .......... 67% 27.5M 2s 2024-03-29T04:11:50.804 INFO:teuthology.orchestra.run.smithi148.stderr:114200K .......... .......... .......... .......... .......... 67% 23.5M 2s 2024-03-29T04:11:50.806 INFO:teuthology.orchestra.run.smithi148.stderr:114250K .......... .......... .......... .......... .......... 67% 27.2M 2s 2024-03-29T04:11:50.807 INFO:teuthology.orchestra.run.smithi148.stderr:114300K .......... .......... .......... .......... .......... 67% 27.4M 2s 2024-03-29T04:11:50.809 INFO:teuthology.orchestra.run.smithi148.stderr:114350K .......... .......... .......... .......... .......... 67% 27.4M 2s 2024-03-29T04:11:50.811 INFO:teuthology.orchestra.run.smithi148.stderr:114400K .......... .......... .......... .......... .......... 67% 22.9M 2s 2024-03-29T04:11:50.813 INFO:teuthology.orchestra.run.smithi148.stderr:114450K .......... .......... .......... .......... .......... 67% 26.5M 2s 2024-03-29T04:11:50.815 INFO:teuthology.orchestra.run.smithi148.stderr:114500K .......... .......... .......... .......... .......... 67% 27.5M 2s 2024-03-29T04:11:50.817 INFO:teuthology.orchestra.run.smithi148.stderr:114550K .......... .......... .......... .......... .......... 67% 27.8M 2s 2024-03-29T04:11:50.819 INFO:teuthology.orchestra.run.smithi148.stderr:114600K .......... .......... .......... .......... .......... 68% 24.1M 2s 2024-03-29T04:11:50.821 INFO:teuthology.orchestra.run.smithi148.stderr:114650K .......... .......... .......... .......... .......... 68% 23.7M 2s 2024-03-29T04:11:50.823 INFO:teuthology.orchestra.run.smithi148.stderr:114700K .......... .......... .......... .......... .......... 68% 28.0M 2s 2024-03-29T04:11:50.824 INFO:teuthology.orchestra.run.smithi148.stderr:114750K .......... .......... .......... .......... .......... 68% 32.1M 2s 2024-03-29T04:11:50.826 INFO:teuthology.orchestra.run.smithi148.stderr:114800K .......... .......... .......... .......... .......... 68% 30.0M 2s 2024-03-29T04:11:50.827 INFO:teuthology.orchestra.run.smithi148.stderr:114850K .......... .......... .......... .......... .......... 68% 42.2M 2s 2024-03-29T04:11:50.828 INFO:teuthology.orchestra.run.smithi148.stderr:114900K .......... .......... .......... .......... .......... 68% 41.0M 2s 2024-03-29T04:11:50.829 INFO:teuthology.orchestra.run.smithi148.stderr:114950K .......... .......... .......... .......... .......... 68% 37.6M 2s 2024-03-29T04:11:50.831 INFO:teuthology.orchestra.run.smithi148.stderr:115000K .......... .......... .......... .......... .......... 68% 34.0M 2s 2024-03-29T04:11:50.832 INFO:teuthology.orchestra.run.smithi148.stderr:115050K .......... .......... .......... .......... .......... 68% 39.5M 2s 2024-03-29T04:11:50.833 INFO:teuthology.orchestra.run.smithi148.stderr:115100K .......... .......... .......... .......... .......... 68% 37.0M 2s 2024-03-29T04:11:50.834 INFO:teuthology.orchestra.run.smithi148.stderr:115150K .......... .......... .......... .......... .......... 68% 40.5M 2s 2024-03-29T04:11:50.836 INFO:teuthology.orchestra.run.smithi148.stderr:115200K .......... .......... .......... .......... .......... 68% 31.0M 2s 2024-03-29T04:11:50.837 INFO:teuthology.orchestra.run.smithi148.stderr:115250K .......... .......... .......... .......... .......... 68% 38.7M 2s 2024-03-29T04:11:50.839 INFO:teuthology.orchestra.run.smithi148.stderr:115300K .......... .......... .......... .......... .......... 68% 39.1M 2s 2024-03-29T04:11:50.840 INFO:teuthology.orchestra.run.smithi148.stderr:115350K .......... .......... .......... .......... .......... 68% 39.6M 2s 2024-03-29T04:11:50.841 INFO:teuthology.orchestra.run.smithi148.stderr:115400K .......... .......... .......... .......... .......... 68% 33.6M 2s 2024-03-29T04:11:50.843 INFO:teuthology.orchestra.run.smithi148.stderr:115450K .......... .......... .......... .......... .......... 68% 39.5M 2s 2024-03-29T04:11:50.844 INFO:teuthology.orchestra.run.smithi148.stderr:115500K .......... .......... .......... .......... .......... 68% 39.7M 2s 2024-03-29T04:11:50.845 INFO:teuthology.orchestra.run.smithi148.stderr:115550K .......... .......... .......... .......... .......... 68% 39.2M 2s 2024-03-29T04:11:50.847 INFO:teuthology.orchestra.run.smithi148.stderr:115600K .......... .......... .......... .......... .......... 68% 33.3M 2s 2024-03-29T04:11:50.848 INFO:teuthology.orchestra.run.smithi148.stderr:115650K .......... .......... .......... .......... .......... 68% 38.1M 2s 2024-03-29T04:11:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:115700K .......... .......... .......... .......... .......... 68% 38.2M 2s 2024-03-29T04:11:50.850 INFO:teuthology.orchestra.run.smithi148.stderr:115750K .......... .......... .......... .......... .......... 68% 37.7M 2s 2024-03-29T04:11:50.852 INFO:teuthology.orchestra.run.smithi148.stderr:115800K .......... .......... .......... .......... .......... 68% 32.2M 2s 2024-03-29T04:11:50.853 INFO:teuthology.orchestra.run.smithi148.stderr:115850K .......... .......... .......... .......... .......... 68% 37.2M 2s 2024-03-29T04:11:50.855 INFO:teuthology.orchestra.run.smithi148.stderr:115900K .......... .......... .......... .......... .......... 68% 37.2M 2s 2024-03-29T04:11:50.856 INFO:teuthology.orchestra.run.smithi148.stderr:115950K .......... .......... .......... .......... .......... 68% 36.9M 2s 2024-03-29T04:11:50.858 INFO:teuthology.orchestra.run.smithi148.stderr:116000K .......... .......... .......... .......... .......... 68% 31.6M 2s 2024-03-29T04:11:50.859 INFO:teuthology.orchestra.run.smithi148.stderr:116050K .......... .......... .......... .......... .......... 68% 37.1M 2s 2024-03-29T04:11:50.860 INFO:teuthology.orchestra.run.smithi148.stderr:116100K .......... .......... .......... .......... .......... 68% 37.4M 2s 2024-03-29T04:11:50.861 INFO:teuthology.orchestra.run.smithi148.stderr:116150K .......... .......... .......... .......... .......... 68% 37.1M 2s 2024-03-29T04:11:50.863 INFO:teuthology.orchestra.run.smithi148.stderr:116200K .......... .......... .......... .......... .......... 68% 31.1M 2s 2024-03-29T04:11:50.864 INFO:teuthology.orchestra.run.smithi148.stderr:116250K .......... .......... .......... .......... .......... 68% 34.4M 2s 2024-03-29T04:11:50.866 INFO:teuthology.orchestra.run.smithi148.stderr:116300K .......... .......... .......... .......... .......... 69% 28.4M 2s 2024-03-29T04:11:50.868 INFO:teuthology.orchestra.run.smithi148.stderr:116350K .......... .......... .......... .......... .......... 69% 29.7M 2s 2024-03-29T04:11:50.870 INFO:teuthology.orchestra.run.smithi148.stderr:116400K .......... .......... .......... .......... .......... 69% 25.4M 2s 2024-03-29T04:11:50.871 INFO:teuthology.orchestra.run.smithi148.stderr:116450K .......... .......... .......... .......... .......... 69% 29.7M 2s 2024-03-29T04:11:50.873 INFO:teuthology.orchestra.run.smithi148.stderr:116500K .......... .......... .......... .......... .......... 69% 30.9M 2s 2024-03-29T04:11:50.874 INFO:teuthology.orchestra.run.smithi148.stderr:116550K .......... .......... .......... .......... .......... 69% 32.9M 2s 2024-03-29T04:11:50.876 INFO:teuthology.orchestra.run.smithi148.stderr:116600K .......... .......... .......... .......... .......... 69% 28.1M 2s 2024-03-29T04:11:50.878 INFO:teuthology.orchestra.run.smithi148.stderr:116650K .......... .......... .......... .......... .......... 69% 24.6M 2s 2024-03-29T04:11:50.880 INFO:teuthology.orchestra.run.smithi148.stderr:116700K .......... .......... .......... .......... .......... 69% 30.4M 2s 2024-03-29T04:11:50.881 INFO:teuthology.orchestra.run.smithi148.stderr:116750K .......... .......... .......... .......... .......... 69% 34.2M 2s 2024-03-29T04:11:50.883 INFO:teuthology.orchestra.run.smithi148.stderr:116800K .......... .......... .......... .......... .......... 69% 28.9M 2s 2024-03-29T04:11:50.884 INFO:teuthology.orchestra.run.smithi148.stderr:116850K .......... .......... .......... .......... .......... 69% 33.1M 2s 2024-03-29T04:11:50.886 INFO:teuthology.orchestra.run.smithi148.stderr:116900K .......... .......... .......... .......... .......... 69% 32.9M 2s 2024-03-29T04:11:50.887 INFO:teuthology.orchestra.run.smithi148.stderr:116950K .......... .......... .......... .......... .......... 69% 31.9M 2s 2024-03-29T04:11:50.889 INFO:teuthology.orchestra.run.smithi148.stderr:117000K .......... .......... .......... .......... .......... 69% 28.5M 2s 2024-03-29T04:11:50.890 INFO:teuthology.orchestra.run.smithi148.stderr:117050K .......... .......... .......... .......... .......... 69% 33.8M 2s 2024-03-29T04:11:50.892 INFO:teuthology.orchestra.run.smithi148.stderr:117100K .......... .......... .......... .......... .......... 69% 33.8M 2s 2024-03-29T04:11:50.893 INFO:teuthology.orchestra.run.smithi148.stderr:117150K .......... .......... .......... .......... .......... 69% 33.4M 2s 2024-03-29T04:11:50.895 INFO:teuthology.orchestra.run.smithi148.stderr:117200K .......... .......... .......... .......... .......... 69% 29.0M 2s 2024-03-29T04:11:50.897 INFO:teuthology.orchestra.run.smithi148.stderr:117250K .......... .......... .......... .......... .......... 69% 32.5M 2s 2024-03-29T04:11:50.898 INFO:teuthology.orchestra.run.smithi148.stderr:117300K .......... .......... .......... .......... .......... 69% 33.0M 2s 2024-03-29T04:11:50.899 INFO:teuthology.orchestra.run.smithi148.stderr:117350K .......... .......... .......... .......... .......... 69% 32.9M 2s 2024-03-29T04:11:50.901 INFO:teuthology.orchestra.run.smithi148.stderr:117400K .......... .......... .......... .......... .......... 69% 28.6M 2s 2024-03-29T04:11:50.903 INFO:teuthology.orchestra.run.smithi148.stderr:117450K .......... .......... .......... .......... .......... 69% 32.7M 2s 2024-03-29T04:11:50.904 INFO:teuthology.orchestra.run.smithi148.stderr:117500K .......... .......... .......... .......... .......... 69% 31.9M 2s 2024-03-29T04:11:50.906 INFO:teuthology.orchestra.run.smithi148.stderr:117550K .......... .......... .......... .......... .......... 69% 31.7M 2s 2024-03-29T04:11:50.908 INFO:teuthology.orchestra.run.smithi148.stderr:117600K .......... .......... .......... .......... .......... 69% 27.2M 2s 2024-03-29T04:11:50.909 INFO:teuthology.orchestra.run.smithi148.stderr:117650K .......... .......... .......... .......... .......... 69% 31.3M 2s 2024-03-29T04:11:50.911 INFO:teuthology.orchestra.run.smithi148.stderr:117700K .......... .......... .......... .......... .......... 69% 31.0M 2s 2024-03-29T04:11:50.912 INFO:teuthology.orchestra.run.smithi148.stderr:117750K .......... .......... .......... .......... .......... 69% 29.7M 2s 2024-03-29T04:11:50.914 INFO:teuthology.orchestra.run.smithi148.stderr:117800K .......... .......... .......... .......... .......... 69% 27.0M 2s 2024-03-29T04:11:50.916 INFO:teuthology.orchestra.run.smithi148.stderr:117850K .......... .......... .......... .......... .......... 69% 32.0M 2s 2024-03-29T04:11:50.917 INFO:teuthology.orchestra.run.smithi148.stderr:117900K .......... .......... .......... .......... .......... 69% 31.1M 2s 2024-03-29T04:11:50.919 INFO:teuthology.orchestra.run.smithi148.stderr:117950K .......... .......... .......... .......... .......... 70% 31.1M 2s 2024-03-29T04:11:50.921 INFO:teuthology.orchestra.run.smithi148.stderr:118000K .......... .......... .......... .......... .......... 70% 26.8M 2s 2024-03-29T04:11:50.922 INFO:teuthology.orchestra.run.smithi148.stderr:118050K .......... .......... .......... .......... .......... 70% 29.7M 2s 2024-03-29T04:11:50.924 INFO:teuthology.orchestra.run.smithi148.stderr:118100K .......... .......... .......... .......... .......... 70% 29.5M 2s 2024-03-29T04:11:50.925 INFO:teuthology.orchestra.run.smithi148.stderr:118150K .......... .......... .......... .......... .......... 70% 29.4M 2s 2024-03-29T04:11:50.927 INFO:teuthology.orchestra.run.smithi148.stderr:118200K .......... .......... .......... .......... .......... 70% 25.1M 2s 2024-03-29T04:11:50.929 INFO:teuthology.orchestra.run.smithi148.stderr:118250K .......... .......... .......... .......... .......... 70% 28.6M 2s 2024-03-29T04:11:50.931 INFO:teuthology.orchestra.run.smithi148.stderr:118300K .......... .......... .......... .......... .......... 70% 28.7M 2s 2024-03-29T04:11:50.933 INFO:teuthology.orchestra.run.smithi148.stderr:118350K .......... .......... .......... .......... .......... 70% 27.8M 2s 2024-03-29T04:11:50.935 INFO:teuthology.orchestra.run.smithi148.stderr:118400K .......... .......... .......... .......... .......... 70% 22.0M 2s 2024-03-29T04:11:50.937 INFO:teuthology.orchestra.run.smithi148.stderr:118450K .......... .......... .......... .......... .......... 70% 27.5M 2s 2024-03-29T04:11:50.939 INFO:teuthology.orchestra.run.smithi148.stderr:118500K .......... .......... .......... .......... .......... 70% 26.9M 2s 2024-03-29T04:11:50.940 INFO:teuthology.orchestra.run.smithi148.stderr:118550K .......... .......... .......... .......... .......... 70% 27.0M 2s 2024-03-29T04:11:50.943 INFO:teuthology.orchestra.run.smithi148.stderr:118600K .......... .......... .......... .......... .......... 70% 22.4M 2s 2024-03-29T04:11:50.944 INFO:teuthology.orchestra.run.smithi148.stderr:118650K .......... .......... .......... .......... .......... 70% 26.7M 2s 2024-03-29T04:11:50.946 INFO:teuthology.orchestra.run.smithi148.stderr:118700K .......... .......... .......... .......... .......... 70% 28.3M 2s 2024-03-29T04:11:50.948 INFO:teuthology.orchestra.run.smithi148.stderr:118750K .......... .......... .......... .......... .......... 70% 24.4M 2s 2024-03-29T04:11:50.951 INFO:teuthology.orchestra.run.smithi148.stderr:118800K .......... .......... .......... .......... .......... 70% 22.2M 2s 2024-03-29T04:11:50.952 INFO:teuthology.orchestra.run.smithi148.stderr:118850K .......... .......... .......... .......... .......... 70% 28.5M 2s 2024-03-29T04:11:50.953 INFO:teuthology.orchestra.run.smithi148.stderr:118900K .......... .......... .......... .......... .......... 70% 31.9M 2s 2024-03-29T04:11:50.955 INFO:teuthology.orchestra.run.smithi148.stderr:118950K .......... .......... .......... .......... .......... 70% 31.6M 2s 2024-03-29T04:11:50.957 INFO:teuthology.orchestra.run.smithi148.stderr:119000K .......... .......... .......... .......... .......... 70% 25.1M 2s 2024-03-29T04:11:50.958 INFO:teuthology.orchestra.run.smithi148.stderr:119050K .......... .......... .......... .......... .......... 70% 29.3M 2s 2024-03-29T04:11:50.960 INFO:teuthology.orchestra.run.smithi148.stderr:119100K .......... .......... .......... .......... .......... 70% 31.8M 2s 2024-03-29T04:11:50.962 INFO:teuthology.orchestra.run.smithi148.stderr:119150K .......... .......... .......... .......... .......... 70% 34.1M 2s 2024-03-29T04:11:50.963 INFO:teuthology.orchestra.run.smithi148.stderr:119200K .......... .......... .......... .......... .......... 70% 30.2M 2s 2024-03-29T04:11:50.965 INFO:teuthology.orchestra.run.smithi148.stderr:119250K .......... .......... .......... .......... .......... 70% 32.3M 2s 2024-03-29T04:11:50.966 INFO:teuthology.orchestra.run.smithi148.stderr:119300K .......... .......... .......... .......... .......... 70% 31.9M 2s 2024-03-29T04:11:50.968 INFO:teuthology.orchestra.run.smithi148.stderr:119350K .......... .......... .......... .......... .......... 70% 36.7M 2s 2024-03-29T04:11:50.969 INFO:teuthology.orchestra.run.smithi148.stderr:119400K .......... .......... .......... .......... .......... 70% 31.6M 2s 2024-03-29T04:11:50.970 INFO:teuthology.orchestra.run.smithi148.stderr:119450K .......... .......... .......... .......... .......... 70% 35.9M 2s 2024-03-29T04:11:50.972 INFO:teuthology.orchestra.run.smithi148.stderr:119500K .......... .......... .......... .......... .......... 70% 37.0M 2s 2024-03-29T04:11:50.973 INFO:teuthology.orchestra.run.smithi148.stderr:119550K .......... .......... .......... .......... .......... 70% 34.6M 2s 2024-03-29T04:11:50.975 INFO:teuthology.orchestra.run.smithi148.stderr:119600K .......... .......... .......... .......... .......... 70% 29.2M 2s 2024-03-29T04:11:50.976 INFO:teuthology.orchestra.run.smithi148.stderr:119650K .......... .......... .......... .......... .......... 71% 34.7M 2s 2024-03-29T04:11:50.978 INFO:teuthology.orchestra.run.smithi148.stderr:119700K .......... .......... .......... .......... .......... 71% 33.1M 2s 2024-03-29T04:11:50.979 INFO:teuthology.orchestra.run.smithi148.stderr:119750K .......... .......... .......... .......... .......... 71% 32.8M 2s 2024-03-29T04:11:50.981 INFO:teuthology.orchestra.run.smithi148.stderr:119800K .......... .......... .......... .......... .......... 71% 29.0M 2s 2024-03-29T04:11:50.982 INFO:teuthology.orchestra.run.smithi148.stderr:119850K .......... .......... .......... .......... .......... 71% 33.7M 2s 2024-03-29T04:11:50.984 INFO:teuthology.orchestra.run.smithi148.stderr:119900K .......... .......... .......... .......... .......... 71% 33.7M 2s 2024-03-29T04:11:50.985 INFO:teuthology.orchestra.run.smithi148.stderr:119950K .......... .......... .......... .......... .......... 71% 32.6M 2s 2024-03-29T04:11:50.987 INFO:teuthology.orchestra.run.smithi148.stderr:120000K .......... .......... .......... .......... .......... 71% 27.8M 2s 2024-03-29T04:11:50.989 INFO:teuthology.orchestra.run.smithi148.stderr:120050K .......... .......... .......... .......... .......... 71% 33.0M 2s 2024-03-29T04:11:50.990 INFO:teuthology.orchestra.run.smithi148.stderr:120100K .......... .......... .......... .......... .......... 71% 33.0M 2s 2024-03-29T04:11:50.992 INFO:teuthology.orchestra.run.smithi148.stderr:120150K .......... .......... .......... .......... .......... 71% 32.9M 2s 2024-03-29T04:11:50.994 INFO:teuthology.orchestra.run.smithi148.stderr:120200K .......... .......... .......... .......... .......... 71% 28.3M 2s 2024-03-29T04:11:50.995 INFO:teuthology.orchestra.run.smithi148.stderr:120250K .......... .......... .......... .......... .......... 71% 32.1M 2s 2024-03-29T04:11:50.996 INFO:teuthology.orchestra.run.smithi148.stderr:120300K .......... .......... .......... .......... .......... 71% 31.3M 2s 2024-03-29T04:11:50.998 INFO:teuthology.orchestra.run.smithi148.stderr:120350K .......... .......... .......... .......... .......... 71% 33.5M 2s 2024-03-29T04:11:51.004 INFO:teuthology.orchestra.run.smithi148.stderr:120400K .......... .......... .......... .......... .......... 71% 28.0M 2s 2024-03-29T04:11:51.004 INFO:teuthology.orchestra.run.smithi148.stderr:120450K .......... .......... .......... .......... .......... 71% 31.9M 2s 2024-03-29T04:11:51.005 INFO:teuthology.orchestra.run.smithi148.stderr:120500K .......... .......... .......... .......... .......... 71% 32.1M 2s 2024-03-29T04:11:51.005 INFO:teuthology.orchestra.run.smithi148.stderr:120550K .......... .......... .......... .......... .......... 71% 32.2M 2s 2024-03-29T04:11:51.006 INFO:teuthology.orchestra.run.smithi148.stderr:120600K .......... .......... .......... .......... .......... 71% 26.6M 2s 2024-03-29T04:11:51.008 INFO:teuthology.orchestra.run.smithi148.stderr:120650K .......... .......... .......... .......... .......... 71% 30.3M 2s 2024-03-29T04:11:51.009 INFO:teuthology.orchestra.run.smithi148.stderr:120700K .......... .......... .......... .......... .......... 71% 31.2M 2s 2024-03-29T04:11:51.011 INFO:teuthology.orchestra.run.smithi148.stderr:120750K .......... .......... .......... .......... .......... 71% 30.3M 2s 2024-03-29T04:11:51.013 INFO:teuthology.orchestra.run.smithi148.stderr:120800K .......... .......... .......... .......... .......... 71% 27.1M 2s 2024-03-29T04:11:51.014 INFO:teuthology.orchestra.run.smithi148.stderr:120850K .......... .......... .......... .......... .......... 71% 30.9M 2s 2024-03-29T04:11:51.016 INFO:teuthology.orchestra.run.smithi148.stderr:120900K .......... .......... .......... .......... .......... 71% 30.7M 2s 2024-03-29T04:11:51.017 INFO:teuthology.orchestra.run.smithi148.stderr:120950K .......... .......... .......... .......... .......... 71% 30.8M 2s 2024-03-29T04:11:51.019 INFO:teuthology.orchestra.run.smithi148.stderr:121000K .......... .......... .......... .......... .......... 71% 26.3M 2s 2024-03-29T04:11:51.021 INFO:teuthology.orchestra.run.smithi148.stderr:121050K .......... .......... .......... .......... .......... 71% 30.3M 2s 2024-03-29T04:11:51.022 INFO:teuthology.orchestra.run.smithi148.stderr:121100K .......... .......... .......... .......... .......... 71% 29.0M 2s 2024-03-29T04:11:51.024 INFO:teuthology.orchestra.run.smithi148.stderr:121150K .......... .......... .......... .......... .......... 71% 31.2M 2s 2024-03-29T04:11:51.026 INFO:teuthology.orchestra.run.smithi148.stderr:121200K .......... .......... .......... .......... .......... 71% 27.4M 2s 2024-03-29T04:11:51.027 INFO:teuthology.orchestra.run.smithi148.stderr:121250K .......... .......... .......... .......... .......... 71% 31.6M 2s 2024-03-29T04:11:51.029 INFO:teuthology.orchestra.run.smithi148.stderr:121300K .......... .......... .......... .......... .......... 71% 28.6M 2s 2024-03-29T04:11:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:121350K .......... .......... .......... .......... .......... 72% 30.4M 2s 2024-03-29T04:11:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:121400K .......... .......... .......... .......... .......... 72% 26.3M 2s 2024-03-29T04:11:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:121450K .......... .......... .......... .......... .......... 72% 31.3M 2s 2024-03-29T04:11:51.036 INFO:teuthology.orchestra.run.smithi148.stderr:121500K .......... .......... .......... .......... .......... 72% 30.1M 2s 2024-03-29T04:11:51.037 INFO:teuthology.orchestra.run.smithi148.stderr:121550K .......... .......... .......... .......... .......... 72% 29.7M 2s 2024-03-29T04:11:51.039 INFO:teuthology.orchestra.run.smithi148.stderr:121600K .......... .......... .......... .......... .......... 72% 25.4M 2s 2024-03-29T04:11:51.041 INFO:teuthology.orchestra.run.smithi148.stderr:121650K .......... .......... .......... .......... .......... 72% 29.6M 2s 2024-03-29T04:11:51.042 INFO:teuthology.orchestra.run.smithi148.stderr:121700K .......... .......... .......... .......... .......... 72% 30.0M 2s 2024-03-29T04:11:51.044 INFO:teuthology.orchestra.run.smithi148.stderr:121750K .......... .......... .......... .......... .......... 72% 30.2M 2s 2024-03-29T04:11:51.046 INFO:teuthology.orchestra.run.smithi148.stderr:121800K .......... .......... .......... .......... .......... 72% 25.1M 2s 2024-03-29T04:11:51.048 INFO:teuthology.orchestra.run.smithi148.stderr:121850K .......... .......... .......... .......... .......... 72% 29.8M 2s 2024-03-29T04:11:51.049 INFO:teuthology.orchestra.run.smithi148.stderr:121900K .......... .......... .......... .......... .......... 72% 29.1M 2s 2024-03-29T04:11:51.051 INFO:teuthology.orchestra.run.smithi148.stderr:121950K .......... .......... .......... .......... .......... 72% 29.2M 2s 2024-03-29T04:11:51.053 INFO:teuthology.orchestra.run.smithi148.stderr:122000K .......... .......... .......... .......... .......... 72% 25.1M 2s 2024-03-29T04:11:51.055 INFO:teuthology.orchestra.run.smithi148.stderr:122050K .......... .......... .......... .......... .......... 72% 28.9M 2s 2024-03-29T04:11:51.056 INFO:teuthology.orchestra.run.smithi148.stderr:122100K .......... .......... .......... .......... .......... 72% 28.6M 2s 2024-03-29T04:11:51.058 INFO:teuthology.orchestra.run.smithi148.stderr:122150K .......... .......... .......... .......... .......... 72% 28.8M 2s 2024-03-29T04:11:51.060 INFO:teuthology.orchestra.run.smithi148.stderr:122200K .......... .......... .......... .......... .......... 72% 25.0M 2s 2024-03-29T04:11:51.062 INFO:teuthology.orchestra.run.smithi148.stderr:122250K .......... .......... .......... .......... .......... 72% 28.4M 2s 2024-03-29T04:11:51.063 INFO:teuthology.orchestra.run.smithi148.stderr:122300K .......... .......... .......... .......... .......... 72% 28.5M 2s 2024-03-29T04:11:51.065 INFO:teuthology.orchestra.run.smithi148.stderr:122350K .......... .......... .......... .......... .......... 72% 28.8M 2s 2024-03-29T04:11:51.067 INFO:teuthology.orchestra.run.smithi148.stderr:122400K .......... .......... .......... .......... .......... 72% 24.3M 2s 2024-03-29T04:11:51.069 INFO:teuthology.orchestra.run.smithi148.stderr:122450K .......... .......... .......... .......... .......... 72% 28.2M 2s 2024-03-29T04:11:51.071 INFO:teuthology.orchestra.run.smithi148.stderr:122500K .......... .......... .......... .......... .......... 72% 28.0M 2s 2024-03-29T04:11:51.072 INFO:teuthology.orchestra.run.smithi148.stderr:122550K .......... .......... .......... .......... .......... 72% 28.1M 2s 2024-03-29T04:11:51.075 INFO:teuthology.orchestra.run.smithi148.stderr:122600K .......... .......... .......... .......... .......... 72% 24.2M 2s 2024-03-29T04:11:51.076 INFO:teuthology.orchestra.run.smithi148.stderr:122650K .......... .......... .......... .......... .......... 72% 28.1M 2s 2024-03-29T04:11:51.078 INFO:teuthology.orchestra.run.smithi148.stderr:122700K .......... .......... .......... .......... .......... 72% 27.6M 2s 2024-03-29T04:11:51.080 INFO:teuthology.orchestra.run.smithi148.stderr:122750K .......... .......... .......... .......... .......... 72% 27.6M 2s 2024-03-29T04:11:51.082 INFO:teuthology.orchestra.run.smithi148.stderr:122800K .......... .......... .......... .......... .......... 72% 23.7M 2s 2024-03-29T04:11:51.084 INFO:teuthology.orchestra.run.smithi148.stderr:122850K .......... .......... .......... .......... .......... 72% 27.1M 2s 2024-03-29T04:11:51.085 INFO:teuthology.orchestra.run.smithi148.stderr:122900K .......... .......... .......... .......... .......... 72% 32.4M 2s 2024-03-29T04:11:51.087 INFO:teuthology.orchestra.run.smithi148.stderr:122950K .......... .......... .......... .......... .......... 72% 30.7M 2s 2024-03-29T04:11:51.089 INFO:teuthology.orchestra.run.smithi148.stderr:123000K .......... .......... .......... .......... .......... 72% 26.7M 2s 2024-03-29T04:11:51.090 INFO:teuthology.orchestra.run.smithi148.stderr:123050K .......... .......... .......... .......... .......... 73% 33.4M 2s 2024-03-29T04:11:51.091 INFO:teuthology.orchestra.run.smithi148.stderr:123100K .......... .......... .......... .......... .......... 73% 32.0M 2s 2024-03-29T04:11:51.093 INFO:teuthology.orchestra.run.smithi148.stderr:123150K .......... .......... .......... .......... .......... 73% 31.7M 2s 2024-03-29T04:11:51.095 INFO:teuthology.orchestra.run.smithi148.stderr:123200K .......... .......... .......... .......... .......... 73% 29.0M 2s 2024-03-29T04:11:51.096 INFO:teuthology.orchestra.run.smithi148.stderr:123250K .......... .......... .......... .......... .......... 73% 39.2M 2s 2024-03-29T04:11:51.098 INFO:teuthology.orchestra.run.smithi148.stderr:123300K .......... .......... .......... .......... .......... 73% 35.2M 2s 2024-03-29T04:11:51.099 INFO:teuthology.orchestra.run.smithi148.stderr:123350K .......... .......... .......... .......... .......... 73% 32.5M 2s 2024-03-29T04:11:51.101 INFO:teuthology.orchestra.run.smithi148.stderr:123400K .......... .......... .......... .......... .......... 73% 27.9M 2s 2024-03-29T04:11:51.102 INFO:teuthology.orchestra.run.smithi148.stderr:123450K .......... .......... .......... .......... .......... 73% 28.6M 2s 2024-03-29T04:11:51.104 INFO:teuthology.orchestra.run.smithi148.stderr:123500K .......... .......... .......... .......... .......... 73% 33.1M 2s 2024-03-29T04:11:51.105 INFO:teuthology.orchestra.run.smithi148.stderr:123550K .......... .......... .......... .......... .......... 73% 33.5M 2s 2024-03-29T04:11:51.107 INFO:teuthology.orchestra.run.smithi148.stderr:123600K .......... .......... .......... .......... .......... 73% 28.4M 2s 2024-03-29T04:11:51.108 INFO:teuthology.orchestra.run.smithi148.stderr:123650K .......... .......... .......... .......... .......... 73% 33.7M 2s 2024-03-29T04:11:51.110 INFO:teuthology.orchestra.run.smithi148.stderr:123700K .......... .......... .......... .......... .......... 73% 32.0M 2s 2024-03-29T04:11:51.111 INFO:teuthology.orchestra.run.smithi148.stderr:123750K .......... .......... .......... .......... .......... 73% 32.5M 2s 2024-03-29T04:11:51.113 INFO:teuthology.orchestra.run.smithi148.stderr:123800K .......... .......... .......... .......... .......... 73% 27.4M 2s 2024-03-29T04:11:51.115 INFO:teuthology.orchestra.run.smithi148.stderr:123850K .......... .......... .......... .......... .......... 73% 32.0M 2s 2024-03-29T04:11:51.116 INFO:teuthology.orchestra.run.smithi148.stderr:123900K .......... .......... .......... .......... .......... 73% 31.1M 2s 2024-03-29T04:11:51.118 INFO:teuthology.orchestra.run.smithi148.stderr:123950K .......... .......... .......... .......... .......... 73% 31.4M 2s 2024-03-29T04:11:51.120 INFO:teuthology.orchestra.run.smithi148.stderr:124000K .......... .......... .......... .......... .......... 73% 27.1M 2s 2024-03-29T04:11:51.121 INFO:teuthology.orchestra.run.smithi148.stderr:124050K .......... .......... .......... .......... .......... 73% 30.9M 2s 2024-03-29T04:11:51.123 INFO:teuthology.orchestra.run.smithi148.stderr:124100K .......... .......... .......... .......... .......... 73% 31.3M 2s 2024-03-29T04:11:51.124 INFO:teuthology.orchestra.run.smithi148.stderr:124150K .......... .......... .......... .......... .......... 73% 31.3M 2s 2024-03-29T04:11:51.126 INFO:teuthology.orchestra.run.smithi148.stderr:124200K .......... .......... .......... .......... .......... 73% 26.2M 2s 2024-03-29T04:11:51.128 INFO:teuthology.orchestra.run.smithi148.stderr:124250K .......... .......... .......... .......... .......... 73% 31.0M 2s 2024-03-29T04:11:51.130 INFO:teuthology.orchestra.run.smithi148.stderr:124300K .......... .......... .......... .......... .......... 73% 29.2M 2s 2024-03-29T04:11:51.131 INFO:teuthology.orchestra.run.smithi148.stderr:124350K .......... .......... .......... .......... .......... 73% 27.3M 2s 2024-03-29T04:11:51.133 INFO:teuthology.orchestra.run.smithi148.stderr:124400K .......... .......... .......... .......... .......... 73% 23.4M 2s 2024-03-29T04:11:51.135 INFO:teuthology.orchestra.run.smithi148.stderr:124450K .......... .......... .......... .......... .......... 73% 26.9M 2s 2024-03-29T04:11:51.137 INFO:teuthology.orchestra.run.smithi148.stderr:124500K .......... .......... .......... .......... .......... 73% 27.2M 2s 2024-03-29T04:11:51.139 INFO:teuthology.orchestra.run.smithi148.stderr:124550K .......... .......... .......... .......... .......... 73% 26.6M 2s 2024-03-29T04:11:51.141 INFO:teuthology.orchestra.run.smithi148.stderr:124600K .......... .......... .......... .......... .......... 73% 23.4M 2s 2024-03-29T04:11:51.143 INFO:teuthology.orchestra.run.smithi148.stderr:124650K .......... .......... .......... .......... .......... 73% 26.9M 2s 2024-03-29T04:11:51.145 INFO:teuthology.orchestra.run.smithi148.stderr:124700K .......... .......... .......... .......... .......... 74% 26.8M 2s 2024-03-29T04:11:51.146 INFO:teuthology.orchestra.run.smithi148.stderr:124750K .......... .......... .......... .......... .......... 74% 26.4M 2s 2024-03-29T04:11:51.148 INFO:teuthology.orchestra.run.smithi148.stderr:124800K .......... .......... .......... .......... .......... 74% 23.3M 2s 2024-03-29T04:11:51.150 INFO:teuthology.orchestra.run.smithi148.stderr:124850K .......... .......... .......... .......... .......... 74% 26.4M 2s 2024-03-29T04:11:51.152 INFO:teuthology.orchestra.run.smithi148.stderr:124900K .......... .......... .......... .......... .......... 74% 26.6M 2s 2024-03-29T04:11:51.154 INFO:teuthology.orchestra.run.smithi148.stderr:124950K .......... .......... .......... .......... .......... 74% 25.6M 2s 2024-03-29T04:11:51.156 INFO:teuthology.orchestra.run.smithi148.stderr:125000K .......... .......... .......... .......... .......... 74% 22.8M 2s 2024-03-29T04:11:51.158 INFO:teuthology.orchestra.run.smithi148.stderr:125050K .......... .......... .......... .......... .......... 74% 26.7M 2s 2024-03-29T04:11:51.160 INFO:teuthology.orchestra.run.smithi148.stderr:125100K .......... .......... .......... .......... .......... 74% 26.7M 2s 2024-03-29T04:11:51.162 INFO:teuthology.orchestra.run.smithi148.stderr:125150K .......... .......... .......... .......... .......... 74% 26.3M 2s 2024-03-29T04:11:51.164 INFO:teuthology.orchestra.run.smithi148.stderr:125200K .......... .......... .......... .......... .......... 74% 22.4M 2s 2024-03-29T04:11:51.166 INFO:teuthology.orchestra.run.smithi148.stderr:125250K .......... .......... .......... .......... .......... 74% 26.0M 2s 2024-03-29T04:11:51.168 INFO:teuthology.orchestra.run.smithi148.stderr:125300K .......... .......... .......... .......... .......... 74% 28.8M 2s 2024-03-29T04:11:51.169 INFO:teuthology.orchestra.run.smithi148.stderr:125350K .......... .......... .......... .......... .......... 74% 28.0M 2s 2024-03-29T04:11:51.172 INFO:teuthology.orchestra.run.smithi148.stderr:125400K .......... .......... .......... .......... .......... 74% 18.5M 2s 2024-03-29T04:11:51.174 INFO:teuthology.orchestra.run.smithi148.stderr:125450K .......... .......... .......... .......... .......... 74% 24.1M 2s 2024-03-29T04:11:51.176 INFO:teuthology.orchestra.run.smithi148.stderr:125500K .......... .......... .......... .......... .......... 74% 24.7M 2s 2024-03-29T04:11:51.178 INFO:teuthology.orchestra.run.smithi148.stderr:125550K .......... .......... .......... .......... .......... 74% 23.1M 2s 2024-03-29T04:11:51.180 INFO:teuthology.orchestra.run.smithi148.stderr:125600K .......... .......... .......... .......... .......... 74% 20.5M 2s 2024-03-29T04:11:51.182 INFO:teuthology.orchestra.run.smithi148.stderr:125650K .......... .......... .......... .......... .......... 74% 24.8M 2s 2024-03-29T04:11:51.184 INFO:teuthology.orchestra.run.smithi148.stderr:125700K .......... .......... .......... .......... .......... 74% 24.2M 2s 2024-03-29T04:11:51.186 INFO:teuthology.orchestra.run.smithi148.stderr:125750K .......... .......... .......... .......... .......... 74% 24.0M 2s 2024-03-29T04:11:51.189 INFO:teuthology.orchestra.run.smithi148.stderr:125800K .......... .......... .......... .......... .......... 74% 20.3M 2s 2024-03-29T04:11:51.191 INFO:teuthology.orchestra.run.smithi148.stderr:125850K .......... .......... .......... .......... .......... 74% 23.2M 2s 2024-03-29T04:11:51.193 INFO:teuthology.orchestra.run.smithi148.stderr:125900K .......... .......... .......... .......... .......... 74% 23.3M 2s 2024-03-29T04:11:51.195 INFO:teuthology.orchestra.run.smithi148.stderr:125950K .......... .......... .......... .......... .......... 74% 23.5M 2s 2024-03-29T04:11:51.198 INFO:teuthology.orchestra.run.smithi148.stderr:126000K .......... .......... .......... .......... .......... 74% 19.1M 2s 2024-03-29T04:11:51.200 INFO:teuthology.orchestra.run.smithi148.stderr:126050K .......... .......... .......... .......... .......... 74% 23.1M 2s 2024-03-29T04:11:51.202 INFO:teuthology.orchestra.run.smithi148.stderr:126100K .......... .......... .......... .......... .......... 74% 22.7M 2s 2024-03-29T04:11:51.207 INFO:teuthology.orchestra.run.smithi148.stderr:126150K .......... .......... .......... .......... .......... 74% 22.5M 2s 2024-03-29T04:11:51.207 INFO:teuthology.orchestra.run.smithi148.stderr:126200K .......... .......... .......... .......... .......... 74% 20.5M 2s 2024-03-29T04:11:51.209 INFO:teuthology.orchestra.run.smithi148.stderr:126250K .......... .......... .......... .......... .......... 74% 25.0M 2s 2024-03-29T04:11:51.210 INFO:teuthology.orchestra.run.smithi148.stderr:126300K .......... .......... .......... .......... .......... 74% 25.6M 1s 2024-03-29T04:11:51.212 INFO:teuthology.orchestra.run.smithi148.stderr:126350K .......... .......... .......... .......... .......... 74% 25.3M 1s 2024-03-29T04:11:51.214 INFO:teuthology.orchestra.run.smithi148.stderr:126400K .......... .......... .......... .......... .......... 75% 21.4M 1s 2024-03-29T04:11:51.216 INFO:teuthology.orchestra.run.smithi148.stderr:126450K .......... .......... .......... .......... .......... 75% 29.4M 1s 2024-03-29T04:11:51.218 INFO:teuthology.orchestra.run.smithi148.stderr:126500K .......... .......... .......... .......... .......... 75% 28.2M 1s 2024-03-29T04:11:51.220 INFO:teuthology.orchestra.run.smithi148.stderr:126550K .......... .......... .......... .......... .......... 75% 28.7M 1s 2024-03-29T04:11:51.222 INFO:teuthology.orchestra.run.smithi148.stderr:126600K .......... .......... .......... .......... .......... 75% 24.4M 1s 2024-03-29T04:11:51.223 INFO:teuthology.orchestra.run.smithi148.stderr:126650K .......... .......... .......... .......... .......... 75% 29.4M 1s 2024-03-29T04:11:51.225 INFO:teuthology.orchestra.run.smithi148.stderr:126700K .......... .......... .......... .......... .......... 75% 28.6M 1s 2024-03-29T04:11:51.227 INFO:teuthology.orchestra.run.smithi148.stderr:126750K .......... .......... .......... .......... .......... 75% 29.6M 1s 2024-03-29T04:11:51.229 INFO:teuthology.orchestra.run.smithi148.stderr:126800K .......... .......... .......... .......... .......... 75% 24.7M 1s 2024-03-29T04:11:51.231 INFO:teuthology.orchestra.run.smithi148.stderr:126850K .......... .......... .......... .......... .......... 75% 23.6M 1s 2024-03-29T04:11:51.233 INFO:teuthology.orchestra.run.smithi148.stderr:126900K .......... .......... .......... .......... .......... 75% 23.9M 1s 2024-03-29T04:11:51.235 INFO:teuthology.orchestra.run.smithi148.stderr:126950K .......... .......... .......... .......... .......... 75% 22.6M 1s 2024-03-29T04:11:51.237 INFO:teuthology.orchestra.run.smithi148.stderr:127000K .......... .......... .......... .......... .......... 75% 22.1M 1s 2024-03-29T04:11:51.240 INFO:teuthology.orchestra.run.smithi148.stderr:127050K .......... .......... .......... .......... .......... 75% 16.0M 1s 2024-03-29T04:11:51.242 INFO:teuthology.orchestra.run.smithi148.stderr:127100K .......... .......... .......... .......... .......... 75% 24.4M 1s 2024-03-29T04:11:51.244 INFO:teuthology.orchestra.run.smithi148.stderr:127150K .......... .......... .......... .......... .......... 75% 29.9M 1s 2024-03-29T04:11:51.245 INFO:teuthology.orchestra.run.smithi148.stderr:127200K .......... .......... .......... .......... .......... 75% 30.9M 1s 2024-03-29T04:11:51.247 INFO:teuthology.orchestra.run.smithi148.stderr:127250K .......... .......... .......... .......... .......... 75% 34.6M 1s 2024-03-29T04:11:51.249 INFO:teuthology.orchestra.run.smithi148.stderr:127300K .......... .......... .......... .......... .......... 75% 26.8M 1s 2024-03-29T04:11:51.250 INFO:teuthology.orchestra.run.smithi148.stderr:127350K .......... .......... .......... .......... .......... 75% 39.5M 1s 2024-03-29T04:11:51.251 INFO:teuthology.orchestra.run.smithi148.stderr:127400K .......... .......... .......... .......... .......... 75% 32.0M 1s 2024-03-29T04:11:51.252 INFO:teuthology.orchestra.run.smithi148.stderr:127450K .......... .......... .......... .......... .......... 75% 38.7M 1s 2024-03-29T04:11:51.254 INFO:teuthology.orchestra.run.smithi148.stderr:127500K .......... .......... .......... .......... .......... 75% 38.0M 1s 2024-03-29T04:11:51.255 INFO:teuthology.orchestra.run.smithi148.stderr:127550K .......... .......... .......... .......... .......... 75% 38.8M 1s 2024-03-29T04:11:51.257 INFO:teuthology.orchestra.run.smithi148.stderr:127600K .......... .......... .......... .......... .......... 75% 32.5M 1s 2024-03-29T04:11:51.258 INFO:teuthology.orchestra.run.smithi148.stderr:127650K .......... .......... .......... .......... .......... 75% 38.0M 1s 2024-03-29T04:11:51.259 INFO:teuthology.orchestra.run.smithi148.stderr:127700K .......... .......... .......... .......... .......... 75% 37.6M 1s 2024-03-29T04:11:51.261 INFO:teuthology.orchestra.run.smithi148.stderr:127750K .......... .......... .......... .......... .......... 75% 37.3M 1s 2024-03-29T04:11:51.262 INFO:teuthology.orchestra.run.smithi148.stderr:127800K .......... .......... .......... .......... .......... 75% 32.1M 1s 2024-03-29T04:11:51.264 INFO:teuthology.orchestra.run.smithi148.stderr:127850K .......... .......... .......... .......... .......... 75% 36.3M 1s 2024-03-29T04:11:51.265 INFO:teuthology.orchestra.run.smithi148.stderr:127900K .......... .......... .......... .......... .......... 75% 36.6M 1s 2024-03-29T04:11:51.266 INFO:teuthology.orchestra.run.smithi148.stderr:127950K .......... .......... .......... .......... .......... 75% 36.8M 1s 2024-03-29T04:11:51.268 INFO:teuthology.orchestra.run.smithi148.stderr:128000K .......... .......... .......... .......... .......... 75% 29.9M 1s 2024-03-29T04:11:51.269 INFO:teuthology.orchestra.run.smithi148.stderr:128050K .......... .......... .......... .......... .......... 75% 35.1M 1s 2024-03-29T04:11:51.270 INFO:teuthology.orchestra.run.smithi148.stderr:128100K .......... .......... .......... .......... .......... 76% 35.2M 1s 2024-03-29T04:11:51.272 INFO:teuthology.orchestra.run.smithi148.stderr:128150K .......... .......... .......... .......... .......... 76% 34.6M 1s 2024-03-29T04:11:51.274 INFO:teuthology.orchestra.run.smithi148.stderr:128200K .......... .......... .......... .......... .......... 76% 29.6M 1s 2024-03-29T04:11:51.275 INFO:teuthology.orchestra.run.smithi148.stderr:128250K .......... .......... .......... .......... .......... 76% 34.0M 1s 2024-03-29T04:11:51.279 INFO:teuthology.orchestra.run.smithi148.stderr:128300K .......... .......... .......... .......... .......... 76% 33.3M 1s 2024-03-29T04:11:51.279 INFO:teuthology.orchestra.run.smithi148.stderr:128350K .......... .......... .......... .......... .......... 76% 34.9M 1s 2024-03-29T04:11:51.281 INFO:teuthology.orchestra.run.smithi148.stderr:128400K .......... .......... .......... .......... .......... 76% 28.2M 1s 2024-03-29T04:11:51.281 INFO:teuthology.orchestra.run.smithi148.stderr:128450K .......... .......... .......... .......... .......... 76% 33.0M 1s 2024-03-29T04:11:51.284 INFO:teuthology.orchestra.run.smithi148.stderr:128500K .......... .......... .......... .......... .......... 76% 33.9M 1s 2024-03-29T04:11:51.285 INFO:teuthology.orchestra.run.smithi148.stderr:128550K .......... .......... .......... .......... .......... 76% 33.9M 1s 2024-03-29T04:11:51.286 INFO:teuthology.orchestra.run.smithi148.stderr:128600K .......... .......... .......... .......... .......... 76% 28.7M 1s 2024-03-29T04:11:51.287 INFO:teuthology.orchestra.run.smithi148.stderr:128650K .......... .......... .......... .......... .......... 76% 33.0M 1s 2024-03-29T04:11:51.289 INFO:teuthology.orchestra.run.smithi148.stderr:128700K .......... .......... .......... .......... .......... 76% 32.8M 1s 2024-03-29T04:11:51.290 INFO:teuthology.orchestra.run.smithi148.stderr:128750K .......... .......... .......... .......... .......... 76% 32.9M 1s 2024-03-29T04:11:51.292 INFO:teuthology.orchestra.run.smithi148.stderr:128800K .......... .......... .......... .......... .......... 76% 27.6M 1s 2024-03-29T04:11:51.294 INFO:teuthology.orchestra.run.smithi148.stderr:128850K .......... .......... .......... .......... .......... 76% 32.6M 1s 2024-03-29T04:11:51.295 INFO:teuthology.orchestra.run.smithi148.stderr:128900K .......... .......... .......... .......... .......... 76% 31.8M 1s 2024-03-29T04:11:51.296 INFO:teuthology.orchestra.run.smithi148.stderr:128950K .......... .......... .......... .......... .......... 76% 33.1M 1s 2024-03-29T04:11:51.298 INFO:teuthology.orchestra.run.smithi148.stderr:129000K .......... .......... .......... .......... .......... 76% 28.6M 1s 2024-03-29T04:11:51.300 INFO:teuthology.orchestra.run.smithi148.stderr:129050K .......... .......... .......... .......... .......... 76% 31.7M 1s 2024-03-29T04:11:51.301 INFO:teuthology.orchestra.run.smithi148.stderr:129100K .......... .......... .......... .......... .......... 76% 30.0M 1s 2024-03-29T04:11:51.303 INFO:teuthology.orchestra.run.smithi148.stderr:129150K .......... .......... .......... .......... .......... 76% 31.3M 1s 2024-03-29T04:11:51.305 INFO:teuthology.orchestra.run.smithi148.stderr:129200K .......... .......... .......... .......... .......... 76% 26.4M 1s 2024-03-29T04:11:51.306 INFO:teuthology.orchestra.run.smithi148.stderr:129250K .......... .......... .......... .......... .......... 76% 31.2M 1s 2024-03-29T04:11:51.308 INFO:teuthology.orchestra.run.smithi148.stderr:129300K .......... .......... .......... .......... .......... 76% 30.9M 1s 2024-03-29T04:11:51.309 INFO:teuthology.orchestra.run.smithi148.stderr:129350K .......... .......... .......... .......... .......... 76% 30.7M 1s 2024-03-29T04:11:51.311 INFO:teuthology.orchestra.run.smithi148.stderr:129400K .......... .......... .......... .......... .......... 76% 26.1M 1s 2024-03-29T04:11:51.313 INFO:teuthology.orchestra.run.smithi148.stderr:129450K .......... .......... .......... .......... .......... 76% 30.9M 1s 2024-03-29T04:11:51.314 INFO:teuthology.orchestra.run.smithi148.stderr:129500K .......... .......... .......... .......... .......... 76% 30.6M 1s 2024-03-29T04:11:51.316 INFO:teuthology.orchestra.run.smithi148.stderr:129550K .......... .......... .......... .......... .......... 76% 29.8M 1s 2024-03-29T04:11:51.318 INFO:teuthology.orchestra.run.smithi148.stderr:129600K .......... .......... .......... .......... .......... 76% 26.1M 1s 2024-03-29T04:11:51.319 INFO:teuthology.orchestra.run.smithi148.stderr:129650K .......... .......... .......... .......... .......... 76% 30.3M 1s 2024-03-29T04:11:51.321 INFO:teuthology.orchestra.run.smithi148.stderr:129700K .......... .......... .......... .......... .......... 76% 30.4M 1s 2024-03-29T04:11:51.323 INFO:teuthology.orchestra.run.smithi148.stderr:129750K .......... .......... .......... .......... .......... 77% 29.8M 1s 2024-03-29T04:11:51.325 INFO:teuthology.orchestra.run.smithi148.stderr:129800K .......... .......... .......... .......... .......... 77% 25.3M 1s 2024-03-29T04:11:51.326 INFO:teuthology.orchestra.run.smithi148.stderr:129850K .......... .......... .......... .......... .......... 77% 29.2M 1s 2024-03-29T04:11:51.328 INFO:teuthology.orchestra.run.smithi148.stderr:129900K .......... .......... .......... .......... .......... 77% 29.6M 1s 2024-03-29T04:11:51.330 INFO:teuthology.orchestra.run.smithi148.stderr:129950K .......... .......... .......... .......... .......... 77% 28.7M 1s 2024-03-29T04:11:51.332 INFO:teuthology.orchestra.run.smithi148.stderr:130000K .......... .......... .......... .......... .......... 77% 24.8M 1s 2024-03-29T04:11:51.334 INFO:teuthology.orchestra.run.smithi148.stderr:130050K .......... .......... .......... .......... .......... 77% 28.8M 1s 2024-03-29T04:11:51.335 INFO:teuthology.orchestra.run.smithi148.stderr:130100K .......... .......... .......... .......... .......... 77% 29.0M 1s 2024-03-29T04:11:51.337 INFO:teuthology.orchestra.run.smithi148.stderr:130150K .......... .......... .......... .......... .......... 77% 29.2M 1s 2024-03-29T04:11:51.339 INFO:teuthology.orchestra.run.smithi148.stderr:130200K .......... .......... .......... .......... .......... 77% 24.7M 1s 2024-03-29T04:11:51.340 INFO:teuthology.orchestra.run.smithi148.stderr:130250K .......... .......... .......... .......... .......... 77% 28.0M 1s 2024-03-29T04:11:51.342 INFO:teuthology.orchestra.run.smithi148.stderr:130300K .......... .......... .......... .......... .......... 77% 28.2M 1s 2024-03-29T04:11:51.344 INFO:teuthology.orchestra.run.smithi148.stderr:130350K .......... .......... .......... .......... .......... 77% 27.9M 1s 2024-03-29T04:11:51.346 INFO:teuthology.orchestra.run.smithi148.stderr:130400K .......... .......... .......... .......... .......... 77% 23.7M 1s 2024-03-29T04:11:51.348 INFO:teuthology.orchestra.run.smithi148.stderr:130450K .......... .......... .......... .......... .......... 77% 28.2M 1s 2024-03-29T04:11:51.349 INFO:teuthology.orchestra.run.smithi148.stderr:130500K .......... .......... .......... .......... .......... 77% 27.5M 1s 2024-03-29T04:11:51.351 INFO:teuthology.orchestra.run.smithi148.stderr:130550K .......... .......... .......... .......... .......... 77% 28.0M 1s 2024-03-29T04:11:51.353 INFO:teuthology.orchestra.run.smithi148.stderr:130600K .......... .......... .......... .......... .......... 77% 23.3M 1s 2024-03-29T04:11:51.355 INFO:teuthology.orchestra.run.smithi148.stderr:130650K .......... .......... .......... .......... .......... 77% 27.3M 1s 2024-03-29T04:11:51.357 INFO:teuthology.orchestra.run.smithi148.stderr:130700K .......... .......... .......... .......... .......... 77% 27.1M 1s 2024-03-29T04:11:51.359 INFO:teuthology.orchestra.run.smithi148.stderr:130750K .......... .......... .......... .......... .......... 77% 26.8M 1s 2024-03-29T04:11:51.361 INFO:teuthology.orchestra.run.smithi148.stderr:130800K .......... .......... .......... .......... .......... 77% 23.3M 1s 2024-03-29T04:11:51.363 INFO:teuthology.orchestra.run.smithi148.stderr:130850K .......... .......... .......... .......... .......... 77% 27.0M 1s 2024-03-29T04:11:51.365 INFO:teuthology.orchestra.run.smithi148.stderr:130900K .......... .......... .......... .......... .......... 77% 26.6M 1s 2024-03-29T04:11:51.366 INFO:teuthology.orchestra.run.smithi148.stderr:130950K .......... .......... .......... .......... .......... 77% 26.3M 1s 2024-03-29T04:11:51.368 INFO:teuthology.orchestra.run.smithi148.stderr:131000K .......... .......... .......... .......... .......... 77% 22.8M 1s 2024-03-29T04:11:51.370 INFO:teuthology.orchestra.run.smithi148.stderr:131050K .......... .......... .......... .......... .......... 77% 25.0M 1s 2024-03-29T04:11:51.372 INFO:teuthology.orchestra.run.smithi148.stderr:131100K .......... .......... .......... .......... .......... 77% 32.3M 1s 2024-03-29T04:11:51.373 INFO:teuthology.orchestra.run.smithi148.stderr:131150K .......... .......... .......... .......... .......... 77% 36.2M 1s 2024-03-29T04:11:51.375 INFO:teuthology.orchestra.run.smithi148.stderr:131200K .......... .......... .......... .......... .......... 77% 23.6M 1s 2024-03-29T04:11:51.377 INFO:teuthology.orchestra.run.smithi148.stderr:131250K .......... .......... .......... .......... .......... 77% 23.4M 1s 2024-03-29T04:11:51.379 INFO:teuthology.orchestra.run.smithi148.stderr:131300K .......... .......... .......... .......... .......... 77% 27.8M 1s 2024-03-29T04:11:51.381 INFO:teuthology.orchestra.run.smithi148.stderr:131350K .......... .......... .......... .......... .......... 77% 28.1M 1s 2024-03-29T04:11:51.383 INFO:teuthology.orchestra.run.smithi148.stderr:131400K .......... .......... .......... .......... .......... 77% 24.9M 1s 2024-03-29T04:11:51.384 INFO:teuthology.orchestra.run.smithi148.stderr:131450K .......... .......... .......... .......... .......... 78% 34.8M 1s 2024-03-29T04:11:51.386 INFO:teuthology.orchestra.run.smithi148.stderr:131500K .......... .......... .......... .......... .......... 78% 26.1M 1s 2024-03-29T04:11:51.388 INFO:teuthology.orchestra.run.smithi148.stderr:131550K .......... .......... .......... .......... .......... 78% 27.9M 1s 2024-03-29T04:11:51.390 INFO:teuthology.orchestra.run.smithi148.stderr:131600K .......... .......... .......... .......... .......... 78% 23.8M 1s 2024-03-29T04:11:51.392 INFO:teuthology.orchestra.run.smithi148.stderr:131650K .......... .......... .......... .......... .......... 78% 28.1M 1s 2024-03-29T04:11:51.394 INFO:teuthology.orchestra.run.smithi148.stderr:131700K .......... .......... .......... .......... .......... 78% 27.3M 1s 2024-03-29T04:11:51.395 INFO:teuthology.orchestra.run.smithi148.stderr:131750K .......... .......... .......... .......... .......... 78% 26.8M 1s 2024-03-29T04:11:51.397 INFO:teuthology.orchestra.run.smithi148.stderr:131800K .......... .......... .......... .......... .......... 78% 23.1M 1s 2024-03-29T04:11:51.399 INFO:teuthology.orchestra.run.smithi148.stderr:131850K .......... .......... .......... .......... .......... 78% 27.7M 1s 2024-03-29T04:11:51.401 INFO:teuthology.orchestra.run.smithi148.stderr:131900K .......... .......... .......... .......... .......... 78% 26.9M 1s 2024-03-29T04:11:51.403 INFO:teuthology.orchestra.run.smithi148.stderr:131950K .......... .......... .......... .......... .......... 78% 26.8M 1s 2024-03-29T04:11:51.405 INFO:teuthology.orchestra.run.smithi148.stderr:132000K .......... .......... .......... .......... .......... 78% 22.3M 1s 2024-03-29T04:11:51.412 INFO:teuthology.orchestra.run.smithi148.stderr:132050K .......... .......... .......... .......... .......... 78% 26.2M 1s 2024-03-29T04:11:51.413 INFO:teuthology.orchestra.run.smithi148.stderr:132100K .......... .......... .......... .......... .......... 78% 28.7M 1s 2024-03-29T04:11:51.413 INFO:teuthology.orchestra.run.smithi148.stderr:132150K .......... .......... .......... .......... .......... 78% 28.5M 1s 2024-03-29T04:11:51.413 INFO:teuthology.orchestra.run.smithi148.stderr:132200K .......... .......... .......... .......... .......... 78% 25.6M 1s 2024-03-29T04:11:51.414 INFO:teuthology.orchestra.run.smithi148.stderr:132250K .......... .......... .......... .......... .......... 78% 27.7M 1s 2024-03-29T04:11:51.416 INFO:teuthology.orchestra.run.smithi148.stderr:132300K .......... .......... .......... .......... .......... 78% 28.1M 1s 2024-03-29T04:11:51.417 INFO:teuthology.orchestra.run.smithi148.stderr:132350K .......... .......... .......... .......... .......... 78% 27.9M 1s 2024-03-29T04:11:51.419 INFO:teuthology.orchestra.run.smithi148.stderr:132400K .......... .......... .......... .......... .......... 78% 24.6M 1s 2024-03-29T04:11:51.421 INFO:teuthology.orchestra.run.smithi148.stderr:132450K .......... .......... .......... .......... .......... 78% 28.7M 1s 2024-03-29T04:11:51.423 INFO:teuthology.orchestra.run.smithi148.stderr:132500K .......... .......... .......... .......... .......... 78% 28.1M 1s 2024-03-29T04:11:51.425 INFO:teuthology.orchestra.run.smithi148.stderr:132550K .......... .......... .......... .......... .......... 78% 27.7M 1s 2024-03-29T04:11:51.427 INFO:teuthology.orchestra.run.smithi148.stderr:132600K .......... .......... .......... .......... .......... 78% 23.8M 1s 2024-03-29T04:11:51.428 INFO:teuthology.orchestra.run.smithi148.stderr:132650K .......... .......... .......... .......... .......... 78% 27.5M 1s 2024-03-29T04:11:51.431 INFO:teuthology.orchestra.run.smithi148.stderr:132700K .......... .......... .......... .......... .......... 78% 25.0M 1s 2024-03-29T04:11:51.433 INFO:teuthology.orchestra.run.smithi148.stderr:132750K .......... .......... .......... .......... .......... 78% 24.0M 1s 2024-03-29T04:11:51.435 INFO:teuthology.orchestra.run.smithi148.stderr:132800K .......... .......... .......... .......... .......... 78% 20.7M 1s 2024-03-29T04:11:51.437 INFO:teuthology.orchestra.run.smithi148.stderr:132850K .......... .......... .......... .......... .......... 78% 23.9M 1s 2024-03-29T04:11:51.439 INFO:teuthology.orchestra.run.smithi148.stderr:132900K .......... .......... .......... .......... .......... 78% 25.1M 1s 2024-03-29T04:11:51.441 INFO:teuthology.orchestra.run.smithi148.stderr:132950K .......... .......... .......... .......... .......... 78% 26.2M 1s 2024-03-29T04:11:51.443 INFO:teuthology.orchestra.run.smithi148.stderr:133000K .......... .......... .......... .......... .......... 78% 22.5M 1s 2024-03-29T04:11:51.445 INFO:teuthology.orchestra.run.smithi148.stderr:133050K .......... .......... .......... .......... .......... 78% 27.2M 1s 2024-03-29T04:11:51.447 INFO:teuthology.orchestra.run.smithi148.stderr:133100K .......... .......... .......... .......... .......... 78% 25.7M 1s 2024-03-29T04:11:51.448 INFO:teuthology.orchestra.run.smithi148.stderr:133150K .......... .......... .......... .......... .......... 79% 25.6M 1s 2024-03-29T04:11:51.451 INFO:teuthology.orchestra.run.smithi148.stderr:133200K .......... .......... .......... .......... .......... 79% 22.1M 1s 2024-03-29T04:11:51.453 INFO:teuthology.orchestra.run.smithi148.stderr:133250K .......... .......... .......... .......... .......... 79% 25.8M 1s 2024-03-29T04:11:51.454 INFO:teuthology.orchestra.run.smithi148.stderr:133300K .......... .......... .......... .......... .......... 79% 26.0M 1s 2024-03-29T04:11:51.456 INFO:teuthology.orchestra.run.smithi148.stderr:133350K .......... .......... .......... .......... .......... 79% 25.6M 1s 2024-03-29T04:11:51.459 INFO:teuthology.orchestra.run.smithi148.stderr:133400K .......... .......... .......... .......... .......... 79% 21.0M 1s 2024-03-29T04:11:51.461 INFO:teuthology.orchestra.run.smithi148.stderr:133450K .......... .......... .......... .......... .......... 79% 23.0M 1s 2024-03-29T04:11:51.463 INFO:teuthology.orchestra.run.smithi148.stderr:133500K .......... .......... .......... .......... .......... 79% 22.4M 1s 2024-03-29T04:11:51.465 INFO:teuthology.orchestra.run.smithi148.stderr:133550K .......... .......... .......... .......... .......... 79% 22.4M 1s 2024-03-29T04:11:51.468 INFO:teuthology.orchestra.run.smithi148.stderr:133600K .......... .......... .......... .......... .......... 79% 19.2M 1s 2024-03-29T04:11:51.470 INFO:teuthology.orchestra.run.smithi148.stderr:133650K .......... .......... .......... .......... .......... 79% 22.0M 1s 2024-03-29T04:11:51.472 INFO:teuthology.orchestra.run.smithi148.stderr:133700K .......... .......... .......... .......... .......... 79% 22.0M 1s 2024-03-29T04:11:51.474 INFO:teuthology.orchestra.run.smithi148.stderr:133750K .......... .......... .......... .......... .......... 79% 22.1M 1s 2024-03-29T04:11:51.477 INFO:teuthology.orchestra.run.smithi148.stderr:133800K .......... .......... .......... .......... .......... 79% 19.6M 1s 2024-03-29T04:11:51.479 INFO:teuthology.orchestra.run.smithi148.stderr:133850K .......... .......... .......... .......... .......... 79% 23.7M 1s 2024-03-29T04:11:51.481 INFO:teuthology.orchestra.run.smithi148.stderr:133900K .......... .......... .......... .......... .......... 79% 24.6M 1s 2024-03-29T04:11:51.483 INFO:teuthology.orchestra.run.smithi148.stderr:133950K .......... .......... .......... .......... .......... 79% 25.4M 1s 2024-03-29T04:11:51.485 INFO:teuthology.orchestra.run.smithi148.stderr:134000K .......... .......... .......... .......... .......... 79% 21.2M 1s 2024-03-29T04:11:51.487 INFO:teuthology.orchestra.run.smithi148.stderr:134050K .......... .......... .......... .......... .......... 79% 24.1M 1s 2024-03-29T04:11:51.489 INFO:teuthology.orchestra.run.smithi148.stderr:134100K .......... .......... .......... .......... .......... 79% 25.5M 1s 2024-03-29T04:11:51.491 INFO:teuthology.orchestra.run.smithi148.stderr:134150K .......... .......... .......... .......... .......... 79% 26.3M 1s 2024-03-29T04:11:51.493 INFO:teuthology.orchestra.run.smithi148.stderr:134200K .......... .......... .......... .......... .......... 79% 22.5M 1s 2024-03-29T04:11:51.495 INFO:teuthology.orchestra.run.smithi148.stderr:134250K .......... .......... .......... .......... .......... 79% 26.9M 1s 2024-03-29T04:11:51.497 INFO:teuthology.orchestra.run.smithi148.stderr:134300K .......... .......... .......... .......... .......... 79% 26.3M 1s 2024-03-29T04:11:51.499 INFO:teuthology.orchestra.run.smithi148.stderr:134350K .......... .......... .......... .......... .......... 79% 26.1M 1s 2024-03-29T04:11:51.501 INFO:teuthology.orchestra.run.smithi148.stderr:134400K .......... .......... .......... .......... .......... 79% 18.6M 1s 2024-03-29T04:11:51.504 INFO:teuthology.orchestra.run.smithi148.stderr:134450K .......... .......... .......... .......... .......... 79% 22.6M 1s 2024-03-29T04:11:51.506 INFO:teuthology.orchestra.run.smithi148.stderr:134500K .......... .......... .......... .......... .......... 79% 23.7M 1s 2024-03-29T04:11:51.508 INFO:teuthology.orchestra.run.smithi148.stderr:134550K .......... .......... .......... .......... .......... 79% 23.8M 1s 2024-03-29T04:11:51.510 INFO:teuthology.orchestra.run.smithi148.stderr:134600K .......... .......... .......... .......... .......... 79% 19.8M 1s 2024-03-29T04:11:51.512 INFO:teuthology.orchestra.run.smithi148.stderr:134650K .......... .......... .......... .......... .......... 79% 23.2M 1s 2024-03-29T04:11:51.514 INFO:teuthology.orchestra.run.smithi148.stderr:134700K .......... .......... .......... .......... .......... 79% 24.2M 1s 2024-03-29T04:11:51.516 INFO:teuthology.orchestra.run.smithi148.stderr:134750K .......... .......... .......... .......... .......... 79% 23.6M 1s 2024-03-29T04:11:51.519 INFO:teuthology.orchestra.run.smithi148.stderr:134800K .......... .......... .......... .......... .......... 79% 20.0M 1s 2024-03-29T04:11:51.521 INFO:teuthology.orchestra.run.smithi148.stderr:134850K .......... .......... .......... .......... .......... 80% 23.1M 1s 2024-03-29T04:11:51.523 INFO:teuthology.orchestra.run.smithi148.stderr:134900K .......... .......... .......... .......... .......... 80% 23.1M 1s 2024-03-29T04:11:51.525 INFO:teuthology.orchestra.run.smithi148.stderr:134950K .......... .......... .......... .......... .......... 80% 23.4M 1s 2024-03-29T04:11:51.527 INFO:teuthology.orchestra.run.smithi148.stderr:135000K .......... .......... .......... .......... .......... 80% 19.9M 1s 2024-03-29T04:11:51.530 INFO:teuthology.orchestra.run.smithi148.stderr:135050K .......... .......... .......... .......... .......... 80% 22.9M 1s 2024-03-29T04:11:51.532 INFO:teuthology.orchestra.run.smithi148.stderr:135100K .......... .......... .......... .......... .......... 80% 23.2M 1s 2024-03-29T04:11:51.533 INFO:teuthology.orchestra.run.smithi148.stderr:135150K .......... .......... .......... .......... .......... 80% 25.3M 1s 2024-03-29T04:11:51.535 INFO:teuthology.orchestra.run.smithi148.stderr:135200K .......... .......... .......... .......... .......... 80% 33.7M 1s 2024-03-29T04:11:51.537 INFO:teuthology.orchestra.run.smithi148.stderr:135250K .......... .......... .......... .......... .......... 80% 27.9M 1s 2024-03-29T04:11:51.539 INFO:teuthology.orchestra.run.smithi148.stderr:135300K .......... .......... .......... .......... .......... 80% 25.3M 1s 2024-03-29T04:11:51.541 INFO:teuthology.orchestra.run.smithi148.stderr:135350K .......... .......... .......... .......... .......... 80% 25.7M 1s 2024-03-29T04:11:51.542 INFO:teuthology.orchestra.run.smithi148.stderr:135400K .......... .......... .......... .......... .......... 80% 27.7M 1s 2024-03-29T04:11:51.544 INFO:teuthology.orchestra.run.smithi148.stderr:135450K .......... .......... .......... .......... .......... 80% 28.7M 1s 2024-03-29T04:11:51.546 INFO:teuthology.orchestra.run.smithi148.stderr:135500K .......... .......... .......... .......... .......... 80% 32.7M 1s 2024-03-29T04:11:51.547 INFO:teuthology.orchestra.run.smithi148.stderr:135550K .......... .......... .......... .......... .......... 80% 31.3M 1s 2024-03-29T04:11:51.549 INFO:teuthology.orchestra.run.smithi148.stderr:135600K .......... .......... .......... .......... .......... 80% 31.2M 1s 2024-03-29T04:11:51.550 INFO:teuthology.orchestra.run.smithi148.stderr:135650K .......... .......... .......... .......... .......... 80% 41.0M 1s 2024-03-29T04:11:51.551 INFO:teuthology.orchestra.run.smithi148.stderr:135700K .......... .......... .......... .......... .......... 80% 39.6M 1s 2024-03-29T04:11:51.552 INFO:teuthology.orchestra.run.smithi148.stderr:135750K .......... .......... .......... .......... .......... 80% 39.1M 1s 2024-03-29T04:11:51.554 INFO:teuthology.orchestra.run.smithi148.stderr:135800K .......... .......... .......... .......... .......... 80% 34.7M 1s 2024-03-29T04:11:51.555 INFO:teuthology.orchestra.run.smithi148.stderr:135850K .......... .......... .......... .......... .......... 80% 40.3M 1s 2024-03-29T04:11:51.556 INFO:teuthology.orchestra.run.smithi148.stderr:135900K .......... .......... .......... .......... .......... 80% 37.4M 1s 2024-03-29T04:11:51.558 INFO:teuthology.orchestra.run.smithi148.stderr:135950K .......... .......... .......... .......... .......... 80% 39.7M 1s 2024-03-29T04:11:51.559 INFO:teuthology.orchestra.run.smithi148.stderr:136000K .......... .......... .......... .......... .......... 80% 33.8M 1s 2024-03-29T04:11:51.560 INFO:teuthology.orchestra.run.smithi148.stderr:136050K .......... .......... .......... .......... .......... 80% 38.9M 1s 2024-03-29T04:11:51.562 INFO:teuthology.orchestra.run.smithi148.stderr:136100K .......... .......... .......... .......... .......... 80% 37.6M 1s 2024-03-29T04:11:51.563 INFO:teuthology.orchestra.run.smithi148.stderr:136150K .......... .......... .......... .......... .......... 80% 39.5M 1s 2024-03-29T04:11:51.564 INFO:teuthology.orchestra.run.smithi148.stderr:136200K .......... .......... .......... .......... .......... 80% 33.4M 1s 2024-03-29T04:11:51.565 INFO:teuthology.orchestra.run.smithi148.stderr:136250K .......... .......... .......... .......... .......... 80% 38.4M 1s 2024-03-29T04:11:51.567 INFO:teuthology.orchestra.run.smithi148.stderr:136300K .......... .......... .......... .......... .......... 80% 37.8M 1s 2024-03-29T04:11:51.568 INFO:teuthology.orchestra.run.smithi148.stderr:136350K .......... .......... .......... .......... .......... 80% 37.8M 1s 2024-03-29T04:11:51.569 INFO:teuthology.orchestra.run.smithi148.stderr:136400K .......... .......... .......... .......... .......... 80% 33.2M 1s 2024-03-29T04:11:51.571 INFO:teuthology.orchestra.run.smithi148.stderr:136450K .......... .......... .......... .......... .......... 80% 37.7M 1s 2024-03-29T04:11:51.572 INFO:teuthology.orchestra.run.smithi148.stderr:136500K .......... .......... .......... .......... .......... 81% 34.6M 1s 2024-03-29T04:11:51.574 INFO:teuthology.orchestra.run.smithi148.stderr:136550K .......... .......... .......... .......... .......... 81% 37.3M 1s 2024-03-29T04:11:51.575 INFO:teuthology.orchestra.run.smithi148.stderr:136600K .......... .......... .......... .......... .......... 81% 32.0M 1s 2024-03-29T04:11:51.576 INFO:teuthology.orchestra.run.smithi148.stderr:136650K .......... .......... .......... .......... .......... 81% 36.4M 1s 2024-03-29T04:11:51.578 INFO:teuthology.orchestra.run.smithi148.stderr:136700K .......... .......... .......... .......... .......... 81% 29.3M 1s 2024-03-29T04:11:51.580 INFO:teuthology.orchestra.run.smithi148.stderr:136750K .......... .......... .......... .......... .......... 81% 29.3M 1s 2024-03-29T04:11:51.582 INFO:teuthology.orchestra.run.smithi148.stderr:136800K .......... .......... .......... .......... .......... 81% 25.7M 1s 2024-03-29T04:11:51.583 INFO:teuthology.orchestra.run.smithi148.stderr:136850K .......... .......... .......... .......... .......... 81% 30.0M 1s 2024-03-29T04:11:51.585 INFO:teuthology.orchestra.run.smithi148.stderr:136900K .......... .......... .......... .......... .......... 81% 32.7M 1s 2024-03-29T04:11:51.586 INFO:teuthology.orchestra.run.smithi148.stderr:136950K .......... .......... .......... .......... .......... 81% 35.8M 1s 2024-03-29T04:11:51.588 INFO:teuthology.orchestra.run.smithi148.stderr:137000K .......... .......... .......... .......... .......... 81% 29.8M 1s 2024-03-29T04:11:51.589 INFO:teuthology.orchestra.run.smithi148.stderr:137050K .......... .......... .......... .......... .......... 81% 35.4M 1s 2024-03-29T04:11:51.590 INFO:teuthology.orchestra.run.smithi148.stderr:137100K .......... .......... .......... .......... .......... 81% 33.6M 1s 2024-03-29T04:11:51.592 INFO:teuthology.orchestra.run.smithi148.stderr:137150K .......... .......... .......... .......... .......... 81% 33.7M 1s 2024-03-29T04:11:51.594 INFO:teuthology.orchestra.run.smithi148.stderr:137200K .......... .......... .......... .......... .......... 81% 28.8M 1s 2024-03-29T04:11:51.595 INFO:teuthology.orchestra.run.smithi148.stderr:137250K .......... .......... .......... .......... .......... 81% 33.8M 1s 2024-03-29T04:11:51.597 INFO:teuthology.orchestra.run.smithi148.stderr:137300K .......... .......... .......... .......... .......... 81% 33.4M 1s 2024-03-29T04:11:51.598 INFO:teuthology.orchestra.run.smithi148.stderr:137350K .......... .......... .......... .......... .......... 81% 33.4M 1s 2024-03-29T04:11:51.600 INFO:teuthology.orchestra.run.smithi148.stderr:137400K .......... .......... .......... .......... .......... 81% 28.1M 1s 2024-03-29T04:11:51.601 INFO:teuthology.orchestra.run.smithi148.stderr:137450K .......... .......... .......... .......... .......... 81% 33.3M 1s 2024-03-29T04:11:51.603 INFO:teuthology.orchestra.run.smithi148.stderr:137500K .......... .......... .......... .......... .......... 81% 32.7M 1s 2024-03-29T04:11:51.604 INFO:teuthology.orchestra.run.smithi148.stderr:137550K .......... .......... .......... .......... .......... 81% 33.0M 1s 2024-03-29T04:11:51.606 INFO:teuthology.orchestra.run.smithi148.stderr:137600K .......... .......... .......... .......... .......... 81% 28.2M 1s 2024-03-29T04:11:51.608 INFO:teuthology.orchestra.run.smithi148.stderr:137650K .......... .......... .......... .......... .......... 81% 32.6M 1s 2024-03-29T04:11:51.609 INFO:teuthology.orchestra.run.smithi148.stderr:137700K .......... .......... .......... .......... .......... 81% 32.8M 1s 2024-03-29T04:11:51.615 INFO:teuthology.orchestra.run.smithi148.stderr:137750K .......... .......... .......... .......... .......... 81% 32.3M 1s 2024-03-29T04:11:51.616 INFO:teuthology.orchestra.run.smithi148.stderr:137800K .......... .......... .......... .......... .......... 81% 27.6M 1s 2024-03-29T04:11:51.616 INFO:teuthology.orchestra.run.smithi148.stderr:137850K .......... .......... .......... .......... .......... 81% 31.9M 1s 2024-03-29T04:11:51.616 INFO:teuthology.orchestra.run.smithi148.stderr:137900K .......... .......... .......... .......... .......... 81% 30.8M 1s 2024-03-29T04:11:51.617 INFO:teuthology.orchestra.run.smithi148.stderr:137950K .......... .......... .......... .......... .......... 81% 30.2M 1s 2024-03-29T04:11:51.619 INFO:teuthology.orchestra.run.smithi148.stderr:138000K .......... .......... .......... .......... .......... 81% 25.9M 1s 2024-03-29T04:11:51.621 INFO:teuthology.orchestra.run.smithi148.stderr:138050K .......... .......... .......... .......... .......... 81% 29.3M 1s 2024-03-29T04:11:51.622 INFO:teuthology.orchestra.run.smithi148.stderr:138100K .......... .......... .......... .......... .......... 81% 30.4M 1s 2024-03-29T04:11:51.624 INFO:teuthology.orchestra.run.smithi148.stderr:138150K .......... .......... .......... .......... .......... 81% 30.3M 1s 2024-03-29T04:11:51.626 INFO:teuthology.orchestra.run.smithi148.stderr:138200K .......... .......... .......... .......... .......... 82% 26.2M 1s 2024-03-29T04:11:51.627 INFO:teuthology.orchestra.run.smithi148.stderr:138250K .......... .......... .......... .......... .......... 82% 28.4M 1s 2024-03-29T04:11:51.629 INFO:teuthology.orchestra.run.smithi148.stderr:138300K .......... .......... .......... .......... .......... 82% 25.5M 1s 2024-03-29T04:11:51.631 INFO:teuthology.orchestra.run.smithi148.stderr:138350K .......... .......... .......... .......... .......... 82% 23.4M 1s 2024-03-29T04:11:51.634 INFO:teuthology.orchestra.run.smithi148.stderr:138400K .......... .......... .......... .......... .......... 82% 21.2M 1s 2024-03-29T04:11:51.636 INFO:teuthology.orchestra.run.smithi148.stderr:138450K .......... .......... .......... .......... .......... 82% 24.6M 1s 2024-03-29T04:11:51.638 INFO:teuthology.orchestra.run.smithi148.stderr:138500K .......... .......... .......... .......... .......... 82% 23.2M 1s 2024-03-29T04:11:51.640 INFO:teuthology.orchestra.run.smithi148.stderr:138550K .......... .......... .......... .......... .......... 82% 25.5M 1s 2024-03-29T04:11:51.642 INFO:teuthology.orchestra.run.smithi148.stderr:138600K .......... .......... .......... .......... .......... 82% 21.3M 1s 2024-03-29T04:11:51.644 INFO:teuthology.orchestra.run.smithi148.stderr:138650K .......... .......... .......... .......... .......... 82% 25.6M 1s 2024-03-29T04:11:51.646 INFO:teuthology.orchestra.run.smithi148.stderr:138700K .......... .......... .......... .......... .......... 82% 24.0M 1s 2024-03-29T04:11:51.648 INFO:teuthology.orchestra.run.smithi148.stderr:138750K .......... .......... .......... .......... .......... 82% 24.9M 1s 2024-03-29T04:11:51.651 INFO:teuthology.orchestra.run.smithi148.stderr:138800K .......... .......... .......... .......... .......... 82% 18.1M 1s 2024-03-29T04:11:51.653 INFO:teuthology.orchestra.run.smithi148.stderr:138850K .......... .......... .......... .......... .......... 82% 21.8M 1s 2024-03-29T04:11:51.655 INFO:teuthology.orchestra.run.smithi148.stderr:138900K .......... .......... .......... .......... .......... 82% 21.0M 1s 2024-03-29T04:11:51.658 INFO:teuthology.orchestra.run.smithi148.stderr:138950K .......... .......... .......... .......... .......... 82% 20.8M 1s 2024-03-29T04:11:51.660 INFO:teuthology.orchestra.run.smithi148.stderr:139000K .......... .......... .......... .......... .......... 82% 17.5M 1s 2024-03-29T04:11:51.664 INFO:teuthology.orchestra.run.smithi148.stderr:139050K .......... .......... .......... .......... .......... 82% 20.7M 1s 2024-03-29T04:11:51.665 INFO:teuthology.orchestra.run.smithi148.stderr:139100K .......... .......... .......... .......... .......... 82% 21.3M 1s 2024-03-29T04:11:51.668 INFO:teuthology.orchestra.run.smithi148.stderr:139150K .......... .......... .......... .......... .......... 82% 20.7M 1s 2024-03-29T04:11:51.670 INFO:teuthology.orchestra.run.smithi148.stderr:139200K .......... .......... .......... .......... .......... 82% 17.6M 1s 2024-03-29T04:11:51.672 INFO:teuthology.orchestra.run.smithi148.stderr:139250K .......... .......... .......... .......... .......... 82% 25.7M 1s 2024-03-29T04:11:51.673 INFO:teuthology.orchestra.run.smithi148.stderr:139300K .......... .......... .......... .......... .......... 82% 38.4M 1s 2024-03-29T04:11:51.675 INFO:teuthology.orchestra.run.smithi148.stderr:139350K .......... .......... .......... .......... .......... 82% 28.6M 1s 2024-03-29T04:11:51.678 INFO:teuthology.orchestra.run.smithi148.stderr:139400K .......... .......... .......... .......... .......... 82% 22.1M 1s 2024-03-29T04:11:51.679 INFO:teuthology.orchestra.run.smithi148.stderr:139450K .......... .......... .......... .......... .......... 82% 27.9M 1s 2024-03-29T04:11:51.681 INFO:teuthology.orchestra.run.smithi148.stderr:139500K .......... .......... .......... .......... .......... 82% 28.9M 1s 2024-03-29T04:11:51.682 INFO:teuthology.orchestra.run.smithi148.stderr:139550K .......... .......... .......... .......... .......... 82% 32.3M 1s 2024-03-29T04:11:51.684 INFO:teuthology.orchestra.run.smithi148.stderr:139600K .......... .......... .......... .......... .......... 82% 26.5M 1s 2024-03-29T04:11:51.685 INFO:teuthology.orchestra.run.smithi148.stderr:139650K .......... .......... .......... .......... .......... 82% 34.3M 1s 2024-03-29T04:11:51.687 INFO:teuthology.orchestra.run.smithi148.stderr:139700K .......... .......... .......... .......... .......... 82% 29.1M 1s 2024-03-29T04:11:51.689 INFO:teuthology.orchestra.run.smithi148.stderr:139750K .......... .......... .......... .......... .......... 82% 35.6M 1s 2024-03-29T04:11:51.690 INFO:teuthology.orchestra.run.smithi148.stderr:139800K .......... .......... .......... .......... .......... 82% 30.2M 1s 2024-03-29T04:11:51.691 INFO:teuthology.orchestra.run.smithi148.stderr:139850K .......... .......... .......... .......... .......... 82% 36.1M 1s 2024-03-29T04:11:51.693 INFO:teuthology.orchestra.run.smithi148.stderr:139900K .......... .......... .......... .......... .......... 83% 36.2M 1s 2024-03-29T04:11:51.694 INFO:teuthology.orchestra.run.smithi148.stderr:139950K .......... .......... .......... .......... .......... 83% 35.8M 1s 2024-03-29T04:11:51.696 INFO:teuthology.orchestra.run.smithi148.stderr:140000K .......... .......... .......... .......... .......... 83% 30.7M 1s 2024-03-29T04:11:51.697 INFO:teuthology.orchestra.run.smithi148.stderr:140050K .......... .......... .......... .......... .......... 83% 36.3M 1s 2024-03-29T04:11:51.699 INFO:teuthology.orchestra.run.smithi148.stderr:140100K .......... .......... .......... .......... .......... 83% 35.7M 1s 2024-03-29T04:11:51.700 INFO:teuthology.orchestra.run.smithi148.stderr:140150K .......... .......... .......... .......... .......... 83% 35.9M 1s 2024-03-29T04:11:51.702 INFO:teuthology.orchestra.run.smithi148.stderr:140200K .......... .......... .......... .......... .......... 83% 30.8M 1s 2024-03-29T04:11:51.703 INFO:teuthology.orchestra.run.smithi148.stderr:140250K .......... .......... .......... .......... .......... 83% 35.9M 1s 2024-03-29T04:11:51.704 INFO:teuthology.orchestra.run.smithi148.stderr:140300K .......... .......... .......... .......... .......... 83% 35.0M 1s 2024-03-29T04:11:51.706 INFO:teuthology.orchestra.run.smithi148.stderr:140350K .......... .......... .......... .......... .......... 83% 34.9M 1s 2024-03-29T04:11:51.707 INFO:teuthology.orchestra.run.smithi148.stderr:140400K .......... .......... .......... .......... .......... 83% 30.5M 1s 2024-03-29T04:11:51.710 INFO:teuthology.orchestra.run.smithi148.stderr:140450K .......... .......... .......... .......... .......... 83% 34.4M 1s 2024-03-29T04:11:51.710 INFO:teuthology.orchestra.run.smithi148.stderr:140500K .......... .......... .......... .......... .......... 83% 34.7M 1s 2024-03-29T04:11:51.712 INFO:teuthology.orchestra.run.smithi148.stderr:140550K .......... .......... .......... .......... .......... 83% 33.8M 1s 2024-03-29T04:11:51.713 INFO:teuthology.orchestra.run.smithi148.stderr:140600K .......... .......... .......... .......... .......... 83% 29.7M 1s 2024-03-29T04:11:51.714 INFO:teuthology.orchestra.run.smithi148.stderr:140650K .......... .......... .......... .......... .......... 83% 33.7M 1s 2024-03-29T04:11:51.716 INFO:teuthology.orchestra.run.smithi148.stderr:140700K .......... .......... .......... .......... .......... 83% 33.5M 1s 2024-03-29T04:11:51.717 INFO:teuthology.orchestra.run.smithi148.stderr:140750K .......... .......... .......... .......... .......... 83% 33.1M 1s 2024-03-29T04:11:51.719 INFO:teuthology.orchestra.run.smithi148.stderr:140800K .......... .......... .......... .......... .......... 83% 28.8M 1s 2024-03-29T04:11:51.721 INFO:teuthology.orchestra.run.smithi148.stderr:140850K .......... .......... .......... .......... .......... 83% 33.4M 1s 2024-03-29T04:11:51.722 INFO:teuthology.orchestra.run.smithi148.stderr:140900K .......... .......... .......... .......... .......... 83% 33.7M 1s 2024-03-29T04:11:51.724 INFO:teuthology.orchestra.run.smithi148.stderr:140950K .......... .......... .......... .......... .......... 83% 32.7M 1s 2024-03-29T04:11:51.725 INFO:teuthology.orchestra.run.smithi148.stderr:141000K .......... .......... .......... .......... .......... 83% 27.7M 1s 2024-03-29T04:11:51.727 INFO:teuthology.orchestra.run.smithi148.stderr:141050K .......... .......... .......... .......... .......... 83% 32.8M 1s 2024-03-29T04:11:51.728 INFO:teuthology.orchestra.run.smithi148.stderr:141100K .......... .......... .......... .......... .......... 83% 33.1M 1s 2024-03-29T04:11:51.731 INFO:teuthology.orchestra.run.smithi148.stderr:141150K .......... .......... .......... .......... .......... 83% 32.5M 1s 2024-03-29T04:11:51.732 INFO:teuthology.orchestra.run.smithi148.stderr:141200K .......... .......... .......... .......... .......... 83% 28.0M 1s 2024-03-29T04:11:51.733 INFO:teuthology.orchestra.run.smithi148.stderr:141250K .......... .......... .......... .......... .......... 83% 33.1M 1s 2024-03-29T04:11:51.735 INFO:teuthology.orchestra.run.smithi148.stderr:141300K .......... .......... .......... .......... .......... 83% 32.0M 1s 2024-03-29T04:11:51.736 INFO:teuthology.orchestra.run.smithi148.stderr:141350K .......... .......... .......... .......... .......... 83% 27.2M 1s 2024-03-29T04:11:51.738 INFO:teuthology.orchestra.run.smithi148.stderr:141400K .......... .......... .......... .......... .......... 83% 28.8M 1s 2024-03-29T04:11:51.739 INFO:teuthology.orchestra.run.smithi148.stderr:141450K .......... .......... .......... .......... .......... 83% 33.4M 1s 2024-03-29T04:11:51.741 INFO:teuthology.orchestra.run.smithi148.stderr:141500K .......... .......... .......... .......... .......... 83% 33.1M 1s 2024-03-29T04:11:51.742 INFO:teuthology.orchestra.run.smithi148.stderr:141550K .......... .......... .......... .......... .......... 84% 32.5M 1s 2024-03-29T04:11:51.744 INFO:teuthology.orchestra.run.smithi148.stderr:141600K .......... .......... .......... .......... .......... 84% 27.9M 1s 2024-03-29T04:11:51.746 INFO:teuthology.orchestra.run.smithi148.stderr:141650K .......... .......... .......... .......... .......... 84% 32.6M 1s 2024-03-29T04:11:51.747 INFO:teuthology.orchestra.run.smithi148.stderr:141700K .......... .......... .......... .......... .......... 84% 29.1M 1s 2024-03-29T04:11:51.749 INFO:teuthology.orchestra.run.smithi148.stderr:141750K .......... .......... .......... .......... .......... 84% 26.9M 1s 2024-03-29T04:11:51.751 INFO:teuthology.orchestra.run.smithi148.stderr:141800K .......... .......... .......... .......... .......... 84% 22.7M 1s 2024-03-29T04:11:51.753 INFO:teuthology.orchestra.run.smithi148.stderr:141850K .......... .......... .......... .......... .......... 84% 28.3M 1s 2024-03-29T04:11:51.755 INFO:teuthology.orchestra.run.smithi148.stderr:141900K .......... .......... .......... .......... .......... 84% 28.3M 1s 2024-03-29T04:11:51.757 INFO:teuthology.orchestra.run.smithi148.stderr:141950K .......... .......... .......... .......... .......... 84% 28.5M 1s 2024-03-29T04:11:51.758 INFO:teuthology.orchestra.run.smithi148.stderr:142000K .......... .......... .......... .......... .......... 84% 24.5M 1s 2024-03-29T04:11:51.760 INFO:teuthology.orchestra.run.smithi148.stderr:142050K .......... .......... .......... .......... .......... 84% 28.4M 1s 2024-03-29T04:11:51.762 INFO:teuthology.orchestra.run.smithi148.stderr:142100K .......... .......... .......... .......... .......... 84% 28.1M 1s 2024-03-29T04:11:51.764 INFO:teuthology.orchestra.run.smithi148.stderr:142150K .......... .......... .......... .......... .......... 84% 28.1M 1s 2024-03-29T04:11:51.766 INFO:teuthology.orchestra.run.smithi148.stderr:142200K .......... .......... .......... .......... .......... 84% 24.1M 1s 2024-03-29T04:11:51.767 INFO:teuthology.orchestra.run.smithi148.stderr:142250K .......... .......... .......... .......... .......... 84% 28.0M 1s 2024-03-29T04:11:51.769 INFO:teuthology.orchestra.run.smithi148.stderr:142300K .......... .......... .......... .......... .......... 84% 28.2M 1s 2024-03-29T04:11:51.771 INFO:teuthology.orchestra.run.smithi148.stderr:142350K .......... .......... .......... .......... .......... 84% 27.8M 1s 2024-03-29T04:11:51.773 INFO:teuthology.orchestra.run.smithi148.stderr:142400K .......... .......... .......... .......... .......... 84% 23.7M 1s 2024-03-29T04:11:51.775 INFO:teuthology.orchestra.run.smithi148.stderr:142450K .......... .......... .......... .......... .......... 84% 27.4M 1s 2024-03-29T04:11:51.777 INFO:teuthology.orchestra.run.smithi148.stderr:142500K .......... .......... .......... .......... .......... 84% 27.7M 1s 2024-03-29T04:11:51.778 INFO:teuthology.orchestra.run.smithi148.stderr:142550K .......... .......... .......... .......... .......... 84% 27.3M 1s 2024-03-29T04:11:51.780 INFO:teuthology.orchestra.run.smithi148.stderr:142600K .......... .......... .......... .......... .......... 84% 23.2M 1s 2024-03-29T04:11:51.782 INFO:teuthology.orchestra.run.smithi148.stderr:142650K .......... .......... .......... .......... .......... 84% 27.1M 1s 2024-03-29T04:11:51.784 INFO:teuthology.orchestra.run.smithi148.stderr:142700K .......... .......... .......... .......... .......... 84% 26.7M 1s 2024-03-29T04:11:51.786 INFO:teuthology.orchestra.run.smithi148.stderr:142750K .......... .......... .......... .......... .......... 84% 26.4M 1s 2024-03-29T04:11:51.788 INFO:teuthology.orchestra.run.smithi148.stderr:142800K .......... .......... .......... .......... .......... 84% 22.4M 1s 2024-03-29T04:11:51.790 INFO:teuthology.orchestra.run.smithi148.stderr:142850K .......... .......... .......... .......... .......... 84% 26.3M 1s 2024-03-29T04:11:51.792 INFO:teuthology.orchestra.run.smithi148.stderr:142900K .......... .......... .......... .......... .......... 84% 25.4M 1s 2024-03-29T04:11:51.794 INFO:teuthology.orchestra.run.smithi148.stderr:142950K .......... .......... .......... .......... .......... 84% 23.1M 1s 2024-03-29T04:11:51.796 INFO:teuthology.orchestra.run.smithi148.stderr:143000K .......... .......... .......... .......... .......... 84% 20.6M 1s 2024-03-29T04:11:51.799 INFO:teuthology.orchestra.run.smithi148.stderr:143050K .......... .......... .......... .......... .......... 84% 22.7M 1s 2024-03-29T04:11:51.801 INFO:teuthology.orchestra.run.smithi148.stderr:143100K .......... .......... .......... .......... .......... 84% 22.3M 1s 2024-03-29T04:11:51.803 INFO:teuthology.orchestra.run.smithi148.stderr:143150K .......... .......... .......... .......... .......... 84% 23.3M 1s 2024-03-29T04:11:51.806 INFO:teuthology.orchestra.run.smithi148.stderr:143200K .......... .......... .......... .......... .......... 84% 19.7M 1s 2024-03-29T04:11:51.810 INFO:teuthology.orchestra.run.smithi148.stderr:143250K .......... .......... .......... .......... .......... 85% 22.4M 1s 2024-03-29T04:11:51.811 INFO:teuthology.orchestra.run.smithi148.stderr:143300K .......... .......... .......... .......... .......... 85% 22.2M 1s 2024-03-29T04:11:51.812 INFO:teuthology.orchestra.run.smithi148.stderr:143350K .......... .......... .......... .......... .......... 85% 26.2M 1s 2024-03-29T04:11:51.813 INFO:teuthology.orchestra.run.smithi148.stderr:143400K .......... .......... .......... .......... .......... 85% 30.4M 1s 2024-03-29T04:11:51.815 INFO:teuthology.orchestra.run.smithi148.stderr:143450K .......... .......... .......... .......... .......... 85% 36.0M 1s 2024-03-29T04:11:51.816 INFO:teuthology.orchestra.run.smithi148.stderr:143500K .......... .......... .......... .......... .......... 85% 39.6M 1s 2024-03-29T04:11:51.817 INFO:teuthology.orchestra.run.smithi148.stderr:143550K .......... .......... .......... .......... .......... 85% 39.8M 1s 2024-03-29T04:11:51.819 INFO:teuthology.orchestra.run.smithi148.stderr:143600K .......... .......... .......... .......... .......... 85% 33.8M 1s 2024-03-29T04:11:51.820 INFO:teuthology.orchestra.run.smithi148.stderr:143650K .......... .......... .......... .......... .......... 85% 39.6M 1s 2024-03-29T04:11:51.821 INFO:teuthology.orchestra.run.smithi148.stderr:143700K .......... .......... .......... .......... .......... 85% 38.7M 1s 2024-03-29T04:11:51.822 INFO:teuthology.orchestra.run.smithi148.stderr:143750K .......... .......... .......... .......... .......... 85% 38.2M 1s 2024-03-29T04:11:51.824 INFO:teuthology.orchestra.run.smithi148.stderr:143800K .......... .......... .......... .......... .......... 85% 33.0M 1s 2024-03-29T04:11:51.825 INFO:teuthology.orchestra.run.smithi148.stderr:143850K .......... .......... .......... .......... .......... 85% 38.9M 1s 2024-03-29T04:11:51.826 INFO:teuthology.orchestra.run.smithi148.stderr:143900K .......... .......... .......... .......... .......... 85% 38.7M 1s 2024-03-29T04:11:51.827 INFO:teuthology.orchestra.run.smithi148.stderr:143950K .......... .......... .......... .......... .......... 85% 35.7M 1s 2024-03-29T04:11:51.829 INFO:teuthology.orchestra.run.smithi148.stderr:144000K .......... .......... .......... .......... .......... 85% 34.5M 1s 2024-03-29T04:11:51.830 INFO:teuthology.orchestra.run.smithi148.stderr:144050K .......... .......... .......... .......... .......... 85% 38.8M 1s 2024-03-29T04:11:51.832 INFO:teuthology.orchestra.run.smithi148.stderr:144100K .......... .......... .......... .......... .......... 85% 31.1M 1s 2024-03-29T04:11:51.833 INFO:teuthology.orchestra.run.smithi148.stderr:144150K .......... .......... .......... .......... .......... 85% 31.8M 1s 2024-03-29T04:11:51.835 INFO:teuthology.orchestra.run.smithi148.stderr:144200K .......... .......... .......... .......... .......... 85% 26.8M 1s 2024-03-29T04:11:51.837 INFO:teuthology.orchestra.run.smithi148.stderr:144250K .......... .......... .......... .......... .......... 85% 31.2M 1s 2024-03-29T04:11:51.838 INFO:teuthology.orchestra.run.smithi148.stderr:144300K .......... .......... .......... .......... .......... 85% 31.2M 1s 2024-03-29T04:11:51.840 INFO:teuthology.orchestra.run.smithi148.stderr:144350K .......... .......... .......... .......... .......... 85% 31.3M 1s 2024-03-29T04:11:51.842 INFO:teuthology.orchestra.run.smithi148.stderr:144400K .......... .......... .......... .......... .......... 85% 26.8M 1s 2024-03-29T04:11:51.843 INFO:teuthology.orchestra.run.smithi148.stderr:144450K .......... .......... .......... .......... .......... 85% 31.8M 1s 2024-03-29T04:11:51.845 INFO:teuthology.orchestra.run.smithi148.stderr:144500K .......... .......... .......... .......... .......... 85% 31.4M 1s 2024-03-29T04:11:51.846 INFO:teuthology.orchestra.run.smithi148.stderr:144550K .......... .......... .......... .......... .......... 85% 31.4M 1s 2024-03-29T04:11:51.848 INFO:teuthology.orchestra.run.smithi148.stderr:144600K .......... .......... .......... .......... .......... 85% 26.7M 1s 2024-03-29T04:11:51.850 INFO:teuthology.orchestra.run.smithi148.stderr:144650K .......... .......... .......... .......... .......... 85% 30.7M 1s 2024-03-29T04:11:51.851 INFO:teuthology.orchestra.run.smithi148.stderr:144700K .......... .......... .......... .......... .......... 85% 30.9M 1s 2024-03-29T04:11:51.853 INFO:teuthology.orchestra.run.smithi148.stderr:144750K .......... .......... .......... .......... .......... 85% 30.6M 1s 2024-03-29T04:11:51.855 INFO:teuthology.orchestra.run.smithi148.stderr:144800K .......... .......... .......... .......... .......... 85% 25.9M 1s 2024-03-29T04:11:51.856 INFO:teuthology.orchestra.run.smithi148.stderr:144850K .......... .......... .......... .......... .......... 85% 30.2M 1s 2024-03-29T04:11:51.858 INFO:teuthology.orchestra.run.smithi148.stderr:144900K .......... .......... .......... .......... .......... 85% 30.1M 1s 2024-03-29T04:11:51.859 INFO:teuthology.orchestra.run.smithi148.stderr:144950K .......... .......... .......... .......... .......... 86% 33.1M 1s 2024-03-29T04:11:51.861 INFO:teuthology.orchestra.run.smithi148.stderr:145000K .......... .......... .......... .......... .......... 86% 29.5M 1s 2024-03-29T04:11:51.862 INFO:teuthology.orchestra.run.smithi148.stderr:145050K .......... .......... .......... .......... .......... 86% 33.3M 1s 2024-03-29T04:11:51.864 INFO:teuthology.orchestra.run.smithi148.stderr:145100K .......... .......... .......... .......... .......... 86% 34.9M 1s 2024-03-29T04:11:51.866 INFO:teuthology.orchestra.run.smithi148.stderr:145150K .......... .......... .......... .......... .......... 86% 33.2M 1s 2024-03-29T04:11:51.868 INFO:teuthology.orchestra.run.smithi148.stderr:145200K .......... .......... .......... .......... .......... 86% 28.8M 1s 2024-03-29T04:11:51.869 INFO:teuthology.orchestra.run.smithi148.stderr:145250K .......... .......... .......... .......... .......... 86% 33.1M 1s 2024-03-29T04:11:51.870 INFO:teuthology.orchestra.run.smithi148.stderr:145300K .......... .......... .......... .......... .......... 86% 32.9M 1s 2024-03-29T04:11:51.872 INFO:teuthology.orchestra.run.smithi148.stderr:145350K .......... .......... .......... .......... .......... 86% 33.2M 1s 2024-03-29T04:11:51.873 INFO:teuthology.orchestra.run.smithi148.stderr:145400K .......... .......... .......... .......... .......... 86% 28.2M 1s 2024-03-29T04:11:51.875 INFO:teuthology.orchestra.run.smithi148.stderr:145450K .......... .......... .......... .......... .......... 86% 32.5M 1s 2024-03-29T04:11:51.876 INFO:teuthology.orchestra.run.smithi148.stderr:145500K .......... .......... .......... .......... .......... 86% 33.3M 1s 2024-03-29T04:11:51.878 INFO:teuthology.orchestra.run.smithi148.stderr:145550K .......... .......... .......... .......... .......... 86% 32.8M 1s 2024-03-29T04:11:51.880 INFO:teuthology.orchestra.run.smithi148.stderr:145600K .......... .......... .......... .......... .......... 86% 28.0M 1s 2024-03-29T04:11:51.881 INFO:teuthology.orchestra.run.smithi148.stderr:145650K .......... .......... .......... .......... .......... 86% 29.4M 1s 2024-03-29T04:11:51.883 INFO:teuthology.orchestra.run.smithi148.stderr:145700K .......... .......... .......... .......... .......... 86% 27.5M 1s 2024-03-29T04:11:51.885 INFO:teuthology.orchestra.run.smithi148.stderr:145750K .......... .......... .......... .......... .......... 86% 24.9M 1s 2024-03-29T04:11:51.887 INFO:teuthology.orchestra.run.smithi148.stderr:145800K .......... .......... .......... .......... .......... 86% 20.0M 1s 2024-03-29T04:11:51.889 INFO:teuthology.orchestra.run.smithi148.stderr:145850K .......... .......... .......... .......... .......... 86% 28.2M 1s 2024-03-29T04:11:51.891 INFO:teuthology.orchestra.run.smithi148.stderr:145900K .......... .......... .......... .......... .......... 86% 33.1M 1s 2024-03-29T04:11:51.892 INFO:teuthology.orchestra.run.smithi148.stderr:145950K .......... .......... .......... .......... .......... 86% 34.5M 1s 2024-03-29T04:11:51.894 INFO:teuthology.orchestra.run.smithi148.stderr:146000K .......... .......... .......... .......... .......... 86% 30.0M 1s 2024-03-29T04:11:51.895 INFO:teuthology.orchestra.run.smithi148.stderr:146050K .......... .......... .......... .......... .......... 86% 34.3M 1s 2024-03-29T04:11:51.897 INFO:teuthology.orchestra.run.smithi148.stderr:146100K .......... .......... .......... .......... .......... 86% 33.5M 1s 2024-03-29T04:11:51.898 INFO:teuthology.orchestra.run.smithi148.stderr:146150K .......... .......... .......... .......... .......... 86% 33.0M 1s 2024-03-29T04:11:51.900 INFO:teuthology.orchestra.run.smithi148.stderr:146200K .......... .......... .......... .......... .......... 86% 29.0M 1s 2024-03-29T04:11:51.901 INFO:teuthology.orchestra.run.smithi148.stderr:146250K .......... .......... .......... .......... .......... 86% 33.7M 1s 2024-03-29T04:11:51.902 INFO:teuthology.orchestra.run.smithi148.stderr:146300K .......... .......... .......... .......... .......... 86% 33.7M 1s 2024-03-29T04:11:51.904 INFO:teuthology.orchestra.run.smithi148.stderr:146350K .......... .......... .......... .......... .......... 86% 33.2M 1s 2024-03-29T04:11:51.906 INFO:teuthology.orchestra.run.smithi148.stderr:146400K .......... .......... .......... .......... .......... 86% 29.5M 1s 2024-03-29T04:11:51.907 INFO:teuthology.orchestra.run.smithi148.stderr:146450K .......... .......... .......... .......... .......... 86% 33.0M 1s 2024-03-29T04:11:51.909 INFO:teuthology.orchestra.run.smithi148.stderr:146500K .......... .......... .......... .......... .......... 86% 33.3M 1s 2024-03-29T04:11:51.910 INFO:teuthology.orchestra.run.smithi148.stderr:146550K .......... .......... .......... .......... .......... 86% 32.6M 1s 2024-03-29T04:11:51.912 INFO:teuthology.orchestra.run.smithi148.stderr:146600K .......... .......... .......... .......... .......... 86% 26.1M 1s 2024-03-29T04:11:51.914 INFO:teuthology.orchestra.run.smithi148.stderr:146650K .......... .......... .......... .......... .......... 87% 29.9M 1s 2024-03-29T04:11:51.915 INFO:teuthology.orchestra.run.smithi148.stderr:146700K .......... .......... .......... .......... .......... 87% 31.4M 1s 2024-03-29T04:11:51.917 INFO:teuthology.orchestra.run.smithi148.stderr:146750K .......... .......... .......... .......... .......... 87% 30.5M 1s 2024-03-29T04:11:51.919 INFO:teuthology.orchestra.run.smithi148.stderr:146800K .......... .......... .......... .......... .......... 87% 26.9M 1s 2024-03-29T04:11:51.920 INFO:teuthology.orchestra.run.smithi148.stderr:146850K .......... .......... .......... .......... .......... 87% 30.0M 1s 2024-03-29T04:11:51.922 INFO:teuthology.orchestra.run.smithi148.stderr:146900K .......... .......... .......... .......... .......... 87% 30.0M 1s 2024-03-29T04:11:51.924 INFO:teuthology.orchestra.run.smithi148.stderr:146950K .......... .......... .......... .......... .......... 87% 30.1M 1s 2024-03-29T04:11:51.925 INFO:teuthology.orchestra.run.smithi148.stderr:147000K .......... .......... .......... .......... .......... 87% 25.9M 1s 2024-03-29T04:11:51.927 INFO:teuthology.orchestra.run.smithi148.stderr:147050K .......... .......... .......... .......... .......... 87% 30.0M 1s 2024-03-29T04:11:51.929 INFO:teuthology.orchestra.run.smithi148.stderr:147100K .......... .......... .......... .......... .......... 87% 30.0M 1s 2024-03-29T04:11:51.930 INFO:teuthology.orchestra.run.smithi148.stderr:147150K .......... .......... .......... .......... .......... 87% 30.2M 1s 2024-03-29T04:11:51.932 INFO:teuthology.orchestra.run.smithi148.stderr:147200K .......... .......... .......... .......... .......... 87% 25.2M 1s 2024-03-29T04:11:51.934 INFO:teuthology.orchestra.run.smithi148.stderr:147250K .......... .......... .......... .......... .......... 87% 28.6M 1s 2024-03-29T04:11:51.936 INFO:teuthology.orchestra.run.smithi148.stderr:147300K .......... .......... .......... .......... .......... 87% 29.4M 1s 2024-03-29T04:11:51.937 INFO:teuthology.orchestra.run.smithi148.stderr:147350K .......... .......... .......... .......... .......... 87% 29.9M 1s 2024-03-29T04:11:51.939 INFO:teuthology.orchestra.run.smithi148.stderr:147400K .......... .......... .......... .......... .......... 87% 24.4M 1s 2024-03-29T04:11:51.940 INFO:teuthology.orchestra.run.smithi148.stderr:147450K .......... .......... .......... .......... .......... 87% 33.2M 1s 2024-03-29T04:11:51.942 INFO:teuthology.orchestra.run.smithi148.stderr:147500K .......... .......... .......... .......... .......... 87% 37.7M 1s 2024-03-29T04:11:51.943 INFO:teuthology.orchestra.run.smithi148.stderr:147550K .......... .......... .......... .......... .......... 87% 40.5M 1s 2024-03-29T04:11:51.944 INFO:teuthology.orchestra.run.smithi148.stderr:147600K .......... .......... .......... .......... .......... 87% 34.8M 1s 2024-03-29T04:11:51.946 INFO:teuthology.orchestra.run.smithi148.stderr:147650K .......... .......... .......... .......... .......... 87% 32.8M 1s 2024-03-29T04:11:51.948 INFO:teuthology.orchestra.run.smithi148.stderr:147700K .......... .......... .......... .......... .......... 87% 31.3M 1s 2024-03-29T04:11:51.950 INFO:teuthology.orchestra.run.smithi148.stderr:147750K .......... .......... .......... .......... .......... 87% 25.6M 1s 2024-03-29T04:11:51.951 INFO:teuthology.orchestra.run.smithi148.stderr:147800K .......... .......... .......... .......... .......... 87% 29.1M 1s 2024-03-29T04:11:51.953 INFO:teuthology.orchestra.run.smithi148.stderr:147850K .......... .......... .......... .......... .......... 87% 35.3M 1s 2024-03-29T04:11:51.954 INFO:teuthology.orchestra.run.smithi148.stderr:147900K .......... .......... .......... .......... .......... 87% 34.9M 1s 2024-03-29T04:11:51.955 INFO:teuthology.orchestra.run.smithi148.stderr:147950K .......... .......... .......... .......... .......... 87% 40.4M 1s 2024-03-29T04:11:51.957 INFO:teuthology.orchestra.run.smithi148.stderr:148000K .......... .......... .......... .......... .......... 87% 28.9M 1s 2024-03-29T04:11:51.958 INFO:teuthology.orchestra.run.smithi148.stderr:148050K .......... .......... .......... .......... .......... 87% 33.8M 1s 2024-03-29T04:11:51.960 INFO:teuthology.orchestra.run.smithi148.stderr:148100K .......... .......... .......... .......... .......... 87% 39.0M 1s 2024-03-29T04:11:51.961 INFO:teuthology.orchestra.run.smithi148.stderr:148150K .......... .......... .......... .......... .......... 87% 38.4M 1s 2024-03-29T04:11:51.962 INFO:teuthology.orchestra.run.smithi148.stderr:148200K .......... .......... .......... .......... .......... 87% 33.5M 1s 2024-03-29T04:11:51.963 INFO:teuthology.orchestra.run.smithi148.stderr:148250K .......... .......... .......... .......... .......... 87% 38.6M 1s 2024-03-29T04:11:51.965 INFO:teuthology.orchestra.run.smithi148.stderr:148300K .......... .......... .......... .......... .......... 88% 38.4M 1s 2024-03-29T04:11:51.966 INFO:teuthology.orchestra.run.smithi148.stderr:148350K .......... .......... .......... .......... .......... 88% 38.4M 1s 2024-03-29T04:11:51.968 INFO:teuthology.orchestra.run.smithi148.stderr:148400K .......... .......... .......... .......... .......... 88% 33.5M 1s 2024-03-29T04:11:51.969 INFO:teuthology.orchestra.run.smithi148.stderr:148450K .......... .......... .......... .......... .......... 88% 38.3M 1s 2024-03-29T04:11:51.970 INFO:teuthology.orchestra.run.smithi148.stderr:148500K .......... .......... .......... .......... .......... 88% 38.9M 1s 2024-03-29T04:11:51.971 INFO:teuthology.orchestra.run.smithi148.stderr:148550K .......... .......... .......... .......... .......... 88% 36.9M 1s 2024-03-29T04:11:51.973 INFO:teuthology.orchestra.run.smithi148.stderr:148600K .......... .......... .......... .......... .......... 88% 32.3M 1s 2024-03-29T04:11:51.974 INFO:teuthology.orchestra.run.smithi148.stderr:148650K .......... .......... .......... .......... .......... 88% 37.2M 1s 2024-03-29T04:11:51.976 INFO:teuthology.orchestra.run.smithi148.stderr:148700K .......... .......... .......... .......... .......... 88% 37.6M 1s 2024-03-29T04:11:51.977 INFO:teuthology.orchestra.run.smithi148.stderr:148750K .......... .......... .......... .......... .......... 88% 37.1M 1s 2024-03-29T04:11:51.978 INFO:teuthology.orchestra.run.smithi148.stderr:148800K .......... .......... .......... .......... .......... 88% 31.2M 1s 2024-03-29T04:11:51.980 INFO:teuthology.orchestra.run.smithi148.stderr:148850K .......... .......... .......... .......... .......... 88% 35.4M 1s 2024-03-29T04:11:51.981 INFO:teuthology.orchestra.run.smithi148.stderr:148900K .......... .......... .......... .......... .......... 88% 35.3M 1s 2024-03-29T04:11:51.982 INFO:teuthology.orchestra.run.smithi148.stderr:148950K .......... .......... .......... .......... .......... 88% 36.1M 1s 2024-03-29T04:11:51.984 INFO:teuthology.orchestra.run.smithi148.stderr:149000K .......... .......... .......... .......... .......... 88% 31.1M 1s 2024-03-29T04:11:51.985 INFO:teuthology.orchestra.run.smithi148.stderr:149050K .......... .......... .......... .......... .......... 88% 35.3M 1s 2024-03-29T04:11:51.987 INFO:teuthology.orchestra.run.smithi148.stderr:149100K .......... .......... .......... .......... .......... 88% 36.4M 1s 2024-03-29T04:11:51.988 INFO:teuthology.orchestra.run.smithi148.stderr:149150K .......... .......... .......... .......... .......... 88% 35.8M 1s 2024-03-29T04:11:51.990 INFO:teuthology.orchestra.run.smithi148.stderr:149200K .......... .......... .......... .......... .......... 88% 30.4M 1s 2024-03-29T04:11:51.991 INFO:teuthology.orchestra.run.smithi148.stderr:149250K .......... .......... .......... .......... .......... 88% 36.7M 1s 2024-03-29T04:11:51.992 INFO:teuthology.orchestra.run.smithi148.stderr:149300K .......... .......... .......... .......... .......... 88% 35.2M 1s 2024-03-29T04:11:51.994 INFO:teuthology.orchestra.run.smithi148.stderr:149350K .......... .......... .......... .......... .......... 88% 36.5M 1s 2024-03-29T04:11:51.996 INFO:teuthology.orchestra.run.smithi148.stderr:149400K .......... .......... .......... .......... .......... 88% 27.1M 1s 2024-03-29T04:11:51.997 INFO:teuthology.orchestra.run.smithi148.stderr:149450K .......... .......... .......... .......... .......... 88% 33.1M 1s 2024-03-29T04:11:51.999 INFO:teuthology.orchestra.run.smithi148.stderr:149500K .......... .......... .......... .......... .......... 88% 32.8M 1s 2024-03-29T04:11:52.000 INFO:teuthology.orchestra.run.smithi148.stderr:149550K .......... .......... .......... .......... .......... 88% 33.0M 1s 2024-03-29T04:11:52.002 INFO:teuthology.orchestra.run.smithi148.stderr:149600K .......... .......... .......... .......... .......... 88% 27.7M 1s 2024-03-29T04:11:52.003 INFO:teuthology.orchestra.run.smithi148.stderr:149650K .......... .......... .......... .......... .......... 88% 33.0M 1s 2024-03-29T04:11:52.005 INFO:teuthology.orchestra.run.smithi148.stderr:149700K .......... .......... .......... .......... .......... 88% 32.4M 1s 2024-03-29T04:11:52.006 INFO:teuthology.orchestra.run.smithi148.stderr:149750K .......... .......... .......... .......... .......... 88% 32.5M 1s 2024-03-29T04:11:52.008 INFO:teuthology.orchestra.run.smithi148.stderr:149800K .......... .......... .......... .......... .......... 88% 27.7M 1s 2024-03-29T04:11:52.010 INFO:teuthology.orchestra.run.smithi148.stderr:149850K .......... .......... .......... .......... .......... 88% 32.1M 1s 2024-03-29T04:11:52.011 INFO:teuthology.orchestra.run.smithi148.stderr:149900K .......... .......... .......... .......... .......... 88% 32.8M 1s 2024-03-29T04:11:52.013 INFO:teuthology.orchestra.run.smithi148.stderr:149950K .......... .......... .......... .......... .......... 88% 32.1M 1s 2024-03-29T04:11:52.014 INFO:teuthology.orchestra.run.smithi148.stderr:150000K .......... .......... .......... .......... .......... 89% 28.0M 1s 2024-03-29T04:11:52.016 INFO:teuthology.orchestra.run.smithi148.stderr:150050K .......... .......... .......... .......... .......... 89% 32.3M 1s 2024-03-29T04:11:52.017 INFO:teuthology.orchestra.run.smithi148.stderr:150100K .......... .......... .......... .......... .......... 89% 32.1M 1s 2024-03-29T04:11:52.023 INFO:teuthology.orchestra.run.smithi148.stderr:150150K .......... .......... .......... .......... .......... 89% 29.6M 1s 2024-03-29T04:11:52.023 INFO:teuthology.orchestra.run.smithi148.stderr:150200K .......... .......... .......... .......... .......... 89% 26.2M 1s 2024-03-29T04:11:52.023 INFO:teuthology.orchestra.run.smithi148.stderr:150250K .......... .......... .......... .......... .......... 89% 30.1M 1s 2024-03-29T04:11:52.024 INFO:teuthology.orchestra.run.smithi148.stderr:150300K .......... .......... .......... .......... .......... 89% 30.0M 1s 2024-03-29T04:11:52.026 INFO:teuthology.orchestra.run.smithi148.stderr:150350K .......... .......... .......... .......... .......... 89% 29.9M 1s 2024-03-29T04:11:52.028 INFO:teuthology.orchestra.run.smithi148.stderr:150400K .......... .......... .......... .......... .......... 89% 25.2M 1s 2024-03-29T04:11:52.030 INFO:teuthology.orchestra.run.smithi148.stderr:150450K .......... .......... .......... .......... .......... 89% 29.3M 1s 2024-03-29T04:11:52.031 INFO:teuthology.orchestra.run.smithi148.stderr:150500K .......... .......... .......... .......... .......... 89% 28.0M 1s 2024-03-29T04:11:52.033 INFO:teuthology.orchestra.run.smithi148.stderr:150550K .......... .......... .......... .......... .......... 89% 29.2M 1s 2024-03-29T04:11:52.035 INFO:teuthology.orchestra.run.smithi148.stderr:150600K .......... .......... .......... .......... .......... 89% 24.1M 1s 2024-03-29T04:11:52.036 INFO:teuthology.orchestra.run.smithi148.stderr:150650K .......... .......... .......... .......... .......... 89% 28.9M 1s 2024-03-29T04:11:52.038 INFO:teuthology.orchestra.run.smithi148.stderr:150700K .......... .......... .......... .......... .......... 89% 28.9M 1s 2024-03-29T04:11:52.040 INFO:teuthology.orchestra.run.smithi148.stderr:150750K .......... .......... .......... .......... .......... 89% 28.7M 1s 2024-03-29T04:11:52.042 INFO:teuthology.orchestra.run.smithi148.stderr:150800K .......... .......... .......... .......... .......... 89% 25.1M 1s 2024-03-29T04:11:52.044 INFO:teuthology.orchestra.run.smithi148.stderr:150850K .......... .......... .......... .......... .......... 89% 28.7M 1s 2024-03-29T04:11:52.045 INFO:teuthology.orchestra.run.smithi148.stderr:150900K .......... .......... .......... .......... .......... 89% 28.9M 1s 2024-03-29T04:11:52.047 INFO:teuthology.orchestra.run.smithi148.stderr:150950K .......... .......... .......... .......... .......... 89% 29.1M 1s 2024-03-29T04:11:52.049 INFO:teuthology.orchestra.run.smithi148.stderr:151000K .......... .......... .......... .......... .......... 89% 24.0M 1s 2024-03-29T04:11:52.051 INFO:teuthology.orchestra.run.smithi148.stderr:151050K .......... .......... .......... .......... .......... 89% 28.3M 1s 2024-03-29T04:11:52.052 INFO:teuthology.orchestra.run.smithi148.stderr:151100K .......... .......... .......... .......... .......... 89% 28.2M 1s 2024-03-29T04:11:52.054 INFO:teuthology.orchestra.run.smithi148.stderr:151150K .......... .......... .......... .......... .......... 89% 28.8M 1s 2024-03-29T04:11:52.056 INFO:teuthology.orchestra.run.smithi148.stderr:151200K .......... .......... .......... .......... .......... 89% 24.1M 1s 2024-03-29T04:11:52.058 INFO:teuthology.orchestra.run.smithi148.stderr:151250K .......... .......... .......... .......... .......... 89% 28.4M 1s 2024-03-29T04:11:52.060 INFO:teuthology.orchestra.run.smithi148.stderr:151300K .......... .......... .......... .......... .......... 89% 27.1M 1s 2024-03-29T04:11:52.061 INFO:teuthology.orchestra.run.smithi148.stderr:151350K .......... .......... .......... .......... .......... 89% 27.1M 1s 2024-03-29T04:11:52.064 INFO:teuthology.orchestra.run.smithi148.stderr:151400K .......... .......... .......... .......... .......... 89% 22.9M 1s 2024-03-29T04:11:52.065 INFO:teuthology.orchestra.run.smithi148.stderr:151450K .......... .......... .......... .......... .......... 89% 26.5M 1s 2024-03-29T04:11:52.067 INFO:teuthology.orchestra.run.smithi148.stderr:151500K .......... .......... .......... .......... .......... 89% 28.8M 1s 2024-03-29T04:11:52.069 INFO:teuthology.orchestra.run.smithi148.stderr:151550K .......... .......... .......... .......... .......... 89% 28.4M 1s 2024-03-29T04:11:52.071 INFO:teuthology.orchestra.run.smithi148.stderr:151600K .......... .......... .......... .......... .......... 89% 24.8M 1s 2024-03-29T04:11:52.072 INFO:teuthology.orchestra.run.smithi148.stderr:151650K .......... .......... .......... .......... .......... 89% 30.1M 1s 2024-03-29T04:11:52.074 INFO:teuthology.orchestra.run.smithi148.stderr:151700K .......... .......... .......... .......... .......... 90% 32.6M 1s 2024-03-29T04:11:52.075 INFO:teuthology.orchestra.run.smithi148.stderr:151750K .......... .......... .......... .......... .......... 90% 35.6M 1s 2024-03-29T04:11:52.077 INFO:teuthology.orchestra.run.smithi148.stderr:151800K .......... .......... .......... .......... .......... 90% 35.4M 1s 2024-03-29T04:11:52.078 INFO:teuthology.orchestra.run.smithi148.stderr:151850K .......... .......... .......... .......... .......... 90% 34.0M 1s 2024-03-29T04:11:52.080 INFO:teuthology.orchestra.run.smithi148.stderr:151900K .......... .......... .......... .......... .......... 90% 28.5M 1s 2024-03-29T04:11:52.081 INFO:teuthology.orchestra.run.smithi148.stderr:151950K .......... .......... .......... .......... .......... 90% 31.9M 1s 2024-03-29T04:11:52.083 INFO:teuthology.orchestra.run.smithi148.stderr:152000K .......... .......... .......... .......... .......... 90% 31.9M 1s 2024-03-29T04:11:52.084 INFO:teuthology.orchestra.run.smithi148.stderr:152050K .......... .......... .......... .......... .......... 90% 37.3M 1s 2024-03-29T04:11:52.085 INFO:teuthology.orchestra.run.smithi148.stderr:152100K .......... .......... .......... .......... .......... 90% 37.3M 1s 2024-03-29T04:11:52.087 INFO:teuthology.orchestra.run.smithi148.stderr:152150K .......... .......... .......... .......... .......... 90% 36.2M 1s 2024-03-29T04:11:52.089 INFO:teuthology.orchestra.run.smithi148.stderr:152200K .......... .......... .......... .......... .......... 90% 29.5M 1s 2024-03-29T04:11:52.090 INFO:teuthology.orchestra.run.smithi148.stderr:152250K .......... .......... .......... .......... .......... 90% 30.7M 1s 2024-03-29T04:11:52.091 INFO:teuthology.orchestra.run.smithi148.stderr:152300K .......... .......... .......... .......... .......... 90% 32.7M 1s 2024-03-29T04:11:52.093 INFO:teuthology.orchestra.run.smithi148.stderr:152350K .......... .......... .......... .......... .......... 90% 36.1M 1s 2024-03-29T04:11:52.095 INFO:teuthology.orchestra.run.smithi148.stderr:152400K .......... .......... .......... .......... .......... 90% 28.0M 1s 2024-03-29T04:11:52.096 INFO:teuthology.orchestra.run.smithi148.stderr:152450K .......... .......... .......... .......... .......... 90% 38.1M 1s 2024-03-29T04:11:52.097 INFO:teuthology.orchestra.run.smithi148.stderr:152500K .......... .......... .......... .......... .......... 90% 37.5M 1s 2024-03-29T04:11:52.099 INFO:teuthology.orchestra.run.smithi148.stderr:152550K .......... .......... .......... .......... .......... 90% 37.9M 1s 2024-03-29T04:11:52.100 INFO:teuthology.orchestra.run.smithi148.stderr:152600K .......... .......... .......... .......... .......... 90% 32.6M 1s 2024-03-29T04:11:52.101 INFO:teuthology.orchestra.run.smithi148.stderr:152650K .......... .......... .......... .......... .......... 90% 38.2M 1s 2024-03-29T04:11:52.103 INFO:teuthology.orchestra.run.smithi148.stderr:152700K .......... .......... .......... .......... .......... 90% 38.1M 1s 2024-03-29T04:11:52.104 INFO:teuthology.orchestra.run.smithi148.stderr:152750K .......... .......... .......... .......... .......... 90% 38.5M 1s 2024-03-29T04:11:52.105 INFO:teuthology.orchestra.run.smithi148.stderr:152800K .......... .......... .......... .......... .......... 90% 33.3M 1s 2024-03-29T04:11:52.107 INFO:teuthology.orchestra.run.smithi148.stderr:152850K .......... .......... .......... .......... .......... 90% 37.3M 1s 2024-03-29T04:11:52.108 INFO:teuthology.orchestra.run.smithi148.stderr:152900K .......... .......... .......... .......... .......... 90% 37.2M 1s 2024-03-29T04:11:52.109 INFO:teuthology.orchestra.run.smithi148.stderr:152950K .......... .......... .......... .......... .......... 90% 37.8M 1s 2024-03-29T04:11:52.111 INFO:teuthology.orchestra.run.smithi148.stderr:153000K .......... .......... .......... .......... .......... 90% 31.8M 1s 2024-03-29T04:11:52.112 INFO:teuthology.orchestra.run.smithi148.stderr:153050K .......... .......... .......... .......... .......... 90% 36.2M 1s 2024-03-29T04:11:52.113 INFO:teuthology.orchestra.run.smithi148.stderr:153100K .......... .......... .......... .......... .......... 90% 36.4M 1s 2024-03-29T04:11:52.115 INFO:teuthology.orchestra.run.smithi148.stderr:153150K .......... .......... .......... .......... .......... 90% 34.4M 1s 2024-03-29T04:11:52.117 INFO:teuthology.orchestra.run.smithi148.stderr:153200K .......... .......... .......... .......... .......... 90% 28.4M 1s 2024-03-29T04:11:52.118 INFO:teuthology.orchestra.run.smithi148.stderr:153250K .......... .......... .......... .......... .......... 90% 31.6M 1s 2024-03-29T04:11:52.120 INFO:teuthology.orchestra.run.smithi148.stderr:153300K .......... .......... .......... .......... .......... 90% 31.0M 1s 2024-03-29T04:11:52.121 INFO:teuthology.orchestra.run.smithi148.stderr:153350K .......... .......... .......... .......... .......... 91% 30.5M 1s 2024-03-29T04:11:52.123 INFO:teuthology.orchestra.run.smithi148.stderr:153400K .......... .......... .......... .......... .......... 91% 26.2M 1s 2024-03-29T04:11:52.125 INFO:teuthology.orchestra.run.smithi148.stderr:153450K .......... .......... .......... .......... .......... 91% 30.5M 1s 2024-03-29T04:11:52.126 INFO:teuthology.orchestra.run.smithi148.stderr:153500K .......... .......... .......... .......... .......... 91% 30.6M 1s 2024-03-29T04:11:52.128 INFO:teuthology.orchestra.run.smithi148.stderr:153550K .......... .......... .......... .......... .......... 91% 29.9M 1s 2024-03-29T04:11:52.130 INFO:teuthology.orchestra.run.smithi148.stderr:153600K .......... .......... .......... .......... .......... 91% 25.6M 1s 2024-03-29T04:11:52.132 INFO:teuthology.orchestra.run.smithi148.stderr:153650K .......... .......... .......... .......... .......... 91% 29.3M 1s 2024-03-29T04:11:52.133 INFO:teuthology.orchestra.run.smithi148.stderr:153700K .......... .......... .......... .......... .......... 91% 28.7M 1s 2024-03-29T04:11:52.135 INFO:teuthology.orchestra.run.smithi148.stderr:153750K .......... .......... .......... .......... .......... 91% 29.6M 1s 2024-03-29T04:11:52.137 INFO:teuthology.orchestra.run.smithi148.stderr:153800K .......... .......... .......... .......... .......... 91% 25.5M 1s 2024-03-29T04:11:52.139 INFO:teuthology.orchestra.run.smithi148.stderr:153850K .......... .......... .......... .......... .......... 91% 28.4M 1s 2024-03-29T04:11:52.140 INFO:teuthology.orchestra.run.smithi148.stderr:153900K .......... .......... .......... .......... .......... 91% 29.5M 1s 2024-03-29T04:11:52.142 INFO:teuthology.orchestra.run.smithi148.stderr:153950K .......... .......... .......... .......... .......... 91% 29.3M 1s 2024-03-29T04:11:52.144 INFO:teuthology.orchestra.run.smithi148.stderr:154000K .......... .......... .......... .......... .......... 91% 25.5M 1s 2024-03-29T04:11:52.146 INFO:teuthology.orchestra.run.smithi148.stderr:154050K .......... .......... .......... .......... .......... 91% 29.7M 1s 2024-03-29T04:11:52.147 INFO:teuthology.orchestra.run.smithi148.stderr:154100K .......... .......... .......... .......... .......... 91% 26.4M 1s 2024-03-29T04:11:52.149 INFO:teuthology.orchestra.run.smithi148.stderr:154150K .......... .......... .......... .......... .......... 91% 28.5M 1s 2024-03-29T04:11:52.151 INFO:teuthology.orchestra.run.smithi148.stderr:154200K .......... .......... .......... .......... .......... 91% 24.8M 1s 2024-03-29T04:11:52.153 INFO:teuthology.orchestra.run.smithi148.stderr:154250K .......... .......... .......... .......... .......... 91% 27.4M 1s 2024-03-29T04:11:52.154 INFO:teuthology.orchestra.run.smithi148.stderr:154300K .......... .......... .......... .......... .......... 91% 28.3M 1s 2024-03-29T04:11:52.156 INFO:teuthology.orchestra.run.smithi148.stderr:154350K .......... .......... .......... .......... .......... 91% 27.8M 0s 2024-03-29T04:11:52.158 INFO:teuthology.orchestra.run.smithi148.stderr:154400K .......... .......... .......... .......... .......... 91% 24.0M 0s 2024-03-29T04:11:52.160 INFO:teuthology.orchestra.run.smithi148.stderr:154450K .......... .......... .......... .......... .......... 91% 27.7M 0s 2024-03-29T04:11:52.162 INFO:teuthology.orchestra.run.smithi148.stderr:154500K .......... .......... .......... .......... .......... 91% 28.0M 0s 2024-03-29T04:11:52.164 INFO:teuthology.orchestra.run.smithi148.stderr:154550K .......... .......... .......... .......... .......... 91% 28.0M 0s 2024-03-29T04:11:52.166 INFO:teuthology.orchestra.run.smithi148.stderr:154600K .......... .......... .......... .......... .......... 91% 23.8M 0s 2024-03-29T04:11:52.167 INFO:teuthology.orchestra.run.smithi148.stderr:154650K .......... .......... .......... .......... .......... 91% 27.1M 0s 2024-03-29T04:11:52.169 INFO:teuthology.orchestra.run.smithi148.stderr:154700K .......... .......... .......... .......... .......... 91% 27.6M 0s 2024-03-29T04:11:52.171 INFO:teuthology.orchestra.run.smithi148.stderr:154750K .......... .......... .......... .......... .......... 91% 27.2M 0s 2024-03-29T04:11:52.173 INFO:teuthology.orchestra.run.smithi148.stderr:154800K .......... .......... .......... .......... .......... 91% 23.4M 0s 2024-03-29T04:11:52.175 INFO:teuthology.orchestra.run.smithi148.stderr:154850K .......... .......... .......... .......... .......... 91% 27.5M 0s 2024-03-29T04:11:52.177 INFO:teuthology.orchestra.run.smithi148.stderr:154900K .......... .......... .......... .......... .......... 91% 27.2M 0s 2024-03-29T04:11:52.178 INFO:teuthology.orchestra.run.smithi148.stderr:154950K .......... .......... .......... .......... .......... 91% 26.9M 0s 2024-03-29T04:11:52.181 INFO:teuthology.orchestra.run.smithi148.stderr:155000K .......... .......... .......... .......... .......... 91% 23.0M 0s 2024-03-29T04:11:52.183 INFO:teuthology.orchestra.run.smithi148.stderr:155050K .......... .......... .......... .......... .......... 92% 25.9M 0s 2024-03-29T04:11:52.184 INFO:teuthology.orchestra.run.smithi148.stderr:155100K .......... .......... .......... .......... .......... 92% 25.9M 0s 2024-03-29T04:11:52.186 INFO:teuthology.orchestra.run.smithi148.stderr:155150K .......... .......... .......... .......... .......... 92% 26.4M 0s 2024-03-29T04:11:52.188 INFO:teuthology.orchestra.run.smithi148.stderr:155200K .......... .......... .......... .......... .......... 92% 22.0M 0s 2024-03-29T04:11:52.190 INFO:teuthology.orchestra.run.smithi148.stderr:155250K .......... .......... .......... .......... .......... 92% 25.0M 0s 2024-03-29T04:11:52.192 INFO:teuthology.orchestra.run.smithi148.stderr:155300K .......... .......... .......... .......... .......... 92% 25.5M 0s 2024-03-29T04:11:52.194 INFO:teuthology.orchestra.run.smithi148.stderr:155350K .......... .......... .......... .......... .......... 92% 24.8M 0s 2024-03-29T04:11:52.197 INFO:teuthology.orchestra.run.smithi148.stderr:155400K .......... .......... .......... .......... .......... 92% 20.7M 0s 2024-03-29T04:11:52.198 INFO:teuthology.orchestra.run.smithi148.stderr:155450K .......... .......... .......... .......... .......... 92% 27.2M 0s 2024-03-29T04:11:52.200 INFO:teuthology.orchestra.run.smithi148.stderr:155500K .......... .......... .......... .......... .......... 92% 29.5M 0s 2024-03-29T04:11:52.202 INFO:teuthology.orchestra.run.smithi148.stderr:155550K .......... .......... .......... .......... .......... 92% 29.6M 0s 2024-03-29T04:11:52.204 INFO:teuthology.orchestra.run.smithi148.stderr:155600K .......... .......... .......... .......... .......... 92% 25.2M 0s 2024-03-29T04:11:52.206 INFO:teuthology.orchestra.run.smithi148.stderr:155650K .......... .......... .......... .......... .......... 92% 24.3M 0s 2024-03-29T04:11:52.208 INFO:teuthology.orchestra.run.smithi148.stderr:155700K .......... .......... .......... .......... .......... 92% 28.4M 0s 2024-03-29T04:11:52.209 INFO:teuthology.orchestra.run.smithi148.stderr:155750K .......... .......... .......... .......... .......... 92% 27.3M 0s 2024-03-29T04:11:52.211 INFO:teuthology.orchestra.run.smithi148.stderr:155800K .......... .......... .......... .......... .......... 92% 24.8M 0s 2024-03-29T04:11:52.213 INFO:teuthology.orchestra.run.smithi148.stderr:155850K .......... .......... .......... .......... .......... 92% 27.6M 0s 2024-03-29T04:11:52.215 INFO:teuthology.orchestra.run.smithi148.stderr:155900K .......... .......... .......... .......... .......... 92% 27.8M 0s 2024-03-29T04:11:52.216 INFO:teuthology.orchestra.run.smithi148.stderr:155950K .......... .......... .......... .......... .......... 92% 30.0M 0s 2024-03-29T04:11:52.218 INFO:teuthology.orchestra.run.smithi148.stderr:156000K .......... .......... .......... .......... .......... 92% 23.6M 0s 2024-03-29T04:11:52.220 INFO:teuthology.orchestra.run.smithi148.stderr:156050K .......... .......... .......... .......... .......... 92% 33.5M 0s 2024-03-29T04:11:52.221 INFO:teuthology.orchestra.run.smithi148.stderr:156100K .......... .......... .......... .......... .......... 92% 32.3M 0s 2024-03-29T04:11:52.228 INFO:teuthology.orchestra.run.smithi148.stderr:156150K .......... .......... .......... .......... .......... 92% 32.9M 0s 2024-03-29T04:11:52.228 INFO:teuthology.orchestra.run.smithi148.stderr:156200K .......... .......... .......... .......... .......... 92% 29.2M 0s 2024-03-29T04:11:52.228 INFO:teuthology.orchestra.run.smithi148.stderr:156250K .......... .......... .......... .......... .......... 92% 33.7M 0s 2024-03-29T04:11:52.228 INFO:teuthology.orchestra.run.smithi148.stderr:156300K .......... .......... .......... .......... .......... 92% 32.6M 0s 2024-03-29T04:11:52.229 INFO:teuthology.orchestra.run.smithi148.stderr:156350K .......... .......... .......... .......... .......... 92% 33.3M 0s 2024-03-29T04:11:52.230 INFO:teuthology.orchestra.run.smithi148.stderr:156400K .......... .......... .......... .......... .......... 92% 29.7M 0s 2024-03-29T04:11:52.232 INFO:teuthology.orchestra.run.smithi148.stderr:156450K .......... .......... .......... .......... .......... 92% 31.8M 0s 2024-03-29T04:11:52.234 INFO:teuthology.orchestra.run.smithi148.stderr:156500K .......... .......... .......... .......... .......... 92% 33.3M 0s 2024-03-29T04:11:52.235 INFO:teuthology.orchestra.run.smithi148.stderr:156550K .......... .......... .......... .......... .......... 92% 32.2M 0s 2024-03-29T04:11:52.237 INFO:teuthology.orchestra.run.smithi148.stderr:156600K .......... .......... .......... .......... .......... 92% 26.7M 0s 2024-03-29T04:11:52.238 INFO:teuthology.orchestra.run.smithi148.stderr:156650K .......... .......... .......... .......... .......... 92% 31.0M 0s 2024-03-29T04:11:52.240 INFO:teuthology.orchestra.run.smithi148.stderr:156700K .......... .......... .......... .......... .......... 92% 34.3M 0s 2024-03-29T04:11:52.241 INFO:teuthology.orchestra.run.smithi148.stderr:156750K .......... .......... .......... .......... .......... 93% 28.7M 0s 2024-03-29T04:11:52.243 INFO:teuthology.orchestra.run.smithi148.stderr:156800K .......... .......... .......... .......... .......... 93% 24.0M 0s 2024-03-29T04:11:52.245 INFO:teuthology.orchestra.run.smithi148.stderr:156850K .......... .......... .......... .......... .......... 93% 27.2M 0s 2024-03-29T04:11:52.247 INFO:teuthology.orchestra.run.smithi148.stderr:156900K .......... .......... .......... .......... .......... 93% 28.4M 0s 2024-03-29T04:11:52.249 INFO:teuthology.orchestra.run.smithi148.stderr:156950K .......... .......... .......... .......... .......... 93% 28.2M 0s 2024-03-29T04:11:52.251 INFO:teuthology.orchestra.run.smithi148.stderr:157000K .......... .......... .......... .......... .......... 93% 24.4M 0s 2024-03-29T04:11:52.253 INFO:teuthology.orchestra.run.smithi148.stderr:157050K .......... .......... .......... .......... .......... 93% 27.1M 0s 2024-03-29T04:11:52.254 INFO:teuthology.orchestra.run.smithi148.stderr:157100K .......... .......... .......... .......... .......... 93% 27.6M 0s 2024-03-29T04:11:52.256 INFO:teuthology.orchestra.run.smithi148.stderr:157150K .......... .......... .......... .......... .......... 93% 27.8M 0s 2024-03-29T04:11:52.258 INFO:teuthology.orchestra.run.smithi148.stderr:157200K .......... .......... .......... .......... .......... 93% 23.1M 0s 2024-03-29T04:11:52.260 INFO:teuthology.orchestra.run.smithi148.stderr:157250K .......... .......... .......... .......... .......... 93% 27.4M 0s 2024-03-29T04:11:52.262 INFO:teuthology.orchestra.run.smithi148.stderr:157300K .......... .......... .......... .......... .......... 93% 27.4M 0s 2024-03-29T04:11:52.264 INFO:teuthology.orchestra.run.smithi148.stderr:157350K .......... .......... .......... .......... .......... 93% 26.3M 0s 2024-03-29T04:11:52.266 INFO:teuthology.orchestra.run.smithi148.stderr:157400K .......... .......... .......... .......... .......... 93% 23.1M 0s 2024-03-29T04:11:52.268 INFO:teuthology.orchestra.run.smithi148.stderr:157450K .......... .......... .......... .......... .......... 93% 26.9M 0s 2024-03-29T04:11:52.269 INFO:teuthology.orchestra.run.smithi148.stderr:157500K .......... .......... .......... .......... .......... 93% 26.6M 0s 2024-03-29T04:11:52.271 INFO:teuthology.orchestra.run.smithi148.stderr:157550K .......... .......... .......... .......... .......... 93% 26.5M 0s 2024-03-29T04:11:52.274 INFO:teuthology.orchestra.run.smithi148.stderr:157600K .......... .......... .......... .......... .......... 93% 22.9M 0s 2024-03-29T04:11:52.275 INFO:teuthology.orchestra.run.smithi148.stderr:157650K .......... .......... .......... .......... .......... 93% 26.5M 0s 2024-03-29T04:11:52.277 INFO:teuthology.orchestra.run.smithi148.stderr:157700K .......... .......... .......... .......... .......... 93% 26.3M 0s 2024-03-29T04:11:52.279 INFO:teuthology.orchestra.run.smithi148.stderr:157750K .......... .......... .......... .......... .......... 93% 25.9M 0s 2024-03-29T04:11:52.281 INFO:teuthology.orchestra.run.smithi148.stderr:157800K .......... .......... .......... .......... .......... 93% 22.1M 0s 2024-03-29T04:11:52.283 INFO:teuthology.orchestra.run.smithi148.stderr:157850K .......... .......... .......... .......... .......... 93% 26.2M 0s 2024-03-29T04:11:52.285 INFO:teuthology.orchestra.run.smithi148.stderr:157900K .......... .......... .......... .......... .......... 93% 25.4M 0s 2024-03-29T04:11:52.287 INFO:teuthology.orchestra.run.smithi148.stderr:157950K .......... .......... .......... .......... .......... 93% 25.2M 0s 2024-03-29T04:11:52.289 INFO:teuthology.orchestra.run.smithi148.stderr:158000K .......... .......... .......... .......... .......... 93% 21.6M 0s 2024-03-29T04:11:52.291 INFO:teuthology.orchestra.run.smithi148.stderr:158050K .......... .......... .......... .......... .......... 93% 25.8M 0s 2024-03-29T04:11:52.293 INFO:teuthology.orchestra.run.smithi148.stderr:158100K .......... .......... .......... .......... .......... 93% 25.5M 0s 2024-03-29T04:11:52.295 INFO:teuthology.orchestra.run.smithi148.stderr:158150K .......... .......... .......... .......... .......... 93% 25.3M 0s 2024-03-29T04:11:52.297 INFO:teuthology.orchestra.run.smithi148.stderr:158200K .......... .......... .......... .......... .......... 93% 21.6M 0s 2024-03-29T04:11:52.299 INFO:teuthology.orchestra.run.smithi148.stderr:158250K .......... .......... .......... .......... .......... 93% 24.9M 0s 2024-03-29T04:11:52.301 INFO:teuthology.orchestra.run.smithi148.stderr:158300K .......... .......... .......... .......... .......... 93% 23.9M 0s 2024-03-29T04:11:52.303 INFO:teuthology.orchestra.run.smithi148.stderr:158350K .......... .......... .......... .......... .......... 93% 24.9M 0s 2024-03-29T04:11:52.305 INFO:teuthology.orchestra.run.smithi148.stderr:158400K .......... .......... .......... .......... .......... 93% 21.0M 0s 2024-03-29T04:11:52.307 INFO:teuthology.orchestra.run.smithi148.stderr:158450K .......... .......... .......... .......... .......... 94% 25.0M 0s 2024-03-29T04:11:52.309 INFO:teuthology.orchestra.run.smithi148.stderr:158500K .......... .......... .......... .......... .......... 94% 24.3M 0s 2024-03-29T04:11:52.311 INFO:teuthology.orchestra.run.smithi148.stderr:158550K .......... .......... .......... .......... .......... 94% 25.1M 0s 2024-03-29T04:11:52.314 INFO:teuthology.orchestra.run.smithi148.stderr:158600K .......... .......... .......... .......... .......... 94% 20.5M 0s 2024-03-29T04:11:52.316 INFO:teuthology.orchestra.run.smithi148.stderr:158650K .......... .......... .......... .......... .......... 94% 24.2M 0s 2024-03-29T04:11:52.318 INFO:teuthology.orchestra.run.smithi148.stderr:158700K .......... .......... .......... .......... .......... 94% 24.4M 0s 2024-03-29T04:11:52.320 INFO:teuthology.orchestra.run.smithi148.stderr:158750K .......... .......... .......... .......... .......... 94% 24.1M 0s 2024-03-29T04:11:52.322 INFO:teuthology.orchestra.run.smithi148.stderr:158800K .......... .......... .......... .......... .......... 94% 20.5M 0s 2024-03-29T04:11:52.324 INFO:teuthology.orchestra.run.smithi148.stderr:158850K .......... .......... .......... .......... .......... 94% 23.9M 0s 2024-03-29T04:11:52.326 INFO:teuthology.orchestra.run.smithi148.stderr:158900K .......... .......... .......... .......... .......... 94% 23.3M 0s 2024-03-29T04:11:52.328 INFO:teuthology.orchestra.run.smithi148.stderr:158950K .......... .......... .......... .......... .......... 94% 25.0M 0s 2024-03-29T04:11:52.331 INFO:teuthology.orchestra.run.smithi148.stderr:159000K .......... .......... .......... .......... .......... 94% 22.2M 0s 2024-03-29T04:11:52.333 INFO:teuthology.orchestra.run.smithi148.stderr:159050K .......... .......... .......... .......... .......... 94% 23.2M 0s 2024-03-29T04:11:52.335 INFO:teuthology.orchestra.run.smithi148.stderr:159100K .......... .......... .......... .......... .......... 94% 23.5M 0s 2024-03-29T04:11:52.337 INFO:teuthology.orchestra.run.smithi148.stderr:159150K .......... .......... .......... .......... .......... 94% 23.6M 0s 2024-03-29T04:11:52.339 INFO:teuthology.orchestra.run.smithi148.stderr:159200K .......... .......... .......... .......... .......... 94% 19.1M 0s 2024-03-29T04:11:52.341 INFO:teuthology.orchestra.run.smithi148.stderr:159250K .......... .......... .......... .......... .......... 94% 25.2M 0s 2024-03-29T04:11:52.343 INFO:teuthology.orchestra.run.smithi148.stderr:159300K .......... .......... .......... .......... .......... 94% 24.6M 0s 2024-03-29T04:11:52.345 INFO:teuthology.orchestra.run.smithi148.stderr:159350K .......... .......... .......... .......... .......... 94% 25.1M 0s 2024-03-29T04:11:52.348 INFO:teuthology.orchestra.run.smithi148.stderr:159400K .......... .......... .......... .......... .......... 94% 21.5M 0s 2024-03-29T04:11:52.350 INFO:teuthology.orchestra.run.smithi148.stderr:159450K .......... .......... .......... .......... .......... 94% 23.9M 0s 2024-03-29T04:11:52.352 INFO:teuthology.orchestra.run.smithi148.stderr:159500K .......... .......... .......... .......... .......... 94% 23.9M 0s 2024-03-29T04:11:52.354 INFO:teuthology.orchestra.run.smithi148.stderr:159550K .......... .......... .......... .......... .......... 94% 24.2M 0s 2024-03-29T04:11:52.356 INFO:teuthology.orchestra.run.smithi148.stderr:159600K .......... .......... .......... .......... .......... 94% 20.4M 0s 2024-03-29T04:11:52.358 INFO:teuthology.orchestra.run.smithi148.stderr:159650K .......... .......... .......... .......... .......... 94% 23.2M 0s 2024-03-29T04:11:52.360 INFO:teuthology.orchestra.run.smithi148.stderr:159700K .......... .......... .......... .......... .......... 94% 24.1M 0s 2024-03-29T04:11:52.362 INFO:teuthology.orchestra.run.smithi148.stderr:159750K .......... .......... .......... .......... .......... 94% 26.1M 0s 2024-03-29T04:11:52.364 INFO:teuthology.orchestra.run.smithi148.stderr:159800K .......... .......... .......... .......... .......... 94% 27.5M 0s 2024-03-29T04:11:52.365 INFO:teuthology.orchestra.run.smithi148.stderr:159850K .......... .......... .......... .......... .......... 94% 33.1M 0s 2024-03-29T04:11:52.367 INFO:teuthology.orchestra.run.smithi148.stderr:159900K .......... .......... .......... .......... .......... 94% 32.7M 0s 2024-03-29T04:11:52.368 INFO:teuthology.orchestra.run.smithi148.stderr:159950K .......... .......... .......... .......... .......... 94% 32.7M 0s 2024-03-29T04:11:52.370 INFO:teuthology.orchestra.run.smithi148.stderr:160000K .......... .......... .......... .......... .......... 94% 27.2M 0s 2024-03-29T04:11:52.371 INFO:teuthology.orchestra.run.smithi148.stderr:160050K .......... .......... .......... .......... .......... 94% 32.6M 0s 2024-03-29T04:11:52.373 INFO:teuthology.orchestra.run.smithi148.stderr:160100K .......... .......... .......... .......... .......... 95% 30.1M 0s 2024-03-29T04:11:52.375 INFO:teuthology.orchestra.run.smithi148.stderr:160150K .......... .......... .......... .......... .......... 95% 26.1M 0s 2024-03-29T04:11:52.377 INFO:teuthology.orchestra.run.smithi148.stderr:160200K .......... .......... .......... .......... .......... 95% 25.5M 0s 2024-03-29T04:11:52.379 INFO:teuthology.orchestra.run.smithi148.stderr:160250K .......... .......... .......... .......... .......... 95% 31.9M 0s 2024-03-29T04:11:52.380 INFO:teuthology.orchestra.run.smithi148.stderr:160300K .......... .......... .......... .......... .......... 95% 32.0M 0s 2024-03-29T04:11:52.382 INFO:teuthology.orchestra.run.smithi148.stderr:160350K .......... .......... .......... .......... .......... 95% 30.3M 0s 2024-03-29T04:11:52.383 INFO:teuthology.orchestra.run.smithi148.stderr:160400K .......... .......... .......... .......... .......... 95% 27.3M 0s 2024-03-29T04:11:52.385 INFO:teuthology.orchestra.run.smithi148.stderr:160450K .......... .......... .......... .......... .......... 95% 32.1M 0s 2024-03-29T04:11:52.387 INFO:teuthology.orchestra.run.smithi148.stderr:160500K .......... .......... .......... .......... .......... 95% 31.6M 0s 2024-03-29T04:11:52.388 INFO:teuthology.orchestra.run.smithi148.stderr:160550K .......... .......... .......... .......... .......... 95% 31.5M 0s 2024-03-29T04:11:52.390 INFO:teuthology.orchestra.run.smithi148.stderr:160600K .......... .......... .......... .......... .......... 95% 26.6M 0s 2024-03-29T04:11:52.391 INFO:teuthology.orchestra.run.smithi148.stderr:160650K .......... .......... .......... .......... .......... 95% 31.8M 0s 2024-03-29T04:11:52.393 INFO:teuthology.orchestra.run.smithi148.stderr:160700K .......... .......... .......... .......... .......... 95% 31.7M 0s 2024-03-29T04:11:52.394 INFO:teuthology.orchestra.run.smithi148.stderr:160750K .......... .......... .......... .......... .......... 95% 31.6M 0s 2024-03-29T04:11:52.396 INFO:teuthology.orchestra.run.smithi148.stderr:160800K .......... .......... .......... .......... .......... 95% 25.9M 0s 2024-03-29T04:11:52.398 INFO:teuthology.orchestra.run.smithi148.stderr:160850K .......... .......... .......... .......... .......... 95% 29.4M 0s 2024-03-29T04:11:52.400 INFO:teuthology.orchestra.run.smithi148.stderr:160900K .......... .......... .......... .......... .......... 95% 28.2M 0s 2024-03-29T04:11:52.402 INFO:teuthology.orchestra.run.smithi148.stderr:160950K .......... .......... .......... .......... .......... 95% 29.2M 0s 2024-03-29T04:11:52.403 INFO:teuthology.orchestra.run.smithi148.stderr:161000K .......... .......... .......... .......... .......... 95% 24.5M 0s 2024-03-29T04:11:52.405 INFO:teuthology.orchestra.run.smithi148.stderr:161050K .......... .......... .......... .......... .......... 95% 28.7M 0s 2024-03-29T04:11:52.407 INFO:teuthology.orchestra.run.smithi148.stderr:161100K .......... .......... .......... .......... .......... 95% 28.0M 0s 2024-03-29T04:11:52.409 INFO:teuthology.orchestra.run.smithi148.stderr:161150K .......... .......... .......... .......... .......... 95% 28.3M 0s 2024-03-29T04:11:52.411 INFO:teuthology.orchestra.run.smithi148.stderr:161200K .......... .......... .......... .......... .......... 95% 24.3M 0s 2024-03-29T04:11:52.413 INFO:teuthology.orchestra.run.smithi148.stderr:161250K .......... .......... .......... .......... .......... 95% 27.0M 0s 2024-03-29T04:11:52.414 INFO:teuthology.orchestra.run.smithi148.stderr:161300K .......... .......... .......... .......... .......... 95% 27.3M 0s 2024-03-29T04:11:52.416 INFO:teuthology.orchestra.run.smithi148.stderr:161350K .......... .......... .......... .......... .......... 95% 27.0M 0s 2024-03-29T04:11:52.418 INFO:teuthology.orchestra.run.smithi148.stderr:161400K .......... .......... .......... .......... .......... 95% 23.8M 0s 2024-03-29T04:11:52.420 INFO:teuthology.orchestra.run.smithi148.stderr:161450K .......... .......... .......... .......... .......... 95% 27.8M 0s 2024-03-29T04:11:52.422 INFO:teuthology.orchestra.run.smithi148.stderr:161500K .......... .......... .......... .......... .......... 95% 26.7M 0s 2024-03-29T04:11:52.423 INFO:teuthology.orchestra.run.smithi148.stderr:161550K .......... .......... .......... .......... .......... 95% 27.3M 0s 2024-03-29T04:11:52.426 INFO:teuthology.orchestra.run.smithi148.stderr:161600K .......... .......... .......... .......... .......... 95% 22.8M 0s 2024-03-29T04:11:52.431 INFO:teuthology.orchestra.run.smithi148.stderr:161650K .......... .......... .......... .......... .......... 95% 27.6M 0s 2024-03-29T04:11:52.431 INFO:teuthology.orchestra.run.smithi148.stderr:161700K .......... .......... .......... .......... .......... 95% 26.6M 0s 2024-03-29T04:11:52.431 INFO:teuthology.orchestra.run.smithi148.stderr:161750K .......... .......... .......... .......... .......... 95% 26.8M 0s 2024-03-29T04:11:52.433 INFO:teuthology.orchestra.run.smithi148.stderr:161800K .......... .......... .......... .......... .......... 96% 22.1M 0s 2024-03-29T04:11:52.435 INFO:teuthology.orchestra.run.smithi148.stderr:161850K .......... .......... .......... .......... .......... 96% 26.5M 0s 2024-03-29T04:11:52.437 INFO:teuthology.orchestra.run.smithi148.stderr:161900K .......... .......... .......... .......... .......... 96% 26.7M 0s 2024-03-29T04:11:52.439 INFO:teuthology.orchestra.run.smithi148.stderr:161950K .......... .......... .......... .......... .......... 96% 26.1M 0s 2024-03-29T04:11:52.441 INFO:teuthology.orchestra.run.smithi148.stderr:162000K .......... .......... .......... .......... .......... 96% 22.4M 0s 2024-03-29T04:11:52.443 INFO:teuthology.orchestra.run.smithi148.stderr:162050K .......... .......... .......... .......... .......... 96% 26.1M 0s 2024-03-29T04:11:52.445 INFO:teuthology.orchestra.run.smithi148.stderr:162100K .......... .......... .......... .......... .......... 96% 26.2M 0s 2024-03-29T04:11:52.447 INFO:teuthology.orchestra.run.smithi148.stderr:162150K .......... .......... .......... .......... .......... 96% 25.9M 0s 2024-03-29T04:11:52.449 INFO:teuthology.orchestra.run.smithi148.stderr:162200K .......... .......... .......... .......... .......... 96% 21.9M 0s 2024-03-29T04:11:52.451 INFO:teuthology.orchestra.run.smithi148.stderr:162250K .......... .......... .......... .......... .......... 96% 25.5M 0s 2024-03-29T04:11:52.452 INFO:teuthology.orchestra.run.smithi148.stderr:162300K .......... .......... .......... .......... .......... 96% 25.8M 0s 2024-03-29T04:11:52.455 INFO:teuthology.orchestra.run.smithi148.stderr:162350K .......... .......... .......... .......... .......... 96% 25.4M 0s 2024-03-29T04:11:52.457 INFO:teuthology.orchestra.run.smithi148.stderr:162400K .......... .......... .......... .......... .......... 96% 22.0M 0s 2024-03-29T04:11:52.459 INFO:teuthology.orchestra.run.smithi148.stderr:162450K .......... .......... .......... .......... .......... 96% 24.9M 0s 2024-03-29T04:11:52.461 INFO:teuthology.orchestra.run.smithi148.stderr:162500K .......... .......... .......... .......... .......... 96% 24.9M 0s 2024-03-29T04:11:52.463 INFO:teuthology.orchestra.run.smithi148.stderr:162550K .......... .......... .......... .......... .......... 96% 24.6M 0s 2024-03-29T04:11:52.465 INFO:teuthology.orchestra.run.smithi148.stderr:162600K .......... .......... .......... .......... .......... 96% 21.0M 0s 2024-03-29T04:11:52.467 INFO:teuthology.orchestra.run.smithi148.stderr:162650K .......... .......... .......... .......... .......... 96% 24.8M 0s 2024-03-29T04:11:52.469 INFO:teuthology.orchestra.run.smithi148.stderr:162700K .......... .......... .......... .......... .......... 96% 24.6M 0s 2024-03-29T04:11:52.471 INFO:teuthology.orchestra.run.smithi148.stderr:162750K .......... .......... .......... .......... .......... 96% 24.6M 0s 2024-03-29T04:11:52.473 INFO:teuthology.orchestra.run.smithi148.stderr:162800K .......... .......... .......... .......... .......... 96% 21.8M 0s 2024-03-29T04:11:52.475 INFO:teuthology.orchestra.run.smithi148.stderr:162850K .......... .......... .......... .......... .......... 96% 23.5M 0s 2024-03-29T04:11:52.477 INFO:teuthology.orchestra.run.smithi148.stderr:162900K .......... .......... .......... .......... .......... 96% 22.2M 0s 2024-03-29T04:11:52.480 INFO:teuthology.orchestra.run.smithi148.stderr:162950K .......... .......... .......... .......... .......... 96% 23.1M 0s 2024-03-29T04:11:52.482 INFO:teuthology.orchestra.run.smithi148.stderr:163000K .......... .......... .......... .......... .......... 96% 19.3M 0s 2024-03-29T04:11:52.484 INFO:teuthology.orchestra.run.smithi148.stderr:163050K .......... .......... .......... .......... .......... 96% 26.7M 0s 2024-03-29T04:11:52.486 INFO:teuthology.orchestra.run.smithi148.stderr:163100K .......... .......... .......... .......... .......... 96% 27.2M 0s 2024-03-29T04:11:52.487 INFO:teuthology.orchestra.run.smithi148.stderr:163150K .......... .......... .......... .......... .......... 96% 27.4M 0s 2024-03-29T04:11:52.490 INFO:teuthology.orchestra.run.smithi148.stderr:163200K .......... .......... .......... .......... .......... 96% 23.5M 0s 2024-03-29T04:11:52.491 INFO:teuthology.orchestra.run.smithi148.stderr:163250K .......... .......... .......... .......... .......... 96% 26.8M 0s 2024-03-29T04:11:52.493 INFO:teuthology.orchestra.run.smithi148.stderr:163300K .......... .......... .......... .......... .......... 96% 27.7M 0s 2024-03-29T04:11:52.495 INFO:teuthology.orchestra.run.smithi148.stderr:163350K .......... .......... .......... .......... .......... 96% 27.7M 0s 2024-03-29T04:11:52.497 INFO:teuthology.orchestra.run.smithi148.stderr:163400K .......... .......... .......... .......... .......... 96% 23.3M 0s 2024-03-29T04:11:52.499 INFO:teuthology.orchestra.run.smithi148.stderr:163450K .......... .......... .......... .......... .......... 96% 27.6M 0s 2024-03-29T04:11:52.501 INFO:teuthology.orchestra.run.smithi148.stderr:163500K .......... .......... .......... .......... .......... 97% 26.7M 0s 2024-03-29T04:11:52.502 INFO:teuthology.orchestra.run.smithi148.stderr:163550K .......... .......... .......... .......... .......... 97% 27.2M 0s 2024-03-29T04:11:52.504 INFO:teuthology.orchestra.run.smithi148.stderr:163600K .......... .......... .......... .......... .......... 97% 23.2M 0s 2024-03-29T04:11:52.506 INFO:teuthology.orchestra.run.smithi148.stderr:163650K .......... .......... .......... .......... .......... 97% 26.8M 0s 2024-03-29T04:11:52.508 INFO:teuthology.orchestra.run.smithi148.stderr:163700K .......... .......... .......... .......... .......... 97% 24.0M 0s 2024-03-29T04:11:52.510 INFO:teuthology.orchestra.run.smithi148.stderr:163750K .......... .......... .......... .......... .......... 97% 24.4M 0s 2024-03-29T04:11:52.513 INFO:teuthology.orchestra.run.smithi148.stderr:163800K .......... .......... .......... .......... .......... 97% 19.8M 0s 2024-03-29T04:11:52.514 INFO:teuthology.orchestra.run.smithi148.stderr:163850K .......... .......... .......... .......... .......... 97% 41.8M 0s 2024-03-29T04:11:52.515 INFO:teuthology.orchestra.run.smithi148.stderr:163900K .......... .......... .......... .......... .......... 97% 42.7M 0s 2024-03-29T04:11:52.517 INFO:teuthology.orchestra.run.smithi148.stderr:163950K .......... .......... .......... .......... .......... 97% 32.9M 0s 2024-03-29T04:11:52.518 INFO:teuthology.orchestra.run.smithi148.stderr:164000K .......... .......... .......... .......... .......... 97% 29.0M 0s 2024-03-29T04:11:52.520 INFO:teuthology.orchestra.run.smithi148.stderr:164050K .......... .......... .......... .......... .......... 97% 31.7M 0s 2024-03-29T04:11:52.521 INFO:teuthology.orchestra.run.smithi148.stderr:164100K .......... .......... .......... .......... .......... 97% 30.6M 0s 2024-03-29T04:11:52.523 INFO:teuthology.orchestra.run.smithi148.stderr:164150K .......... .......... .......... .......... .......... 97% 30.3M 0s 2024-03-29T04:11:52.525 INFO:teuthology.orchestra.run.smithi148.stderr:164200K .......... .......... .......... .......... .......... 97% 25.7M 0s 2024-03-29T04:11:52.526 INFO:teuthology.orchestra.run.smithi148.stderr:164250K .......... .......... .......... .......... .......... 97% 35.3M 0s 2024-03-29T04:11:52.528 INFO:teuthology.orchestra.run.smithi148.stderr:164300K .......... .......... .......... .......... .......... 97% 36.1M 0s 2024-03-29T04:11:52.529 INFO:teuthology.orchestra.run.smithi148.stderr:164350K .......... .......... .......... .......... .......... 97% 31.6M 0s 2024-03-29T04:11:52.531 INFO:teuthology.orchestra.run.smithi148.stderr:164400K .......... .......... .......... .......... .......... 97% 32.3M 0s 2024-03-29T04:11:52.532 INFO:teuthology.orchestra.run.smithi148.stderr:164450K .......... .......... .......... .......... .......... 97% 33.6M 0s 2024-03-29T04:11:52.534 INFO:teuthology.orchestra.run.smithi148.stderr:164500K .......... .......... .......... .......... .......... 97% 32.4M 0s 2024-03-29T04:11:52.535 INFO:teuthology.orchestra.run.smithi148.stderr:164550K .......... .......... .......... .......... .......... 97% 37.6M 0s 2024-03-29T04:11:52.536 INFO:teuthology.orchestra.run.smithi148.stderr:164600K .......... .......... .......... .......... .......... 97% 32.1M 0s 2024-03-29T04:11:52.538 INFO:teuthology.orchestra.run.smithi148.stderr:164650K .......... .......... .......... .......... .......... 97% 37.4M 0s 2024-03-29T04:11:52.539 INFO:teuthology.orchestra.run.smithi148.stderr:164700K .......... .......... .......... .......... .......... 97% 38.2M 0s 2024-03-29T04:11:52.540 INFO:teuthology.orchestra.run.smithi148.stderr:164750K .......... .......... .......... .......... .......... 97% 37.7M 0s 2024-03-29T04:11:52.542 INFO:teuthology.orchestra.run.smithi148.stderr:164800K .......... .......... .......... .......... .......... 97% 32.4M 0s 2024-03-29T04:11:52.543 INFO:teuthology.orchestra.run.smithi148.stderr:164850K .......... .......... .......... .......... .......... 97% 36.5M 0s 2024-03-29T04:11:52.545 INFO:teuthology.orchestra.run.smithi148.stderr:164900K .......... .......... .......... .......... .......... 97% 37.0M 0s 2024-03-29T04:11:52.546 INFO:teuthology.orchestra.run.smithi148.stderr:164950K .......... .......... .......... .......... .......... 97% 37.0M 0s 2024-03-29T04:11:52.547 INFO:teuthology.orchestra.run.smithi148.stderr:165000K .......... .......... .......... .......... .......... 97% 31.1M 0s 2024-03-29T04:11:52.549 INFO:teuthology.orchestra.run.smithi148.stderr:165050K .......... .......... .......... .......... .......... 97% 35.8M 0s 2024-03-29T04:11:52.550 INFO:teuthology.orchestra.run.smithi148.stderr:165100K .......... .......... .......... .......... .......... 97% 35.9M 0s 2024-03-29T04:11:52.552 INFO:teuthology.orchestra.run.smithi148.stderr:165150K .......... .......... .......... .......... .......... 98% 36.0M 0s 2024-03-29T04:11:52.553 INFO:teuthology.orchestra.run.smithi148.stderr:165200K .......... .......... .......... .......... .......... 98% 30.5M 0s 2024-03-29T04:11:52.555 INFO:teuthology.orchestra.run.smithi148.stderr:165250K .......... .......... .......... .......... .......... 98% 34.7M 0s 2024-03-29T04:11:52.556 INFO:teuthology.orchestra.run.smithi148.stderr:165300K .......... .......... .......... .......... .......... 98% 34.6M 0s 2024-03-29T04:11:52.557 INFO:teuthology.orchestra.run.smithi148.stderr:165350K .......... .......... .......... .......... .......... 98% 34.3M 0s 2024-03-29T04:11:52.559 INFO:teuthology.orchestra.run.smithi148.stderr:165400K .......... .......... .......... .......... .......... 98% 30.0M 0s 2024-03-29T04:11:52.560 INFO:teuthology.orchestra.run.smithi148.stderr:165450K .......... .......... .......... .......... .......... 98% 34.2M 0s 2024-03-29T04:11:52.562 INFO:teuthology.orchestra.run.smithi148.stderr:165500K .......... .......... .......... .......... .......... 98% 33.6M 0s 2024-03-29T04:11:52.563 INFO:teuthology.orchestra.run.smithi148.stderr:165550K .......... .......... .......... .......... .......... 98% 34.1M 0s 2024-03-29T04:11:52.565 INFO:teuthology.orchestra.run.smithi148.stderr:165600K .......... .......... .......... .......... .......... 98% 29.2M 0s 2024-03-29T04:11:52.566 INFO:teuthology.orchestra.run.smithi148.stderr:165650K .......... .......... .......... .......... .......... 98% 33.6M 0s 2024-03-29T04:11:52.568 INFO:teuthology.orchestra.run.smithi148.stderr:165700K .......... .......... .......... .......... .......... 98% 33.8M 0s 2024-03-29T04:11:52.569 INFO:teuthology.orchestra.run.smithi148.stderr:165750K .......... .......... .......... .......... .......... 98% 33.4M 0s 2024-03-29T04:11:52.571 INFO:teuthology.orchestra.run.smithi148.stderr:165800K .......... .......... .......... .......... .......... 98% 28.0M 0s 2024-03-29T04:11:52.573 INFO:teuthology.orchestra.run.smithi148.stderr:165850K .......... .......... .......... .......... .......... 98% 28.0M 0s 2024-03-29T04:11:52.575 INFO:teuthology.orchestra.run.smithi148.stderr:165900K .......... .......... .......... .......... .......... 98% 24.8M 0s 2024-03-29T04:11:52.577 INFO:teuthology.orchestra.run.smithi148.stderr:165950K .......... .......... .......... .......... .......... 98% 25.1M 0s 2024-03-29T04:11:52.579 INFO:teuthology.orchestra.run.smithi148.stderr:166000K .......... .......... .......... .......... .......... 98% 21.6M 0s 2024-03-29T04:11:52.581 INFO:teuthology.orchestra.run.smithi148.stderr:166050K .......... .......... .......... .......... .......... 98% 25.3M 0s 2024-03-29T04:11:52.583 INFO:teuthology.orchestra.run.smithi148.stderr:166100K .......... .......... .......... .......... .......... 98% 24.0M 0s 2024-03-29T04:11:52.585 INFO:teuthology.orchestra.run.smithi148.stderr:166150K .......... .......... .......... .......... .......... 98% 25.1M 0s 2024-03-29T04:11:52.588 INFO:teuthology.orchestra.run.smithi148.stderr:166200K .......... .......... .......... .......... .......... 98% 21.1M 0s 2024-03-29T04:11:52.589 INFO:teuthology.orchestra.run.smithi148.stderr:166250K .......... .......... .......... .......... .......... 98% 23.4M 0s 2024-03-29T04:11:52.591 INFO:teuthology.orchestra.run.smithi148.stderr:166300K .......... .......... .......... .......... .......... 98% 24.5M 0s 2024-03-29T04:11:52.594 INFO:teuthology.orchestra.run.smithi148.stderr:166350K .......... .......... .......... .......... .......... 98% 24.8M 0s 2024-03-29T04:11:52.596 INFO:teuthology.orchestra.run.smithi148.stderr:166400K .......... .......... .......... .......... .......... 98% 20.5M 0s 2024-03-29T04:11:52.598 INFO:teuthology.orchestra.run.smithi148.stderr:166450K .......... .......... .......... .......... .......... 98% 24.3M 0s 2024-03-29T04:11:52.600 INFO:teuthology.orchestra.run.smithi148.stderr:166500K .......... .......... .......... .......... .......... 98% 24.0M 0s 2024-03-29T04:11:52.602 INFO:teuthology.orchestra.run.smithi148.stderr:166550K .......... .......... .......... .......... .......... 98% 24.3M 0s 2024-03-29T04:11:52.604 INFO:teuthology.orchestra.run.smithi148.stderr:166600K .......... .......... .......... .......... .......... 98% 20.7M 0s 2024-03-29T04:11:52.606 INFO:teuthology.orchestra.run.smithi148.stderr:166650K .......... .......... .......... .......... .......... 98% 24.2M 0s 2024-03-29T04:11:52.608 INFO:teuthology.orchestra.run.smithi148.stderr:166700K .......... .......... .......... .......... .......... 98% 24.1M 0s 2024-03-29T04:11:52.610 INFO:teuthology.orchestra.run.smithi148.stderr:166750K .......... .......... .......... .......... .......... 98% 23.9M 0s 2024-03-29T04:11:52.613 INFO:teuthology.orchestra.run.smithi148.stderr:166800K .......... .......... .......... .......... .......... 98% 19.9M 0s 2024-03-29T04:11:52.615 INFO:teuthology.orchestra.run.smithi148.stderr:166850K .......... .......... .......... .......... .......... 99% 25.9M 0s 2024-03-29T04:11:52.616 INFO:teuthology.orchestra.run.smithi148.stderr:166900K .......... .......... .......... .......... .......... 99% 26.2M 0s 2024-03-29T04:11:52.618 INFO:teuthology.orchestra.run.smithi148.stderr:166950K .......... .......... .......... .......... .......... 99% 25.8M 0s 2024-03-29T04:11:52.621 INFO:teuthology.orchestra.run.smithi148.stderr:167000K .......... .......... .......... .......... .......... 99% 22.6M 0s 2024-03-29T04:11:52.622 INFO:teuthology.orchestra.run.smithi148.stderr:167050K .......... .......... .......... .......... .......... 99% 26.2M 0s 2024-03-29T04:11:52.624 INFO:teuthology.orchestra.run.smithi148.stderr:167100K .......... .......... .......... .......... .......... 99% 26.3M 0s 2024-03-29T04:11:52.626 INFO:teuthology.orchestra.run.smithi148.stderr:167150K .......... .......... .......... .......... .......... 99% 26.2M 0s 2024-03-29T04:11:52.628 INFO:teuthology.orchestra.run.smithi148.stderr:167200K .......... .......... .......... .......... .......... 99% 22.5M 0s 2024-03-29T04:11:52.630 INFO:teuthology.orchestra.run.smithi148.stderr:167250K .......... .......... .......... .......... .......... 99% 25.9M 0s 2024-03-29T04:11:52.632 INFO:teuthology.orchestra.run.smithi148.stderr:167300K .......... .......... .......... .......... .......... 99% 25.4M 0s 2024-03-29T04:11:52.634 INFO:teuthology.orchestra.run.smithi148.stderr:167350K .......... .......... .......... .......... .......... 99% 23.5M 0s 2024-03-29T04:11:52.636 INFO:teuthology.orchestra.run.smithi148.stderr:167400K .......... .......... .......... .......... .......... 99% 23.8M 0s 2024-03-29T04:11:52.638 INFO:teuthology.orchestra.run.smithi148.stderr:167450K .......... .......... .......... .......... .......... 99% 28.2M 0s 2024-03-29T04:11:52.640 INFO:teuthology.orchestra.run.smithi148.stderr:167500K .......... .......... .......... .......... .......... 99% 28.2M 0s 2024-03-29T04:11:52.642 INFO:teuthology.orchestra.run.smithi148.stderr:167550K .......... .......... .......... .......... .......... 99% 23.0M 0s 2024-03-29T04:11:52.644 INFO:teuthology.orchestra.run.smithi148.stderr:167600K .......... .......... .......... .......... .......... 99% 20.6M 0s 2024-03-29T04:11:52.646 INFO:teuthology.orchestra.run.smithi148.stderr:167650K .......... .......... .......... .......... .......... 99% 24.3M 0s 2024-03-29T04:11:52.648 INFO:teuthology.orchestra.run.smithi148.stderr:167700K .......... .......... .......... .......... .......... 99% 23.4M 0s 2024-03-29T04:11:52.650 INFO:teuthology.orchestra.run.smithi148.stderr:167750K .......... .......... .......... .......... .......... 99% 25.0M 0s 2024-03-29T04:11:52.652 INFO:teuthology.orchestra.run.smithi148.stderr:167800K .......... .......... .......... .......... .......... 99% 23.2M 0s 2024-03-29T04:11:52.654 INFO:teuthology.orchestra.run.smithi148.stderr:167850K .......... .......... .......... .......... .......... 99% 27.2M 0s 2024-03-29T04:11:52.657 INFO:teuthology.orchestra.run.smithi148.stderr:167900K .......... .......... .......... .......... .......... 99% 23.3M 0s 2024-03-29T04:11:52.658 INFO:teuthology.orchestra.run.smithi148.stderr:167950K .......... .......... .......... .......... .......... 99% 37.6M 0s 2024-03-29T04:11:52.659 INFO:teuthology.orchestra.run.smithi148.stderr:168000K .......... .......... .......... .......... .......... 99% 32.6M 0s 2024-03-29T04:11:52.660 INFO:teuthology.orchestra.run.smithi148.stderr:168050K .......... .......... .......... .......... .......... 99% 37.5M 0s 2024-03-29T04:11:52.662 INFO:teuthology.orchestra.run.smithi148.stderr:168100K .......... .......... .......... .......... .......... 99% 38.1M 0s 2024-03-29T04:11:52.663 INFO:teuthology.orchestra.run.smithi148.stderr:168150K .......... .......... .......... .......... .......... 99% 32.2M 0s 2024-03-29T04:11:52.665 INFO:teuthology.orchestra.run.smithi148.stderr:168200K .......... .......... .......... .......... .......... 99% 27.0M 0s 2024-03-29T04:11:52.666 INFO:teuthology.orchestra.run.smithi148.stderr:168250K .......... .......... .......... .......... .......... 99% 29.4M 0s 2024-03-29T04:11:52.668 INFO:teuthology.orchestra.run.smithi148.stderr:168300K .......... .......... .......... .......... .......... 99% 28.8M 0s 2024-03-29T04:11:52.670 INFO:teuthology.orchestra.run.smithi148.stderr:168350K .......... .......... .......... .......... .......... 99% 28.9M 0s 2024-03-29T04:11:52.672 INFO:teuthology.orchestra.run.smithi148.stderr:168400K .......... .......... .......... .......... .......... 99% 24.2M 0s 2024-03-29T04:11:52.674 INFO:teuthology.orchestra.run.smithi148.stderr:168450K .......... .......... .......... .......... .......... 99% 26.6M 0s 2024-03-29T04:11:52.675 INFO:teuthology.orchestra.run.smithi148.stderr:168500K .......... .......... .......... .......... .......... 99% 31.6M 0s 2024-03-29T04:11:52.676 INFO:teuthology.orchestra.run.smithi148.stderr:168550K .......... .......... . 100% 27.4M=6.0s 2024-03-29T04:11:52.676 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-03-29T04:11:52.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29 04:11:52 (27.7 MB/s) - ‘linux-5.4.tar.gz’ saved [172616875/172616875] 2024-03-29T04:11:52.677 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-03-29T04:11:52.678 INFO:tasks.cephfs.test_uninlining:Running Data Uninlining test with: without clearing cache, without unmount client, without mount.open_background, without multimds, without snapshots 2024-03-29T04:11:52.678 DEBUG:tasks.cephfs.test_uninlining:setting max_mds:1 2024-03-29T04:11:52.678 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 1 2024-03-29T04:11:52.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.843+0000 7f9c0b688640 1 Processor -- start 2024-03-29T04:11:52.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.843+0000 7f9c0b688640 1 -- start start 2024-03-29T04:11:52.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.844+0000 7f9c0b688640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9c040b2f00 0x7f9c04162680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:52.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.844+0000 7f9c0b688640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9c040b3430 0x7f9c0406b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:52.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.844+0000 7f9c0b688640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c040cf330 con 0x7f9c0415e3c0 2024-03-29T04:11:52.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.844+0000 7f9c0b688640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f9c040cf960 con 0x7f9c040b2f00 2024-03-29T04:11:52.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.844+0000 7f9c0b688640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9c04162cf0 con 0x7f9c040b3430 2024-03-29T04:11:52.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.844+0000 7f9c08c23640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9c040b2f00 0x7f9c04162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:52.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.844+0000 7f9c08c23640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9c040b2f00 0x7f9c04162680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39092/0 (socket says 172.21.15.148:39092) 2024-03-29T04:11:52.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.844+0000 7f9c08c23640 1 -- 172.21.15.148:0/1619989208 learned_addr learned my addr 172.21.15.148:0/1619989208 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:52.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.844+0000 7f9c09c25640 1 --2- 172.21.15.148:0/1619989208 >> v2:172.21.15.157:3300/0 conn(0x7f9c040b3430 0x7f9c0406b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:52.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.844+0000 7f9c08c23640 1 -- 172.21.15.148:0/1619989208 >> v2:172.21.15.157:3300/0 conn(0x7f9c040b3430 msgr2=0x7f9c0406b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:52.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.844+0000 7f9c08c23640 1 --2- 172.21.15.148:0/1619989208 >> v2:172.21.15.157:3300/0 conn(0x7f9c040b3430 0x7f9c0406b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:52.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.844+0000 7f9c08c23640 1 -- 172.21.15.148:0/1619989208 >> v1:172.21.15.157:6789/0 conn(0x7f9c0415e3c0 legacy=0x7f9c04161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:52.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.844+0000 7f9c08c23640 1 -- 172.21.15.148:0/1619989208 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c040cf5f0 con 0x7f9c040b2f00 2024-03-29T04:11:52.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.845+0000 7f9c08c23640 1 --2- 172.21.15.148:0/1619989208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c040b2f00 0x7f9c04162680 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9c040cf8d0 tx=0x7f9c0002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9f3b431f5c14b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:52.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.845+0000 7f9bfbfff640 1 -- 172.21.15.148:0/1619989208 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0002e070 con 0x7f9c040b2f00 2024-03-29T04:11:52.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.845+0000 7f9bfbfff640 1 -- 172.21.15.148:0/1619989208 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c00037070 con 0x7f9c040b2f00 2024-03-29T04:11:52.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.845+0000 7f9bfbfff640 1 -- 172.21.15.148:0/1619989208 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00031a00 con 0x7f9c040b2f00 2024-03-29T04:11:52.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.846+0000 7f9c0b688640 1 -- 172.21.15.148:0/1619989208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c040b2f00 msgr2=0x7f9c04162680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:52.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.846+0000 7f9c0b688640 1 --2- 172.21.15.148:0/1619989208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c040b2f00 0x7f9c04162680 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9c040cf8d0 tx=0x7f9c0002b880 comp rx=0 tx=0).stop 2024-03-29T04:11:52.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.846+0000 7f9c0b688640 1 -- 172.21.15.148:0/1619989208 shutdown_connections 2024-03-29T04:11:52.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.846+0000 7f9c0b688640 1 --2- 172.21.15.148:0/1619989208 >> v2:172.21.15.157:3300/0 conn(0x7f9c040b3430 0x7f9c0406b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:52.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.846+0000 7f9c0b688640 1 --2- 172.21.15.148:0/1619989208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c040b2f00 0x7f9c04162680 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:52.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.846+0000 7f9c0b688640 1 -- 172.21.15.148:0/1619989208 >> 172.21.15.148:0/1619989208 conn(0x7f9c040b6430 msgr2=0x7f9c040d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:52.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.846+0000 7f9c0b688640 1 -- 172.21.15.148:0/1619989208 shutdown_connections 2024-03-29T04:11:52.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.846+0000 7f9c0b688640 1 -- 172.21.15.148:0/1619989208 wait complete. 2024-03-29T04:11:52.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.847+0000 7f9c0b688640 1 Processor -- start 2024-03-29T04:11:52.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.847+0000 7f9c0b688640 1 -- start start 2024-03-29T04:11:52.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.848+0000 7f9c0b688640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c040b3430 0x7f9c041f3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:52.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.848+0000 7f9c0b688640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c0415e3c0 0x7f9c0415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:52.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.848+0000 7f9c0b688640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c040ca850 0x7f9c0415ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:52.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.848+0000 7f9c0b688640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9c040b3b20 con 0x7f9c040b3430 2024-03-29T04:11:52.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.848+0000 7f9c0b688640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9c040b39a0 con 0x7f9c0415e3c0 2024-03-29T04:11:52.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.848+0000 7f9c0b688640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9c0406bd30 con 0x7f9c040ca850 2024-03-29T04:11:52.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.848+0000 7f9c09424640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c040b3430 0x7f9c041f3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:52.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.848+0000 7f9c09424640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c040b3430 0x7f9c041f3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39108/0 (socket says 172.21.15.148:39108) 2024-03-29T04:11:52.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.848+0000 7f9c09424640 1 -- 172.21.15.148:0/2425017706 learned_addr learned my addr 172.21.15.148:0/2425017706 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:52.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.848+0000 7f9c09424640 1 -- 172.21.15.148:0/2425017706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c040ca850 msgr2=0x7f9c0415ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:52.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.848+0000 7f9c09424640 1 --2- 172.21.15.148:0/2425017706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c040ca850 0x7f9c0415ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:52.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.848+0000 7f9c09424640 1 -- 172.21.15.148:0/2425017706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c0415e3c0 msgr2=0x7f9c0415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:52.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.848+0000 7f9c09424640 1 --2- 172.21.15.148:0/2425017706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c0415e3c0 0x7f9c0415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:52.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.848+0000 7f9c09424640 1 -- 172.21.15.148:0/2425017706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0415ec20 con 0x7f9c040b3430 2024-03-29T04:11:52.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.849+0000 7f9c09424640 1 --2- 172.21.15.148:0/2425017706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c040b3430 0x7f9c041f3e70 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9bf400b3f0 tx=0x7f9bf400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:52.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.849+0000 7f9bf9ffb640 1 -- 172.21.15.148:0/2425017706 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4015020 con 0x7f9c040b3430 2024-03-29T04:11:52.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.849+0000 7f9bf9ffb640 1 -- 172.21.15.148:0/2425017706 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bf4011830 con 0x7f9c040b3430 2024-03-29T04:11:52.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.849+0000 7f9bf9ffb640 1 -- 172.21.15.148:0/2425017706 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4011c20 con 0x7f9c040b3430 2024-03-29T04:11:52.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.849+0000 7f9c0b688640 1 -- 172.21.15.148:0/2425017706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c041f4450 con 0x7f9c040b3430 2024-03-29T04:11:52.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.849+0000 7f9c0b688640 1 -- 172.21.15.148:0/2425017706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c041f4a10 con 0x7f9c040b3430 2024-03-29T04:11:52.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.850+0000 7f9c0b688640 1 -- 172.21.15.148:0/2425017706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c04068850 con 0x7f9c040b3430 2024-03-29T04:11:52.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.851+0000 7f9bf9ffb640 1 -- 172.21.15.148:0/2425017706 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9bf4018020 con 0x7f9c040b3430 2024-03-29T04:11:52.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.851+0000 7f9bf9ffb640 1 --2- 172.21.15.148:0/2425017706 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9be806c530 0x7f9be806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:52.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.851+0000 7f9bf9ffb640 1 -- 172.21.15.148:0/2425017706 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f9bf408b420 con 0x7f9c040b3430 2024-03-29T04:11:52.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.851+0000 7f9c08c23640 1 --2- 172.21.15.148:0/2425017706 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9be806c530 0x7f9be806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:52.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.852+0000 7f9c08c23640 1 --2- 172.21.15.148:0/2425017706 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9be806c530 0x7f9be806e9f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9c00036000 tx=0x7f9c00030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:52.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:52.856+0000 7f9bf9ffb640 1 -- 172.21.15.148:0/2425017706 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9bf401f030 con 0x7f9c040b3430 2024-03-29T04:11:53.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:53.075+0000 7f9c0b688640 1 -- 172.21.15.148:0/2425017706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) -- 0x7f9c040743c0 con 0x7f9c040b3430 2024-03-29T04:11:53.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:53.654+0000 7f9bf9ffb640 1 -- 172.21.15.148:0/2425017706 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f9bf40172c0 con 0x7f9c040b3430 2024-03-29T04:11:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:53.657+0000 7f9c0b688640 1 -- 172.21.15.148:0/2425017706 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9be806c530 msgr2=0x7f9be806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:53.657+0000 7f9c0b688640 1 --2- 172.21.15.148:0/2425017706 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9be806c530 0x7f9be806e9f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9c00036000 tx=0x7f9c00030000 comp rx=0 tx=0).stop 2024-03-29T04:11:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:53.657+0000 7f9c0b688640 1 -- 172.21.15.148:0/2425017706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c040b3430 msgr2=0x7f9c041f3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:53.657+0000 7f9c0b688640 1 --2- 172.21.15.148:0/2425017706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c040b3430 0x7f9c041f3e70 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9bf400b3f0 tx=0x7f9bf400b8b0 comp rx=0 tx=0).stop 2024-03-29T04:11:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:53.657+0000 7f9c0b688640 1 -- 172.21.15.148:0/2425017706 shutdown_connections 2024-03-29T04:11:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:53.657+0000 7f9c0b688640 1 --2- 172.21.15.148:0/2425017706 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9be806c530 0x7f9be806e9f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:53.657+0000 7f9c0b688640 1 --2- 172.21.15.148:0/2425017706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c040ca850 0x7f9c0415ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:53.657+0000 7f9c0b688640 1 --2- 172.21.15.148:0/2425017706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c0415e3c0 0x7f9c0415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:53.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:53.657+0000 7f9c0b688640 1 --2- 172.21.15.148:0/2425017706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c040b3430 0x7f9c041f3e70 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:53.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:53.657+0000 7f9c0b688640 1 -- 172.21.15.148:0/2425017706 >> 172.21.15.148:0/2425017706 conn(0x7f9c040b6430 msgr2=0x7f9c040b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:53.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:53.657+0000 7f9c0b688640 1 -- 172.21.15.148:0/2425017706 shutdown_connections 2024-03-29T04:11:53.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:53.657+0000 7f9c0b688640 1 -- 172.21.15.148:0/2425017706 wait complete. 2024-03-29T04:11:53.647 DEBUG:tasks.cephfs.test_uninlining:getting client_max_inline_size 2024-03-29T04:11:53.647 DEBUG:tasks.cephfs.test_uninlining:got client_max_inline_size:4096 2024-03-29T04:11:53.647 DEBUG:tasks.cephfs.test_uninlining:extracting tarball 2024-03-29T04:11:53.647 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/mnt.0 && tar -x -z -f linux-5.4.tar.gz linux-5.4/fs/ceph linux-5.4/fs/orangefs linux-5.4/fs/ext2 2024-03-29T04:11:59.027 DEBUG:tasks.cephfs.test_uninlining:fetching dir list 2024-03-29T04:11:59.027 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/mnt.0 && find linux-5.4/ -mindepth 2 -maxdepth 2 -type d 2024-03-29T04:11:59.065 INFO:teuthology.orchestra.run.smithi148.stdout:linux-5.4/fs/ceph 2024-03-29T04:11:59.065 INFO:teuthology.orchestra.run.smithi148.stdout:linux-5.4/fs/ext2 2024-03-29T04:11:59.065 INFO:teuthology.orchestra.run.smithi148.stdout:linux-5.4/fs/orangefs 2024-03-29T04:11:59.066 DEBUG:tasks.cephfs.test_uninlining:distributing dir pins 2024-03-29T04:11:59.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:59.066 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/ceph) 2024-03-29T04:11:59.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:59.123 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/ext2) 2024-03-29T04:11:59.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:59.180 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/orangefs) 2024-03-29T04:11:59.238 DEBUG:tasks.cephfs.test_uninlining:listing files with size <= 4096 2024-03-29T04:11:59.238 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/mnt.0 && find linux-5.4/ -type f -size -4097c 2024-03-29T04:11:59.326 INFO:teuthology.orchestra.run.smithi148.stdout:linux-5.4/fs/ceph/Kconfig 2024-03-29T04:11:59.326 INFO:teuthology.orchestra.run.smithi148.stdout:linux-5.4/fs/ceph/Makefile 2024-03-29T04:11:59.326 INFO:teuthology.orchestra.run.smithi148.stdout:linux-5.4/fs/ceph/ceph_frag.c 2024-03-29T04:11:59.326 INFO:teuthology.orchestra.run.smithi148.stdout:linux-5.4/fs/ceph/io.h 2024-03-29T04:11:59.326 INFO:teuthology.orchestra.run.smithi148.stdout:linux-5.4/fs/ceph/ioctl.h 2024-03-29T04:11:59.326 INFO:teuthology.orchestra.run.smithi148.stdout:linux-5.4/fs/ext2/Kconfig 2024-03-29T04:11:59.326 INFO:teuthology.orchestra.run.smithi148.stdout:linux-5.4/fs/ext2/xattr_user.c 2024-03-29T04:11:59.326 INFO:teuthology.orchestra.run.smithi148.stdout:linux-5.4/fs/ext2/xattr_security.c 2024-03-29T04:11:59.326 INFO:teuthology.orchestra.run.smithi148.stdout:linux-5.4/fs/ext2/Makefile 2024-03-29T04:11:59.326 INFO:teuthology.orchestra.run.smithi148.stdout:linux-5.4/fs/ext2/symlink.c 2024-03-29T04:11:59.326 INFO:teuthology.orchestra.run.smithi148.stdout:linux-5.4/fs/ext2/acl.h 2024-03-29T04:11:59.327 INFO:teuthology.orchestra.run.smithi148.stdout:linux-5.4/fs/ext2/xattr_trusted.c 2024-03-29T04:11:59.327 INFO:teuthology.orchestra.run.smithi148.stdout:linux-5.4/fs/ext2/xattr.h 2024-03-29T04:11:59.327 INFO:teuthology.orchestra.run.smithi148.stdout:linux-5.4/fs/orangefs/Kconfig 2024-03-29T04:11:59.327 INFO:teuthology.orchestra.run.smithi148.stdout:linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-29T04:11:59.327 INFO:teuthology.orchestra.run.smithi148.stdout:linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-29T04:11:59.327 INFO:teuthology.orchestra.run.smithi148.stdout:linux-5.4/fs/orangefs/dcache.c 2024-03-29T04:11:59.327 INFO:teuthology.orchestra.run.smithi148.stdout:linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-29T04:11:59.327 INFO:teuthology.orchestra.run.smithi148.stdout:linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-29T04:11:59.327 INFO:teuthology.orchestra.run.smithi148.stdout:linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-29T04:11:59.327 INFO:teuthology.orchestra.run.smithi148.stdout:linux-5.4/fs/orangefs/Makefile 2024-03-29T04:11:59.327 INFO:teuthology.orchestra.run.smithi148.stdout:linux-5.4/fs/orangefs/symlink.c 2024-03-29T04:11:59.327 INFO:teuthology.orchestra.run.smithi148.stdout:linux-5.4/fs/orangefs/downcall.h 2024-03-29T04:11:59.327 INFO:tasks.cephfs.test_uninlining:Found 23 inlined files 2024-03-29T04:11:59.327 DEBUG:tasks.cephfs.test_uninlining:starting scrub 2024-03-29T04:11:59.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:59.328 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub start / recursive 2024-03-29T04:11:59.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.453+0000 7f464cfcd640 1 Processor -- start 2024-03-29T04:11:59.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.454+0000 7f464cfcd640 1 -- start start 2024-03-29T04:11:59.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.454+0000 7f464cfcd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f46480b36e0 0x7f46481615d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.454+0000 7f464cfcd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46480d3c20 0x7f4648161b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.454+0000 7f464cfcd640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46480d2a50 con 0x7f46480d3840 2024-03-29T04:11:59.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.454+0000 7f464cfcd640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f46480d2d30 con 0x7f46480b36e0 2024-03-29T04:11:59.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.454+0000 7f464cfcd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f46480d3010 con 0x7f46480d3c20 2024-03-29T04:11:59.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.455+0000 7f463ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f46480b36e0 0x7f46481615d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.455+0000 7f464659c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f46480d3840 0x7f46480d2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49242/0 (socket says 172.21.15.148:49242) 2024-03-29T04:11:59.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.455+0000 7f464659c640 1 -- 172.21.15.148:0/1059323535 learned_addr learned my addr 172.21.15.148:0/1059323535 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:59.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.455+0000 7f4646d9d640 1 --2- 172.21.15.148:0/1059323535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46480d3c20 0x7f4648161b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.455+0000 7f463ffff640 1 -- 172.21.15.148:0/1059323535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46480d3c20 msgr2=0x7f4648161b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:59.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.455+0000 7f463ffff640 1 --2- 172.21.15.148:0/1059323535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46480d3c20 0x7f4648161b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.455+0000 7f463ffff640 1 -- 172.21.15.148:0/1059323535 >> v1:172.21.15.148:6789/0 conn(0x7f46480d3840 legacy=0x7f46480d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:59.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.455+0000 7f463ffff640 1 -- 172.21.15.148:0/1059323535 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f464806b7c0 con 0x7f46480b36e0 2024-03-29T04:11:59.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.455+0000 7f4646d9d640 1 --2- 172.21.15.148:0/1059323535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46480d3c20 0x7f4648161b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:59.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.456+0000 7f463ffff640 1 --2- 172.21.15.148:0/1059323535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46480b36e0 0x7f46481615d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f46380097a0 tx=0x7f463802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a8f8133bcab977 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:59.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.456+0000 7f4645d9b640 1 -- 172.21.15.148:0/1059323535 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638035020 con 0x7f46480b36e0 2024-03-29T04:11:59.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.456+0000 7f4645d9b640 1 -- 172.21.15.148:0/1059323535 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4638031830 con 0x7f46480b36e0 2024-03-29T04:11:59.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.456+0000 7f4645d9b640 1 -- 172.21.15.148:0/1059323535 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638031c00 con 0x7f46480b36e0 2024-03-29T04:11:59.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.456+0000 7f464cfcd640 1 -- 172.21.15.148:0/1059323535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46480b36e0 msgr2=0x7f46481615d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:59.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.456+0000 7f464cfcd640 1 --2- 172.21.15.148:0/1059323535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46480b36e0 0x7f46481615d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f46380097a0 tx=0x7f463802b8d0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.456+0000 7f464cfcd640 1 -- 172.21.15.148:0/1059323535 shutdown_connections 2024-03-29T04:11:59.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.456+0000 7f464cfcd640 1 --2- 172.21.15.148:0/1059323535 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46480d3c20 0x7f4648161b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.456+0000 7f464cfcd640 1 --2- 172.21.15.148:0/1059323535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46480b36e0 0x7f46481615d0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.456+0000 7f464cfcd640 1 -- 172.21.15.148:0/1059323535 >> 172.21.15.148:0/1059323535 conn(0x7f46480d42d0 msgr2=0x7f46480d46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:59.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.456+0000 7f464cfcd640 1 -- 172.21.15.148:0/1059323535 shutdown_connections 2024-03-29T04:11:59.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.456+0000 7f464cfcd640 1 -- 172.21.15.148:0/1059323535 wait complete. 2024-03-29T04:11:59.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.458+0000 7f464cfcd640 1 Processor -- start 2024-03-29T04:11:59.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.458+0000 7f464cfcd640 1 -- start start 2024-03-29T04:11:59.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.458+0000 7f464cfcd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46480b36e0 0x7f464815af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.458+0000 7f464cfcd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46480d3840 0x7f46480cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.458+0000 7f464cfcd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46480d3c20 0x7f464814c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.458+0000 7f464cfcd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f464806c0f0 con 0x7f46480d3840 2024-03-29T04:11:59.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.458+0000 7f464cfcd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f464806bf70 con 0x7f46480b36e0 2024-03-29T04:11:59.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.458+0000 7f464cfcd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f464806c270 con 0x7f46480d3c20 2024-03-29T04:11:59.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.459+0000 7f463ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46480d3840 0x7f46480cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.459+0000 7f463ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46480d3840 0x7f46480cb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40792/0 (socket says 172.21.15.148:40792) 2024-03-29T04:11:59.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.459+0000 7f463ffff640 1 -- 172.21.15.148:0/2993553652 learned_addr learned my addr 172.21.15.148:0/2993553652 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:59.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.459+0000 7f464659c640 1 --2- 172.21.15.148:0/2993553652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46480b36e0 0x7f464815af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.459+0000 7f4646d9d640 1 --2- 172.21.15.148:0/2993553652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46480d3c20 0x7f464814c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.459+0000 7f463ffff640 1 -- 172.21.15.148:0/2993553652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46480d3c20 msgr2=0x7f464814c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:59.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.459+0000 7f463ffff640 1 --2- 172.21.15.148:0/2993553652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46480d3c20 0x7f464814c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.459+0000 7f463ffff640 1 -- 172.21.15.148:0/2993553652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46480b36e0 msgr2=0x7f464815af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:59.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.459+0000 7f463ffff640 1 --2- 172.21.15.148:0/2993553652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46480b36e0 0x7f464815af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.459+0000 7f463ffff640 1 -- 172.21.15.148:0/2993553652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f464815b790 con 0x7f46480d3840 2024-03-29T04:11:59.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.459+0000 7f4646d9d640 1 --2- 172.21.15.148:0/2993553652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46480d3c20 0x7f464814c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:59.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.459+0000 7f463ffff640 1 --2- 172.21.15.148:0/2993553652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46480d3840 0x7f46480cb610 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f46380098c0 tx=0x7f46380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:59.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.459+0000 7f463effd640 1 -- 172.21.15.148:0/2993553652 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638041070 con 0x7f46480d3840 2024-03-29T04:11:59.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.459+0000 7f463effd640 1 -- 172.21.15.148:0/2993553652 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f463803a9b0 con 0x7f46480d3840 2024-03-29T04:11:59.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.459+0000 7f464cfcd640 1 -- 172.21.15.148:0/2993553652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f464815ba70 con 0x7f46480d3840 2024-03-29T04:11:59.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.459+0000 7f463effd640 1 -- 172.21.15.148:0/2993553652 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638035040 con 0x7f46480d3840 2024-03-29T04:11:59.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.459+0000 7f464cfcd640 1 -- 172.21.15.148:0/2993553652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f464815c030 con 0x7f46480d3840 2024-03-29T04:11:59.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.461+0000 7f463effd640 1 -- 172.21.15.148:0/2993553652 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f463804c060 con 0x7f46480d3840 2024-03-29T04:11:59.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.461+0000 7f463effd640 1 --2- 172.21.15.148:0/2993553652 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f462806c530 0x7f462806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.461+0000 7f464659c640 1 --2- 172.21.15.148:0/2993553652 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f462806c530 0x7f462806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.461+0000 7f463effd640 1 -- 172.21.15.148:0/2993553652 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f46380ad250 con 0x7f46480d3840 2024-03-29T04:11:59.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.462+0000 7f464659c640 1 --2- 172.21.15.148:0/2993553652 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f462806c530 0x7f462806e9f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4630000f60 tx=0x7f4630009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:59.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.462+0000 7f464fbb9740 1 Processor -- start 2024-03-29T04:11:59.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.462+0000 7f464fbb9740 1 -- start start 2024-03-29T04:11:59.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.463+0000 7f464fbb9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5564f4058b20 0x5564f402bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.463+0000 7f464fbb9740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5564f4058f00 0x5564f405b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.463+0000 7f464fbb9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564f402c000 0x5564f405bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.463+0000 7f464fbb9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5564f3f95410 con 0x5564f402c000 2024-03-29T04:11:59.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.463+0000 7f464fbb9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5564f3f33a80 con 0x5564f4058b20 2024-03-29T04:11:59.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.463+0000 7f464fbb9740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x5564f3f913c0 con 0x5564f4058f00 2024-03-29T04:11:59.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.463+0000 7f4646d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564f402c000 0x5564f405bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.463+0000 7f4646d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564f402c000 0x5564f405bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40804/0 (socket says 172.21.15.148:40804) 2024-03-29T04:11:59.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.463+0000 7f4646d9d640 1 -- 172.21.15.148:0/519218329 learned_addr learned my addr 172.21.15.148:0/519218329 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:59.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.463+0000 7f464659c640 1 --2- 172.21.15.148:0/519218329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5564f4058b20 0x5564f402bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.463+0000 7f463ffff640 1 --2- 172.21.15.148:0/519218329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5564f4058f00 0x5564f405b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.464+0000 7f4646d9d640 1 -- 172.21.15.148:0/519218329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5564f4058f00 msgr2=0x5564f405b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:59.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.464+0000 7f4646d9d640 1 --2- 172.21.15.148:0/519218329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5564f4058f00 0x5564f405b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.464+0000 7f4646d9d640 1 -- 172.21.15.148:0/519218329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5564f4058b20 msgr2=0x5564f402bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:59.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.464+0000 7f4646d9d640 1 --2- 172.21.15.148:0/519218329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5564f4058b20 0x5564f402bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.464+0000 7f4646d9d640 1 -- 172.21.15.148:0/519218329 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564f3d757b0 con 0x5564f402c000 2024-03-29T04:11:59.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.464+0000 7f464659c640 1 --2- 172.21.15.148:0/519218329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5564f4058b20 0x5564f402bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:59.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.464+0000 7f4646d9d640 1 --2- 172.21.15.148:0/519218329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564f402c000 0x5564f405bdf0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f4640009fd0 tx=0x7f46400077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b18151927ec91b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:59.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.464+0000 7f4627fff640 1 -- 172.21.15.148:0/519218329 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46400137d0 con 0x5564f402c000 2024-03-29T04:11:59.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.464+0000 7f4627fff640 1 -- 172.21.15.148:0/519218329 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4640019070 con 0x5564f402c000 2024-03-29T04:11:59.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.464+0000 7f4627fff640 1 -- 172.21.15.148:0/519218329 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640013b70 con 0x5564f402c000 2024-03-29T04:11:59.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.465+0000 7f464fbb9740 1 -- 172.21.15.148:0/519218329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564f402c000 msgr2=0x5564f405bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:59.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.465+0000 7f464fbb9740 1 --2- 172.21.15.148:0/519218329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564f402c000 0x5564f405bdf0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f4640009fd0 tx=0x7f46400077c0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.465+0000 7f464fbb9740 1 -- 172.21.15.148:0/519218329 shutdown_connections 2024-03-29T04:11:59.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.465+0000 7f464fbb9740 1 --2- 172.21.15.148:0/519218329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564f402c000 0x5564f405bdf0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.465+0000 7f464fbb9740 1 --2- 172.21.15.148:0/519218329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5564f4058f00 0x5564f405b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.465+0000 7f464fbb9740 1 --2- 172.21.15.148:0/519218329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5564f4058b20 0x5564f402bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.465+0000 7f464fbb9740 1 -- 172.21.15.148:0/519218329 >> 172.21.15.148:0/519218329 conn(0x5564f402dd30 msgr2=0x5564f40768b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:59.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.465+0000 7f464fbb9740 1 -- 172.21.15.148:0/519218329 shutdown_connections 2024-03-29T04:11:59.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.465+0000 7f464fbb9740 1 -- 172.21.15.148:0/519218329 wait complete. 2024-03-29T04:11:59.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.466+0000 7f464fbb9740 1 -- start start 2024-03-29T04:11:59.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.466+0000 7f464fbb9740 1 Processor -- start 2024-03-29T04:11:59.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.466+0000 7f464fbb9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5564f4058b20 0x5564f4045ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.466+0000 7f464fbb9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564f4058f00 0x5564f4046210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.466+0000 7f464fbb9740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5564f404bdd0 0x5564f405e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.467+0000 7f464fbb9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5564f40303d0 con 0x5564f4058f00 2024-03-29T04:11:59.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.467+0000 7f464fbb9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5564f3f31740 con 0x5564f4058b20 2024-03-29T04:11:59.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.467+0000 7f464fbb9740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x5564f4032c10 con 0x5564f404bdd0 2024-03-29T04:11:59.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.467+0000 7f464fbb9740 10 client.0 _ll_register_callbacks cb 0x5564f3f3eb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:11:59.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.467+0000 7f463ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564f4058f00 0x5564f4046210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.467+0000 7f463ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564f4058f00 0x5564f4046210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40810/0 (socket says 172.21.15.148:40810) 2024-03-29T04:11:59.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.467+0000 7f463ffff640 1 -- 172.21.15.148:0/1199346788 learned_addr learned my addr 172.21.15.148:0/1199346788 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:59.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.467+0000 7f464659c640 1 --2- 172.21.15.148:0/1199346788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5564f4058b20 0x5564f4045ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.467+0000 7f4646d9d640 1 --2- 172.21.15.148:0/1199346788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5564f404bdd0 0x5564f405e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.467+0000 7f463ffff640 1 -- 172.21.15.148:0/1199346788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5564f404bdd0 msgr2=0x5564f405e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:59.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.467+0000 7f463ffff640 1 --2- 172.21.15.148:0/1199346788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5564f404bdd0 0x5564f405e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.467+0000 7f463ffff640 1 -- 172.21.15.148:0/1199346788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5564f4058b20 msgr2=0x5564f4045ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:59.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.467+0000 7f463ffff640 1 --2- 172.21.15.148:0/1199346788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5564f4058b20 0x5564f4045ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.467+0000 7f463ffff640 1 -- 172.21.15.148:0/1199346788 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564f3dd5f40 con 0x5564f4058f00 2024-03-29T04:11:59.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.467+0000 7f4646d9d640 1 --2- 172.21.15.148:0/1199346788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5564f404bdd0 0x5564f405e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:59.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.467+0000 7f463ffff640 1 --2- 172.21.15.148:0/1199346788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564f4058f00 0x5564f4046210 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f463802bee0 tx=0x7f463807b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97fe37ee061d5c3f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:59.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.467+0000 7f46257fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-03-29T04:11:59.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.467+0000 7f46257fa640 1 -- 172.21.15.148:0/1199346788 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463807f380 con 0x5564f4058f00 2024-03-29T04:11:59.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.467+0000 7f46257fa640 1 -- 172.21.15.148:0/1199346788 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f463803ada0 con 0x5564f4058f00 2024-03-29T04:11:59.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.467+0000 7f464fbb9740 1 -- 172.21.15.148:0/1199346788 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4640019070 con 0x5564f4058f00 2024-03-29T04:11:59.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.467+0000 7f46257fa640 1 -- 172.21.15.148:0/1199346788 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f4638083bb0 con 0x5564f4058f00 2024-03-29T04:11:59.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.467+0000 7f464fbb9740 10 client.5829 fetch_fsmap learned FSMap version 19 2024-03-29T04:11:59.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.467+0000 7f464fbb9740 1 -- 172.21.15.148:0/1199346788 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x5564f3c690a0 con 0x5564f4058f00 2024-03-29T04:11:59.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.468+0000 7f46257fa640 1 -- 172.21.15.148:0/1199346788 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4640019070 con 0x5564f4058f00 2024-03-29T04:11:59.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.468+0000 7f46257fa640 1 -- 172.21.15.148:0/1199346788 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f4638077640 con 0x5564f4058f00 2024-03-29T04:11:59.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.468+0000 7f464fbb9740 10 client.5829 fetch_fsmap finished waiting for FSMap version 19 2024-03-29T04:11:59.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.468+0000 7f464fbb9740 10 client.5829 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-29T04:11:59.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.468+0000 7f464fbb9740 20 client.5829 populate_metadata read hostname 'smithi148' 2024-03-29T04:11:59.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.468+0000 7f464fbb9740 1 --2- 172.21.15.148:0/1199346788 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x5564f4048ea0 0x5564f4062eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.468+0000 7f464fbb9740 4 client.5829 mds_command: new command op to 5583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T04:11:59.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.468+0000 7f464fbb9740 1 -- 172.21.15.148:0/1199346788 --> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564f3fa2950 con 0x5564f4048ea0 2024-03-29T04:11:59.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.468+0000 7f464659c640 1 --2- 172.21.15.148:0/1199346788 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x5564f4048ea0 0x5564f4062eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.468+0000 7f46257fa640 10 client.5829 ms_handle_connect on v2:172.21.15.148:6836/473448013 2024-03-29T04:11:59.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.468+0000 7f464659c640 1 --2- 172.21.15.148:0/1199346788 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x5564f4048ea0 0x5564f4062eb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b8241dbfb5d39f server_cookie=9c3f4b158dba2087 in_seq=0 out_seq=0 2024-03-29T04:11:59.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.469+0000 7f46257fa640 1 -- 172.21.15.148:0/1199346788 <== mds.0 v2:172.21.15.148:6836/473448013 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x5564f3fa2950 con 0x5564f4048ea0 2024-03-29T04:11:59.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.469+0000 7f46257fa640 10 client.5829 handle_command_reply: tid=0 2024-03-29T04:11:59.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.469+0000 7f464fbb9740 1 client.5829 shutdown 2024-03-29T04:11:59.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.469+0000 7f464fbb9740 1 -- 172.21.15.148:0/1199346788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564f4058f00 msgr2=0x5564f4046210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:59.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.469+0000 7f464fbb9740 1 --2- 172.21.15.148:0/1199346788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564f4058f00 0x5564f4046210 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f463802bee0 tx=0x7f463807b790 comp rx=0 tx=0).stop 2024-03-29T04:11:59.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.469+0000 7f464fbb9740 1 -- 172.21.15.148:0/1199346788 shutdown_connections 2024-03-29T04:11:59.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.469+0000 7f464fbb9740 1 --2- 172.21.15.148:0/1199346788 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x5564f4048ea0 0x5564f4062eb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.469+0000 7f464fbb9740 1 --2- 172.21.15.148:0/1199346788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5564f404bdd0 0x5564f405e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.469+0000 7f464fbb9740 1 --2- 172.21.15.148:0/1199346788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564f4058f00 0x5564f4046210 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.469+0000 7f464fbb9740 1 --2- 172.21.15.148:0/1199346788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5564f4058b20 0x5564f4045ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.469+0000 7f46257fa640 0 client.5829 ms_handle_reset on v2:172.21.15.148:6836/473448013 2024-03-29T04:11:59.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.469+0000 7f464fbb9740 1 -- 172.21.15.148:0/1199346788 >> 172.21.15.148:0/1199346788 conn(0x5564f402c000 msgr2=0x5564f4078c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:59.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.469+0000 7f464fbb9740 1 -- 172.21.15.148:0/1199346788 shutdown_connections 2024-03-29T04:11:59.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.469+0000 7f464fbb9740 1 -- 172.21.15.148:0/1199346788 wait complete. 2024-03-29T04:11:59.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.469+0000 7f464fbb9740 20 client.5829 trim_cache size 0 max 16384 2024-03-29T04:11:59.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.487+0000 7f464fbb9740 1 Processor -- start 2024-03-29T04:11:59.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.487+0000 7f464fbb9740 1 -- start start 2024-03-29T04:11:59.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.487+0000 7f464fbb9740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5564f404bdd0 0x5564f409ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.487+0000 7f464fbb9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564f4058b20 0x5564f40636d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.487+0000 7f464fbb9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5564f4058f00 0x5564f40458f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.487+0000 7f464fbb9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4648201180 con 0x5564f4058b20 2024-03-29T04:11:59.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.487+0000 7f464fbb9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f46482001e0 con 0x5564f4058f00 2024-03-29T04:11:59.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.487+0000 7f464fbb9740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f46480c9660 con 0x5564f404bdd0 2024-03-29T04:11:59.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.487+0000 7f463ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564f4058b20 0x5564f40636d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.487+0000 7f463ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564f4058b20 0x5564f40636d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40814/0 (socket says 172.21.15.148:40814) 2024-03-29T04:11:59.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.487+0000 7f463ffff640 1 -- 172.21.15.148:0/1795808256 learned_addr learned my addr 172.21.15.148:0/1795808256 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:59.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.488+0000 7f464659c640 1 --2- 172.21.15.148:0/1795808256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5564f404bdd0 0x5564f409ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.488+0000 7f4646d9d640 1 --2- 172.21.15.148:0/1795808256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5564f4058f00 0x5564f40458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.488+0000 7f463ffff640 1 -- 172.21.15.148:0/1795808256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5564f404bdd0 msgr2=0x5564f409ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:59.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.488+0000 7f463ffff640 1 --2- 172.21.15.148:0/1795808256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5564f404bdd0 0x5564f409ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.488+0000 7f463ffff640 1 -- 172.21.15.148:0/1795808256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5564f4058f00 msgr2=0x5564f40458f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:59.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.488+0000 7f463ffff640 1 --2- 172.21.15.148:0/1795808256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5564f4058f00 0x5564f40458f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.488+0000 7f463ffff640 1 -- 172.21.15.148:0/1795808256 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564f407a6d0 con 0x5564f4058b20 2024-03-29T04:11:59.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.488+0000 7f463ffff640 1 --2- 172.21.15.148:0/1795808256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564f4058b20 0x5564f40636d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f463807acd0 tx=0x7f463803d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55301b9b3e09eb6e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:59.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.488+0000 7f46277fe640 1 -- 172.21.15.148:0/1795808256 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463807f380 con 0x5564f4058b20 2024-03-29T04:11:59.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.488+0000 7f46277fe640 1 -- 172.21.15.148:0/1795808256 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46380ab340 con 0x5564f4058b20 2024-03-29T04:11:59.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.488+0000 7f46277fe640 1 -- 172.21.15.148:0/1795808256 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638077640 con 0x5564f4058b20 2024-03-29T04:11:59.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.488+0000 7f464fbb9740 1 -- 172.21.15.148:0/1795808256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564f4058b20 msgr2=0x5564f40636d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:59.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.488+0000 7f464fbb9740 1 --2- 172.21.15.148:0/1795808256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564f4058b20 0x5564f40636d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f463807acd0 tx=0x7f463803d070 comp rx=0 tx=0).stop 2024-03-29T04:11:59.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.488+0000 7f464fbb9740 1 -- 172.21.15.148:0/1795808256 shutdown_connections 2024-03-29T04:11:59.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.488+0000 7f464fbb9740 1 --2- 172.21.15.148:0/1795808256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5564f4058f00 0x5564f40458f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.488+0000 7f464fbb9740 1 --2- 172.21.15.148:0/1795808256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564f4058b20 0x5564f40636d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.488+0000 7f464fbb9740 1 --2- 172.21.15.148:0/1795808256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5564f404bdd0 0x5564f409ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.488+0000 7f464fbb9740 1 -- 172.21.15.148:0/1795808256 >> 172.21.15.148:0/1795808256 conn(0x5564f402c000 msgr2=0x5564f40621f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:59.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.489+0000 7f464fbb9740 1 -- 172.21.15.148:0/1795808256 shutdown_connections 2024-03-29T04:11:59.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.489+0000 7f464fbb9740 1 -- 172.21.15.148:0/1795808256 wait complete. 2024-03-29T04:11:59.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.489+0000 7f464fbb9740 1 -- start start 2024-03-29T04:11:59.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.489+0000 7f464fbb9740 1 Processor -- start 2024-03-29T04:11:59.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.490+0000 7f464fbb9740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5564f404bdd0 0x5564f4045cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.490+0000 7f464fbb9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564f4058b20 0x5564f405b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.490+0000 7f464fbb9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5564f4058f00 0x5564f405b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.490+0000 7f464fbb9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5564f3fa3540 con 0x5564f4058b20 2024-03-29T04:11:59.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.490+0000 7f464fbb9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f46481ffbf0 con 0x5564f4058f00 2024-03-29T04:11:59.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.490+0000 7f464fbb9740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x5564f40789d0 con 0x5564f404bdd0 2024-03-29T04:11:59.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.490+0000 7f464fbb9740 10 client.0 _ll_register_callbacks cb 0x7f46380094a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:11:59.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.490+0000 7f463ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564f4058b20 0x5564f405b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.490+0000 7f463ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564f4058b20 0x5564f405b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40826/0 (socket says 172.21.15.148:40826) 2024-03-29T04:11:59.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.490+0000 7f463ffff640 1 -- 172.21.15.148:0/3309918435 learned_addr learned my addr 172.21.15.148:0/3309918435 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:59.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.490+0000 7f4646d9d640 1 --2- 172.21.15.148:0/3309918435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5564f4058f00 0x5564f405b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.490+0000 7f464659c640 1 --2- 172.21.15.148:0/3309918435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5564f404bdd0 0x5564f4045cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.490+0000 7f463ffff640 1 -- 172.21.15.148:0/3309918435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5564f404bdd0 msgr2=0x5564f4045cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:59.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.490+0000 7f463ffff640 1 --2- 172.21.15.148:0/3309918435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5564f404bdd0 0x5564f4045cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.490+0000 7f463ffff640 1 -- 172.21.15.148:0/3309918435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5564f4058f00 msgr2=0x5564f405b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:59.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.490+0000 7f463ffff640 1 --2- 172.21.15.148:0/3309918435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5564f4058f00 0x5564f405b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.490+0000 7f463ffff640 1 -- 172.21.15.148:0/3309918435 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564f4046230 con 0x5564f4058b20 2024-03-29T04:11:59.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.491+0000 7f463ffff640 1 --2- 172.21.15.148:0/3309918435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564f4058b20 0x5564f405b2f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f463807acd0 tx=0x7f46380ab530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f771c7eb40351c87 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:59.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.491+0000 7f46257fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-03-29T04:11:59.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.491+0000 7f46257fa640 1 -- 172.21.15.148:0/3309918435 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463807f380 con 0x5564f4058b20 2024-03-29T04:11:59.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.491+0000 7f46257fa640 1 -- 172.21.15.148:0/3309918435 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f463803ada0 con 0x5564f4058b20 2024-03-29T04:11:59.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.491+0000 7f464fbb9740 1 -- 172.21.15.148:0/3309918435 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46380ab340 con 0x5564f4058b20 2024-03-29T04:11:59.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.491+0000 7f46257fa640 1 -- 172.21.15.148:0/3309918435 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f46380932c0 con 0x5564f4058b20 2024-03-29T04:11:59.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.491+0000 7f464fbb9740 10 client.5835 fetch_fsmap learned FSMap version 19 2024-03-29T04:11:59.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.491+0000 7f464fbb9740 1 -- 172.21.15.148:0/3309918435 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x5564f40f3620 con 0x5564f4058b20 2024-03-29T04:11:59.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.491+0000 7f46257fa640 1 -- 172.21.15.148:0/3309918435 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46380ab340 con 0x5564f4058b20 2024-03-29T04:11:59.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.491+0000 7f46257fa640 1 -- 172.21.15.148:0/3309918435 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f4638076d20 con 0x5564f4058b20 2024-03-29T04:11:59.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.491+0000 7f464fbb9740 10 client.5835 fetch_fsmap finished waiting for FSMap version 19 2024-03-29T04:11:59.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.491+0000 7f464fbb9740 10 client.5835 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-29T04:11:59.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.491+0000 7f464fbb9740 20 client.5835 populate_metadata read hostname 'smithi148' 2024-03-29T04:11:59.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.491+0000 7f464fbb9740 1 --2- 172.21.15.148:0/3309918435 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x5564f402f8e0 0x5564f405bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.491+0000 7f464fbb9740 4 client.5835 mds_command: new command op to 5583 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-03-29T04:11:59.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.491+0000 7f464fbb9740 1 -- 172.21.15.148:0/3309918435 --> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) -- 0x5564f4064770 con 0x5564f402f8e0 2024-03-29T04:11:59.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.491+0000 7f464659c640 1 --2- 172.21.15.148:0/3309918435 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x5564f402f8e0 0x5564f405bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.492+0000 7f46257fa640 10 client.5835 ms_handle_connect on v2:172.21.15.148:6836/473448013 2024-03-29T04:11:59.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.492+0000 7f464659c640 1 --2- 172.21.15.148:0/3309918435 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x5564f402f8e0 0x5564f405bd70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8ba3620cf89a547 server_cookie=bd0a21fa5453d102 in_seq=0 out_seq=0 2024-03-29T04:11:59.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.493+0000 7f46257fa640 1 -- 172.21.15.148:0/3309918435 <== mds.0 v2:172.21.15.148:6836/473448013 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5564f4064770 con 0x5564f402f8e0 2024-03-29T04:11:59.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.493+0000 7f46257fa640 10 client.5835 handle_command_reply: tid=0 2024-03-29T04:11:59.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.493+0000 7f464fbb9740 1 client.5835 shutdown 2024-03-29T04:11:59.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.493+0000 7f464fbb9740 1 -- 172.21.15.148:0/3309918435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564f4058b20 msgr2=0x5564f405b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:59.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.493+0000 7f464fbb9740 1 --2- 172.21.15.148:0/3309918435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564f4058b20 0x5564f405b2f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f463807acd0 tx=0x7f46380ab530 comp rx=0 tx=0).stop 2024-03-29T04:11:59.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.493+0000 7f464fbb9740 1 -- 172.21.15.148:0/3309918435 shutdown_connections 2024-03-29T04:11:59.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.493+0000 7f464fbb9740 1 --2- 172.21.15.148:0/3309918435 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x5564f402f8e0 0x5564f405bd70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.493+0000 7f464fbb9740 1 --2- 172.21.15.148:0/3309918435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5564f4058f00 0x5564f405b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.493+0000 7f464fbb9740 1 --2- 172.21.15.148:0/3309918435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564f4058b20 0x5564f405b2f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.493+0000 7f464fbb9740 1 --2- 172.21.15.148:0/3309918435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5564f404bdd0 0x5564f4045cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.493+0000 7f464fbb9740 1 -- 172.21.15.148:0/3309918435 >> 172.21.15.148:0/3309918435 conn(0x5564f402c000 msgr2=0x5564f4031c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:59.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.493+0000 7f46257fa640 0 client.5835 ms_handle_reset on v2:172.21.15.148:6836/473448013 2024-03-29T04:11:59.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.493+0000 7f464fbb9740 1 -- 172.21.15.148:0/3309918435 shutdown_connections 2024-03-29T04:11:59.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.493+0000 7f464fbb9740 1 -- 172.21.15.148:0/3309918435 wait complete. 2024-03-29T04:11:59.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.493+0000 7f464fbb9740 20 client.5835 trim_cache size 0 max 16384 2024-03-29T04:11:59.478 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-03-29T04:11:59.478 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0, 2024-03-29T04:11:59.478 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_tag": "713fb6cc-5d83-4906-8a07-c0c0f0760e64", 2024-03-29T04:11:59.479 INFO:teuthology.orchestra.run.smithi148.stdout: "mode": "asynchronous" 2024-03-29T04:11:59.479 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-03-29T04:11:59.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.494+0000 7f464cfcd640 1 -- 172.21.15.148:0/2993553652 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f462806c530 msgr2=0x7f462806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:59.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.494+0000 7f464cfcd640 1 --2- 172.21.15.148:0/2993553652 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f462806c530 0x7f462806e9f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4630000f60 tx=0x7f4630009040 comp rx=0 tx=0).stop 2024-03-29T04:11:59.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.494+0000 7f464cfcd640 1 -- 172.21.15.148:0/2993553652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46480d3840 msgr2=0x7f46480cb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:59.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.494+0000 7f464cfcd640 1 --2- 172.21.15.148:0/2993553652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46480d3840 0x7f46480cb610 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f46380098c0 tx=0x7f46380046c0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.494+0000 7f464cfcd640 1 -- 172.21.15.148:0/2993553652 shutdown_connections 2024-03-29T04:11:59.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.494+0000 7f464cfcd640 1 --2- 172.21.15.148:0/2993553652 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f462806c530 0x7f462806e9f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.494+0000 7f464cfcd640 1 --2- 172.21.15.148:0/2993553652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46480d3c20 0x7f464814c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.494+0000 7f464cfcd640 1 --2- 172.21.15.148:0/2993553652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46480d3840 0x7f46480cb610 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.494+0000 7f464cfcd640 1 --2- 172.21.15.148:0/2993553652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46480b36e0 0x7f464815af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.494+0000 7f464cfcd640 1 -- 172.21.15.148:0/2993553652 >> 172.21.15.148:0/2993553652 conn(0x7f46480d42d0 msgr2=0x7f46480d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:59.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.494+0000 7f464cfcd640 1 -- 172.21.15.148:0/2993553652 shutdown_connections 2024-03-29T04:11:59.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.494+0000 7f464cfcd640 1 -- 172.21.15.148:0/2993553652 wait complete. 2024-03-29T04:11:59.489 DEBUG:tasks.cephfs.test_uninlining:scrub start response: {'return_code': 0, 'scrub_tag': '713fb6cc-5d83-4906-8a07-c0c0f0760e64', 'mode': 'asynchronous'} 2024-03-29T04:11:59.489 DEBUG:tasks.cephfs.test_uninlining:waiting for scrub to complete 2024-03-29T04:11:59.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:59.489 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-03-29T04:11:59.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.654+0000 7f9f3e4a5640 1 Processor -- start 2024-03-29T04:11:59.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.654+0000 7f9f3e4a5640 1 -- start start 2024-03-29T04:11:59.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.655+0000 7f9f3e4a5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9f380b3430 0x7f9f38162680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.655+0000 7f9f3e4a5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f380cf330 con 0x7f9f380b2f00 2024-03-29T04:11:59.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.655+0000 7f9f3e4a5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f38162cf0 con 0x7f9f3815e3c0 2024-03-29T04:11:59.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.655+0000 7f9f377fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9f380b2f00 0x7f9f3806b7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49246/0 (socket says 172.21.15.148:49246) 2024-03-29T04:11:59.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.655+0000 7f9f3e4a5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9f380cf960 con 0x7f9f380b3430 2024-03-29T04:11:59.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.655+0000 7f9f377fe640 1 -- 172.21.15.148:0/1608921652 learned_addr learned my addr 172.21.15.148:0/1608921652 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:11:59.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.655+0000 7f9f3ca42640 1 --2- 172.21.15.148:0/1608921652 >> v2:172.21.15.157:3300/0 conn(0x7f9f380b3430 0x7f9f38162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.655+0000 7f9f36ffd640 1 -- 172.21.15.148:0/1608921652 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 686208022 0 0) 0x7f9f38162cf0 con 0x7f9f3815e3c0 2024-03-29T04:11:59.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.655+0000 7f9f36ffd640 1 -- 172.21.15.148:0/1608921652 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f20003520 con 0x7f9f3815e3c0 2024-03-29T04:11:59.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.655+0000 7f9f36ffd640 1 -- 172.21.15.148:0/1608921652 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2519529099 0 0) 0x7f9f380cf330 con 0x7f9f380b2f00 2024-03-29T04:11:59.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.655+0000 7f9f3ca42640 1 -- 172.21.15.148:0/1608921652 >> v1:172.21.15.157:6789/0 conn(0x7f9f3815e3c0 legacy=0x7f9f38161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:59.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.655+0000 7f9f3ca42640 1 -- 172.21.15.148:0/1608921652 >> v1:172.21.15.148:6789/0 conn(0x7f9f380b2f00 legacy=0x7f9f3806b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:59.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.655+0000 7f9f3ca42640 1 -- 172.21.15.148:0/1608921652 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f380cf5f0 con 0x7f9f380b3430 2024-03-29T04:11:59.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.656+0000 7f9f3ca42640 1 --2- 172.21.15.148:0/1608921652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f380b3430 0x7f9f38162680 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9f30009d90 tx=0x7f9f3002beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e82b1cdcac3eed85 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:59.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.656+0000 7f9f36ffd640 1 -- 172.21.15.148:0/1608921652 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30039070 con 0x7f9f380b3430 2024-03-29T04:11:59.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.656+0000 7f9f36ffd640 1 -- 172.21.15.148:0/1608921652 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f3002cc60 con 0x7f9f380b3430 2024-03-29T04:11:59.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.656+0000 7f9f36ffd640 1 -- 172.21.15.148:0/1608921652 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30034660 con 0x7f9f380b3430 2024-03-29T04:11:59.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.657+0000 7f9f3e4a5640 1 -- 172.21.15.148:0/1608921652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f380b3430 msgr2=0x7f9f38162680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:59.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.657+0000 7f9f3e4a5640 1 --2- 172.21.15.148:0/1608921652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f380b3430 0x7f9f38162680 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9f30009d90 tx=0x7f9f3002beb0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.657+0000 7f9f3e4a5640 1 -- 172.21.15.148:0/1608921652 shutdown_connections 2024-03-29T04:11:59.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.657+0000 7f9f3e4a5640 1 --2- 172.21.15.148:0/1608921652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f380b3430 0x7f9f38162680 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.657+0000 7f9f3e4a5640 1 -- 172.21.15.148:0/1608921652 >> 172.21.15.148:0/1608921652 conn(0x7f9f380b6430 msgr2=0x7f9f380d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:59.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.657+0000 7f9f3e4a5640 1 -- 172.21.15.148:0/1608921652 shutdown_connections 2024-03-29T04:11:59.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.657+0000 7f9f3e4a5640 1 -- 172.21.15.148:0/1608921652 wait complete. 2024-03-29T04:11:59.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.659+0000 7f9f3e4a5640 1 Processor -- start 2024-03-29T04:11:59.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.659+0000 7f9f3e4a5640 1 -- start start 2024-03-29T04:11:59.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.660+0000 7f9f3e4a5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f380b3430 0x7f9f380cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.660+0000 7f9f3e4a5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f3815e3c0 0x7f9f38151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.660+0000 7f9f3e4a5640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f38152010 0x7f9f380b8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.660+0000 7f9f3e4a5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9f380b3b20 con 0x7f9f380b3430 2024-03-29T04:11:59.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.660+0000 7f9f3e4a5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9f380b39a0 con 0x7f9f3815e3c0 2024-03-29T04:11:59.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.660+0000 7f9f3e4a5640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9f3806bf00 con 0x7f9f38152010 2024-03-29T04:11:59.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.660+0000 7f9f37fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f380b3430 0x7f9f380cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.660+0000 7f9f377fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f3815e3c0 0x7f9f38151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.661+0000 7f9f3ca42640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f38152010 0x7f9f380b8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.661+0000 7f9f377fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f3815e3c0 0x7f9f38151840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.148:55106/0 (socket says 172.21.15.148:55106) 2024-03-29T04:11:59.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.661+0000 7f9f37fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f380b3430 0x7f9f380cb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40836/0 (socket says 172.21.15.148:40836) 2024-03-29T04:11:59.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.661+0000 7f9f377fe640 1 -- 172.21.15.148:0/3677336393 learned_addr learned my addr 172.21.15.148:0/3677336393 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:59.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.661+0000 7f9f377fe640 1 -- 172.21.15.148:0/3677336393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f38152010 msgr2=0x7f9f380b8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:59.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.661+0000 7f9f377fe640 1 --2- 172.21.15.148:0/3677336393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f38152010 0x7f9f380b8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.661+0000 7f9f377fe640 1 -- 172.21.15.148:0/3677336393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f380b3430 msgr2=0x7f9f380cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:59.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.661+0000 7f9f377fe640 1 --2- 172.21.15.148:0/3677336393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f380b3430 0x7f9f380cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.661+0000 7f9f377fe640 1 -- 172.21.15.148:0/3677336393 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f380cb980 con 0x7f9f3815e3c0 2024-03-29T04:11:59.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.661+0000 7f9f3ca42640 1 --2- 172.21.15.148:0/3677336393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f38152010 0x7f9f380b8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:11:59.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.661+0000 7f9f377fe640 1 --2- 172.21.15.148:0/3677336393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f3815e3c0 0x7f9f38151840 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00c550 tx=0x7f9f2c00ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:59.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.661+0000 7f9f34ff9640 1 -- 172.21.15.148:0/3677336393 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c018070 con 0x7f9f3815e3c0 2024-03-29T04:11:59.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.661+0000 7f9f34ff9640 1 -- 172.21.15.148:0/3677336393 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f2c0074e0 con 0x7f9f3815e3c0 2024-03-29T04:11:59.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.661+0000 7f9f3e4a5640 1 -- 172.21.15.148:0/3677336393 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f380cbc60 con 0x7f9f3815e3c0 2024-03-29T04:11:59.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.661+0000 7f9f34ff9640 1 -- 172.21.15.148:0/3677336393 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c004a20 con 0x7f9f3815e3c0 2024-03-29T04:11:59.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.661+0000 7f9f3e4a5640 1 -- 172.21.15.148:0/3677336393 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f380cc220 con 0x7f9f3815e3c0 2024-03-29T04:11:59.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.663+0000 7f9f34ff9640 1 -- 172.21.15.148:0/3677336393 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9f2c01b070 con 0x7f9f3815e3c0 2024-03-29T04:11:59.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.664+0000 7f9f34ff9640 1 --2- 172.21.15.148:0/3677336393 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9f2406c530 0x7f9f2406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.664+0000 7f9f37fff640 1 --2- 172.21.15.148:0/3677336393 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9f2406c530 0x7f9f2406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.664+0000 7f9f34ff9640 1 -- 172.21.15.148:0/3677336393 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f9f2c08aba0 con 0x7f9f3815e3c0 2024-03-29T04:11:59.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.664+0000 7f9f37fff640 1 --2- 172.21.15.148:0/3677336393 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9f2406c530 0x7f9f2406e9f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9f280011e0 tx=0x7f9f28009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:59.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.665+0000 7f9f4116e740 1 Processor -- start 2024-03-29T04:11:59.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.665+0000 7f9f4116e740 1 -- start start 2024-03-29T04:11:59.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.666+0000 7f9f4116e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d06b05b200 0x55d06b02e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.666+0000 7f9f4116e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d06b05b5e0 0x55d06b05df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.666+0000 7f9f4116e740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d06b02e6e0 0x55d06b05e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.667+0000 7f9f4116e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d06af98880 con 0x55d06b05b5e0 2024-03-29T04:11:59.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.667+0000 7f9f4116e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d06af408f0 con 0x55d06b05b200 2024-03-29T04:11:59.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.667+0000 7f9f4116e740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55d06af94830 con 0x55d06b02e6e0 2024-03-29T04:11:59.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.667+0000 7f9f377fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d06b05b5e0 0x55d06b05df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.667+0000 7f9f377fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d06b05b5e0 0x55d06b05df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40842/0 (socket says 172.21.15.148:40842) 2024-03-29T04:11:59.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.667+0000 7f9f377fe640 1 -- 172.21.15.148:0/1259097264 learned_addr learned my addr 172.21.15.148:0/1259097264 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:59.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.667+0000 7f9f37fff640 1 --2- 172.21.15.148:0/1259097264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d06b05b200 0x55d06b02e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.667+0000 7f9f3ca42640 1 --2- 172.21.15.148:0/1259097264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d06b02e6e0 0x55d06b05e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.667+0000 7f9f377fe640 1 -- 172.21.15.148:0/1259097264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d06b02e6e0 msgr2=0x55d06b05e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:59.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.667+0000 7f9f377fe640 1 --2- 172.21.15.148:0/1259097264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d06b02e6e0 0x55d06b05e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.667+0000 7f9f377fe640 1 -- 172.21.15.148:0/1259097264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d06b05b200 msgr2=0x55d06b02e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:59.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.667+0000 7f9f377fe640 1 --2- 172.21.15.148:0/1259097264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d06b05b200 0x55d06b02e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.667+0000 7f9f377fe640 1 -- 172.21.15.148:0/1259097264 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d06af4b140 con 0x55d06b05b5e0 2024-03-29T04:11:59.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.667+0000 7f9f3ca42640 1 --2- 172.21.15.148:0/1259097264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d06b02e6e0 0x55d06b05e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:59.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.667+0000 7f9f37fff640 1 --2- 172.21.15.148:0/1259097264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d06b05b200 0x55d06b02e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:11:59.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.667+0000 7f9f377fe640 1 --2- 172.21.15.148:0/1259097264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d06b05b5e0 0x55d06b05df90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c088f50 tx=0x7f9f2c05cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f359d4e15206ca server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:59.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.669+0000 7f9f1dffb640 1 -- 172.21.15.148:0/1259097264 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c0550c0 con 0x55d06b05b5e0 2024-03-29T04:11:59.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.669+0000 7f9f1dffb640 1 -- 172.21.15.148:0/1259097264 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f2c05f070 con 0x55d06b05b5e0 2024-03-29T04:11:59.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.670+0000 7f9f1dffb640 1 -- 172.21.15.148:0/1259097264 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c055260 con 0x55d06b05b5e0 2024-03-29T04:11:59.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.670+0000 7f9f4116e740 1 -- 172.21.15.148:0/1259097264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d06b05b5e0 msgr2=0x55d06b05df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:59.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.670+0000 7f9f4116e740 1 --2- 172.21.15.148:0/1259097264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d06b05b5e0 0x55d06b05df90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c088f50 tx=0x7f9f2c05cd50 comp rx=0 tx=0).stop 2024-03-29T04:11:59.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.670+0000 7f9f4116e740 1 -- 172.21.15.148:0/1259097264 shutdown_connections 2024-03-29T04:11:59.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.670+0000 7f9f4116e740 1 --2- 172.21.15.148:0/1259097264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d06b02e6e0 0x55d06b05e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.670+0000 7f9f4116e740 1 --2- 172.21.15.148:0/1259097264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d06b05b5e0 0x55d06b05df90 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.670+0000 7f9f4116e740 1 --2- 172.21.15.148:0/1259097264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d06b05b200 0x55d06b02e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.670+0000 7f9f4116e740 1 -- 172.21.15.148:0/1259097264 >> 172.21.15.148:0/1259097264 conn(0x55d06b030410 msgr2=0x55d06b078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:59.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.670+0000 7f9f4116e740 1 -- 172.21.15.148:0/1259097264 shutdown_connections 2024-03-29T04:11:59.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.670+0000 7f9f4116e740 1 -- 172.21.15.148:0/1259097264 wait complete. 2024-03-29T04:11:59.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.671+0000 7f9f4116e740 1 -- start start 2024-03-29T04:11:59.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.671+0000 7f9f4116e740 1 Processor -- start 2024-03-29T04:11:59.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.672+0000 7f9f4116e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d06b05b200 0x55d06b049490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.672+0000 7f9f4116e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d06b05b5e0 0x55d06b05cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.672+0000 7f9f4116e740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d06b089b50 0x55d06b05d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.672+0000 7f9f4116e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d06b032ab0 con 0x55d06b05b200 2024-03-29T04:11:59.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.672+0000 7f9f4116e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d06af53cf0 con 0x55d06b05b5e0 2024-03-29T04:11:59.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.672+0000 7f9f37fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d06b05b200 0x55d06b049490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.672+0000 7f9f37fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d06b05b200 0x55d06b049490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40854/0 (socket says 172.21.15.148:40854) 2024-03-29T04:11:59.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.672+0000 7f9f37fff640 1 -- 172.21.15.148:0/3281239625 learned_addr learned my addr 172.21.15.148:0/3281239625 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:59.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.672+0000 7f9f4116e740 1 -- 172.21.15.148:0/3281239625 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55d06b0352f0 con 0x55d06b089b50 2024-03-29T04:11:59.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.672+0000 7f9f377fe640 1 --2- 172.21.15.148:0/3281239625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d06b05b5e0 0x55d06b05cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.673+0000 7f9f3ca42640 1 --2- 172.21.15.148:0/3281239625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d06b089b50 0x55d06b05d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.673+0000 7f9f4116e740 10 client.0 _ll_register_callbacks cb 0x55d06af45f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:11:59.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.673+0000 7f9f377fe640 1 -- 172.21.15.148:0/3281239625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d06b089b50 msgr2=0x55d06b05d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:59.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.673+0000 7f9f377fe640 1 --2- 172.21.15.148:0/3281239625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d06b089b50 0x55d06b05d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.673+0000 7f9f377fe640 1 -- 172.21.15.148:0/3281239625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d06b05b200 msgr2=0x55d06b049490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:59.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.673+0000 7f9f377fe640 1 --2- 172.21.15.148:0/3281239625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d06b05b200 0x55d06b049490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.673+0000 7f9f377fe640 1 -- 172.21.15.148:0/3281239625 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d06ad767d0 con 0x55d06b05b5e0 2024-03-29T04:11:59.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.673+0000 7f9f377fe640 1 --2- 172.21.15.148:0/3281239625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d06b05b5e0 0x55d06b05cd50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c054eb0 tx=0x7f9f2c01f040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=605c5f9f9fa3edb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:59.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.673+0000 7f9f16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-03-29T04:11:59.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.673+0000 7f9f16ffd640 1 -- 172.21.15.148:0/3281239625 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c004bc0 con 0x55d06b05b5e0 2024-03-29T04:11:59.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.673+0000 7f9f16ffd640 1 -- 172.21.15.148:0/3281239625 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f9f2c004d60 con 0x55d06b05b5e0 2024-03-29T04:11:59.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.673+0000 7f9f4116e740 1 -- 172.21.15.148:0/3281239625 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f2c05f070 con 0x55d06b05b5e0 2024-03-29T04:11:59.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.673+0000 7f9f16ffd640 1 -- 172.21.15.148:0/3281239625 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f9f2c075050 con 0x55d06b05b5e0 2024-03-29T04:11:59.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.673+0000 7f9f4116e740 10 client.5464 fetch_fsmap learned FSMap version 19 2024-03-29T04:11:59.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.673+0000 7f9f4116e740 1 -- 172.21.15.148:0/3281239625 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55d06acac3f0 con 0x55d06b05b5e0 2024-03-29T04:11:59.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.674+0000 7f9f16ffd640 1 -- 172.21.15.148:0/3281239625 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f2c05f070 con 0x55d06b05b5e0 2024-03-29T04:11:59.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.674+0000 7f9f16ffd640 1 -- 172.21.15.148:0/3281239625 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f9f2c057ea0 con 0x55d06b05b5e0 2024-03-29T04:11:59.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.674+0000 7f9f4116e740 10 client.5464 fetch_fsmap finished waiting for FSMap version 19 2024-03-29T04:11:59.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.674+0000 7f9f4116e740 10 client.5464 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-29T04:11:59.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.674+0000 7f9f4116e740 20 client.5464 populate_metadata read hostname 'smithi148' 2024-03-29T04:11:59.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.674+0000 7f9f4116e740 1 --2- 172.21.15.148:0/3281239625 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x55d06b079330 0x55d06b076b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.674+0000 7f9f4116e740 4 client.5464 mds_command: new command op to 5583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T04:11:59.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.674+0000 7f9f4116e740 1 -- 172.21.15.148:0/3281239625 --> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d06afa5dc0 con 0x55d06b079330 2024-03-29T04:11:59.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.674+0000 7f9f37fff640 1 --2- 172.21.15.148:0/3281239625 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x55d06b079330 0x55d06b076b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.675+0000 7f9f37fff640 1 --2- 172.21.15.148:0/3281239625 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x55d06b079330 0x55d06b076b40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83223de76d956bd3 server_cookie=d8322bf62b034702 in_seq=0 out_seq=0 2024-03-29T04:11:59.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.675+0000 7f9f16ffd640 10 client.5464 ms_handle_connect on v2:172.21.15.148:6836/473448013 2024-03-29T04:11:59.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.677+0000 7f9f16ffd640 1 -- 172.21.15.148:0/3281239625 <== mds.0 v2:172.21.15.148:6836/473448013 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55d06afa5dc0 con 0x55d06b079330 2024-03-29T04:11:59.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.677+0000 7f9f16ffd640 10 client.5464 handle_command_reply: tid=0 2024-03-29T04:11:59.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.677+0000 7f9f4116e740 1 client.5464 shutdown 2024-03-29T04:11:59.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.678+0000 7f9f4116e740 1 -- 172.21.15.148:0/3281239625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d06b05b5e0 msgr2=0x55d06b05cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:59.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.678+0000 7f9f4116e740 1 --2- 172.21.15.148:0/3281239625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d06b05b5e0 0x55d06b05cd50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c054eb0 tx=0x7f9f2c01f040 comp rx=0 tx=0).stop 2024-03-29T04:11:59.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.678+0000 7f9f4116e740 1 -- 172.21.15.148:0/3281239625 shutdown_connections 2024-03-29T04:11:59.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.678+0000 7f9f4116e740 1 --2- 172.21.15.148:0/3281239625 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x55d06b079330 0x55d06b076b40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.678+0000 7f9f4116e740 1 --2- 172.21.15.148:0/3281239625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d06b089b50 0x55d06b05d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.678+0000 7f9f4116e740 1 --2- 172.21.15.148:0/3281239625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d06b05b5e0 0x55d06b05cd50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.678+0000 7f9f4116e740 1 --2- 172.21.15.148:0/3281239625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d06b05b200 0x55d06b049490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.678+0000 7f9f16ffd640 0 client.5464 ms_handle_reset on v2:172.21.15.148:6836/473448013 2024-03-29T04:11:59.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.678+0000 7f9f4116e740 1 -- 172.21.15.148:0/3281239625 >> 172.21.15.148:0/3281239625 conn(0x55d06b02e6e0 msgr2=0x55d06b0320b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:59.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.678+0000 7f9f4116e740 1 -- 172.21.15.148:0/3281239625 shutdown_connections 2024-03-29T04:11:59.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.679+0000 7f9f4116e740 1 -- 172.21.15.148:0/3281239625 wait complete. 2024-03-29T04:11:59.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.679+0000 7f9f4116e740 20 client.5464 trim_cache size 0 max 16384 2024-03-29T04:11:59.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.702+0000 7f9f4116e740 1 Processor -- start 2024-03-29T04:11:59.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.702+0000 7f9f4116e740 1 -- start start 2024-03-29T04:11:59.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.703+0000 7f9f4116e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d06b05b200 0x55d06b05cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.703+0000 7f9f4116e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d06b05b5e0 0x55d06b07c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.703+0000 7f9f4116e740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d06b079330 0x55d06b07c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.703+0000 7f9f4116e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9f38200db0 con 0x55d06b05b200 2024-03-29T04:11:59.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.703+0000 7f9f4116e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9f381ffdd0 con 0x55d06b05b5e0 2024-03-29T04:11:59.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.703+0000 7f9f4116e740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9f380cdae0 con 0x55d06b079330 2024-03-29T04:11:59.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.703+0000 7f9f37fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d06b05b200 0x55d06b05cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.703+0000 7f9f37fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d06b05b200 0x55d06b05cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40866/0 (socket says 172.21.15.148:40866) 2024-03-29T04:11:59.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.703+0000 7f9f37fff640 1 -- 172.21.15.148:0/2200770707 learned_addr learned my addr 172.21.15.148:0/2200770707 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:59.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.703+0000 7f9f3ca42640 1 --2- 172.21.15.148:0/2200770707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d06b079330 0x55d06b07c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.703+0000 7f9f377fe640 1 --2- 172.21.15.148:0/2200770707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d06b05b5e0 0x55d06b07c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.703+0000 7f9f37fff640 1 -- 172.21.15.148:0/2200770707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d06b079330 msgr2=0x55d06b07c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:59.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.703+0000 7f9f37fff640 1 --2- 172.21.15.148:0/2200770707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d06b079330 0x55d06b07c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.703+0000 7f9f37fff640 1 -- 172.21.15.148:0/2200770707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d06b05b5e0 msgr2=0x55d06b07c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:59.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.703+0000 7f9f37fff640 1 --2- 172.21.15.148:0/2200770707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d06b05b5e0 0x55d06b07c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.703+0000 7f9f37fff640 1 -- 172.21.15.148:0/2200770707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d06ac6c0f0 con 0x55d06b05b200 2024-03-29T04:11:59.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.703+0000 7f9f37fff640 1 --2- 172.21.15.148:0/2200770707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d06b05b200 0x55d06b05cc60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800fbc0 tx=0x7f9f28012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f0d43fb63c7cc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:59.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.703+0000 7f9f1d7fa640 1 -- 172.21.15.148:0/2200770707 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2802ab40 con 0x55d06b05b200 2024-03-29T04:11:59.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.703+0000 7f9f1d7fa640 1 -- 172.21.15.148:0/2200770707 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f2801b040 con 0x55d06b05b200 2024-03-29T04:11:59.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.703+0000 7f9f1d7fa640 1 -- 172.21.15.148:0/2200770707 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2802ad00 con 0x55d06b05b200 2024-03-29T04:11:59.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.704+0000 7f9f4116e740 1 -- 172.21.15.148:0/2200770707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d06b05b200 msgr2=0x55d06b05cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:59.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.704+0000 7f9f4116e740 1 --2- 172.21.15.148:0/2200770707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d06b05b200 0x55d06b05cc60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800fbc0 tx=0x7f9f28012000 comp rx=0 tx=0).stop 2024-03-29T04:11:59.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.704+0000 7f9f4116e740 1 -- 172.21.15.148:0/2200770707 shutdown_connections 2024-03-29T04:11:59.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.704+0000 7f9f4116e740 1 --2- 172.21.15.148:0/2200770707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d06b079330 0x55d06b07c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.704+0000 7f9f4116e740 1 --2- 172.21.15.148:0/2200770707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d06b05b5e0 0x55d06b07c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.704+0000 7f9f4116e740 1 --2- 172.21.15.148:0/2200770707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d06b05b200 0x55d06b05cc60 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.704+0000 7f9f4116e740 1 -- 172.21.15.148:0/2200770707 >> 172.21.15.148:0/2200770707 conn(0x55d06b02e6e0 msgr2=0x55d06b05ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:59.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.704+0000 7f9f4116e740 1 -- 172.21.15.148:0/2200770707 shutdown_connections 2024-03-29T04:11:59.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.704+0000 7f9f4116e740 1 -- 172.21.15.148:0/2200770707 wait complete. 2024-03-29T04:11:59.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.705+0000 7f9f4116e740 1 -- start start 2024-03-29T04:11:59.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.705+0000 7f9f4116e740 1 Processor -- start 2024-03-29T04:11:59.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.706+0000 7f9f4116e740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d06b05b200 0x55d06b07ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.706+0000 7f9f4116e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d06b05b5e0 0x55d06b07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.706+0000 7f9f4116e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d06b079330 0x55d06b0478e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.706+0000 7f9f4116e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d06b02fcb0 con 0x55d06b079330 2024-03-29T04:11:59.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.706+0000 7f9f4116e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9f380ce130 con 0x55d06b05b5e0 2024-03-29T04:11:59.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.706+0000 7f9f4116e740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55d06b08cb60 con 0x55d06b05b200 2024-03-29T04:11:59.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.706+0000 7f9f4116e740 10 client.0 _ll_register_callbacks cb 0x7f9f2c015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:11:59.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.706+0000 7f9f3ca42640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d06b079330 0x55d06b0478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.706+0000 7f9f3ca42640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d06b079330 0x55d06b0478e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40880/0 (socket says 172.21.15.148:40880) 2024-03-29T04:11:59.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.706+0000 7f9f3ca42640 1 -- 172.21.15.148:0/3353621195 learned_addr learned my addr 172.21.15.148:0/3353621195 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:11:59.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.706+0000 7f9f377fe640 1 --2- 172.21.15.148:0/3353621195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d06b05b5e0 0x55d06b07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.706+0000 7f9f37fff640 1 --2- 172.21.15.148:0/3353621195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d06b05b200 0x55d06b07ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.706+0000 7f9f3ca42640 1 -- 172.21.15.148:0/3353621195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d06b05b200 msgr2=0x55d06b07ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:59.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.706+0000 7f9f3ca42640 1 --2- 172.21.15.148:0/3353621195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d06b05b200 0x55d06b07ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.706+0000 7f9f3ca42640 1 -- 172.21.15.148:0/3353621195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d06b05b5e0 msgr2=0x55d06b07b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:59.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.706+0000 7f9f3ca42640 1 --2- 172.21.15.148:0/3353621195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d06b05b5e0 0x55d06b07b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.706+0000 7f9f3ca42640 1 -- 172.21.15.148:0/3353621195 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d06b048520 con 0x55d06b079330 2024-03-29T04:11:59.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.706+0000 7f9f377fe640 1 --2- 172.21.15.148:0/3353621195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d06b05b5e0 0x55d06b07b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:11:59.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.706+0000 7f9f3ca42640 1 --2- 172.21.15.148:0/3353621195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d06b079330 0x55d06b0478e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9f30009e80 tx=0x7f9f30004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91e5b176b6615f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:59.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.706+0000 7f9f16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-03-29T04:11:59.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.707+0000 7f9f16ffd640 1 -- 172.21.15.148:0/3353621195 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30039040 con 0x55d06b079330 2024-03-29T04:11:59.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.707+0000 7f9f16ffd640 1 -- 172.21.15.148:0/3353621195 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f9f30041650 con 0x55d06b079330 2024-03-29T04:11:59.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.707+0000 7f9f4116e740 1 -- 172.21.15.148:0/3353621195 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f2801b040 con 0x55d06b079330 2024-03-29T04:11:59.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.707+0000 7f9f16ffd640 1 -- 172.21.15.148:0/3353621195 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f9f30054050 con 0x55d06b079330 2024-03-29T04:11:59.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.707+0000 7f9f4116e740 10 client.5853 fetch_fsmap learned FSMap version 19 2024-03-29T04:11:59.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.707+0000 7f9f4116e740 1 -- 172.21.15.148:0/3353621195 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55d06b05e4d0 con 0x55d06b079330 2024-03-29T04:11:59.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.707+0000 7f9f16ffd640 1 -- 172.21.15.148:0/3353621195 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f2801b040 con 0x55d06b079330 2024-03-29T04:11:59.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.707+0000 7f9f16ffd640 1 -- 172.21.15.148:0/3353621195 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f9f30057020 con 0x55d06b079330 2024-03-29T04:11:59.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.707+0000 7f9f4116e740 10 client.5853 fetch_fsmap finished waiting for FSMap version 19 2024-03-29T04:11:59.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.707+0000 7f9f4116e740 10 client.5853 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-29T04:11:59.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.707+0000 7f9f4116e740 20 client.5853 populate_metadata read hostname 'smithi148' 2024-03-29T04:11:59.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.707+0000 7f9f4116e740 1 --2- 172.21.15.148:0/3353621195 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x55d06b089b50 0x55d06b047e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:59.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.707+0000 7f9f4116e740 4 client.5853 mds_command: new command op to 5583 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-29T04:11:59.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.707+0000 7f9f4116e740 1 -- 172.21.15.148:0/3353621195 --> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d06afa0110 con 0x55d06b089b50 2024-03-29T04:11:59.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.707+0000 7f9f37fff640 1 --2- 172.21.15.148:0/3353621195 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x55d06b089b50 0x55d06b047e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:59.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.707+0000 7f9f16ffd640 10 client.5853 ms_handle_connect on v2:172.21.15.148:6836/473448013 2024-03-29T04:11:59.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.707+0000 7f9f37fff640 1 --2- 172.21.15.148:0/3353621195 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x55d06b089b50 0x55d06b047e20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ca8dc97b1b3f82d server_cookie=133d3ebce6cf2f05 in_seq=0 out_seq=0 2024-03-29T04:11:59.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.708+0000 7f9f16ffd640 1 -- 172.21.15.148:0/3353621195 <== mds.0 v2:172.21.15.148:6836/473448013 1 ==== command_reply(tid 0: 0 ) ==== 8+0+564 (crc 0 0 0) 0x55d06afa0110 con 0x55d06b089b50 2024-03-29T04:11:59.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.708+0000 7f9f16ffd640 10 client.5853 handle_command_reply: tid=0 2024-03-29T04:11:59.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.708+0000 7f9f4116e740 1 client.5853 shutdown 2024-03-29T04:11:59.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.708+0000 7f9f4116e740 1 -- 172.21.15.148:0/3353621195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d06b079330 msgr2=0x55d06b0478e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:59.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.708+0000 7f9f4116e740 1 --2- 172.21.15.148:0/3353621195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d06b079330 0x55d06b0478e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9f30009e80 tx=0x7f9f30004ab0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.708+0000 7f9f4116e740 1 -- 172.21.15.148:0/3353621195 shutdown_connections 2024-03-29T04:11:59.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.708+0000 7f9f4116e740 1 --2- 172.21.15.148:0/3353621195 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x55d06b089b50 0x55d06b047e20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.708+0000 7f9f4116e740 1 --2- 172.21.15.148:0/3353621195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d06b079330 0x55d06b0478e0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.708+0000 7f9f4116e740 1 --2- 172.21.15.148:0/3353621195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d06b05b5e0 0x55d06b07b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.708+0000 7f9f4116e740 1 --2- 172.21.15.148:0/3353621195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d06b05b200 0x55d06b07ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.708+0000 7f9f4116e740 1 -- 172.21.15.148:0/3353621195 >> 172.21.15.148:0/3353621195 conn(0x55d06b02e6e0 msgr2=0x55d06b07c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:59.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.708+0000 7f9f16ffd640 0 client.5853 ms_handle_reset on v2:172.21.15.148:6836/473448013 2024-03-29T04:11:59.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.708+0000 7f9f4116e740 1 -- 172.21.15.148:0/3353621195 shutdown_connections 2024-03-29T04:11:59.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.708+0000 7f9f4116e740 1 -- 172.21.15.148:0/3353621195 wait complete. 2024-03-29T04:11:59.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.708+0000 7f9f4116e740 20 client.5853 trim_cache size 0 max 16384 2024-03-29T04:11:59.694 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-03-29T04:11:59.694 INFO:teuthology.orchestra.run.smithi148.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-29T04:11:59.694 INFO:teuthology.orchestra.run.smithi148.stdout: "scrubs": { 2024-03-29T04:11:59.694 INFO:teuthology.orchestra.run.smithi148.stdout: "713fb6cc-5d83-4906-8a07-c0c0f0760e64": { 2024-03-29T04:11:59.694 INFO:teuthology.orchestra.run.smithi148.stdout: "path": "/", 2024-03-29T04:11:59.694 INFO:teuthology.orchestra.run.smithi148.stdout: "tag": "713fb6cc-5d83-4906-8a07-c0c0f0760e64", 2024-03-29T04:11:59.694 INFO:teuthology.orchestra.run.smithi148.stdout: "options": "recursive" 2024-03-29T04:11:59.694 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:59.694 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T04:11:59.694 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_stats": { 2024-03-29T04:11:59.695 INFO:teuthology.orchestra.run.smithi148.stdout: "713fb6cc-5d83-4906-8a07-c0c0f0760e64": { 2024-03-29T04:11:59.695 INFO:teuthology.orchestra.run.smithi148.stdout: "start_time": "2024-03-29T04:11:59.492927+0000", 2024-03-29T04:11:59.695 INFO:teuthology.orchestra.run.smithi148.stdout: "path": "/", 2024-03-29T04:11:59.695 INFO:teuthology.orchestra.run.smithi148.stdout: "uninline_started": 0, 2024-03-29T04:11:59.695 INFO:teuthology.orchestra.run.smithi148.stdout: "uninline_passed": 0, 2024-03-29T04:11:59.695 INFO:teuthology.orchestra.run.smithi148.stdout: "uninline_failed": 0, 2024-03-29T04:11:59.695 INFO:teuthology.orchestra.run.smithi148.stdout: "uninline_skipped": 0 2024-03-29T04:11:59.695 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:59.695 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:11:59.695 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-03-29T04:11:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.709+0000 7f9f3e4a5640 1 -- 172.21.15.148:0/3677336393 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9f2406c530 msgr2=0x7f9f2406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.709+0000 7f9f3e4a5640 1 --2- 172.21.15.148:0/3677336393 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9f2406c530 0x7f9f2406e9f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9f280011e0 tx=0x7f9f28009040 comp rx=0 tx=0).stop 2024-03-29T04:11:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.709+0000 7f9f3e4a5640 1 -- 172.21.15.148:0/3677336393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f3815e3c0 msgr2=0x7f9f38151840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.709+0000 7f9f3e4a5640 1 --2- 172.21.15.148:0/3677336393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f3815e3c0 0x7f9f38151840 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00c550 tx=0x7f9f2c00ca10 comp rx=0 tx=0).stop 2024-03-29T04:11:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.709+0000 7f9f3e4a5640 1 -- 172.21.15.148:0/3677336393 shutdown_connections 2024-03-29T04:11:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.709+0000 7f9f3e4a5640 1 --2- 172.21.15.148:0/3677336393 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9f2406c530 0x7f9f2406e9f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.709+0000 7f9f3e4a5640 1 --2- 172.21.15.148:0/3677336393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f38152010 0x7f9f380b8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.709+0000 7f9f3e4a5640 1 --2- 172.21.15.148:0/3677336393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f3815e3c0 0x7f9f38151840 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.709+0000 7f9f3e4a5640 1 --2- 172.21.15.148:0/3677336393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f380b3430 0x7f9f380cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:59.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.709+0000 7f9f3e4a5640 1 -- 172.21.15.148:0/3677336393 >> 172.21.15.148:0/3677336393 conn(0x7f9f380b6430 msgr2=0x7f9f38162450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:59.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.709+0000 7f9f3e4a5640 1 -- 172.21.15.148:0/3677336393 shutdown_connections 2024-03-29T04:11:59.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:11:59.709+0000 7f9f3e4a5640 1 -- 172.21.15.148:0/3677336393 wait complete. 2024-03-29T04:11:59.699 INFO:tasks.cephfs.filesystem:scrub status for tag:713fb6cc-5d83-4906-8a07-c0c0f0760e64 - {'path': '/', 'tag': '713fb6cc-5d83-4906-8a07-c0c0f0760e64', 'options': 'recursive'} 2024-03-29T04:12:01.859 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:12:01.862 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:12:29.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:12:29.700 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-03-29T04:12:29.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.825+0000 7f8380474640 1 Processor -- start 2024-03-29T04:12:29.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.825+0000 7f8380474640 1 -- start start 2024-03-29T04:12:29.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.826+0000 7f8380474640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f83780cfda0 0x7f8378166150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:29.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.826+0000 7f8380474640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83780cc660 con 0x7f8378144980 2024-03-29T04:12:29.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.826+0000 7f8380474640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83780cc940 con 0x7f83780cc0f0 2024-03-29T04:12:29.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.826+0000 7f8380474640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f83780701c0 con 0x7f83780cfda0 2024-03-29T04:12:29.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.826+0000 7f837ea11640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f83780cfda0 0x7f8378166150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:29.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.826+0000 7f837e210640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8378144980 0x7f837806fab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39334/0 (socket says 172.21.15.148:39334) 2024-03-29T04:12:29.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.826+0000 7f837e210640 1 -- 172.21.15.148:0/497095194 learned_addr learned my addr 172.21.15.148:0/497095194 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:29.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.826+0000 7f837ea11640 1 -- 172.21.15.148:0/497095194 >> v1:172.21.15.157:6789/0 conn(0x7f83780cc0f0 legacy=0x7f83780cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:29.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.826+0000 7f837d20e640 1 -- 172.21.15.148:0/497095194 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4288522432 0 0) 0x7f83780cc660 con 0x7f8378144980 2024-03-29T04:12:29.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.826+0000 7f837ea11640 1 -- 172.21.15.148:0/497095194 >> v1:172.21.15.148:6789/0 conn(0x7f8378144980 legacy=0x7f837806fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:29.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.826+0000 7f837ea11640 1 -- 172.21.15.148:0/497095194 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83780ccc20 con 0x7f83780cfda0 2024-03-29T04:12:29.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.827+0000 7f837ea11640 1 --2- 172.21.15.148:0/497095194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83780cfda0 0x7f8378166150 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8370009d90 tx=0x7f837002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e1dabed23a54da server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:29.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.827+0000 7f837d20e640 1 -- 172.21.15.148:0/497095194 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370039070 con 0x7f83780cfda0 2024-03-29T04:12:29.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.827+0000 7f837d20e640 1 -- 172.21.15.148:0/497095194 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f837002cd00 con 0x7f83780cfda0 2024-03-29T04:12:29.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.827+0000 7f837d20e640 1 -- 172.21.15.148:0/497095194 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370034660 con 0x7f83780cfda0 2024-03-29T04:12:29.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.828+0000 7f8380474640 1 -- 172.21.15.148:0/497095194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83780cfda0 msgr2=0x7f8378166150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:29.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.828+0000 7f8380474640 1 --2- 172.21.15.148:0/497095194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83780cfda0 0x7f8378166150 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8370009d90 tx=0x7f837002bf50 comp rx=0 tx=0).stop 2024-03-29T04:12:29.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.828+0000 7f8380474640 1 -- 172.21.15.148:0/497095194 shutdown_connections 2024-03-29T04:12:29.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.828+0000 7f8380474640 1 --2- 172.21.15.148:0/497095194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83780cfda0 0x7f8378166150 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.828+0000 7f8380474640 1 -- 172.21.15.148:0/497095194 >> 172.21.15.148:0/497095194 conn(0x7f83780b2c60 msgr2=0x7f83780b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:29.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.828+0000 7f8380474640 1 -- 172.21.15.148:0/497095194 shutdown_connections 2024-03-29T04:12:29.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.828+0000 7f8380474640 1 -- 172.21.15.148:0/497095194 wait complete. 2024-03-29T04:12:29.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.829+0000 7f8380474640 1 Processor -- start 2024-03-29T04:12:29.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.829+0000 7f8380474640 1 -- start start 2024-03-29T04:12:29.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.830+0000 7f8380474640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83780cc0f0 0x7f837815d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:29.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.830+0000 7f8380474640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83780cfda0 0x7f837813ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:29.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.830+0000 7f8380474640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8378144980 0x7f8378141f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:29.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.830+0000 7f8380474640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f83780cd270 con 0x7f83780cfda0 2024-03-29T04:12:29.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.830+0000 7f8380474640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8378144d60 con 0x7f8378144980 2024-03-29T04:12:29.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.830+0000 7f8380474640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f83780cd700 con 0x7f83780cc0f0 2024-03-29T04:12:29.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.830+0000 7f837da0f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83780cfda0 0x7f837813ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:29.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.830+0000 7f837da0f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83780cfda0 0x7f837813ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36198/0 (socket says 172.21.15.148:36198) 2024-03-29T04:12:29.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.830+0000 7f837da0f640 1 -- 172.21.15.148:0/3678128194 learned_addr learned my addr 172.21.15.148:0/3678128194 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:29.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.830+0000 7f837ea11640 1 --2- 172.21.15.148:0/3678128194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8378144980 0x7f8378141f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:29.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.830+0000 7f837e210640 1 --2- 172.21.15.148:0/3678128194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83780cc0f0 0x7f837815d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:29.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.830+0000 7f837da0f640 1 -- 172.21.15.148:0/3678128194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83780cc0f0 msgr2=0x7f837815d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:29.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.830+0000 7f837da0f640 1 --2- 172.21.15.148:0/3678128194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83780cc0f0 0x7f837815d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.830+0000 7f837da0f640 1 -- 172.21.15.148:0/3678128194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8378144980 msgr2=0x7f8378141f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:29.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.830+0000 7f837da0f640 1 --2- 172.21.15.148:0/3678128194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8378144980 0x7f8378141f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.830+0000 7f837da0f640 1 -- 172.21.15.148:0/3678128194 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f837815c6a0 con 0x7f83780cfda0 2024-03-29T04:12:29.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.830+0000 7f837da0f640 1 --2- 172.21.15.148:0/3678128194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83780cfda0 0x7f837813ecd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f836800c480 tx=0x7f836800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:29.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.830+0000 7f836effd640 1 -- 172.21.15.148:0/3678128194 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83680052d0 con 0x7f83780cfda0 2024-03-29T04:12:29.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.831+0000 7f836effd640 1 -- 172.21.15.148:0/3678128194 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8368005470 con 0x7f83780cfda0 2024-03-29T04:12:29.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.831+0000 7f8380474640 1 -- 172.21.15.148:0/3678128194 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f837815c980 con 0x7f83780cfda0 2024-03-29T04:12:29.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.831+0000 7f836effd640 1 -- 172.21.15.148:0/3678128194 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368012870 con 0x7f83780cfda0 2024-03-29T04:12:29.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.831+0000 7f8380474640 1 -- 172.21.15.148:0/3678128194 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8378142480 con 0x7f83780cfda0 2024-03-29T04:12:29.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.832+0000 7f836effd640 1 -- 172.21.15.148:0/3678128194 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8368012a10 con 0x7f83780cfda0 2024-03-29T04:12:29.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.833+0000 7f836effd640 1 --2- 172.21.15.148:0/3678128194 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f835c06c530 0x7f835c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:29.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.833+0000 7f836effd640 1 -- 172.21.15.148:0/3678128194 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f836808b4f0 con 0x7f83780cfda0 2024-03-29T04:12:29.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.833+0000 7f837e210640 1 --2- 172.21.15.148:0/3678128194 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f835c06c530 0x7f835c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:29.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.833+0000 7f837e210640 1 --2- 172.21.15.148:0/3678128194 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f835c06c530 0x7f835c06e9f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8374000f60 tx=0x7f8374009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:29.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.834+0000 7f8383065740 1 Processor -- start 2024-03-29T04:12:29.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.834+0000 7f8383065740 1 -- start start 2024-03-29T04:12:29.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.834+0000 7f8383065740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615ed673200 0x5615ed6461a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:29.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.834+0000 7f8383065740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615ed6735e0 0x5615ed675f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:29.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.834+0000 7f8383065740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5615ed6466e0 0x5615ed6764d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:29.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.834+0000 7f8383065740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5615ed5b0880 con 0x5615ed673200 2024-03-29T04:12:29.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.834+0000 7f8383065740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5615ed5588f0 con 0x5615ed6735e0 2024-03-29T04:12:29.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.834+0000 7f8383065740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x5615ed5ac830 con 0x5615ed6466e0 2024-03-29T04:12:29.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.835+0000 7f837e210640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615ed673200 0x5615ed6461a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:29.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.835+0000 7f837e210640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615ed673200 0x5615ed6461a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36200/0 (socket says 172.21.15.148:36200) 2024-03-29T04:12:29.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.835+0000 7f837e210640 1 -- 172.21.15.148:0/3804829740 learned_addr learned my addr 172.21.15.148:0/3804829740 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:29.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.835+0000 7f837da0f640 1 --2- 172.21.15.148:0/3804829740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615ed6735e0 0x5615ed675f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:29.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.835+0000 7f837ea11640 1 --2- 172.21.15.148:0/3804829740 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5615ed6466e0 0x5615ed6764d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:29.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.835+0000 7f837e210640 1 -- 172.21.15.148:0/3804829740 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5615ed6466e0 msgr2=0x5615ed6764d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:29.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.835+0000 7f837e210640 1 --2- 172.21.15.148:0/3804829740 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5615ed6466e0 0x5615ed6764d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.835+0000 7f837e210640 1 -- 172.21.15.148:0/3804829740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615ed6735e0 msgr2=0x5615ed675f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:29.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.835+0000 7f837e210640 1 --2- 172.21.15.148:0/3804829740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615ed6735e0 0x5615ed675f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.835+0000 7f837e210640 1 -- 172.21.15.148:0/3804829740 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615ed563140 con 0x5615ed673200 2024-03-29T04:12:29.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.835+0000 7f837e210640 1 --2- 172.21.15.148:0/3804829740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615ed673200 0x5615ed6461a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8374007500 tx=0x7f8374010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38843e4c024e69b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:29.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.835+0000 7f835bfff640 1 -- 172.21.15.148:0/3804829740 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8374005c80 con 0x5615ed673200 2024-03-29T04:12:29.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.835+0000 7f835bfff640 1 -- 172.21.15.148:0/3804829740 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8374005e20 con 0x5615ed673200 2024-03-29T04:12:29.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.835+0000 7f835bfff640 1 -- 172.21.15.148:0/3804829740 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8374014bd0 con 0x5615ed673200 2024-03-29T04:12:29.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.836+0000 7f8383065740 1 -- 172.21.15.148:0/3804829740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615ed673200 msgr2=0x5615ed6461a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:29.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.836+0000 7f8383065740 1 --2- 172.21.15.148:0/3804829740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615ed673200 0x5615ed6461a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8374007500 tx=0x7f8374010000 comp rx=0 tx=0).stop 2024-03-29T04:12:29.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.836+0000 7f8383065740 1 -- 172.21.15.148:0/3804829740 shutdown_connections 2024-03-29T04:12:29.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.836+0000 7f8383065740 1 --2- 172.21.15.148:0/3804829740 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5615ed6466e0 0x5615ed6764d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.836+0000 7f8383065740 1 --2- 172.21.15.148:0/3804829740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615ed6735e0 0x5615ed675f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.836+0000 7f8383065740 1 --2- 172.21.15.148:0/3804829740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615ed673200 0x5615ed6461a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.836+0000 7f8383065740 1 -- 172.21.15.148:0/3804829740 >> 172.21.15.148:0/3804829740 conn(0x5615ed648410 msgr2=0x5615ed690f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:29.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.836+0000 7f8383065740 1 -- 172.21.15.148:0/3804829740 shutdown_connections 2024-03-29T04:12:29.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.836+0000 7f8383065740 1 -- 172.21.15.148:0/3804829740 wait complete. 2024-03-29T04:12:29.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.837+0000 7f8383065740 1 -- start start 2024-03-29T04:12:29.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.837+0000 7f8383065740 1 Processor -- start 2024-03-29T04:12:29.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.838+0000 7f8383065740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615ed673200 0x5615ed6a4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:29.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.838+0000 7f8383065740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5615ed6735e0 0x5615ed6a20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:29.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.838+0000 7f8383065740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615ed679820 0x5615ed679ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:29.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.838+0000 7f8383065740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5615ed64aab0 con 0x5615ed673200 2024-03-29T04:12:29.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.838+0000 7f8383065740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5615ed56bcf0 con 0x5615ed679820 2024-03-29T04:12:29.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.838+0000 7f8383065740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x5615ed64d2f0 con 0x5615ed6735e0 2024-03-29T04:12:29.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.838+0000 7f8383065740 10 client.0 _ll_register_callbacks cb 0x5615ed55df00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:12:29.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.838+0000 7f837e210640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615ed673200 0x5615ed6a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:29.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.838+0000 7f837e210640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615ed673200 0x5615ed6a4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36212/0 (socket says 172.21.15.148:36212) 2024-03-29T04:12:29.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.838+0000 7f837e210640 1 -- 172.21.15.148:0/3221782431 learned_addr learned my addr 172.21.15.148:0/3221782431 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:29.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.838+0000 7f837da0f640 1 --2- 172.21.15.148:0/3221782431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5615ed6735e0 0x5615ed6a20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:29.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.838+0000 7f837ea11640 1 --2- 172.21.15.148:0/3221782431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615ed679820 0x5615ed679ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:29.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.838+0000 7f837e210640 1 -- 172.21.15.148:0/3221782431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5615ed6735e0 msgr2=0x5615ed6a20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:29.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.838+0000 7f837e210640 1 --2- 172.21.15.148:0/3221782431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5615ed6735e0 0x5615ed6a20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.838+0000 7f837e210640 1 -- 172.21.15.148:0/3221782431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615ed679820 msgr2=0x5615ed679ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:29.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.838+0000 7f837e210640 1 --2- 172.21.15.148:0/3221782431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615ed679820 0x5615ed679ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.838+0000 7f837e210640 1 -- 172.21.15.148:0/3221782431 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615ed38e7d0 con 0x5615ed673200 2024-03-29T04:12:29.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.838+0000 7f837e210640 1 --2- 172.21.15.148:0/3221782431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615ed673200 0x5615ed6a4560 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8374014be0 tx=0x7f8374007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98932cdb6a81b1ff server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:29.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.838+0000 7f83597fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-03-29T04:12:29.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.838+0000 7f83597fa640 1 -- 172.21.15.148:0/3221782431 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8374020450 con 0x5615ed673200 2024-03-29T04:12:29.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.838+0000 7f83597fa640 1 -- 172.21.15.148:0/3221782431 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f83740205f0 con 0x5615ed673200 2024-03-29T04:12:29.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.838+0000 7f8383065740 1 -- 172.21.15.148:0/3221782431 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8374005e20 con 0x5615ed673200 2024-03-29T04:12:29.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.839+0000 7f83597fa640 1 -- 172.21.15.148:0/3221782431 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f8374035050 con 0x5615ed673200 2024-03-29T04:12:29.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.839+0000 7f8383065740 10 client.5868 fetch_fsmap learned FSMap version 19 2024-03-29T04:12:29.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.839+0000 7f8383065740 1 -- 172.21.15.148:0/3221782431 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x5615ed2c43f0 con 0x5615ed673200 2024-03-29T04:12:29.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.839+0000 7f83597fa640 1 -- 172.21.15.148:0/3221782431 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8374005e20 con 0x5615ed673200 2024-03-29T04:12:29.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.839+0000 7f83597fa640 1 -- 172.21.15.148:0/3221782431 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f8374038020 con 0x5615ed673200 2024-03-29T04:12:29.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.839+0000 7f8383065740 10 client.5868 fetch_fsmap finished waiting for FSMap version 19 2024-03-29T04:12:29.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.839+0000 7f8383065740 10 client.5868 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-29T04:12:29.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.839+0000 7f8383065740 20 client.5868 populate_metadata read hostname 'smithi148' 2024-03-29T04:12:29.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.839+0000 7f8383065740 1 --2- 172.21.15.148:0/3221782431 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x5615ed6a2830 0x5615ed6a5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:29.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.839+0000 7f8383065740 4 client.5868 mds_command: new command op to 5583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T04:12:29.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.839+0000 7f8383065740 1 -- 172.21.15.148:0/3221782431 --> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615ed5bddc0 con 0x5615ed6a2830 2024-03-29T04:12:29.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.839+0000 7f837da0f640 1 --2- 172.21.15.148:0/3221782431 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x5615ed6a2830 0x5615ed6a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:29.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.839+0000 7f83597fa640 10 client.5868 ms_handle_connect on v2:172.21.15.148:6836/473448013 2024-03-29T04:12:29.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.839+0000 7f837da0f640 1 --2- 172.21.15.148:0/3221782431 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x5615ed6a2830 0x5615ed6a5130 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4739bc352200d5fa server_cookie=2d118b3901511b01 in_seq=0 out_seq=0 2024-03-29T04:12:29.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.840+0000 7f83597fa640 1 -- 172.21.15.148:0/3221782431 <== mds.0 v2:172.21.15.148:6836/473448013 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x5615ed5bddc0 con 0x5615ed6a2830 2024-03-29T04:12:29.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.840+0000 7f83597fa640 10 client.5868 handle_command_reply: tid=0 2024-03-29T04:12:29.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.840+0000 7f8383065740 1 client.5868 shutdown 2024-03-29T04:12:29.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.840+0000 7f8383065740 1 -- 172.21.15.148:0/3221782431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615ed673200 msgr2=0x5615ed6a4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:29.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.840+0000 7f8383065740 1 --2- 172.21.15.148:0/3221782431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615ed673200 0x5615ed6a4560 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8374014be0 tx=0x7f8374007830 comp rx=0 tx=0).stop 2024-03-29T04:12:29.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.840+0000 7f8383065740 1 -- 172.21.15.148:0/3221782431 shutdown_connections 2024-03-29T04:12:29.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.840+0000 7f8383065740 1 --2- 172.21.15.148:0/3221782431 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x5615ed6a2830 0x5615ed6a5130 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.840+0000 7f8383065740 1 --2- 172.21.15.148:0/3221782431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615ed679820 0x5615ed679ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.840+0000 7f8383065740 1 --2- 172.21.15.148:0/3221782431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5615ed6735e0 0x5615ed6a20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.840+0000 7f8383065740 1 --2- 172.21.15.148:0/3221782431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615ed673200 0x5615ed6a4560 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.840+0000 7f83597fa640 0 client.5868 ms_handle_reset on v2:172.21.15.148:6836/473448013 2024-03-29T04:12:29.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.840+0000 7f8383065740 1 -- 172.21.15.148:0/3221782431 >> 172.21.15.148:0/3221782431 conn(0x5615ed6466e0 msgr2=0x5615ed64a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:29.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.840+0000 7f8383065740 1 -- 172.21.15.148:0/3221782431 shutdown_connections 2024-03-29T04:12:29.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.840+0000 7f8383065740 1 -- 172.21.15.148:0/3221782431 wait complete. 2024-03-29T04:12:29.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.841+0000 7f8383065740 20 client.5868 trim_cache size 0 max 16384 2024-03-29T04:12:29.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.858+0000 7f8383065740 1 Processor -- start 2024-03-29T04:12:29.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.858+0000 7f8383065740 1 -- start start 2024-03-29T04:12:29.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.859+0000 7f8383065740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5615ed673200 0x5615ed6a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:29.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.859+0000 7f8383065740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615ed6735e0 0x5615ed69e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:29.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.859+0000 7f8383065740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615ed679820 0x5615ed68e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:29.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.859+0000 7f8383065740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f83782010a0 con 0x5615ed679820 2024-03-29T04:12:29.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.859+0000 7f8383065740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f83782000c0 con 0x5615ed6735e0 2024-03-29T04:12:29.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.859+0000 7f8383065740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f83781434c0 con 0x5615ed673200 2024-03-29T04:12:29.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.859+0000 7f837ea11640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615ed679820 0x5615ed68e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:29.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.859+0000 7f837ea11640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615ed679820 0x5615ed68e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36222/0 (socket says 172.21.15.148:36222) 2024-03-29T04:12:29.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.859+0000 7f837ea11640 1 -- 172.21.15.148:0/3186806842 learned_addr learned my addr 172.21.15.148:0/3186806842 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:29.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.859+0000 7f837da0f640 1 --2- 172.21.15.148:0/3186806842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615ed6735e0 0x5615ed69e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:29.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.859+0000 7f837e210640 1 --2- 172.21.15.148:0/3186806842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5615ed673200 0x5615ed6a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:29.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.859+0000 7f837ea11640 1 -- 172.21.15.148:0/3186806842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5615ed673200 msgr2=0x5615ed6a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:29.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.859+0000 7f837ea11640 1 --2- 172.21.15.148:0/3186806842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5615ed673200 0x5615ed6a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.859+0000 7f837ea11640 1 -- 172.21.15.148:0/3186806842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615ed6735e0 msgr2=0x5615ed69e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:29.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.859+0000 7f837ea11640 1 --2- 172.21.15.148:0/3186806842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615ed6735e0 0x5615ed69e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.859+0000 7f837ea11640 1 -- 172.21.15.148:0/3186806842 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615ed2840f0 con 0x5615ed679820 2024-03-29T04:12:29.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.859+0000 7f837e210640 1 --2- 172.21.15.148:0/3186806842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5615ed673200 0x5615ed6a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:12:29.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.859+0000 7f837ea11640 1 --2- 172.21.15.148:0/3186806842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615ed679820 0x5615ed68e960 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8370037040 tx=0x7f837002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df73cc3458fb5a5f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:29.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.859+0000 7f835b7fe640 1 -- 172.21.15.148:0/3186806842 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370046070 con 0x5615ed679820 2024-03-29T04:12:29.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.859+0000 7f835b7fe640 1 -- 172.21.15.148:0/3186806842 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f837003a040 con 0x5615ed679820 2024-03-29T04:12:29.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.859+0000 7f835b7fe640 1 -- 172.21.15.148:0/3186806842 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370039040 con 0x5615ed679820 2024-03-29T04:12:29.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.860+0000 7f8383065740 1 -- 172.21.15.148:0/3186806842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615ed679820 msgr2=0x5615ed68e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:29.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.860+0000 7f8383065740 1 --2- 172.21.15.148:0/3186806842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615ed679820 0x5615ed68e960 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8370037040 tx=0x7f837002e000 comp rx=0 tx=0).stop 2024-03-29T04:12:29.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.860+0000 7f8383065740 1 -- 172.21.15.148:0/3186806842 shutdown_connections 2024-03-29T04:12:29.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.860+0000 7f8383065740 1 --2- 172.21.15.148:0/3186806842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615ed679820 0x5615ed68e960 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.860+0000 7f8383065740 1 --2- 172.21.15.148:0/3186806842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615ed6735e0 0x5615ed69e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.860+0000 7f8383065740 1 --2- 172.21.15.148:0/3186806842 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5615ed673200 0x5615ed6a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.860+0000 7f8383065740 1 -- 172.21.15.148:0/3186806842 >> 172.21.15.148:0/3186806842 conn(0x5615ed6466e0 msgr2=0x5615ed6a0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:29.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.860+0000 7f8383065740 1 -- 172.21.15.148:0/3186806842 shutdown_connections 2024-03-29T04:12:29.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.860+0000 7f8383065740 1 -- 172.21.15.148:0/3186806842 wait complete. 2024-03-29T04:12:29.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.861+0000 7f8383065740 1 -- start start 2024-03-29T04:12:29.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.861+0000 7f8383065740 1 Processor -- start 2024-03-29T04:12:29.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.861+0000 7f8383065740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5615ed673200 0x5615ed70d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:29.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.861+0000 7f8383065740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615ed6735e0 0x5615ed68e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:29.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.861+0000 7f8383065740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615ed679820 0x5615ed68eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:29.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.861+0000 7f8383065740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5615ed673ec0 con 0x5615ed6735e0 2024-03-29T04:12:29.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.861+0000 7f8383065740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f83781ffaa0 con 0x5615ed679820 2024-03-29T04:12:29.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.861+0000 7f8383065740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x5615ed70d680 con 0x5615ed673200 2024-03-29T04:12:29.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.862+0000 7f8383065740 10 client.0 _ll_register_callbacks cb 0x7f83740073a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:12:29.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.862+0000 7f837da0f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615ed6735e0 0x5615ed68e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:29.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.862+0000 7f837da0f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615ed6735e0 0x5615ed68e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36238/0 (socket says 172.21.15.148:36238) 2024-03-29T04:12:29.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.862+0000 7f837da0f640 1 -- 172.21.15.148:0/2863688977 learned_addr learned my addr 172.21.15.148:0/2863688977 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:29.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.862+0000 7f837ea11640 1 --2- 172.21.15.148:0/2863688977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615ed679820 0x5615ed68eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:29.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.862+0000 7f837e210640 1 --2- 172.21.15.148:0/2863688977 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5615ed673200 0x5615ed70d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:29.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.862+0000 7f837da0f640 1 -- 172.21.15.148:0/2863688977 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5615ed673200 msgr2=0x5615ed70d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:29.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.862+0000 7f837da0f640 1 --2- 172.21.15.148:0/2863688977 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5615ed673200 0x5615ed70d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.862+0000 7f837da0f640 1 -- 172.21.15.148:0/2863688977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615ed679820 msgr2=0x5615ed68eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:29.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.862+0000 7f837da0f640 1 --2- 172.21.15.148:0/2863688977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615ed679820 0x5615ed68eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.862+0000 7f837da0f640 1 -- 172.21.15.148:0/2863688977 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615ed64c060 con 0x5615ed6735e0 2024-03-29T04:12:29.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.862+0000 7f837da0f640 1 --2- 172.21.15.148:0/2863688977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615ed6735e0 0x5615ed68e960 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f83680591f0 tx=0x7f8368059030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a73eaddb6b34406 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:29.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.862+0000 7f83597fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-03-29T04:12:29.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.862+0000 7f83597fa640 1 -- 172.21.15.148:0/2863688977 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368012d00 con 0x5615ed6735e0 2024-03-29T04:12:29.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.862+0000 7f83597fa640 1 -- 172.21.15.148:0/2863688977 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f83680558d0 con 0x5615ed6735e0 2024-03-29T04:12:29.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.862+0000 7f8383065740 1 -- 172.21.15.148:0/2863688977 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f837003a040 con 0x5615ed6735e0 2024-03-29T04:12:29.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.862+0000 7f83597fa640 1 -- 172.21.15.148:0/2863688977 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f83680619b0 con 0x5615ed6735e0 2024-03-29T04:12:29.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.862+0000 7f8383065740 10 client.5874 fetch_fsmap learned FSMap version 19 2024-03-29T04:12:29.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.862+0000 7f8383065740 1 -- 172.21.15.148:0/2863688977 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x5615ed69eae0 con 0x5615ed6735e0 2024-03-29T04:12:29.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.863+0000 7f83597fa640 1 -- 172.21.15.148:0/2863688977 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f837003a040 con 0x5615ed6735e0 2024-03-29T04:12:29.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.863+0000 7f83597fa640 1 -- 172.21.15.148:0/2863688977 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 19) ==== 3494+0+0 (secure 0 0 0) 0x7f836808b0b0 con 0x5615ed6735e0 2024-03-29T04:12:29.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.863+0000 7f8383065740 10 client.5874 fetch_fsmap finished waiting for FSMap version 19 2024-03-29T04:12:29.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.863+0000 7f8383065740 10 client.5874 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-29T04:12:29.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.863+0000 7f8383065740 20 client.5874 populate_metadata read hostname 'smithi148' 2024-03-29T04:12:29.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.863+0000 7f8383065740 1 --2- 172.21.15.148:0/2863688977 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x5615ed6b41f0 0x5615ed6b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:29.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.863+0000 7f8383065740 4 client.5874 mds_command: new command op to 5583 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-29T04:12:29.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.863+0000 7f8383065740 1 -- 172.21.15.148:0/2863688977 --> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5615ed5b8110 con 0x5615ed6b41f0 2024-03-29T04:12:29.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.863+0000 7f837e210640 1 --2- 172.21.15.148:0/2863688977 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x5615ed6b41f0 0x5615ed6b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:29.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.863+0000 7f83597fa640 10 client.5874 ms_handle_connect on v2:172.21.15.148:6836/473448013 2024-03-29T04:12:29.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.863+0000 7f837e210640 1 --2- 172.21.15.148:0/2863688977 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x5615ed6b41f0 0x5615ed6b4610 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59d351b519e6e1b4 server_cookie=e5edd07fa8734157 in_seq=0 out_seq=0 2024-03-29T04:12:29.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.863+0000 7f83597fa640 1 -- 172.21.15.148:0/2863688977 <== mds.0 v2:172.21.15.148:6836/473448013 1 ==== command_reply(tid 0: 0 ) ==== 8+0+376 (crc 0 0 0) 0x5615ed5b8110 con 0x5615ed6b41f0 2024-03-29T04:12:29.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.863+0000 7f83597fa640 10 client.5874 handle_command_reply: tid=0 2024-03-29T04:12:29.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.863+0000 7f8383065740 1 client.5874 shutdown 2024-03-29T04:12:29.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.863+0000 7f8383065740 1 -- 172.21.15.148:0/2863688977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615ed6735e0 msgr2=0x5615ed68e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:29.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.863+0000 7f8383065740 1 --2- 172.21.15.148:0/2863688977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615ed6735e0 0x5615ed68e960 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f83680591f0 tx=0x7f8368059030 comp rx=0 tx=0).stop 2024-03-29T04:12:29.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.864+0000 7f8383065740 1 -- 172.21.15.148:0/2863688977 shutdown_connections 2024-03-29T04:12:29.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.864+0000 7f8383065740 1 --2- 172.21.15.148:0/2863688977 >> [v2:172.21.15.148:6836/473448013,v1:172.21.15.148:6837/473448013] conn(0x5615ed6b41f0 0x5615ed6b4610 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.864+0000 7f8383065740 1 --2- 172.21.15.148:0/2863688977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5615ed679820 0x5615ed68eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.864+0000 7f8383065740 1 --2- 172.21.15.148:0/2863688977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615ed6735e0 0x5615ed68e960 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.864+0000 7f83597fa640 0 client.5874 ms_handle_reset on v2:172.21.15.148:6836/473448013 2024-03-29T04:12:29.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.864+0000 7f8383065740 1 --2- 172.21.15.148:0/2863688977 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5615ed673200 0x5615ed70d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.864+0000 7f8383065740 1 -- 172.21.15.148:0/2863688977 >> 172.21.15.148:0/2863688977 conn(0x5615ed6466e0 msgr2=0x5615ed6760f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:29.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.864+0000 7f8383065740 1 -- 172.21.15.148:0/2863688977 shutdown_connections 2024-03-29T04:12:29.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.864+0000 7f8383065740 1 -- 172.21.15.148:0/2863688977 wait complete. 2024-03-29T04:12:29.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.864+0000 7f8383065740 20 client.5874 trim_cache size 0 max 16384 2024-03-29T04:12:29.853 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-03-29T04:12:29.853 INFO:teuthology.orchestra.run.smithi148.stdout: "status": "no active scrubs running", 2024-03-29T04:12:29.853 INFO:teuthology.orchestra.run.smithi148.stdout: "scrubs": {}, 2024-03-29T04:12:29.853 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_stats": { 2024-03-29T04:12:29.853 INFO:teuthology.orchestra.run.smithi148.stdout: "713fb6cc-5d83-4906-8a07-c0c0f0760e64": { 2024-03-29T04:12:29.853 INFO:teuthology.orchestra.run.smithi148.stdout: "start_time": "2024-03-29T04:11:59.492927+0000", 2024-03-29T04:12:29.853 INFO:teuthology.orchestra.run.smithi148.stdout: "path": "/", 2024-03-29T04:12:29.853 INFO:teuthology.orchestra.run.smithi148.stdout: "uninline_started": 23, 2024-03-29T04:12:29.854 INFO:teuthology.orchestra.run.smithi148.stdout: "uninline_passed": 23, 2024-03-29T04:12:29.854 INFO:teuthology.orchestra.run.smithi148.stdout: "uninline_failed": 0, 2024-03-29T04:12:29.854 INFO:teuthology.orchestra.run.smithi148.stdout: "uninline_skipped": 0 2024-03-29T04:12:29.854 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:12:29.854 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T04:12:29.854 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-03-29T04:12:29.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.864+0000 7f8380474640 1 -- 172.21.15.148:0/3678128194 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f835c06c530 msgr2=0x7f835c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:29.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.864+0000 7f8380474640 1 --2- 172.21.15.148:0/3678128194 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f835c06c530 0x7f835c06e9f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8374000f60 tx=0x7f8374009040 comp rx=0 tx=0).stop 2024-03-29T04:12:29.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.864+0000 7f8380474640 1 -- 172.21.15.148:0/3678128194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83780cfda0 msgr2=0x7f837813ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:29.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.864+0000 7f8380474640 1 --2- 172.21.15.148:0/3678128194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83780cfda0 0x7f837813ecd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f836800c480 tx=0x7f836800c940 comp rx=0 tx=0).stop 2024-03-29T04:12:29.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.864+0000 7f8380474640 1 -- 172.21.15.148:0/3678128194 shutdown_connections 2024-03-29T04:12:29.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.864+0000 7f8380474640 1 --2- 172.21.15.148:0/3678128194 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f835c06c530 0x7f835c06e9f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.864+0000 7f8380474640 1 --2- 172.21.15.148:0/3678128194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8378144980 0x7f8378141f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.864+0000 7f8380474640 1 --2- 172.21.15.148:0/3678128194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83780cfda0 0x7f837813ecd0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.864+0000 7f8380474640 1 --2- 172.21.15.148:0/3678128194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83780cc0f0 0x7f837815d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:29.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.864+0000 7f8380474640 1 -- 172.21.15.148:0/3678128194 >> 172.21.15.148:0/3678128194 conn(0x7f83780b2c60 msgr2=0x7f837806f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:29.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.864+0000 7f8380474640 1 -- 172.21.15.148:0/3678128194 shutdown_connections 2024-03-29T04:12:29.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:29.864+0000 7f8380474640 1 -- 172.21.15.148:0/3678128194 wait complete. 2024-03-29T04:12:29.857 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-29T04:12:29.857 DEBUG:tasks.cephfs.test_uninlining:extracting inodes 2024-03-29T04:12:29.857 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/Kconfig 2024-03-29T04:12:29.857 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/Kconfig 2024-03-29T04:12:29.874 INFO:teuthology.orchestra.run.smithi148.stdout:1099511627780 linux-5.4/fs/ceph/Kconfig 2024-03-29T04:12:29.874 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/Makefile 2024-03-29T04:12:29.874 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/Makefile 2024-03-29T04:12:29.932 INFO:teuthology.orchestra.run.smithi148.stdout:1099511628281 linux-5.4/fs/ceph/Makefile 2024-03-29T04:12:29.932 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/ceph_frag.c 2024-03-29T04:12:29.933 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/ceph_frag.c 2024-03-29T04:12:29.989 INFO:teuthology.orchestra.run.smithi148.stdout:1099511628287 linux-5.4/fs/ceph/ceph_frag.c 2024-03-29T04:12:29.990 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/io.h 2024-03-29T04:12:29.990 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/io.h 2024-03-29T04:12:30.046 INFO:teuthology.orchestra.run.smithi148.stdout:1099511628294 linux-5.4/fs/ceph/io.h 2024-03-29T04:12:30.046 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/ioctl.h 2024-03-29T04:12:30.046 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/ioctl.h 2024-03-29T04:12:30.103 INFO:teuthology.orchestra.run.smithi148.stdout:1099511628296 linux-5.4/fs/ceph/ioctl.h 2024-03-29T04:12:30.103 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/Kconfig 2024-03-29T04:12:30.103 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/Kconfig 2024-03-29T04:12:30.160 INFO:teuthology.orchestra.run.smithi148.stdout:1099511628308 linux-5.4/fs/ext2/Kconfig 2024-03-29T04:12:30.160 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr_user.c 2024-03-29T04:12:30.160 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr_user.c 2024-03-29T04:12:30.217 INFO:teuthology.orchestra.run.smithi148.stdout:1099511628326 linux-5.4/fs/ext2/xattr_user.c 2024-03-29T04:12:30.217 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr_security.c 2024-03-29T04:12:30.217 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr_security.c 2024-03-29T04:12:30.272 INFO:teuthology.orchestra.run.smithi148.stdout:1099511628324 linux-5.4/fs/ext2/xattr_security.c 2024-03-29T04:12:30.272 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/Makefile 2024-03-29T04:12:30.273 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/Makefile 2024-03-29T04:12:30.329 INFO:teuthology.orchestra.run.smithi148.stdout:1099511628309 linux-5.4/fs/ext2/Makefile 2024-03-29T04:12:30.329 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/symlink.c 2024-03-29T04:12:30.329 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/symlink.c 2024-03-29T04:12:30.386 INFO:teuthology.orchestra.run.smithi148.stdout:1099511628321 linux-5.4/fs/ext2/symlink.c 2024-03-29T04:12:30.386 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/acl.h 2024-03-29T04:12:30.386 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/acl.h 2024-03-29T04:12:30.443 INFO:teuthology.orchestra.run.smithi148.stdout:1099511628311 linux-5.4/fs/ext2/acl.h 2024-03-29T04:12:30.444 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr_trusted.c 2024-03-29T04:12:30.444 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr_trusted.c 2024-03-29T04:12:30.501 INFO:teuthology.orchestra.run.smithi148.stdout:1099511628325 linux-5.4/fs/ext2/xattr_trusted.c 2024-03-29T04:12:30.501 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr.h 2024-03-29T04:12:30.501 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr.h 2024-03-29T04:12:30.558 INFO:teuthology.orchestra.run.smithi148.stdout:1099511628323 linux-5.4/fs/ext2/xattr.h 2024-03-29T04:12:30.558 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/Kconfig 2024-03-29T04:12:30.558 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/Kconfig 2024-03-29T04:12:30.614 INFO:teuthology.orchestra.run.smithi148.stdout:1099511628328 linux-5.4/fs/orangefs/Kconfig 2024-03-29T04:12:30.614 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-29T04:12:30.615 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-29T04:12:30.671 INFO:teuthology.orchestra.run.smithi148.stdout:1099511628341 linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-29T04:12:30.672 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-29T04:12:30.672 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-29T04:12:30.728 INFO:teuthology.orchestra.run.smithi148.stdout:1099511628348 linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-29T04:12:30.728 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/dcache.c 2024-03-29T04:12:30.728 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/dcache.c 2024-03-29T04:12:30.785 INFO:teuthology.orchestra.run.smithi148.stdout:1099511628331 linux-5.4/fs/orangefs/dcache.c 2024-03-29T04:12:30.785 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-29T04:12:30.785 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-29T04:12:30.842 INFO:teuthology.orchestra.run.smithi148.stdout:1099511628339 linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-29T04:12:30.842 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-29T04:12:30.842 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-29T04:12:30.899 INFO:teuthology.orchestra.run.smithi148.stdout:1099511628343 linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-29T04:12:30.899 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-29T04:12:30.899 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-29T04:12:30.956 INFO:teuthology.orchestra.run.smithi148.stdout:1099511628344 linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-29T04:12:30.956 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/Makefile 2024-03-29T04:12:30.956 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/Makefile 2024-03-29T04:12:31.013 INFO:teuthology.orchestra.run.smithi148.stdout:1099511628329 linux-5.4/fs/orangefs/Makefile 2024-03-29T04:12:31.013 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/symlink.c 2024-03-29T04:12:31.013 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/symlink.c 2024-03-29T04:12:31.070 INFO:teuthology.orchestra.run.smithi148.stdout:1099511628352 linux-5.4/fs/orangefs/symlink.c 2024-03-29T04:12:31.071 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/downcall.h 2024-03-29T04:12:31.071 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/downcall.h 2024-03-29T04:12:31.127 INFO:teuthology.orchestra.run.smithi148.stdout:1099511628334 linux-5.4/fs/orangefs/downcall.h 2024-03-29T04:12:31.127 DEBUG:tasks.cephfs.test_uninlining:getting inline data info 2024-03-29T04:12:31.127 DEBUG:tasks.cephfs.test_uninlining:getting inode info #1 of 23:1099511627780 2024-03-29T04:12:31.127 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511627780 2024-03-29T04:12:31.128 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-29T04:12:31.128 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T04:12:31.215 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T04:12:31.261 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-29T04:12:31.261 INFO:tasks.cephfs.mount:Terminating background process 2024-03-29T04:12:31.261 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:12:31.261 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:12:31.277 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:12:31.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.439+0000 7f2ff3d90640 1 Processor -- start 2024-03-29T04:12:31.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.440+0000 7f2ff3d90640 1 -- start start 2024-03-29T04:12:31.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.440+0000 7f2ff3d90640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fec15e3e0 0x7f2fec0d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:31.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.440+0000 7f2ff3d90640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2fec0d10d0 0x7f2fec0d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:31.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.440+0000 7f2ff3d90640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fec15e7c0 con 0x7f2fec0ba030 2024-03-29T04:12:31.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.440+0000 7f2ff3d90640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f2fec0d3840 con 0x7f2fec0d10d0 2024-03-29T04:12:31.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.440+0000 7f2ff3d90640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2fec0d2c30 con 0x7f2fec15e3e0 2024-03-29T04:12:31.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.441+0000 7f2ff232d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2fec0d10d0 0x7f2fec0d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:31.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.441+0000 7f2ff232d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2fec0d10d0 0x7f2fec0d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36246/0 (socket says 172.21.15.148:36246) 2024-03-29T04:12:31.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.441+0000 7f2ff232d640 1 -- 172.21.15.148:0/3375068446 learned_addr learned my addr 172.21.15.148:0/3375068446 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:31.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.441+0000 7f2ff1b2c640 1 --2- 172.21.15.148:0/3375068446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fec15e3e0 0x7f2fec0d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:31.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.441+0000 7f2ff232d640 1 -- 172.21.15.148:0/3375068446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fec15e3e0 msgr2=0x7f2fec0d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:31.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.441+0000 7f2ff232d640 1 --2- 172.21.15.148:0/3375068446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fec15e3e0 0x7f2fec0d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:31.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.441+0000 7f2ff232d640 1 -- 172.21.15.148:0/3375068446 >> v1:172.21.15.157:6789/0 conn(0x7f2fec0ba030 legacy=0x7f2fec0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:31.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.441+0000 7f2ff232d640 1 -- 172.21.15.148:0/3375068446 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fec0d2f10 con 0x7f2fec0d10d0 2024-03-29T04:12:31.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.441+0000 7f2ff1b2c640 1 --2- 172.21.15.148:0/3375068446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fec15e3e0 0x7f2fec0d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:12:31.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.442+0000 7f2ff232d640 1 --2- 172.21.15.148:0/3375068446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fec0d10d0 0x7f2fec0d1580 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009e50 tx=0x7f2fe402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7e1c18d8d651c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:31.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.442+0000 7f2ff0b2a640 1 -- 172.21.15.148:0/3375068446 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe402cb00 con 0x7f2fec0d10d0 2024-03-29T04:12:31.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.442+0000 7f2ff0b2a640 1 -- 172.21.15.148:0/3375068446 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fe402cca0 con 0x7f2fec0d10d0 2024-03-29T04:12:31.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.442+0000 7f2ff0b2a640 1 -- 172.21.15.148:0/3375068446 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4034610 con 0x7f2fec0d10d0 2024-03-29T04:12:31.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.443+0000 7f2ff3d90640 1 -- 172.21.15.148:0/3375068446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fec0d10d0 msgr2=0x7f2fec0d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:31.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.443+0000 7f2ff3d90640 1 --2- 172.21.15.148:0/3375068446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fec0d10d0 0x7f2fec0d1580 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009e50 tx=0x7f2fe402bf60 comp rx=0 tx=0).stop 2024-03-29T04:12:31.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.443+0000 7f2ff3d90640 1 -- 172.21.15.148:0/3375068446 shutdown_connections 2024-03-29T04:12:31.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.443+0000 7f2ff3d90640 1 --2- 172.21.15.148:0/3375068446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fec0d10d0 0x7f2fec0d1580 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:31.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.443+0000 7f2ff3d90640 1 --2- 172.21.15.148:0/3375068446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fec15e3e0 0x7f2fec0d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:31.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.443+0000 7f2ff3d90640 1 -- 172.21.15.148:0/3375068446 >> 172.21.15.148:0/3375068446 conn(0x7f2fec0b6210 msgr2=0x7f2fec0b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:31.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.443+0000 7f2ff3d90640 1 -- 172.21.15.148:0/3375068446 shutdown_connections 2024-03-29T04:12:31.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.443+0000 7f2ff3d90640 1 -- 172.21.15.148:0/3375068446 wait complete. 2024-03-29T04:12:31.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.444+0000 7f2ff3d90640 1 Processor -- start 2024-03-29T04:12:31.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.444+0000 7f2ff3d90640 1 -- start start 2024-03-29T04:12:31.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.446+0000 7f2ff3d90640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fec0ba030 0x7f2fec150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:31.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.446+0000 7f2ff3d90640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fec0d10d0 0x7f2fec0cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:31.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.446+0000 7f2ff3d90640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fec15e3e0 0x7f2fec1522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:31.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.446+0000 7f2ff3d90640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2fec167f50 con 0x7f2fec0ba030 2024-03-29T04:12:31.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.446+0000 7f2ff3d90640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2fec0d34d0 con 0x7f2fec0d10d0 2024-03-29T04:12:31.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.446+0000 7f2ff3d90640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2fec1680d0 con 0x7f2fec15e3e0 2024-03-29T04:12:31.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.446+0000 7f2ff1b2c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fec0ba030 0x7f2fec150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:31.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.446+0000 7f2ff1b2c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fec0ba030 0x7f2fec150580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36262/0 (socket says 172.21.15.148:36262) 2024-03-29T04:12:31.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.446+0000 7f2ff1b2c640 1 -- 172.21.15.148:0/3179209856 learned_addr learned my addr 172.21.15.148:0/3179209856 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:31.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.446+0000 7f2ff132b640 1 --2- 172.21.15.148:0/3179209856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fec0d10d0 0x7f2fec0cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:31.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.446+0000 7f2ff232d640 1 --2- 172.21.15.148:0/3179209856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fec15e3e0 0x7f2fec1522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:31.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.446+0000 7f2ff1b2c640 1 -- 172.21.15.148:0/3179209856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fec15e3e0 msgr2=0x7f2fec1522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:31.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.446+0000 7f2ff1b2c640 1 --2- 172.21.15.148:0/3179209856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fec15e3e0 0x7f2fec1522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:31.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.446+0000 7f2ff1b2c640 1 -- 172.21.15.148:0/3179209856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fec0d10d0 msgr2=0x7f2fec0cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:31.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.446+0000 7f2ff1b2c640 1 --2- 172.21.15.148:0/3179209856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fec0d10d0 0x7f2fec0cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:31.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.446+0000 7f2ff1b2c640 1 -- 172.21.15.148:0/3179209856 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fec15f140 con 0x7f2fec0ba030 2024-03-29T04:12:31.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.446+0000 7f2ff232d640 1 --2- 172.21.15.148:0/3179209856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fec15e3e0 0x7f2fec1522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:31.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.446+0000 7f2ff132b640 1 --2- 172.21.15.148:0/3179209856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fec0d10d0 0x7f2fec0cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:31.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.446+0000 7f2ff1b2c640 1 --2- 172.21.15.148:0/3179209856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fec0ba030 0x7f2fec150580 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800c7f0 tx=0x7f2fe800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:31.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.446+0000 7f2fe27fc640 1 -- 172.21.15.148:0/3179209856 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8015070 con 0x7f2fec0ba030 2024-03-29T04:12:31.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.446+0000 7f2fe27fc640 1 -- 172.21.15.148:0/3179209856 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fe8007820 con 0x7f2fec0ba030 2024-03-29T04:12:31.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.446+0000 7f2ff3d90640 1 -- 172.21.15.148:0/3179209856 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fec15f420 con 0x7f2fec0ba030 2024-03-29T04:12:31.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.446+0000 7f2fe27fc640 1 -- 172.21.15.148:0/3179209856 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8007bd0 con 0x7f2fec0ba030 2024-03-29T04:12:31.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.447+0000 7f2ff3d90640 1 -- 172.21.15.148:0/3179209856 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fec150ac0 con 0x7f2fec0ba030 2024-03-29T04:12:31.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.448+0000 7f2ff3d90640 1 -- 172.21.15.148:0/3179209856 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fec0d0720 con 0x7f2fec0ba030 2024-03-29T04:12:31.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.448+0000 7f2fe27fc640 1 -- 172.21.15.148:0/3179209856 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2fe8018070 con 0x7f2fec0ba030 2024-03-29T04:12:31.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.449+0000 7f2fe27fc640 1 --2- 172.21.15.148:0/3179209856 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f2fd006c530 0x7f2fd006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:31.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.449+0000 7f2ff132b640 1 --2- 172.21.15.148:0/3179209856 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f2fd006c530 0x7f2fd006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:31.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.449+0000 7f2fe27fc640 1 -- 172.21.15.148:0/3179209856 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f2fe808b750 con 0x7f2fec0ba030 2024-03-29T04:12:31.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.449+0000 7f2ff132b640 1 --2- 172.21.15.148:0/3179209856 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f2fd006c530 0x7f2fd006e9f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2fec160ba0 tx=0x7f2fdc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:31.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.453+0000 7f2fe27fc640 1 -- 172.21.15.148:0/3179209856 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2fec0d0720 con 0x7f2fec0ba030 2024-03-29T04:12:31.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.781+0000 7f2ff3d90640 1 -- 172.21.15.148:0/3179209856 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2fec0687e0 con 0x7f2fec0ba030 2024-03-29T04:12:31.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.782+0000 7f2fe27fc640 1 -- 172.21.15.148:0/3179209856 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+17003 (secure 0 0 0) 0x7f2fe8059c50 con 0x7f2fec0ba030 2024-03-29T04:12:31.761 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:12:31.761 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":41,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:11:42.632417+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:11:30.794790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T04:11:31.907100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:12:31.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.784+0000 7f2ff3d90640 1 -- 172.21.15.148:0/3179209856 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f2fd006c530 msgr2=0x7f2fd006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:31.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.784+0000 7f2ff3d90640 1 --2- 172.21.15.148:0/3179209856 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f2fd006c530 0x7f2fd006e9f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2fec160ba0 tx=0x7f2fdc009040 comp rx=0 tx=0).stop 2024-03-29T04:12:31.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.784+0000 7f2ff3d90640 1 -- 172.21.15.148:0/3179209856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fec0ba030 msgr2=0x7f2fec150580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:31.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.784+0000 7f2ff3d90640 1 --2- 172.21.15.148:0/3179209856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fec0ba030 0x7f2fec150580 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800c7f0 tx=0x7f2fe800ccb0 comp rx=0 tx=0).stop 2024-03-29T04:12:31.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.784+0000 7f2ff3d90640 1 -- 172.21.15.148:0/3179209856 shutdown_connections 2024-03-29T04:12:31.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.784+0000 7f2ff3d90640 1 --2- 172.21.15.148:0/3179209856 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f2fd006c530 0x7f2fd006e9f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:31.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.784+0000 7f2ff3d90640 1 --2- 172.21.15.148:0/3179209856 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fec15e3e0 0x7f2fec1522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:31.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.784+0000 7f2ff3d90640 1 --2- 172.21.15.148:0/3179209856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fec0d10d0 0x7f2fec0cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:31.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.784+0000 7f2ff3d90640 1 --2- 172.21.15.148:0/3179209856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fec0ba030 0x7f2fec150580 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:31.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.784+0000 7f2ff3d90640 1 -- 172.21.15.148:0/3179209856 >> 172.21.15.148:0/3179209856 conn(0x7f2fec0b6210 msgr2=0x7f2fec0b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:31.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.784+0000 7f2ff3d90640 1 -- 172.21.15.148:0/3179209856 shutdown_connections 2024-03-29T04:12:31.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.784+0000 7f2ff3d90640 1 -- 172.21.15.148:0/3179209856 wait complete. 2024-03-29T04:12:31.774 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:47.898351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:11:30.794790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:11:31.907100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:12:31.774 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:12:31.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.901+0000 7f2df929a640 1 Processor -- start 2024-03-29T04:12:31.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.901+0000 7f2df929a640 1 -- start start 2024-03-29T04:12:31.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.901+0000 7f2df929a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2df415e3c0 0x7f2df4161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:31.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.901+0000 7f2df929a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2df40cf330 con 0x7f2df40b2f00 2024-03-29T04:12:31.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.901+0000 7f2df929a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2df4162cf0 con 0x7f2df40b3430 2024-03-29T04:12:31.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.901+0000 7f2df929a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f2df40cf960 con 0x7f2df415e3c0 2024-03-29T04:12:31.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.902+0000 7f2df2ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2df415e3c0 0x7f2df4161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:31.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.902+0000 7f2dea7fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2df40b2f00 0x7f2df41624b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39342/0 (socket says 172.21.15.148:39342) 2024-03-29T04:12:31.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.902+0000 7f2dea7fc640 1 -- 172.21.15.148:0/1441509818 learned_addr learned my addr 172.21.15.148:0/1441509818 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:31.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.902+0000 7f2df2ffd640 1 -- 172.21.15.148:0/1441509818 >> v1:172.21.15.157:6789/0 conn(0x7f2df40b3430 legacy=0x7f2df406b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:31.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.902+0000 7f2df2ffd640 1 -- 172.21.15.148:0/1441509818 >> v1:172.21.15.148:6789/0 conn(0x7f2df40b2f00 legacy=0x7f2df41624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:31.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.902+0000 7f2df2ffd640 1 -- 172.21.15.148:0/1441509818 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df40cf5f0 con 0x7f2df415e3c0 2024-03-29T04:12:31.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.902+0000 7f2df27fc640 1 -- 172.21.15.148:0/1441509818 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1746511592 0 0) 0x7f2df40cf330 con 0x7f2df40b2f00 2024-03-29T04:12:31.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.903+0000 7f2df2ffd640 1 --2- 172.21.15.148:0/1441509818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2df415e3c0 0x7f2df4161f70 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2de40097a0 tx=0x7f2de402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab855af4d245ff3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:31.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.903+0000 7f2df27fc640 1 -- 172.21.15.148:0/1441509818 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de4035020 con 0x7f2df415e3c0 2024-03-29T04:12:31.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.903+0000 7f2df27fc640 1 -- 172.21.15.148:0/1441509818 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2de4031830 con 0x7f2df415e3c0 2024-03-29T04:12:31.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.903+0000 7f2df27fc640 1 -- 172.21.15.148:0/1441509818 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de4031c00 con 0x7f2df415e3c0 2024-03-29T04:12:31.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.903+0000 7f2df929a640 1 -- 172.21.15.148:0/1441509818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2df415e3c0 msgr2=0x7f2df4161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:31.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.903+0000 7f2df929a640 1 --2- 172.21.15.148:0/1441509818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2df415e3c0 0x7f2df4161f70 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2de40097a0 tx=0x7f2de402b8d0 comp rx=0 tx=0).stop 2024-03-29T04:12:31.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.903+0000 7f2df929a640 1 -- 172.21.15.148:0/1441509818 shutdown_connections 2024-03-29T04:12:31.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.903+0000 7f2df929a640 1 --2- 172.21.15.148:0/1441509818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2df415e3c0 0x7f2df4161f70 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:31.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.903+0000 7f2df929a640 1 -- 172.21.15.148:0/1441509818 >> 172.21.15.148:0/1441509818 conn(0x7f2df40b6430 msgr2=0x7f2df40d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:31.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.903+0000 7f2df929a640 1 -- 172.21.15.148:0/1441509818 shutdown_connections 2024-03-29T04:12:31.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.903+0000 7f2df929a640 1 -- 172.21.15.148:0/1441509818 wait complete. 2024-03-29T04:12:31.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.905+0000 7f2df929a640 1 Processor -- start 2024-03-29T04:12:31.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.905+0000 7f2df929a640 1 -- start start 2024-03-29T04:12:31.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.905+0000 7f2df929a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2df40b3430 0x7f2df4152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:31.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.905+0000 7f2df929a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2df415e3c0 0x7f2df40cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:31.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.905+0000 7f2df929a640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2df40cd110 0x7f2df40c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:31.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.905+0000 7f2df929a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2df40b3b20 con 0x7f2df40b3430 2024-03-29T04:12:31.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.905+0000 7f2df929a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2df40b39a0 con 0x7f2df415e3c0 2024-03-29T04:12:31.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.905+0000 7f2df929a640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2df406bf00 con 0x7f2df40cd110 2024-03-29T04:12:31.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.906+0000 7f2df2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2df40b3430 0x7f2df4152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:31.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.906+0000 7f2df2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2df40b3430 0x7f2df4152420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36280/0 (socket says 172.21.15.148:36280) 2024-03-29T04:12:31.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.906+0000 7f2df2ffd640 1 -- 172.21.15.148:0/403633422 learned_addr learned my addr 172.21.15.148:0/403633422 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:31.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.906+0000 7f2dea7fc640 1 --2- 172.21.15.148:0/403633422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2df415e3c0 0x7f2df40cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:31.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.906+0000 7f2df37fe640 1 --2- 172.21.15.148:0/403633422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2df40cd110 0x7f2df40c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:31.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.906+0000 7f2df2ffd640 1 -- 172.21.15.148:0/403633422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2df40cd110 msgr2=0x7f2df40c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:31.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.906+0000 7f2df2ffd640 1 --2- 172.21.15.148:0/403633422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2df40cd110 0x7f2df40c9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:31.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.906+0000 7f2df2ffd640 1 -- 172.21.15.148:0/403633422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2df415e3c0 msgr2=0x7f2df40cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:31.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.906+0000 7f2df2ffd640 1 --2- 172.21.15.148:0/403633422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2df415e3c0 0x7f2df40cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:31.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.906+0000 7f2df2ffd640 1 -- 172.21.15.148:0/403633422 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df40ccdd0 con 0x7f2df40b3430 2024-03-29T04:12:31.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.906+0000 7f2dea7fc640 1 --2- 172.21.15.148:0/403633422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2df415e3c0 0x7f2df40cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:12:31.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.906+0000 7f2df2ffd640 1 --2- 172.21.15.148:0/403633422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2df40b3430 0x7f2df4152420 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2de40097a0 tx=0x7f2de40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:31.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.906+0000 7f2debfff640 1 -- 172.21.15.148:0/403633422 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de4041070 con 0x7f2df40b3430 2024-03-29T04:12:31.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.906+0000 7f2debfff640 1 -- 172.21.15.148:0/403633422 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2de4004230 con 0x7f2df40b3430 2024-03-29T04:12:31.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.906+0000 7f2df929a640 1 -- 172.21.15.148:0/403633422 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df40cddb0 con 0x7f2df40b3430 2024-03-29T04:12:31.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.906+0000 7f2debfff640 1 -- 172.21.15.148:0/403633422 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de4035040 con 0x7f2df40b3430 2024-03-29T04:12:31.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.906+0000 7f2df929a640 1 -- 172.21.15.148:0/403633422 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df40cbdb0 con 0x7f2df40b3430 2024-03-29T04:12:31.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.907+0000 7f2df929a640 1 -- 172.21.15.148:0/403633422 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df4068850 con 0x7f2df40b3430 2024-03-29T04:12:31.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.908+0000 7f2debfff640 1 -- 172.21.15.148:0/403633422 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2de4038020 con 0x7f2df40b3430 2024-03-29T04:12:31.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.908+0000 7f2debfff640 1 --2- 172.21.15.148:0/403633422 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f2dd406c530 0x7f2dd406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:31.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.908+0000 7f2debfff640 1 -- 172.21.15.148:0/403633422 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f2de40ad310 con 0x7f2df40b3430 2024-03-29T04:12:31.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.908+0000 7f2dea7fc640 1 --2- 172.21.15.148:0/403633422 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f2dd406c530 0x7f2dd406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:31.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.909+0000 7f2dea7fc640 1 --2- 172.21.15.148:0/403633422 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f2dd406c530 0x7f2dd406e9f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2df4161a30 tx=0x7f2ddc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:31.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:31.913+0000 7f2debfff640 1 -- 172.21.15.148:0/403633422 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2de407b780 con 0x7f2df40b3430 2024-03-29T04:12:31.927 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:12:31.929 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:12:32.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.147+0000 7f2df929a640 1 -- 172.21.15.148:0/403633422 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2df40d2670 con 0x7f2df40b3430 2024-03-29T04:12:32.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.147+0000 7f2debfff640 1 -- 172.21.15.148:0/403633422 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7f2de407b080 con 0x7f2df40b3430 2024-03-29T04:12:32.126 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-29T04:12:32.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.150+0000 7f2df929a640 1 -- 172.21.15.148:0/403633422 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f2dd406c530 msgr2=0x7f2dd406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:32.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.150+0000 7f2df929a640 1 --2- 172.21.15.148:0/403633422 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f2dd406c530 0x7f2dd406e9f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2df4161a30 tx=0x7f2ddc009040 comp rx=0 tx=0).stop 2024-03-29T04:12:32.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.150+0000 7f2df929a640 1 -- 172.21.15.148:0/403633422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2df40b3430 msgr2=0x7f2df4152420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:32.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.150+0000 7f2df929a640 1 --2- 172.21.15.148:0/403633422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2df40b3430 0x7f2df4152420 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2de40097a0 tx=0x7f2de40046c0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.150+0000 7f2df929a640 1 -- 172.21.15.148:0/403633422 shutdown_connections 2024-03-29T04:12:32.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.150+0000 7f2df929a640 1 --2- 172.21.15.148:0/403633422 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f2dd406c530 0x7f2dd406e9f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.150+0000 7f2df929a640 1 --2- 172.21.15.148:0/403633422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2df40cd110 0x7f2df40c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.150+0000 7f2df929a640 1 --2- 172.21.15.148:0/403633422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2df415e3c0 0x7f2df40cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.150+0000 7f2df929a640 1 --2- 172.21.15.148:0/403633422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2df40b3430 0x7f2df4152420 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.150+0000 7f2df929a640 1 -- 172.21.15.148:0/403633422 >> 172.21.15.148:0/403633422 conn(0x7f2df40b6430 msgr2=0x7f2df415c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:32.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.150+0000 7f2df929a640 1 -- 172.21.15.148:0/403633422 shutdown_connections 2024-03-29T04:12:32.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.150+0000 7f2df929a640 1 -- 172.21.15.148:0/403633422 wait complete. 2024-03-29T04:12:32.140 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:12:32.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.264+0000 7f8f1b455640 1 Processor -- start 2024-03-29T04:12:32.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.264+0000 7f8f1b455640 1 -- start start 2024-03-29T04:12:32.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.266+0000 7f8f1b455640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f140b2f00 0x7f8f14162680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:32.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.266+0000 7f8f1b455640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8f140b3430 0x7f8f1406b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:32.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.266+0000 7f8f1b455640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f140cf330 con 0x7f8f1415e3c0 2024-03-29T04:12:32.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.266+0000 7f8f1b455640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f8f140cf960 con 0x7f8f140b3430 2024-03-29T04:12:32.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.266+0000 7f8f1b455640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8f14162cf0 con 0x7f8f140b2f00 2024-03-29T04:12:32.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.266+0000 7f8f199f2640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8f140b3430 0x7f8f1406b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:32.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.266+0000 7f8f191f1640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8f1415e3c0 0x7f8f14161f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39354/0 (socket says 172.21.15.148:39354) 2024-03-29T04:12:32.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.266+0000 7f8f191f1640 1 -- 172.21.15.148:0/231704231 learned_addr learned my addr 172.21.15.148:0/231704231 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:32.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.266+0000 7f8f199f2640 1 -- 172.21.15.148:0/231704231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f140b2f00 msgr2=0x7f8f14162680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:32.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.266+0000 7f8f199f2640 1 --2- 172.21.15.148:0/231704231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f140b2f00 0x7f8f14162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.266+0000 7f8f199f2640 1 -- 172.21.15.148:0/231704231 >> v1:172.21.15.148:6789/0 conn(0x7f8f1415e3c0 legacy=0x7f8f14161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:32.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.266+0000 7f8f199f2640 1 -- 172.21.15.148:0/231704231 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f140cf5f0 con 0x7f8f140b3430 2024-03-29T04:12:32.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.266+0000 7f8f0bfff640 1 -- 172.21.15.148:0/231704231 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1959331333 0 0) 0x7f8f140cf330 con 0x7f8f1415e3c0 2024-03-29T04:12:32.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.267+0000 7f8f199f2640 1 --2- 172.21.15.148:0/231704231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f140b3430 0x7f8f1406b7f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c009d90 tx=0x7f8f0c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f761e9609ce4a1e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:32.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.267+0000 7f8f0bfff640 1 -- 172.21.15.148:0/231704231 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c039070 con 0x7f8f140b3430 2024-03-29T04:12:32.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.267+0000 7f8f0bfff640 1 -- 172.21.15.148:0/231704231 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f0c02cbc0 con 0x7f8f140b3430 2024-03-29T04:12:32.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.267+0000 7f8f0bfff640 1 -- 172.21.15.148:0/231704231 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c034440 con 0x7f8f140b3430 2024-03-29T04:12:32.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.268+0000 7f8f1b455640 1 -- 172.21.15.148:0/231704231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f140b3430 msgr2=0x7f8f1406b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:32.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.268+0000 7f8f1b455640 1 --2- 172.21.15.148:0/231704231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f140b3430 0x7f8f1406b7f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c009d90 tx=0x7f8f0c02be10 comp rx=0 tx=0).stop 2024-03-29T04:12:32.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.268+0000 7f8f1b455640 1 -- 172.21.15.148:0/231704231 shutdown_connections 2024-03-29T04:12:32.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.268+0000 7f8f1b455640 1 --2- 172.21.15.148:0/231704231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f140b3430 0x7f8f1406b7f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.268+0000 7f8f1b455640 1 --2- 172.21.15.148:0/231704231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f140b2f00 0x7f8f14162680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.268+0000 7f8f1b455640 1 -- 172.21.15.148:0/231704231 >> 172.21.15.148:0/231704231 conn(0x7f8f140b6430 msgr2=0x7f8f140d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:32.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.268+0000 7f8f1b455640 1 -- 172.21.15.148:0/231704231 shutdown_connections 2024-03-29T04:12:32.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.268+0000 7f8f1b455640 1 -- 172.21.15.148:0/231704231 wait complete. 2024-03-29T04:12:32.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.269+0000 7f8f1b455640 1 Processor -- start 2024-03-29T04:12:32.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.269+0000 7f8f1b455640 1 -- start start 2024-03-29T04:12:32.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.270+0000 7f8f1b455640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f140b3430 0x7f8f14148640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:32.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.270+0000 7f8f1b455640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f1415e3c0 0x7f8f1415e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:32.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.270+0000 7f8f1b455640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f141526f0 0x7f8f14155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:32.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.270+0000 7f8f1b455640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8f140b3b20 con 0x7f8f1415e3c0 2024-03-29T04:12:32.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.270+0000 7f8f1b455640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8f140b39a0 con 0x7f8f140b3430 2024-03-29T04:12:32.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.270+0000 7f8f1b455640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8f1406bd30 con 0x7f8f141526f0 2024-03-29T04:12:32.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.270+0000 7f8f189f0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f1415e3c0 0x7f8f1415e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:32.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.270+0000 7f8f189f0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f1415e3c0 0x7f8f1415e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36304/0 (socket says 172.21.15.148:36304) 2024-03-29T04:12:32.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.270+0000 7f8f189f0640 1 -- 172.21.15.148:0/1256440012 learned_addr learned my addr 172.21.15.148:0/1256440012 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:32.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.270+0000 7f8f191f1640 1 --2- 172.21.15.148:0/1256440012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f140b3430 0x7f8f14148640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:32.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.270+0000 7f8f189f0640 1 -- 172.21.15.148:0/1256440012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f141526f0 msgr2=0x7f8f14155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:32.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.270+0000 7f8f199f2640 1 --2- 172.21.15.148:0/1256440012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f141526f0 0x7f8f14155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:32.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.270+0000 7f8f189f0640 1 --2- 172.21.15.148:0/1256440012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f141526f0 0x7f8f14155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.270+0000 7f8f189f0640 1 -- 172.21.15.148:0/1256440012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f140b3430 msgr2=0x7f8f14148640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:32.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.270+0000 7f8f189f0640 1 --2- 172.21.15.148:0/1256440012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f140b3430 0x7f8f14148640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.270+0000 7f8f189f0640 1 -- 172.21.15.148:0/1256440012 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1415eef0 con 0x7f8f1415e3c0 2024-03-29T04:12:32.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.270+0000 7f8f191f1640 1 --2- 172.21.15.148:0/1256440012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f140b3430 0x7f8f14148640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:12:32.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.270+0000 7f8f189f0640 1 --2- 172.21.15.148:0/1256440012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f1415e3c0 0x7f8f1415e9b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8f1000b4d0 tx=0x7f8f1000b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:32.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.270+0000 7f8f09ffb640 1 -- 172.21.15.148:0/1256440012 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f10014070 con 0x7f8f1415e3c0 2024-03-29T04:12:32.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.270+0000 7f8f09ffb640 1 -- 172.21.15.148:0/1256440012 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f10010610 con 0x7f8f1415e3c0 2024-03-29T04:12:32.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.270+0000 7f8f09ffb640 1 -- 172.21.15.148:0/1256440012 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f10010a00 con 0x7f8f1415e3c0 2024-03-29T04:12:32.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.270+0000 7f8f1b455640 1 -- 172.21.15.148:0/1256440012 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f14152ef0 con 0x7f8f1415e3c0 2024-03-29T04:12:32.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.271+0000 7f8f1b455640 1 -- 172.21.15.148:0/1256440012 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f140cc450 con 0x7f8f1415e3c0 2024-03-29T04:12:32.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.272+0000 7f8f09ffb640 1 -- 172.21.15.148:0/1256440012 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8f10017070 con 0x7f8f1415e3c0 2024-03-29T04:12:32.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.272+0000 7f8f1b455640 1 -- 172.21.15.148:0/1256440012 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f14068850 con 0x7f8f1415e3c0 2024-03-29T04:12:32.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.272+0000 7f8f09ffb640 1 --2- 172.21.15.148:0/1256440012 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f8ef806c560 0x7f8ef806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:32.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.272+0000 7f8f09ffb640 1 -- 172.21.15.148:0/1256440012 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f8f1008b220 con 0x7f8f1415e3c0 2024-03-29T04:12:32.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.272+0000 7f8f191f1640 1 --2- 172.21.15.148:0/1256440012 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f8ef806c560 0x7f8ef806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:32.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.273+0000 7f8f191f1640 1 --2- 172.21.15.148:0/1256440012 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f8ef806c560 0x7f8ef806ea20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8f04000f60 tx=0x7f8f04009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:32.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.277+0000 7f8f09ffb640 1 -- 172.21.15.148:0/1256440012 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8f10059690 con 0x7f8f1415e3c0 2024-03-29T04:12:32.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.523+0000 7f8f1b455640 1 -- 172.21.15.148:0/1256440012 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8f140d2940 con 0x7f8f1415e3c0 2024-03-29T04:12:32.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.523+0000 7f8f09ffb640 1 -- 172.21.15.148:0/1256440012 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) ==== 104+0+11 (secure 0 0 0) 0x7f8f10058f90 con 0x7f8f1415e3c0 2024-03-29T04:12:32.502 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:12:32.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.525+0000 7f8f1b455640 1 -- 172.21.15.148:0/1256440012 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f8ef806c560 msgr2=0x7f8ef806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:32.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.525+0000 7f8f1b455640 1 --2- 172.21.15.148:0/1256440012 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f8ef806c560 0x7f8ef806ea20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8f04000f60 tx=0x7f8f04009040 comp rx=0 tx=0).stop 2024-03-29T04:12:32.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.525+0000 7f8f1b455640 1 -- 172.21.15.148:0/1256440012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f1415e3c0 msgr2=0x7f8f1415e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:32.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.525+0000 7f8f1b455640 1 --2- 172.21.15.148:0/1256440012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f1415e3c0 0x7f8f1415e9b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8f1000b4d0 tx=0x7f8f1000b990 comp rx=0 tx=0).stop 2024-03-29T04:12:32.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.525+0000 7f8f1b455640 1 -- 172.21.15.148:0/1256440012 shutdown_connections 2024-03-29T04:12:32.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.525+0000 7f8f1b455640 1 --2- 172.21.15.148:0/1256440012 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f8ef806c560 0x7f8ef806ea20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.525+0000 7f8f1b455640 1 --2- 172.21.15.148:0/1256440012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f141526f0 0x7f8f14155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.525+0000 7f8f1b455640 1 --2- 172.21.15.148:0/1256440012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f1415e3c0 0x7f8f1415e9b0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.525+0000 7f8f1b455640 1 --2- 172.21.15.148:0/1256440012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f140b3430 0x7f8f14148640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.525+0000 7f8f1b455640 1 -- 172.21.15.148:0/1256440012 >> 172.21.15.148:0/1256440012 conn(0x7f8f140b6430 msgr2=0x7f8f140b47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:32.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.525+0000 7f8f1b455640 1 -- 172.21.15.148:0/1256440012 shutdown_connections 2024-03-29T04:12:32.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.525+0000 7f8f1b455640 1 -- 172.21.15.148:0/1256440012 wait complete. 2024-03-29T04:12:32.515 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:12:32.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.642+0000 7fdfa820d640 1 Processor -- start 2024-03-29T04:12:32.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.642+0000 7fdfa820d640 1 -- start start 2024-03-29T04:12:32.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.643+0000 7fdfa820d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdfa015e3c0 0x7fdfa0161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:32.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.643+0000 7fdfa820d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdfa00b3430 0x7fdfa006b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:32.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.643+0000 7fdfa820d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfa00cf330 con 0x7fdfa00b2f00 2024-03-29T04:12:32.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.643+0000 7fdfa820d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fdfa00cf960 con 0x7fdfa00b3430 2024-03-29T04:12:32.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.643+0000 7fdfa820d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fdfa0162cf0 con 0x7fdfa015e3c0 2024-03-29T04:12:32.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.643+0000 7fdfa67aa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdfa00b3430 0x7fdfa006b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:32.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.643+0000 7fdfa67aa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdfa00b3430 0x7fdfa006b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36318/0 (socket says 172.21.15.148:36318) 2024-03-29T04:12:32.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.643+0000 7fdfa67aa640 1 -- 172.21.15.148:0/1788081221 learned_addr learned my addr 172.21.15.148:0/1788081221 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:32.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.643+0000 7fdfa5fa9640 1 --2- 172.21.15.148:0/1788081221 >> v2:172.21.15.157:3300/0 conn(0x7fdfa015e3c0 0x7fdfa0161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:32.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.643+0000 7fdfa67aa640 1 -- 172.21.15.148:0/1788081221 >> v2:172.21.15.157:3300/0 conn(0x7fdfa015e3c0 msgr2=0x7fdfa0161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:32.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.643+0000 7fdfa67aa640 1 --2- 172.21.15.148:0/1788081221 >> v2:172.21.15.157:3300/0 conn(0x7fdfa015e3c0 0x7fdfa0161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.643+0000 7fdfa67aa640 1 -- 172.21.15.148:0/1788081221 >> v1:172.21.15.157:6789/0 conn(0x7fdfa00b2f00 legacy=0x7fdfa01624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:32.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.643+0000 7fdfa67aa640 1 -- 172.21.15.148:0/1788081221 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa00cf5f0 con 0x7fdfa00b3430 2024-03-29T04:12:32.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.644+0000 7fdfa67aa640 1 --2- 172.21.15.148:0/1788081221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfa00b3430 0x7fdfa006b7f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdfa00d2670 tx=0x7fdf9802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f175fa6a2b426a74 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:32.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.644+0000 7fdfa4fa7640 1 -- 172.21.15.148:0/1788081221 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9802ca80 con 0x7fdfa00b3430 2024-03-29T04:12:32.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.644+0000 7fdfa4fa7640 1 -- 172.21.15.148:0/1788081221 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf9802cc20 con 0x7fdfa00b3430 2024-03-29T04:12:32.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.644+0000 7fdfa4fa7640 1 -- 172.21.15.148:0/1788081221 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf980343f0 con 0x7fdfa00b3430 2024-03-29T04:12:32.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.645+0000 7fdfa820d640 1 -- 172.21.15.148:0/1788081221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfa00b3430 msgr2=0x7fdfa006b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:32.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.645+0000 7fdfa820d640 1 --2- 172.21.15.148:0/1788081221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfa00b3430 0x7fdfa006b7f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdfa00d2670 tx=0x7fdf9802bee0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.645+0000 7fdfa820d640 1 -- 172.21.15.148:0/1788081221 shutdown_connections 2024-03-29T04:12:32.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.645+0000 7fdfa820d640 1 --2- 172.21.15.148:0/1788081221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfa00b3430 0x7fdfa006b7f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.645+0000 7fdfa820d640 1 --2- 172.21.15.148:0/1788081221 >> v2:172.21.15.157:3300/0 conn(0x7fdfa015e3c0 0x7fdfa0161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.645+0000 7fdfa820d640 1 -- 172.21.15.148:0/1788081221 >> 172.21.15.148:0/1788081221 conn(0x7fdfa00b6430 msgr2=0x7fdfa00d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:32.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.645+0000 7fdfa820d640 1 -- 172.21.15.148:0/1788081221 shutdown_connections 2024-03-29T04:12:32.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.645+0000 7fdfa820d640 1 -- 172.21.15.148:0/1788081221 wait complete. 2024-03-29T04:12:32.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.646+0000 7fdfa820d640 1 Processor -- start 2024-03-29T04:12:32.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.646+0000 7fdfa820d640 1 -- start start 2024-03-29T04:12:32.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.647+0000 7fdfa820d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfa00b3430 0x7fdfa015b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:32.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.647+0000 7fdfa820d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfa015e3c0 0x7fdfa015b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:32.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.647+0000 7fdfa820d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfa0161570 0x7fdfa015fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:32.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.647+0000 7fdfa820d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa00b3b20 con 0x7fdfa00b3430 2024-03-29T04:12:32.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.647+0000 7fdfa820d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa00b39a0 con 0x7fdfa0161570 2024-03-29T04:12:32.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.647+0000 7fdfa820d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdfa006bd30 con 0x7fdfa015e3c0 2024-03-29T04:12:32.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.647+0000 7fdfa5fa9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfa00b3430 0x7fdfa015b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:32.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.647+0000 7fdfa5fa9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfa00b3430 0x7fdfa015b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36330/0 (socket says 172.21.15.148:36330) 2024-03-29T04:12:32.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.647+0000 7fdfa5fa9640 1 -- 172.21.15.148:0/3373614091 learned_addr learned my addr 172.21.15.148:0/3373614091 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:32.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.647+0000 7fdfa67aa640 1 --2- 172.21.15.148:0/3373614091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfa0161570 0x7fdfa015fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:32.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.647+0000 7fdfa57a8640 1 --2- 172.21.15.148:0/3373614091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfa015e3c0 0x7fdfa015b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:32.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.647+0000 7fdfa5fa9640 1 -- 172.21.15.148:0/3373614091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfa015e3c0 msgr2=0x7fdfa015b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:32.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.647+0000 7fdfa5fa9640 1 --2- 172.21.15.148:0/3373614091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfa015e3c0 0x7fdfa015b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.647+0000 7fdfa5fa9640 1 -- 172.21.15.148:0/3373614091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfa0161570 msgr2=0x7fdfa015fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:32.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.647+0000 7fdfa5fa9640 1 --2- 172.21.15.148:0/3373614091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfa0161570 0x7fdfa015fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.647+0000 7fdfa5fa9640 1 -- 172.21.15.148:0/3373614091 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa00c7f30 con 0x7fdfa00b3430 2024-03-29T04:12:32.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.647+0000 7fdfa57a8640 1 --2- 172.21.15.148:0/3373614091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfa015e3c0 0x7fdfa015b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:12:32.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.647+0000 7fdfa5fa9640 1 --2- 172.21.15.148:0/3373614091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfa00b3430 0x7fdfa015b360 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c00e8d0 tx=0x7fdf9c00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:32.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.647+0000 7fdf967fc640 1 -- 172.21.15.148:0/3373614091 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c017070 con 0x7fdfa00b3430 2024-03-29T04:12:32.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.647+0000 7fdf967fc640 1 -- 172.21.15.148:0/3373614091 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf9c00c8e0 con 0x7fdfa00b3430 2024-03-29T04:12:32.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.647+0000 7fdf967fc640 1 -- 172.21.15.148:0/3373614091 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c00ccb0 con 0x7fdfa00b3430 2024-03-29T04:12:32.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.647+0000 7fdfa820d640 1 -- 172.21.15.148:0/3373614091 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa00c8210 con 0x7fdfa00b3430 2024-03-29T04:12:32.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.647+0000 7fdfa820d640 1 -- 172.21.15.148:0/3373614091 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa00c87d0 con 0x7fdfa00b3430 2024-03-29T04:12:32.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.648+0000 7fdfa820d640 1 -- 172.21.15.148:0/3373614091 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfa0068850 con 0x7fdfa00b3430 2024-03-29T04:12:32.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.649+0000 7fdf967fc640 1 -- 172.21.15.148:0/3373614091 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdf9c01a070 con 0x7fdfa00b3430 2024-03-29T04:12:32.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.649+0000 7fdf967fc640 1 --2- 172.21.15.148:0/3373614091 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fdf8406c560 0x7fdf8406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:32.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.650+0000 7fdfa57a8640 1 --2- 172.21.15.148:0/3373614091 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fdf8406c560 0x7fdf8406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:32.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.650+0000 7fdf967fc640 1 -- 172.21.15.148:0/3373614091 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7fdf9c08b8f0 con 0x7fdfa00b3430 2024-03-29T04:12:32.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.650+0000 7fdfa57a8640 1 --2- 172.21.15.148:0/3373614091 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fdf8406c560 0x7fdf8406ea20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdfa015dab0 tx=0x7fdf90009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:32.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.657+0000 7fdf967fc640 1 -- 172.21.15.148:0/3373614091 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fdf9c059df0 con 0x7fdfa00b3430 2024-03-29T04:12:32.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.841+0000 7fdfa820d640 1 -- 172.21.15.148:0/3373614091 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdfa00c75b0 con 0x7fdfa00b3430 2024-03-29T04:12:32.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.842+0000 7fdf967fc640 1 -- 172.21.15.148:0/3373614091 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) ==== 100+0+11 (secure 0 0 0) 0x7fdf9c0596f0 con 0x7fdfa00b3430 2024-03-29T04:12:32.821 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-29T04:12:32.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.844+0000 7fdfa820d640 1 -- 172.21.15.148:0/3373614091 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fdf8406c560 msgr2=0x7fdf8406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:32.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.844+0000 7fdfa820d640 1 --2- 172.21.15.148:0/3373614091 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fdf8406c560 0x7fdf8406ea20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdfa015dab0 tx=0x7fdf90009040 comp rx=0 tx=0).stop 2024-03-29T04:12:32.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.844+0000 7fdfa820d640 1 -- 172.21.15.148:0/3373614091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfa00b3430 msgr2=0x7fdfa015b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:32.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.844+0000 7fdfa820d640 1 --2- 172.21.15.148:0/3373614091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfa00b3430 0x7fdfa015b360 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c00e8d0 tx=0x7fdf9c00ed90 comp rx=0 tx=0).stop 2024-03-29T04:12:32.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.844+0000 7fdfa820d640 1 -- 172.21.15.148:0/3373614091 shutdown_connections 2024-03-29T04:12:32.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.844+0000 7fdfa820d640 1 --2- 172.21.15.148:0/3373614091 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fdf8406c560 0x7fdf8406ea20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.844+0000 7fdfa820d640 1 --2- 172.21.15.148:0/3373614091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfa0161570 0x7fdfa015fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.844+0000 7fdfa820d640 1 --2- 172.21.15.148:0/3373614091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdfa015e3c0 0x7fdfa015b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.844+0000 7fdfa820d640 1 --2- 172.21.15.148:0/3373614091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfa00b3430 0x7fdfa015b360 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.844+0000 7fdfa820d640 1 -- 172.21.15.148:0/3373614091 >> 172.21.15.148:0/3373614091 conn(0x7fdfa00b6430 msgr2=0x7fdfa00d0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:32.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.844+0000 7fdfa820d640 1 -- 172.21.15.148:0/3373614091 shutdown_connections 2024-03-29T04:12:32.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.844+0000 7fdfa820d640 1 -- 172.21.15.148:0/3373614091 wait complete. 2024-03-29T04:12:32.834 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:12:32.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.958+0000 7f97e3ac8640 1 Processor -- start 2024-03-29T04:12:32.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.958+0000 7f97e3ac8640 1 -- start start 2024-03-29T04:12:32.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.958+0000 7f97e3ac8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f97dc15e3e0 0x7f97dc0d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:32.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.958+0000 7f97e3ac8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97dc0d10d0 0x7f97dc0d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:32.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.959+0000 7f97e3ac8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97dc15e7c0 con 0x7f97dc0ba030 2024-03-29T04:12:32.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.959+0000 7f97e3ac8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f97dc0d3840 con 0x7f97dc15e3e0 2024-03-29T04:12:32.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.959+0000 7f97e3ac8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f97dc0d2c30 con 0x7f97dc0d10d0 2024-03-29T04:12:32.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.959+0000 7f97e1864640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f97dc15e3e0 0x7f97dc0d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:32.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.959+0000 7f97e1063640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f97dc0ba030 0x7f97dc0d2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39366/0 (socket says 172.21.15.148:39366) 2024-03-29T04:12:32.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.959+0000 7f97e1063640 1 -- 172.21.15.148:0/13221915 learned_addr learned my addr 172.21.15.148:0/13221915 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:32.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.959+0000 7f97e2065640 1 --2- 172.21.15.148:0/13221915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97dc0d10d0 0x7f97dc0d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:32.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.959+0000 7f97e1864640 1 -- 172.21.15.148:0/13221915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97dc0d10d0 msgr2=0x7f97dc0d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:32.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.959+0000 7f97e0862640 1 -- 172.21.15.148:0/13221915 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4171394366 0 0) 0x7f97dc15e7c0 con 0x7f97dc0ba030 2024-03-29T04:12:32.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.959+0000 7f97e1864640 1 --2- 172.21.15.148:0/13221915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97dc0d10d0 0x7f97dc0d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.959+0000 7f97e1864640 1 -- 172.21.15.148:0/13221915 >> v1:172.21.15.148:6789/0 conn(0x7f97dc0ba030 legacy=0x7f97dc0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:32.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.959+0000 7f97e1864640 1 -- 172.21.15.148:0/13221915 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97dc0d2f10 con 0x7f97dc15e3e0 2024-03-29T04:12:32.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.960+0000 7f97e1864640 1 --2- 172.21.15.148:0/13221915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97dc15e3e0 0x7f97dc0d3cd0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f97d80097a0 tx=0x7f97d802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc04a55759f6f38e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:32.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.960+0000 7f97e0862640 1 -- 172.21.15.148:0/13221915 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d8034070 con 0x7f97dc15e3e0 2024-03-29T04:12:32.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.960+0000 7f97e0862640 1 -- 172.21.15.148:0/13221915 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97d8030610 con 0x7f97dc15e3e0 2024-03-29T04:12:32.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.960+0000 7f97e0862640 1 -- 172.21.15.148:0/13221915 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d80309e0 con 0x7f97dc15e3e0 2024-03-29T04:12:32.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.961+0000 7f97e3ac8640 1 -- 172.21.15.148:0/13221915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97dc15e3e0 msgr2=0x7f97dc0d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:32.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.961+0000 7f97e3ac8640 1 --2- 172.21.15.148:0/13221915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97dc15e3e0 0x7f97dc0d3cd0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f97d80097a0 tx=0x7f97d802b960 comp rx=0 tx=0).stop 2024-03-29T04:12:32.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.961+0000 7f97e3ac8640 1 -- 172.21.15.148:0/13221915 shutdown_connections 2024-03-29T04:12:32.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.961+0000 7f97e3ac8640 1 --2- 172.21.15.148:0/13221915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97dc0d10d0 0x7f97dc0d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.961+0000 7f97e3ac8640 1 --2- 172.21.15.148:0/13221915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97dc15e3e0 0x7f97dc0d3cd0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.961+0000 7f97e3ac8640 1 -- 172.21.15.148:0/13221915 >> 172.21.15.148:0/13221915 conn(0x7f97dc0b6210 msgr2=0x7f97dc0b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:32.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.961+0000 7f97e3ac8640 1 -- 172.21.15.148:0/13221915 shutdown_connections 2024-03-29T04:12:32.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.961+0000 7f97e3ac8640 1 -- 172.21.15.148:0/13221915 wait complete. 2024-03-29T04:12:32.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.963+0000 7f97e3ac8640 1 Processor -- start 2024-03-29T04:12:32.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.963+0000 7f97e3ac8640 1 -- start start 2024-03-29T04:12:32.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.964+0000 7f97e3ac8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97dc0ba030 0x7f97dc15d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:32.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.964+0000 7f97e3ac8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97dc0d10d0 0x7f97dc149700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:32.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.964+0000 7f97e3ac8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97dc15e3e0 0x7f97dc161160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:32.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.964+0000 7f97e3ac8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc167f50 con 0x7f97dc15e3e0 2024-03-29T04:12:32.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.964+0000 7f97e3ac8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc0d34d0 con 0x7f97dc0d10d0 2024-03-29T04:12:32.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.964+0000 7f97e3ac8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f97dc1680d0 con 0x7f97dc0ba030 2024-03-29T04:12:32.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.964+0000 7f97e2065640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97dc15e3e0 0x7f97dc161160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:32.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.964+0000 7f97e2065640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97dc15e3e0 0x7f97dc161160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36340/0 (socket says 172.21.15.148:36340) 2024-03-29T04:12:32.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.964+0000 7f97e2065640 1 -- 172.21.15.148:0/3265766772 learned_addr learned my addr 172.21.15.148:0/3265766772 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:32.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.964+0000 7f97e1864640 1 --2- 172.21.15.148:0/3265766772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97dc0ba030 0x7f97dc15d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:32.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.964+0000 7f97e1063640 1 --2- 172.21.15.148:0/3265766772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97dc0d10d0 0x7f97dc149700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:32.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.964+0000 7f97e2065640 1 -- 172.21.15.148:0/3265766772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97dc0ba030 msgr2=0x7f97dc15d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:32.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.964+0000 7f97e2065640 1 --2- 172.21.15.148:0/3265766772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97dc0ba030 0x7f97dc15d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.964+0000 7f97e2065640 1 -- 172.21.15.148:0/3265766772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97dc0d10d0 msgr2=0x7f97dc149700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:32.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.964+0000 7f97e2065640 1 --2- 172.21.15.148:0/3265766772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97dc0d10d0 0x7f97dc149700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:32.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.964+0000 7f97e2065640 1 -- 172.21.15.148:0/3265766772 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97dc14a730 con 0x7f97dc15e3e0 2024-03-29T04:12:32.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.965+0000 7f97e2065640 1 --2- 172.21.15.148:0/3265766772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97dc15e3e0 0x7f97dc161160 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f97d400cd70 tx=0x7f97d400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:32.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.965+0000 7f97d27fc640 1 -- 172.21.15.148:0/3265766772 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d4003b90 con 0x7f97dc15e3e0 2024-03-29T04:12:32.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.965+0000 7f97d27fc640 1 -- 172.21.15.148:0/3265766772 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97d4003d30 con 0x7f97dc15e3e0 2024-03-29T04:12:32.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.965+0000 7f97e3ac8640 1 -- 172.21.15.148:0/3265766772 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97dc0ca0b0 con 0x7f97dc15e3e0 2024-03-29T04:12:32.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.965+0000 7f97d27fc640 1 -- 172.21.15.148:0/3265766772 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d4004240 con 0x7f97dc15e3e0 2024-03-29T04:12:32.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.965+0000 7f97e3ac8640 1 -- 172.21.15.148:0/3265766772 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97dc160c40 con 0x7f97dc15e3e0 2024-03-29T04:12:32.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.966+0000 7f97e3ac8640 1 -- 172.21.15.148:0/3265766772 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97dc160a70 con 0x7f97dc15e3e0 2024-03-29T04:12:32.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.966+0000 7f97d27fc640 1 -- 172.21.15.148:0/3265766772 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f97d40043e0 con 0x7f97dc15e3e0 2024-03-29T04:12:32.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.967+0000 7f97d27fc640 1 --2- 172.21.15.148:0/3265766772 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f97c006c530 0x7f97c006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:32.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.967+0000 7f97d27fc640 1 -- 172.21.15.148:0/3265766772 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f97d408c110 con 0x7f97dc15e3e0 2024-03-29T04:12:32.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.967+0000 7f97e1864640 1 --2- 172.21.15.148:0/3265766772 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f97c006c530 0x7f97c006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:32.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.967+0000 7f97e1864640 1 --2- 172.21.15.148:0/3265766772 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f97c006c530 0x7f97c006e9f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f97d8036000 tx=0x7f97d8008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:32.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:32.971+0000 7f97d27fc640 1 -- 172.21.15.148:0/3265766772 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f97d4059fc0 con 0x7f97dc15e3e0 2024-03-29T04:12:33.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.201+0000 7f97e3ac8640 1 -- 172.21.15.148:0/3265766772 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f97dc160e10 con 0x7f97dc15e3e0 2024-03-29T04:12:33.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.202+0000 7f97d27fc640 1 -- 172.21.15.148:0/3265766772 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) ==== 80+0+0 (secure 0 0 0) 0x7f97d405e2c0 con 0x7f97dc15e3e0 2024-03-29T04:12:33.181 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-03-29T04:12:33.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.204+0000 7f97e3ac8640 1 -- 172.21.15.148:0/3265766772 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f97c006c530 msgr2=0x7f97c006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:33.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.204+0000 7f97e3ac8640 1 --2- 172.21.15.148:0/3265766772 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f97c006c530 0x7f97c006e9f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f97d8036000 tx=0x7f97d8008030 comp rx=0 tx=0).stop 2024-03-29T04:12:33.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.204+0000 7f97e3ac8640 1 -- 172.21.15.148:0/3265766772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97dc15e3e0 msgr2=0x7f97dc161160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:33.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.204+0000 7f97e3ac8640 1 --2- 172.21.15.148:0/3265766772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97dc15e3e0 0x7f97dc161160 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f97d400cd70 tx=0x7f97d400a660 comp rx=0 tx=0).stop 2024-03-29T04:12:33.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.204+0000 7f97e3ac8640 1 -- 172.21.15.148:0/3265766772 shutdown_connections 2024-03-29T04:12:33.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.204+0000 7f97e3ac8640 1 --2- 172.21.15.148:0/3265766772 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f97c006c530 0x7f97c006e9f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:33.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.204+0000 7f97e3ac8640 1 --2- 172.21.15.148:0/3265766772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97dc15e3e0 0x7f97dc161160 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:33.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.204+0000 7f97e3ac8640 1 --2- 172.21.15.148:0/3265766772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97dc0d10d0 0x7f97dc149700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:33.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.204+0000 7f97e3ac8640 1 --2- 172.21.15.148:0/3265766772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97dc0ba030 0x7f97dc15d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:33.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.204+0000 7f97e3ac8640 1 -- 172.21.15.148:0/3265766772 >> 172.21.15.148:0/3265766772 conn(0x7f97dc0b6210 msgr2=0x7f97dc153bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:33.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.204+0000 7f97e3ac8640 1 -- 172.21.15.148:0/3265766772 shutdown_connections 2024-03-29T04:12:33.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.204+0000 7f97e3ac8640 1 -- 172.21.15.148:0/3265766772 wait complete. 2024-03-29T04:12:33.194 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi148.front.sepia.ceph.com... 2024-03-29T04:12:33.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:12:33.194 DEBUG:teuthology.orchestra.run.smithi148:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T04:12:33.219 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi148.stderr:ceph-fuse[54627]: fuse finished with error 0 and tester_r 0 2024-03-29T04:12:33.231 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-03-29T04:12:33.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:12:33.231 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T04:12:33.374 INFO:tasks.cephfs.mount:Terminating background process 2024-03-29T04:12:33.374 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:12:33.374 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:12:33.392 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:12:33.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.514+0000 7fd52d9af640 1 Processor -- start 2024-03-29T04:12:33.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.514+0000 7fd52d9af640 1 -- start start 2024-03-29T04:12:33.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.515+0000 7fd52d9af640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5280cef80 0x7fd5280b7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:33.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.515+0000 7fd52d9af640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd5280d3430 0x7fd5280cf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:33.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.515+0000 7fd52d9af640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5280b7c50 con 0x7fd5280cf8a0 2024-03-29T04:12:33.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.515+0000 7fd52d9af640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd52815ec20 con 0x7fd5280d3430 2024-03-29T04:12:33.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.515+0000 7fd52d9af640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd5280b47b0 con 0x7fd5280cef80 2024-03-29T04:12:33.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.515+0000 7fd526ffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5280cef80 0x7fd5280b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:33.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.515+0000 7fd5267fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd5280d3430 0x7fd5280cf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:33.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.515+0000 7fd5277fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd5280cf8a0 0x7fd5280b4c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.148:44338/0 (socket says 172.21.15.148:44338) 2024-03-29T04:12:33.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.515+0000 7fd5277fe640 1 -- 172.21.15.148:0/2351902915 learned_addr learned my addr 172.21.15.148:0/2351902915 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:33.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.515+0000 7fd526ffd640 1 -- 172.21.15.148:0/2351902915 >> v2:172.21.15.157:3300/0 conn(0x7fd5280d3430 msgr2=0x7fd5280cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:33.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.515+0000 7fd526ffd640 1 --2- 172.21.15.148:0/2351902915 >> v2:172.21.15.157:3300/0 conn(0x7fd5280d3430 0x7fd5280cf360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:33.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.515+0000 7fd526ffd640 1 -- 172.21.15.148:0/2351902915 >> v1:172.21.15.157:6789/0 conn(0x7fd5280cf8a0 legacy=0x7fd5280b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:33.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.515+0000 7fd526ffd640 1 -- 172.21.15.148:0/2351902915 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5280b3ba0 con 0x7fd5280cef80 2024-03-29T04:12:33.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.515+0000 7fd5267fc640 1 --2- 172.21.15.148:0/2351902915 >> v2:172.21.15.157:3300/0 conn(0x7fd5280d3430 0x7fd5280cf360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:12:33.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.516+0000 7fd526ffd640 1 --2- 172.21.15.148:0/2351902915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5280cef80 0x7fd5280b7710 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd5180097a0 tx=0x7fd51802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3c0ce1ebf388999 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:33.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.516+0000 7fd525ffb640 1 -- 172.21.15.148:0/2351902915 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518035020 con 0x7fd5280cef80 2024-03-29T04:12:33.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.516+0000 7fd525ffb640 1 -- 172.21.15.148:0/2351902915 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd518031830 con 0x7fd5280cef80 2024-03-29T04:12:33.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.516+0000 7fd525ffb640 1 -- 172.21.15.148:0/2351902915 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518031c00 con 0x7fd5280cef80 2024-03-29T04:12:33.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.517+0000 7fd52d9af640 1 -- 172.21.15.148:0/2351902915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5280cef80 msgr2=0x7fd5280b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:33.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.517+0000 7fd52d9af640 1 --2- 172.21.15.148:0/2351902915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5280cef80 0x7fd5280b7710 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd5180097a0 tx=0x7fd51802b8d0 comp rx=0 tx=0).stop 2024-03-29T04:12:33.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.517+0000 7fd52d9af640 1 -- 172.21.15.148:0/2351902915 shutdown_connections 2024-03-29T04:12:33.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.517+0000 7fd52d9af640 1 --2- 172.21.15.148:0/2351902915 >> v2:172.21.15.157:3300/0 conn(0x7fd5280d3430 0x7fd5280cf360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:33.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.517+0000 7fd52d9af640 1 --2- 172.21.15.148:0/2351902915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5280cef80 0x7fd5280b7710 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:33.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.517+0000 7fd52d9af640 1 -- 172.21.15.148:0/2351902915 >> 172.21.15.148:0/2351902915 conn(0x7fd5280d4300 msgr2=0x7fd5280d4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:33.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.517+0000 7fd52d9af640 1 -- 172.21.15.148:0/2351902915 shutdown_connections 2024-03-29T04:12:33.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.517+0000 7fd52d9af640 1 -- 172.21.15.148:0/2351902915 wait complete. 2024-03-29T04:12:33.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.518+0000 7fd52d9af640 1 Processor -- start 2024-03-29T04:12:33.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.518+0000 7fd52d9af640 1 -- start start 2024-03-29T04:12:33.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.519+0000 7fd52d9af640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5280cef80 0x7fd528160b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:33.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.519+0000 7fd52d9af640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5280cf8a0 0x7fd5280c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:33.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.519+0000 7fd52d9af640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5280d3430 0x7fd5280b6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:33.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.519+0000 7fd52d9af640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd5280b4440 con 0x7fd5280cef80 2024-03-29T04:12:33.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.519+0000 7fd52d9af640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd5280b4160 con 0x7fd5280cf8a0 2024-03-29T04:12:33.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.519+0000 7fd52d9af640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd5280d26b0 con 0x7fd5280d3430 2024-03-29T04:12:33.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.519+0000 7fd526ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5280cef80 0x7fd528160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:33.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.519+0000 7fd526ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5280cef80 0x7fd528160b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36342/0 (socket says 172.21.15.148:36342) 2024-03-29T04:12:33.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.519+0000 7fd526ffd640 1 -- 172.21.15.148:0/3325065554 learned_addr learned my addr 172.21.15.148:0/3325065554 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:33.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.519+0000 7fd5267fc640 1 --2- 172.21.15.148:0/3325065554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5280cf8a0 0x7fd5280c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:33.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.519+0000 7fd5277fe640 1 --2- 172.21.15.148:0/3325065554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5280d3430 0x7fd5280b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:33.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.519+0000 7fd526ffd640 1 -- 172.21.15.148:0/3325065554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5280d3430 msgr2=0x7fd5280b6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:33.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.519+0000 7fd526ffd640 1 --2- 172.21.15.148:0/3325065554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5280d3430 0x7fd5280b6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:33.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.519+0000 7fd526ffd640 1 -- 172.21.15.148:0/3325065554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5280cf8a0 msgr2=0x7fd5280c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:33.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.519+0000 7fd526ffd640 1 --2- 172.21.15.148:0/3325065554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5280cf8a0 0x7fd5280c9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:33.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.519+0000 7fd526ffd640 1 -- 172.21.15.148:0/3325065554 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52815ee40 con 0x7fd5280cef80 2024-03-29T04:12:33.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.520+0000 7fd526ffd640 1 --2- 172.21.15.148:0/3325065554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5280cef80 0x7fd528160b00 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd5180098c0 tx=0x7fd5180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:33.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.520+0000 7fd50ffff640 1 -- 172.21.15.148:0/3325065554 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518041070 con 0x7fd5280cef80 2024-03-29T04:12:33.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.520+0000 7fd50ffff640 1 -- 172.21.15.148:0/3325065554 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd51803b920 con 0x7fd5280cef80 2024-03-29T04:12:33.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.520+0000 7fd52d9af640 1 -- 172.21.15.148:0/3325065554 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd52815f180 con 0x7fd5280cef80 2024-03-29T04:12:33.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.520+0000 7fd50ffff640 1 -- 172.21.15.148:0/3325065554 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518035040 con 0x7fd5280cef80 2024-03-29T04:12:33.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.520+0000 7fd52d9af640 1 -- 172.21.15.148:0/3325065554 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd52815f9c0 con 0x7fd5280cef80 2024-03-29T04:12:33.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.521+0000 7fd52d9af640 1 -- 172.21.15.148:0/3325065554 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5280b3830 con 0x7fd5280cef80 2024-03-29T04:12:33.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.521+0000 7fd50ffff640 1 -- 172.21.15.148:0/3325065554 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd518038020 con 0x7fd5280cef80 2024-03-29T04:12:33.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.522+0000 7fd50ffff640 1 --2- 172.21.15.148:0/3325065554 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd51406c530 0x7fd51406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:33.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.522+0000 7fd50ffff640 1 -- 172.21.15.148:0/3325065554 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7fd5180ad140 con 0x7fd5280cef80 2024-03-29T04:12:33.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.522+0000 7fd5267fc640 1 --2- 172.21.15.148:0/3325065554 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd51406c530 0x7fd51406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:33.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.522+0000 7fd5267fc640 1 --2- 172.21.15.148:0/3325065554 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd51406c530 0x7fd51406e9f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd51c009560 tx=0x7fd51c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:33.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.527+0000 7fd50ffff640 1 -- 172.21.15.148:0/3325065554 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd5280b3830 con 0x7fd5280cef80 2024-03-29T04:12:33.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.748+0000 7fd52d9af640 1 -- 172.21.15.148:0/3325065554 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5280ce400 con 0x7fd5280cef80 2024-03-29T04:12:33.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.750+0000 7fd50ffff640 1 -- 172.21.15.148:0/3325065554 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+17003 (secure 0 0 0) 0x7fd5180a4e20 con 0x7fd5280cef80 2024-03-29T04:12:33.729 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:12:33.729 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":41,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:11:42.632417+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:11:30.794790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T04:11:31.907100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:12:33.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.752+0000 7fd52d9af640 1 -- 172.21.15.148:0/3325065554 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd51406c530 msgr2=0x7fd51406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:33.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.752+0000 7fd52d9af640 1 --2- 172.21.15.148:0/3325065554 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd51406c530 0x7fd51406e9f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd51c009560 tx=0x7fd51c009190 comp rx=0 tx=0).stop 2024-03-29T04:12:33.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.752+0000 7fd52d9af640 1 -- 172.21.15.148:0/3325065554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5280cef80 msgr2=0x7fd528160b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:33.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.752+0000 7fd52d9af640 1 --2- 172.21.15.148:0/3325065554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5280cef80 0x7fd528160b00 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd5180098c0 tx=0x7fd5180046c0 comp rx=0 tx=0).stop 2024-03-29T04:12:33.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.752+0000 7fd52d9af640 1 -- 172.21.15.148:0/3325065554 shutdown_connections 2024-03-29T04:12:33.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.752+0000 7fd52d9af640 1 --2- 172.21.15.148:0/3325065554 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fd51406c530 0x7fd51406e9f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:33.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.752+0000 7fd52d9af640 1 --2- 172.21.15.148:0/3325065554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5280d3430 0x7fd5280b6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:33.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.752+0000 7fd52d9af640 1 --2- 172.21.15.148:0/3325065554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5280cf8a0 0x7fd5280c9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:33.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.752+0000 7fd52d9af640 1 --2- 172.21.15.148:0/3325065554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5280cef80 0x7fd528160b00 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:33.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.752+0000 7fd52d9af640 1 -- 172.21.15.148:0/3325065554 >> 172.21.15.148:0/3325065554 conn(0x7fd5280d4300 msgr2=0x7fd52806c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:33.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.753+0000 7fd52d9af640 1 -- 172.21.15.148:0/3325065554 shutdown_connections 2024-03-29T04:12:33.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.753+0000 7fd52d9af640 1 -- 172.21.15.148:0/3325065554 wait complete. 2024-03-29T04:12:33.746 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:47.898351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:11:30.794790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:11:31.907100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:12:33.746 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:12:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.871+0000 7efe7b8eb640 1 Processor -- start 2024-03-29T04:12:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.871+0000 7efe7b8eb640 1 -- start start 2024-03-29T04:12:33.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.873+0000 7efe7b8eb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efe740b2f00 0x7efe74162680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:33.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.873+0000 7efe7b8eb640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efe740b3430 0x7efe7406b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:33.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.873+0000 7efe7b8eb640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe740cf330 con 0x7efe7415e3c0 2024-03-29T04:12:33.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.873+0000 7efe7b8eb640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7efe740cf960 con 0x7efe740b3430 2024-03-29T04:12:33.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.873+0000 7efe7b8eb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7efe74162cf0 con 0x7efe740b2f00 2024-03-29T04:12:33.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.873+0000 7efe79e88640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efe740b3430 0x7efe7406b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:33.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.873+0000 7efe79687640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7efe7415e3c0 0x7efe74161f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39376/0 (socket says 172.21.15.148:39376) 2024-03-29T04:12:33.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.873+0000 7efe79687640 1 -- 172.21.15.148:0/1131184509 learned_addr learned my addr 172.21.15.148:0/1131184509 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:33.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.873+0000 7efe6bfff640 1 -- 172.21.15.148:0/1131184509 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3435161069 0 0) 0x7efe740cf330 con 0x7efe7415e3c0 2024-03-29T04:12:33.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.873+0000 7efe6bfff640 1 -- 172.21.15.148:0/1131184509 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe60003520 con 0x7efe7415e3c0 2024-03-29T04:12:33.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.873+0000 7efe78e86640 1 --2- 172.21.15.148:0/1131184509 >> v2:172.21.15.157:3300/0 conn(0x7efe740b2f00 0x7efe74162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:33.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.873+0000 7efe79e88640 1 -- 172.21.15.148:0/1131184509 >> v2:172.21.15.157:3300/0 conn(0x7efe740b2f00 msgr2=0x7efe74162680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:33.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.873+0000 7efe79e88640 1 --2- 172.21.15.148:0/1131184509 >> v2:172.21.15.157:3300/0 conn(0x7efe740b2f00 0x7efe74162680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:33.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.873+0000 7efe79e88640 1 -- 172.21.15.148:0/1131184509 >> v1:172.21.15.148:6789/0 conn(0x7efe7415e3c0 legacy=0x7efe74161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:33.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.873+0000 7efe79e88640 1 -- 172.21.15.148:0/1131184509 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe740cf5f0 con 0x7efe740b3430 2024-03-29T04:12:33.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.873+0000 7efe78e86640 1 --2- 172.21.15.148:0/1131184509 >> v2:172.21.15.157:3300/0 conn(0x7efe740b2f00 0x7efe74162680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:12:33.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.874+0000 7efe79e88640 1 --2- 172.21.15.148:0/1131184509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe740b3430 0x7efe7406b7f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7efe740c7be0 tx=0x7efe6c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11ec191eff573ab server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:33.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.874+0000 7efe6bfff640 1 -- 172.21.15.148:0/1131184509 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c039070 con 0x7efe740b3430 2024-03-29T04:12:33.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.874+0000 7efe6bfff640 1 -- 172.21.15.148:0/1131184509 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe6c02cbf0 con 0x7efe740b3430 2024-03-29T04:12:33.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.874+0000 7efe6bfff640 1 -- 172.21.15.148:0/1131184509 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c034440 con 0x7efe740b3430 2024-03-29T04:12:33.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.875+0000 7efe7b8eb640 1 -- 172.21.15.148:0/1131184509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe740b3430 msgr2=0x7efe7406b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:33.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.875+0000 7efe7b8eb640 1 --2- 172.21.15.148:0/1131184509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe740b3430 0x7efe7406b7f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7efe740c7be0 tx=0x7efe6c02be40 comp rx=0 tx=0).stop 2024-03-29T04:12:33.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.875+0000 7efe7b8eb640 1 -- 172.21.15.148:0/1131184509 shutdown_connections 2024-03-29T04:12:33.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.875+0000 7efe7b8eb640 1 --2- 172.21.15.148:0/1131184509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe740b3430 0x7efe7406b7f0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:33.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.875+0000 7efe7b8eb640 1 --2- 172.21.15.148:0/1131184509 >> v2:172.21.15.157:3300/0 conn(0x7efe740b2f00 0x7efe74162680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:33.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.875+0000 7efe7b8eb640 1 -- 172.21.15.148:0/1131184509 >> 172.21.15.148:0/1131184509 conn(0x7efe740b6430 msgr2=0x7efe740d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:33.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.875+0000 7efe7b8eb640 1 -- 172.21.15.148:0/1131184509 shutdown_connections 2024-03-29T04:12:33.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.875+0000 7efe7b8eb640 1 -- 172.21.15.148:0/1131184509 wait complete. 2024-03-29T04:12:33.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.876+0000 7efe7b8eb640 1 Processor -- start 2024-03-29T04:12:33.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.876+0000 7efe7b8eb640 1 -- start start 2024-03-29T04:12:33.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.877+0000 7efe7b8eb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe740b3430 0x7efe741f7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:33.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.877+0000 7efe7b8eb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe7415e3c0 0x7efe74161350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:33.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.877+0000 7efe7b8eb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe74152a70 0x7efe740c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:33.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.877+0000 7efe7b8eb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efe740b3b20 con 0x7efe7415e3c0 2024-03-29T04:12:33.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.877+0000 7efe7b8eb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efe740b39a0 con 0x7efe74152a70 2024-03-29T04:12:33.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.877+0000 7efe7b8eb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efe7406bd30 con 0x7efe740b3430 2024-03-29T04:12:33.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.877+0000 7efe78e86640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe7415e3c0 0x7efe74161350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:33.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.877+0000 7efe78e86640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe7415e3c0 0x7efe74161350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36362/0 (socket says 172.21.15.148:36362) 2024-03-29T04:12:33.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.877+0000 7efe78e86640 1 -- 172.21.15.148:0/253745600 learned_addr learned my addr 172.21.15.148:0/253745600 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:33.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.877+0000 7efe78e86640 1 -- 172.21.15.148:0/253745600 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe740b3430 msgr2=0x7efe741f7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:33.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.877+0000 7efe79e88640 1 --2- 172.21.15.148:0/253745600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe74152a70 0x7efe740c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:33.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.877+0000 7efe78e86640 1 --2- 172.21.15.148:0/253745600 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe740b3430 0x7efe741f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:33.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.877+0000 7efe78e86640 1 -- 172.21.15.148:0/253745600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe74152a70 msgr2=0x7efe740c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:33.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.877+0000 7efe78e86640 1 --2- 172.21.15.148:0/253745600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe74152a70 0x7efe740c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:33.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.877+0000 7efe78e86640 1 -- 172.21.15.148:0/253745600 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe741f7b50 con 0x7efe7415e3c0 2024-03-29T04:12:33.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.878+0000 7efe78e86640 1 --2- 172.21.15.148:0/253745600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe7415e3c0 0x7efe74161350 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7efe6400c7f0 tx=0x7efe6400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:33.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.878+0000 7efe69ffb640 1 -- 172.21.15.148:0/253745600 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe64015070 con 0x7efe7415e3c0 2024-03-29T04:12:33.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.878+0000 7efe69ffb640 1 -- 172.21.15.148:0/253745600 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe64007a30 con 0x7efe7415e3c0 2024-03-29T04:12:33.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.878+0000 7efe7b8eb640 1 -- 172.21.15.148:0/253745600 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe741f7e30 con 0x7efe7415e3c0 2024-03-29T04:12:33.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.878+0000 7efe69ffb640 1 -- 172.21.15.148:0/253745600 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe64007e00 con 0x7efe7415e3c0 2024-03-29T04:12:33.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.878+0000 7efe7b8eb640 1 -- 172.21.15.148:0/253745600 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe74152f20 con 0x7efe7415e3c0 2024-03-29T04:12:33.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.879+0000 7efe7b8eb640 1 -- 172.21.15.148:0/253745600 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe741f8350 con 0x7efe7415e3c0 2024-03-29T04:12:33.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.880+0000 7efe69ffb640 1 -- 172.21.15.148:0/253745600 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efe64018070 con 0x7efe7415e3c0 2024-03-29T04:12:33.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.880+0000 7efe69ffb640 1 --2- 172.21.15.148:0/253745600 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efe5806c530 0x7efe5806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:33.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.880+0000 7efe69ffb640 1 -- 172.21.15.148:0/253745600 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7efe6408bb50 con 0x7efe7415e3c0 2024-03-29T04:12:33.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.881+0000 7efe79687640 1 --2- 172.21.15.148:0/253745600 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efe5806c530 0x7efe5806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:33.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.881+0000 7efe79687640 1 --2- 172.21.15.148:0/253745600 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efe5806c530 0x7efe5806e9f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efe70000f60 tx=0x7efe700078f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:33.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:33.884+0000 7efe69ffb640 1 -- 172.21.15.148:0/253745600 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efe741f8350 con 0x7efe7415e3c0 2024-03-29T04:12:34.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.066+0000 7efe7b8eb640 1 -- 172.21.15.148:0/253745600 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efe74068850 con 0x7efe7415e3c0 2024-03-29T04:12:34.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.066+0000 7efe69ffb640 1 -- 172.21.15.148:0/253745600 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7efe6405a130 con 0x7efe7415e3c0 2024-03-29T04:12:34.045 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-29T04:12:34.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.068+0000 7efe7b8eb640 1 -- 172.21.15.148:0/253745600 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efe5806c530 msgr2=0x7efe5806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:34.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.068+0000 7efe7b8eb640 1 --2- 172.21.15.148:0/253745600 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efe5806c530 0x7efe5806e9f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efe70000f60 tx=0x7efe700078f0 comp rx=0 tx=0).stop 2024-03-29T04:12:34.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.068+0000 7efe7b8eb640 1 -- 172.21.15.148:0/253745600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe7415e3c0 msgr2=0x7efe74161350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:34.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.068+0000 7efe7b8eb640 1 --2- 172.21.15.148:0/253745600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe7415e3c0 0x7efe74161350 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7efe6400c7f0 tx=0x7efe6400cbf0 comp rx=0 tx=0).stop 2024-03-29T04:12:34.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.068+0000 7efe7b8eb640 1 -- 172.21.15.148:0/253745600 shutdown_connections 2024-03-29T04:12:34.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.068+0000 7efe7b8eb640 1 --2- 172.21.15.148:0/253745600 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7efe5806c530 0x7efe5806e9f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:34.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.068+0000 7efe7b8eb640 1 --2- 172.21.15.148:0/253745600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe74152a70 0x7efe740c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:34.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.068+0000 7efe7b8eb640 1 --2- 172.21.15.148:0/253745600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe7415e3c0 0x7efe74161350 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:34.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.068+0000 7efe7b8eb640 1 --2- 172.21.15.148:0/253745600 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe740b3430 0x7efe741f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:34.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.068+0000 7efe7b8eb640 1 -- 172.21.15.148:0/253745600 >> 172.21.15.148:0/253745600 conn(0x7efe740b6430 msgr2=0x7efe740b51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:34.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.068+0000 7efe7b8eb640 1 -- 172.21.15.148:0/253745600 shutdown_connections 2024-03-29T04:12:34.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.068+0000 7efe7b8eb640 1 -- 172.21.15.148:0/253745600 wait complete. 2024-03-29T04:12:34.058 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:12:34.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.184+0000 7fe597ea0640 1 Processor -- start 2024-03-29T04:12:34.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.184+0000 7fe597ea0640 1 -- start start 2024-03-29T04:12:34.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.185+0000 7fe597ea0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe5900b2f00 0x7fe590162680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:34.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.185+0000 7fe597ea0640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5900cf330 con 0x7fe59015e3c0 2024-03-29T04:12:34.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.185+0000 7fe597ea0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe590162cf0 con 0x7fe5900b3430 2024-03-29T04:12:34.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.185+0000 7fe597ea0640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fe5900cf960 con 0x7fe5900b2f00 2024-03-29T04:12:34.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.185+0000 7fe59543b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe5900b2f00 0x7fe590162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:34.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.185+0000 7fe595c3c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe59015e3c0 0x7fe590161f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39380/0 (socket says 172.21.15.148:39380) 2024-03-29T04:12:34.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.185+0000 7fe595c3c640 1 -- 172.21.15.148:0/2302930770 learned_addr learned my addr 172.21.15.148:0/2302930770 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:34.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.185+0000 7fe59543b640 1 -- 172.21.15.148:0/2302930770 >> v1:172.21.15.157:6789/0 conn(0x7fe5900b3430 legacy=0x7fe59006b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:34.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.185+0000 7fe59543b640 1 -- 172.21.15.148:0/2302930770 >> v1:172.21.15.148:6789/0 conn(0x7fe59015e3c0 legacy=0x7fe590161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:34.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.185+0000 7fe594c3a640 1 -- 172.21.15.148:0/2302930770 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1501268406 0 0) 0x7fe5900cf330 con 0x7fe59015e3c0 2024-03-29T04:12:34.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.185+0000 7fe59543b640 1 -- 172.21.15.148:0/2302930770 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5900cf5f0 con 0x7fe5900b2f00 2024-03-29T04:12:34.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.186+0000 7fe59543b640 1 --2- 172.21.15.148:0/2302930770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5900b2f00 0x7fe590162680 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe5800097a0 tx=0x7fe58002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee64e9e88d1ee90 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:34.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.186+0000 7fe594c3a640 1 -- 172.21.15.148:0/2302930770 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5800315a0 con 0x7fe5900b2f00 2024-03-29T04:12:34.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.186+0000 7fe594c3a640 1 -- 172.21.15.148:0/2302930770 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe580037070 con 0x7fe5900b2f00 2024-03-29T04:12:34.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.186+0000 7fe594c3a640 1 -- 172.21.15.148:0/2302930770 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe580031960 con 0x7fe5900b2f00 2024-03-29T04:12:34.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.187+0000 7fe597ea0640 1 -- 172.21.15.148:0/2302930770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5900b2f00 msgr2=0x7fe590162680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:34.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.187+0000 7fe597ea0640 1 --2- 172.21.15.148:0/2302930770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5900b2f00 0x7fe590162680 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe5800097a0 tx=0x7fe58002b840 comp rx=0 tx=0).stop 2024-03-29T04:12:34.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.187+0000 7fe597ea0640 1 -- 172.21.15.148:0/2302930770 shutdown_connections 2024-03-29T04:12:34.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.187+0000 7fe597ea0640 1 --2- 172.21.15.148:0/2302930770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5900b2f00 0x7fe590162680 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:34.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.187+0000 7fe597ea0640 1 -- 172.21.15.148:0/2302930770 >> 172.21.15.148:0/2302930770 conn(0x7fe5900b6430 msgr2=0x7fe5900d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:34.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.187+0000 7fe597ea0640 1 -- 172.21.15.148:0/2302930770 shutdown_connections 2024-03-29T04:12:34.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.187+0000 7fe597ea0640 1 -- 172.21.15.148:0/2302930770 wait complete. 2024-03-29T04:12:34.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.189+0000 7fe597ea0640 1 Processor -- start 2024-03-29T04:12:34.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.189+0000 7fe597ea0640 1 -- start start 2024-03-29T04:12:34.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.190+0000 7fe597ea0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5900b3430 0x7fe5901f83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:34.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.190+0000 7fe597ea0640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe59015e3c0 0x7fe5900c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:34.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.190+0000 7fe597ea0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe590159370 0x7fe5900d2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:34.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.190+0000 7fe597ea0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe5900b3b20 con 0x7fe5900b3430 2024-03-29T04:12:34.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.190+0000 7fe597ea0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe5900b39a0 con 0x7fe590159370 2024-03-29T04:12:34.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.190+0000 7fe597ea0640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe59006bf00 con 0x7fe59015e3c0 2024-03-29T04:12:34.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.190+0000 7fe595c3c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5900b3430 0x7fe5901f83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:34.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.190+0000 7fe595c3c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5900b3430 0x7fe5901f83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36372/0 (socket says 172.21.15.148:36372) 2024-03-29T04:12:34.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.190+0000 7fe595c3c640 1 -- 172.21.15.148:0/4134064995 learned_addr learned my addr 172.21.15.148:0/4134064995 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:34.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.190+0000 7fe59643d640 1 --2- 172.21.15.148:0/4134064995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe590159370 0x7fe5900d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:34.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.190+0000 7fe595c3c640 1 -- 172.21.15.148:0/4134064995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe59015e3c0 msgr2=0x7fe5900c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:34.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.190+0000 7fe595c3c640 1 --2- 172.21.15.148:0/4134064995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe59015e3c0 0x7fe5900c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:34.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.190+0000 7fe595c3c640 1 -- 172.21.15.148:0/4134064995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe590159370 msgr2=0x7fe5900d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:34.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.190+0000 7fe595c3c640 1 --2- 172.21.15.148:0/4134064995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe590159370 0x7fe5900d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:34.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.190+0000 7fe595c3c640 1 -- 172.21.15.148:0/4134064995 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5900ca1f0 con 0x7fe5900b3430 2024-03-29T04:12:34.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.190+0000 7fe59643d640 1 --2- 172.21.15.148:0/4134064995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe590159370 0x7fe5900d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:12:34.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.191+0000 7fe595c3c640 1 --2- 172.21.15.148:0/4134064995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5900b3430 0x7fe5901f83a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe58c00c460 tx=0x7fe58c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:34.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.191+0000 7fe5867fc640 1 -- 172.21.15.148:0/4134064995 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c018070 con 0x7fe5900b3430 2024-03-29T04:12:34.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.191+0000 7fe5867fc640 1 -- 172.21.15.148:0/4134064995 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe58c007500 con 0x7fe5900b3430 2024-03-29T04:12:34.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.191+0000 7fe597ea0640 1 -- 172.21.15.148:0/4134064995 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5900ca4d0 con 0x7fe5900b3430 2024-03-29T04:12:34.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.191+0000 7fe5867fc640 1 -- 172.21.15.148:0/4134064995 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c004a20 con 0x7fe5900b3430 2024-03-29T04:12:34.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.191+0000 7fe597ea0640 1 -- 172.21.15.148:0/4134064995 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5900caa90 con 0x7fe5900b3430 2024-03-29T04:12:34.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.192+0000 7fe597ea0640 1 -- 172.21.15.148:0/4134064995 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe590068850 con 0x7fe5900b3430 2024-03-29T04:12:34.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.192+0000 7fe5867fc640 1 -- 172.21.15.148:0/4134064995 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe58c01b070 con 0x7fe5900b3430 2024-03-29T04:12:34.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.193+0000 7fe5867fc640 1 --2- 172.21.15.148:0/4134064995 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe57406c530 0x7fe57406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:34.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.193+0000 7fe59543b640 1 --2- 172.21.15.148:0/4134064995 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe57406c530 0x7fe57406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:34.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.193+0000 7fe5867fc640 1 -- 172.21.15.148:0/4134064995 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7fe58c08ab80 con 0x7fe5900b3430 2024-03-29T04:12:34.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.193+0000 7fe59543b640 1 --2- 172.21.15.148:0/4134064995 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe57406c530 0x7fe57406e9f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe580036000 tx=0x7fe580030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:34.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.197+0000 7fe5867fc640 1 -- 172.21.15.148:0/4134064995 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe58c058ff0 con 0x7fe5900b3430 2024-03-29T04:12:34.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.371+0000 7fe597ea0640 1 -- 172.21.15.148:0/4134064995 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe59015d2a0 con 0x7fe5900b3430 2024-03-29T04:12:34.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.371+0000 7fe5867fc640 1 -- 172.21.15.148:0/4134064995 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) ==== 104+0+11 (secure 0 0 0) 0x7fe58c0588f0 con 0x7fe5900b3430 2024-03-29T04:12:34.350 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:12:34.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.373+0000 7fe597ea0640 1 -- 172.21.15.148:0/4134064995 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe57406c530 msgr2=0x7fe57406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:34.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.373+0000 7fe597ea0640 1 --2- 172.21.15.148:0/4134064995 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe57406c530 0x7fe57406e9f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe580036000 tx=0x7fe580030000 comp rx=0 tx=0).stop 2024-03-29T04:12:34.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.373+0000 7fe597ea0640 1 -- 172.21.15.148:0/4134064995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5900b3430 msgr2=0x7fe5901f83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:34.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.373+0000 7fe597ea0640 1 --2- 172.21.15.148:0/4134064995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5900b3430 0x7fe5901f83a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe58c00c460 tx=0x7fe58c00c920 comp rx=0 tx=0).stop 2024-03-29T04:12:34.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.374+0000 7fe597ea0640 1 -- 172.21.15.148:0/4134064995 shutdown_connections 2024-03-29T04:12:34.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.374+0000 7fe597ea0640 1 --2- 172.21.15.148:0/4134064995 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe57406c530 0x7fe57406e9f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:34.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.374+0000 7fe597ea0640 1 --2- 172.21.15.148:0/4134064995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe590159370 0x7fe5900d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:34.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.374+0000 7fe597ea0640 1 --2- 172.21.15.148:0/4134064995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe59015e3c0 0x7fe5900c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:34.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.374+0000 7fe597ea0640 1 --2- 172.21.15.148:0/4134064995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5900b3430 0x7fe5901f83a0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:34.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.374+0000 7fe597ea0640 1 -- 172.21.15.148:0/4134064995 >> 172.21.15.148:0/4134064995 conn(0x7fe5900b6430 msgr2=0x7fe5900b54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:34.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.374+0000 7fe597ea0640 1 -- 172.21.15.148:0/4134064995 shutdown_connections 2024-03-29T04:12:34.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.374+0000 7fe597ea0640 1 -- 172.21.15.148:0/4134064995 wait complete. 2024-03-29T04:12:34.364 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:12:34.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.490+0000 7f7c5e31b640 1 Processor -- start 2024-03-29T04:12:34.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.490+0000 7f7c5e31b640 1 -- start start 2024-03-29T04:12:34.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.491+0000 7f7c5e31b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7c580b2f00 0x7f7c58162680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:34.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.491+0000 7f7c5e31b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c580cf330 con 0x7f7c580b3430 2024-03-29T04:12:34.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.491+0000 7f7c5e31b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c58162cf0 con 0x7f7c5815e3c0 2024-03-29T04:12:34.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.491+0000 7f7c5e31b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f7c580cf960 con 0x7f7c580b2f00 2024-03-29T04:12:34.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.491+0000 7f7c577fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7c580b2f00 0x7f7c58162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:34.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.491+0000 7f7c5c8b8640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7c580b3430 0x7f7c5806b7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39394/0 (socket says 172.21.15.148:39394) 2024-03-29T04:12:34.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.491+0000 7f7c5c8b8640 1 -- 172.21.15.148:0/2982075825 learned_addr learned my addr 172.21.15.148:0/2982075825 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:34.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.491+0000 7f7c577fe640 1 -- 172.21.15.148:0/2982075825 >> v1:172.21.15.157:6789/0 conn(0x7f7c5815e3c0 legacy=0x7f7c58161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:34.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.491+0000 7f7c577fe640 1 -- 172.21.15.148:0/2982075825 >> v1:172.21.15.148:6789/0 conn(0x7f7c580b3430 legacy=0x7f7c5806b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:34.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.491+0000 7f7c56ffd640 1 -- 172.21.15.148:0/2982075825 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1908836170 0 0) 0x7f7c580cf330 con 0x7f7c580b3430 2024-03-29T04:12:34.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.491+0000 7f7c577fe640 1 -- 172.21.15.148:0/2982075825 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c580cf5f0 con 0x7f7c580b2f00 2024-03-29T04:12:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.492+0000 7f7c577fe640 1 --2- 172.21.15.148:0/2982075825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c580b2f00 0x7f7c58162680 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f7c400097a0 tx=0x7f7c4002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544ffe1a4e28603e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.492+0000 7f7c56ffd640 1 -- 172.21.15.148:0/2982075825 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c400315a0 con 0x7f7c580b2f00 2024-03-29T04:12:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.492+0000 7f7c56ffd640 1 -- 172.21.15.148:0/2982075825 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c40037070 con 0x7f7c580b2f00 2024-03-29T04:12:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.492+0000 7f7c56ffd640 1 -- 172.21.15.148:0/2982075825 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40031960 con 0x7f7c580b2f00 2024-03-29T04:12:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.493+0000 7f7c5e31b640 1 -- 172.21.15.148:0/2982075825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c580b2f00 msgr2=0x7f7c58162680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.493+0000 7f7c5e31b640 1 --2- 172.21.15.148:0/2982075825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c580b2f00 0x7f7c58162680 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f7c400097a0 tx=0x7f7c4002b840 comp rx=0 tx=0).stop 2024-03-29T04:12:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.493+0000 7f7c5e31b640 1 -- 172.21.15.148:0/2982075825 shutdown_connections 2024-03-29T04:12:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.493+0000 7f7c5e31b640 1 --2- 172.21.15.148:0/2982075825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c580b2f00 0x7f7c58162680 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.493+0000 7f7c5e31b640 1 -- 172.21.15.148:0/2982075825 >> 172.21.15.148:0/2982075825 conn(0x7f7c580b6430 msgr2=0x7f7c580d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.493+0000 7f7c5e31b640 1 -- 172.21.15.148:0/2982075825 shutdown_connections 2024-03-29T04:12:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.493+0000 7f7c5e31b640 1 -- 172.21.15.148:0/2982075825 wait complete. 2024-03-29T04:12:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.494+0000 7f7c5e31b640 1 Processor -- start 2024-03-29T04:12:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.494+0000 7f7c5e31b640 1 -- start start 2024-03-29T04:12:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.495+0000 7f7c5e31b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c580b3430 0x7f7c580cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.495+0000 7f7c5e31b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c5815e3c0 0x7f7c5814a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.495+0000 7f7c5e31b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c580b88d0 0x7f7c580b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.495+0000 7f7c5e31b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7c580b3b20 con 0x7f7c5815e3c0 2024-03-29T04:12:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.495+0000 7f7c5e31b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7c580b39a0 con 0x7f7c580b88d0 2024-03-29T04:12:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.495+0000 7f7c5e31b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7c5806bf00 con 0x7f7c580b3430 2024-03-29T04:12:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.495+0000 7f7c577fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c5815e3c0 0x7f7c5814a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.495+0000 7f7c577fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c5815e3c0 0x7f7c5814a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36392/0 (socket says 172.21.15.148:36392) 2024-03-29T04:12:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.495+0000 7f7c577fe640 1 -- 172.21.15.148:0/579621442 learned_addr learned my addr 172.21.15.148:0/579621442 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.495+0000 7f7c5c8b8640 1 --2- 172.21.15.148:0/579621442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c580b88d0 0x7f7c580b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.495+0000 7f7c577fe640 1 -- 172.21.15.148:0/579621442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c580b3430 msgr2=0x7f7c580cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.495+0000 7f7c577fe640 1 --2- 172.21.15.148:0/579621442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c580b3430 0x7f7c580cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.495+0000 7f7c577fe640 1 -- 172.21.15.148:0/579621442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c580b88d0 msgr2=0x7f7c580b8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.495+0000 7f7c577fe640 1 --2- 172.21.15.148:0/579621442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c580b88d0 0x7f7c580b8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.495+0000 7f7c577fe640 1 -- 172.21.15.148:0/579621442 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c58161250 con 0x7f7c5815e3c0 2024-03-29T04:12:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.495+0000 7f7c5c8b8640 1 --2- 172.21.15.148:0/579621442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c580b88d0 0x7f7c580b8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:12:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.496+0000 7f7c577fe640 1 --2- 172.21.15.148:0/579621442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c5815e3c0 0x7f7c5814a7d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7c400097a0 tx=0x7f7c400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.496+0000 7f7c54ff9640 1 -- 172.21.15.148:0/579621442 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40004030 con 0x7f7c5815e3c0 2024-03-29T04:12:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.496+0000 7f7c54ff9640 1 -- 172.21.15.148:0/579621442 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c400041f0 con 0x7f7c5815e3c0 2024-03-29T04:12:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.496+0000 7f7c5e31b640 1 -- 172.21.15.148:0/579621442 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c58161530 con 0x7f7c5815e3c0 2024-03-29T04:12:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.496+0000 7f7c54ff9640 1 -- 172.21.15.148:0/579621442 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40032640 con 0x7f7c5815e3c0 2024-03-29T04:12:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.496+0000 7f7c5e31b640 1 -- 172.21.15.148:0/579621442 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c58161810 con 0x7f7c5815e3c0 2024-03-29T04:12:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.497+0000 7f7c5e31b640 1 -- 172.21.15.148:0/579621442 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c58068850 con 0x7f7c5815e3c0 2024-03-29T04:12:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.497+0000 7f7c54ff9640 1 -- 172.21.15.148:0/579621442 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7c400327e0 con 0x7f7c5815e3c0 2024-03-29T04:12:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.498+0000 7f7c54ff9640 1 --2- 172.21.15.148:0/579621442 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7c3806c560 0x7f7c3806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.498+0000 7f7c54ff9640 1 -- 172.21.15.148:0/579621442 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f7c4002f030 con 0x7f7c5815e3c0 2024-03-29T04:12:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.498+0000 7f7c57fff640 1 --2- 172.21.15.148:0/579621442 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7c3806c560 0x7f7c3806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:34.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.498+0000 7f7c57fff640 1 --2- 172.21.15.148:0/579621442 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7c3806c560 0x7f7c3806ea20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7c48006fd0 tx=0x7f7c48006bc0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:34.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.502+0000 7f7c54ff9640 1 -- 172.21.15.148:0/579621442 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7c40041070 con 0x7f7c5815e3c0 2024-03-29T04:12:34.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.697+0000 7f7c5e31b640 1 -- 172.21.15.148:0/579621442 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7c580d3b50 con 0x7f7c5815e3c0 2024-03-29T04:12:34.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.697+0000 7f7c54ff9640 1 -- 172.21.15.148:0/579621442 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) ==== 100+0+11 (secure 0 0 0) 0x7f7c4007bdf0 con 0x7f7c5815e3c0 2024-03-29T04:12:34.676 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-29T04:12:34.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.700+0000 7f7c5e31b640 1 -- 172.21.15.148:0/579621442 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7c3806c560 msgr2=0x7f7c3806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:34.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.700+0000 7f7c5e31b640 1 --2- 172.21.15.148:0/579621442 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7c3806c560 0x7f7c3806ea20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7c48006fd0 tx=0x7f7c48006bc0 comp rx=0 tx=0).stop 2024-03-29T04:12:34.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.700+0000 7f7c5e31b640 1 -- 172.21.15.148:0/579621442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c5815e3c0 msgr2=0x7f7c5814a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:34.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.700+0000 7f7c5e31b640 1 --2- 172.21.15.148:0/579621442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c5815e3c0 0x7f7c5814a7d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7c400097a0 tx=0x7f7c400046c0 comp rx=0 tx=0).stop 2024-03-29T04:12:34.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.700+0000 7f7c5e31b640 1 -- 172.21.15.148:0/579621442 shutdown_connections 2024-03-29T04:12:34.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.700+0000 7f7c5e31b640 1 --2- 172.21.15.148:0/579621442 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f7c3806c560 0x7f7c3806ea20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:34.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.700+0000 7f7c5e31b640 1 --2- 172.21.15.148:0/579621442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c580b88d0 0x7f7c580b8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:34.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.700+0000 7f7c5e31b640 1 --2- 172.21.15.148:0/579621442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c5815e3c0 0x7f7c5814a7d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:34.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.700+0000 7f7c5e31b640 1 --2- 172.21.15.148:0/579621442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c580b3430 0x7f7c580cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:34.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.700+0000 7f7c5e31b640 1 -- 172.21.15.148:0/579621442 >> 172.21.15.148:0/579621442 conn(0x7f7c580b6430 msgr2=0x7f7c580b4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:34.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.700+0000 7f7c5e31b640 1 -- 172.21.15.148:0/579621442 shutdown_connections 2024-03-29T04:12:34.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.700+0000 7f7c5e31b640 1 -- 172.21.15.148:0/579621442 wait complete. 2024-03-29T04:12:34.690 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:12:34.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.814+0000 7f804fb90640 1 Processor -- start 2024-03-29T04:12:34.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.814+0000 7f804fb90640 1 -- start start 2024-03-29T04:12:34.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.815+0000 7f804fb90640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f80480b3ef0 0x7f8048165760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:34.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.815+0000 7f804fb90640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80480cf790 con 0x7f804815a740 2024-03-29T04:12:34.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.815+0000 7f804fb90640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80480b4350 con 0x7f804806e6b0 2024-03-29T04:12:34.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.815+0000 7f804fb90640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f804815f1f0 con 0x7f80480b3ef0 2024-03-29T04:12:34.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.815+0000 7f804d92c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f804815a740 0x7f804815e3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39404/0 (socket says 172.21.15.148:39404) 2024-03-29T04:12:34.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.815+0000 7f804d92c640 1 -- 172.21.15.148:0/1804088213 learned_addr learned my addr 172.21.15.148:0/1804088213 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:34.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.815+0000 7f804e12d640 1 --2- 172.21.15.148:0/1804088213 >> v2:172.21.15.157:3300/0 conn(0x7f80480b3ef0 0x7f8048165760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:34.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.815+0000 7f804c92a640 1 -- 172.21.15.148:0/1804088213 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 374313887 0 0) 0x7f8038001110 con 0x7f804815a740 2024-03-29T04:12:34.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.815+0000 7f804c92a640 1 -- 172.21.15.148:0/1804088213 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8034004630 con 0x7f804815a740 2024-03-29T04:12:34.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.816+0000 7f804c92a640 1 -- 172.21.15.148:0/1804088213 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3978355564 0 0) 0x7f8034004630 con 0x7f804815a740 2024-03-29T04:12:34.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.816+0000 7f804c92a640 1 -- 172.21.15.148:0/1804088213 >> v2:172.21.15.157:3300/0 conn(0x7f80480b3ef0 msgr2=0x7f8048165760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:34.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.816+0000 7f804c92a640 1 --2- 172.21.15.148:0/1804088213 >> v2:172.21.15.157:3300/0 conn(0x7f80480b3ef0 0x7f8048165760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:34.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.816+0000 7f804c92a640 1 -- 172.21.15.148:0/1804088213 >> v1:172.21.15.157:6789/0 conn(0x7f804806e6b0 legacy=0x7f804815eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:34.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.816+0000 7f804c92a640 1 -- 172.21.15.148:0/1804088213 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80480728d0 con 0x7f804815a740 2024-03-29T04:12:34.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.816+0000 7f804c92a640 1 -- 172.21.15.148:0/1804088213 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f8038001840 con 0x7f804815a740 2024-03-29T04:12:34.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.816+0000 7f804c92a640 1 -- 172.21.15.148:0/1804088213 >> v1:172.21.15.148:6789/0 conn(0x7f804815a740 legacy=0x7f804815e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:34.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.816+0000 7f804c92a640 1 --2- 172.21.15.148:0/1804088213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8034004ac0 0x7f8034004ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:34.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.816+0000 7f804c92a640 1 --2- 172.21.15.148:0/1804088213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8034005910 0x7f8034005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:34.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.816+0000 7f804c92a640 1 --2- 172.21.15.148:0/1804088213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80340062d0 0x7f803400cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:34.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.816+0000 7f804c92a640 1 -- 172.21.15.148:0/1804088213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f804815f1f0 con 0x7f8034004ac0 2024-03-29T04:12:34.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.816+0000 7f804c92a640 1 -- 172.21.15.148:0/1804088213 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f803400d270 con 0x7f8034005910 2024-03-29T04:12:34.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.816+0000 7f804c92a640 1 -- 172.21.15.148:0/1804088213 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f803400d470 con 0x7f80340062d0 2024-03-29T04:12:34.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.816+0000 7f804d92c640 1 --2- 172.21.15.148:0/1804088213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8034004ac0 0x7f8034004ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:34.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.816+0000 7f804d12b640 1 --2- 172.21.15.148:0/1804088213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8034005910 0x7f8034005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:34.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.816+0000 7f804e12d640 1 --2- 172.21.15.148:0/1804088213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80340062d0 0x7f803400cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:34.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.816+0000 7f804d92c640 1 -- 172.21.15.148:0/1804088213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80340062d0 msgr2=0x7f803400cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:34.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.816+0000 7f804d92c640 1 --2- 172.21.15.148:0/1804088213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80340062d0 0x7f803400cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:34.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.816+0000 7f804d92c640 1 -- 172.21.15.148:0/1804088213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8034005910 msgr2=0x7f8034005d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:34.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.816+0000 7f804d92c640 1 --2- 172.21.15.148:0/1804088213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8034005910 0x7f8034005d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:34.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.816+0000 7f804d92c640 1 -- 172.21.15.148:0/1804088213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8034004630 con 0x7f8034004ac0 2024-03-29T04:12:34.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.816+0000 7f804e12d640 1 -- 172.21.15.148:0/1804088213 reap_dead start 2024-03-29T04:12:34.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.817+0000 7f804d92c640 1 --2- 172.21.15.148:0/1804088213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8034004ac0 0x7f8034004ee0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f803800c4c0 tx=0x7f8034006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32fbf740ac5c31bd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:34.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.817+0000 7f804c92a640 1 -- 172.21.15.148:0/1804088213 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803800c0e0 con 0x7f8034004ac0 2024-03-29T04:12:34.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.817+0000 7f804c92a640 1 -- 172.21.15.148:0/1804088213 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8038030070 con 0x7f8034004ac0 2024-03-29T04:12:34.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.817+0000 7f804c92a640 1 -- 172.21.15.148:0/1804088213 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80380058d0 con 0x7f8034004ac0 2024-03-29T04:12:34.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.818+0000 7f804fb90640 1 -- 172.21.15.148:0/1804088213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8034004ac0 msgr2=0x7f8034004ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:34.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.818+0000 7f804fb90640 1 --2- 172.21.15.148:0/1804088213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8034004ac0 0x7f8034004ee0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f803800c4c0 tx=0x7f8034006940 comp rx=0 tx=0).stop 2024-03-29T04:12:34.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.818+0000 7f804fb90640 1 -- 172.21.15.148:0/1804088213 shutdown_connections 2024-03-29T04:12:34.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.818+0000 7f804fb90640 1 --2- 172.21.15.148:0/1804088213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8034004ac0 0x7f8034004ee0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:34.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.818+0000 7f804fb90640 1 -- 172.21.15.148:0/1804088213 >> 172.21.15.148:0/1804088213 conn(0x7f80480cfed0 msgr2=0x7f80480d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:34.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.818+0000 7f804fb90640 1 -- 172.21.15.148:0/1804088213 shutdown_connections 2024-03-29T04:12:34.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.818+0000 7f804fb90640 1 -- 172.21.15.148:0/1804088213 wait complete. 2024-03-29T04:12:34.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.819+0000 7f804fb90640 1 Processor -- start 2024-03-29T04:12:34.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.819+0000 7f804fb90640 1 -- start start 2024-03-29T04:12:34.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.820+0000 7f804fb90640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8034004ac0 0x7f804814e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:34.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.820+0000 7f804fb90640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f804815c6a0 0x7f80480ae580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:34.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.820+0000 7f804fb90640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f804814f0c0 0x7f80480aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:34.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.820+0000 7f804fb90640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8048073170 con 0x7f804815c6a0 2024-03-29T04:12:34.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.820+0000 7f804fb90640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8048072e90 con 0x7f8034004ac0 2024-03-29T04:12:34.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.820+0000 7f804fb90640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f804815ab20 con 0x7f804814f0c0 2024-03-29T04:12:34.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.820+0000 7f804d12b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f804815c6a0 0x7f80480ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:34.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.820+0000 7f804d12b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f804815c6a0 0x7f80480ae580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36422/0 (socket says 172.21.15.148:36422) 2024-03-29T04:12:34.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.820+0000 7f804d12b640 1 -- 172.21.15.148:0/4023637517 learned_addr learned my addr 172.21.15.148:0/4023637517 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:34.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.820+0000 7f804d92c640 1 --2- 172.21.15.148:0/4023637517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8034004ac0 0x7f804814e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:34.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.820+0000 7f804e12d640 1 --2- 172.21.15.148:0/4023637517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f804814f0c0 0x7f80480aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:34.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.820+0000 7f804d12b640 1 -- 172.21.15.148:0/4023637517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f804814f0c0 msgr2=0x7f80480aeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:34.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.821+0000 7f804d12b640 1 --2- 172.21.15.148:0/4023637517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f804814f0c0 0x7f80480aeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:34.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.821+0000 7f804d12b640 1 -- 172.21.15.148:0/4023637517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8034004ac0 msgr2=0x7f804814e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:34.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.821+0000 7f804d12b640 1 --2- 172.21.15.148:0/4023637517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8034004ac0 0x7f804814e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:34.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.821+0000 7f804d12b640 1 -- 172.21.15.148:0/4023637517 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804815cb60 con 0x7f804815c6a0 2024-03-29T04:12:34.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.821+0000 7f804d12b640 1 --2- 172.21.15.148:0/4023637517 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f804815c6a0 0x7f80480ae580 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f804400f5c0 tx=0x7f804400fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:34.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.821+0000 7f803e7fc640 1 -- 172.21.15.148:0/4023637517 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80440052e0 con 0x7f804815c6a0 2024-03-29T04:12:34.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.821+0000 7f803e7fc640 1 -- 172.21.15.148:0/4023637517 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8044018070 con 0x7f804815c6a0 2024-03-29T04:12:34.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.821+0000 7f804fb90640 1 -- 172.21.15.148:0/4023637517 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804815b960 con 0x7f804815c6a0 2024-03-29T04:12:34.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.821+0000 7f803e7fc640 1 -- 172.21.15.148:0/4023637517 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044003830 con 0x7f804815c6a0 2024-03-29T04:12:34.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.821+0000 7f804fb90640 1 -- 172.21.15.148:0/4023637517 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80480af160 con 0x7f804815c6a0 2024-03-29T04:12:34.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.822+0000 7f804fb90640 1 -- 172.21.15.148:0/4023637517 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80480687e0 con 0x7f804815c6a0 2024-03-29T04:12:34.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.822+0000 7f803e7fc640 1 -- 172.21.15.148:0/4023637517 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f80440039d0 con 0x7f804815c6a0 2024-03-29T04:12:34.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.823+0000 7f803e7fc640 1 --2- 172.21.15.148:0/4023637517 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f802c06c530 0x7f802c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:34.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.823+0000 7f804d92c640 1 --2- 172.21.15.148:0/4023637517 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f802c06c530 0x7f802c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:34.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.823+0000 7f803e7fc640 1 -- 172.21.15.148:0/4023637517 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f804408b420 con 0x7f804815c6a0 2024-03-29T04:12:34.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.823+0000 7f804d92c640 1 --2- 172.21.15.148:0/4023637517 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f802c06c530 0x7f802c06e9f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8038034000 tx=0x7f8038007000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:34.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:34.827+0000 7f803e7fc640 1 -- 172.21.15.148:0/4023637517 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f80480b4350 con 0x7f804815c6a0 2024-03-29T04:12:35.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.112+0000 7f804fb90640 1 -- 172.21.15.148:0/4023637517 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f80480af420 con 0x7f804815c6a0 2024-03-29T04:12:35.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.113+0000 7f803e7fc640 1 -- 172.21.15.148:0/4023637517 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) ==== 80+0+0 (secure 0 0 0) 0x7f80440592d0 con 0x7f804815c6a0 2024-03-29T04:12:35.095 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-03-29T04:12:35.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.115+0000 7f804fb90640 1 -- 172.21.15.148:0/4023637517 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f802c06c530 msgr2=0x7f802c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:35.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.115+0000 7f804fb90640 1 --2- 172.21.15.148:0/4023637517 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f802c06c530 0x7f802c06e9f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8038034000 tx=0x7f8038007000 comp rx=0 tx=0).stop 2024-03-29T04:12:35.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.115+0000 7f804fb90640 1 -- 172.21.15.148:0/4023637517 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f804815c6a0 msgr2=0x7f80480ae580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:35.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.115+0000 7f804fb90640 1 --2- 172.21.15.148:0/4023637517 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f804815c6a0 0x7f80480ae580 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f804400f5c0 tx=0x7f804400fa80 comp rx=0 tx=0).stop 2024-03-29T04:12:35.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.115+0000 7f804fb90640 1 -- 172.21.15.148:0/4023637517 shutdown_connections 2024-03-29T04:12:35.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.115+0000 7f804fb90640 1 --2- 172.21.15.148:0/4023637517 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f802c06c530 0x7f802c06e9f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:35.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.115+0000 7f804fb90640 1 --2- 172.21.15.148:0/4023637517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f804814f0c0 0x7f80480aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:35.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.115+0000 7f804fb90640 1 --2- 172.21.15.148:0/4023637517 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f804815c6a0 0x7f80480ae580 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:35.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.115+0000 7f804fb90640 1 --2- 172.21.15.148:0/4023637517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8034004ac0 0x7f804814e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:35.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.115+0000 7f804fb90640 1 -- 172.21.15.148:0/4023637517 >> 172.21.15.148:0/4023637517 conn(0x7f80480cfed0 msgr2=0x7f804806d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:35.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.115+0000 7f804fb90640 1 -- 172.21.15.148:0/4023637517 shutdown_connections 2024-03-29T04:12:35.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.115+0000 7f804fb90640 1 -- 172.21.15.148:0/4023637517 wait complete. 2024-03-29T04:12:35.105 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2024-03-29T04:12:35.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:12:35.105 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-03-29T04:12:35.136 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[44482]: fuse finished with error 0 and tester_r 0 2024-03-29T04:12:35.138 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-03-29T04:12:35.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:12:35.138 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-29T04:12:35.231 INFO:tasks.cephfs.mount:Terminating background process 2024-03-29T04:12:35.232 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:12:35.232 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:12:35.247 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18560 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T04:12:35.247 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-03-29T04:12:35.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:12:35.247 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-29T04:12:35.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:12:35.303 INFO:teuthology.orchestra.run.smithi148.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-03-29T04:12:35.304 INFO:tasks.cephfs.mount:Terminating background process 2024-03-29T04:12:35.304 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:12:35.304 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:12:35.321 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16940 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T04:12:35.321 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-03-29T04:12:35.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:12:35.322 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-29T04:12:35.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:12:35.377 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-03-29T04:12:35.378 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:12:35.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.502+0000 7f997ef11640 1 Processor -- start 2024-03-29T04:12:35.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.502+0000 7f997ef11640 1 -- start start 2024-03-29T04:12:35.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.502+0000 7f997ef11640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f997815e3e0 0x7f99780d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:35.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.502+0000 7f997ef11640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f99780ba030 0x7f99780d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:35.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.502+0000 7f997ef11640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f997815e7c0 con 0x7f99780d10d0 2024-03-29T04:12:35.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.502+0000 7f997ef11640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f99780d3840 con 0x7f99780ba030 2024-03-29T04:12:35.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.502+0000 7f997ef11640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f99780d2c30 con 0x7f997815e3e0 2024-03-29T04:12:35.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.502+0000 7f9977fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f99780ba030 0x7f99780d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:35.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.502+0000 7f997d4ae640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f99780d10d0 0x7f99780b72d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39418/0 (socket says 172.21.15.148:39418) 2024-03-29T04:12:35.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.503+0000 7f997d4ae640 1 -- 172.21.15.148:0/2857056270 learned_addr learned my addr 172.21.15.148:0/2857056270 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:35.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.503+0000 7f997ccad640 1 --2- 172.21.15.148:0/2857056270 >> v2:172.21.15.157:3300/0 conn(0x7f997815e3e0 0x7f99780d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:35.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.503+0000 7f9977fff640 1 -- 172.21.15.148:0/2857056270 >> v2:172.21.15.157:3300/0 conn(0x7f997815e3e0 msgr2=0x7f99780d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:35.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.503+0000 7f9977fff640 1 --2- 172.21.15.148:0/2857056270 >> v2:172.21.15.157:3300/0 conn(0x7f997815e3e0 0x7f99780d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:35.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.503+0000 7f9977fff640 1 -- 172.21.15.148:0/2857056270 >> v1:172.21.15.148:6789/0 conn(0x7f99780d10d0 legacy=0x7f99780b72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:35.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.503+0000 7f9977fff640 1 -- 172.21.15.148:0/2857056270 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99780d2f10 con 0x7f99780ba030 2024-03-29T04:12:35.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.503+0000 7f99777fe640 1 -- 172.21.15.148:0/2857056270 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3064653020 0 0) 0x7f997815e7c0 con 0x7f99780d10d0 2024-03-29T04:12:35.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.504+0000 7f9977fff640 1 --2- 172.21.15.148:0/2857056270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99780ba030 0x7f99780d2330 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f99780b4eb0 tx=0x7f996802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74a78ba97511e9c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:35.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.504+0000 7f99777fe640 1 -- 172.21.15.148:0/2857056270 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9968034070 con 0x7f99780ba030 2024-03-29T04:12:35.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.504+0000 7f99777fe640 1 -- 172.21.15.148:0/2857056270 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9968030610 con 0x7f99780ba030 2024-03-29T04:12:35.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.504+0000 7f99777fe640 1 -- 172.21.15.148:0/2857056270 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99680309e0 con 0x7f99780ba030 2024-03-29T04:12:35.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.504+0000 7f997ef11640 1 -- 172.21.15.148:0/2857056270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99780ba030 msgr2=0x7f99780d2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:35.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.504+0000 7f997ef11640 1 --2- 172.21.15.148:0/2857056270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99780ba030 0x7f99780d2330 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f99780b4eb0 tx=0x7f996802b960 comp rx=0 tx=0).stop 2024-03-29T04:12:35.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.504+0000 7f997ef11640 1 -- 172.21.15.148:0/2857056270 shutdown_connections 2024-03-29T04:12:35.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.504+0000 7f997ef11640 1 --2- 172.21.15.148:0/2857056270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99780ba030 0x7f99780d2330 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:35.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.504+0000 7f997ef11640 1 --2- 172.21.15.148:0/2857056270 >> v2:172.21.15.157:3300/0 conn(0x7f997815e3e0 0x7f99780d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:35.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.504+0000 7f997ef11640 1 -- 172.21.15.148:0/2857056270 >> 172.21.15.148:0/2857056270 conn(0x7f99780b6210 msgr2=0x7f99780b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:35.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.504+0000 7f997ef11640 1 -- 172.21.15.148:0/2857056270 shutdown_connections 2024-03-29T04:12:35.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.504+0000 7f997ef11640 1 -- 172.21.15.148:0/2857056270 wait complete. 2024-03-29T04:12:35.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.505+0000 7f997ef11640 1 Processor -- start 2024-03-29T04:12:35.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.506+0000 7f997ef11640 1 -- start start 2024-03-29T04:12:35.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.506+0000 7f997ef11640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99780ba030 0x7f997815a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:35.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.506+0000 7f997ef11640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99780d10d0 0x7f99780b4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:35.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.506+0000 7f997ef11640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997815e3e0 0x7f997806c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:35.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.506+0000 7f997ef11640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f99780d2870 con 0x7f99780d10d0 2024-03-29T04:12:35.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.506+0000 7f997ef11640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f99780d34d0 con 0x7f99780ba030 2024-03-29T04:12:35.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.506+0000 7f997ef11640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f99780d29f0 con 0x7f997815e3e0 2024-03-29T04:12:35.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.506+0000 7f9977fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99780d10d0 0x7f99780b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:35.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.506+0000 7f9977fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99780d10d0 0x7f99780b4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60522/0 (socket says 172.21.15.148:60522) 2024-03-29T04:12:35.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.507+0000 7f9977fff640 1 -- 172.21.15.148:0/190288561 learned_addr learned my addr 172.21.15.148:0/190288561 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:35.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.507+0000 7f997ccad640 1 --2- 172.21.15.148:0/190288561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99780ba030 0x7f997815a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:35.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.507+0000 7f997d4ae640 1 --2- 172.21.15.148:0/190288561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997815e3e0 0x7f997806c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:35.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.507+0000 7f9977fff640 1 -- 172.21.15.148:0/190288561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997815e3e0 msgr2=0x7f997806c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:35.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.507+0000 7f9977fff640 1 --2- 172.21.15.148:0/190288561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997815e3e0 0x7f997806c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:35.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.507+0000 7f9977fff640 1 -- 172.21.15.148:0/190288561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99780ba030 msgr2=0x7f997815a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:35.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.507+0000 7f9977fff640 1 --2- 172.21.15.148:0/190288561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99780ba030 0x7f997815a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:35.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.507+0000 7f9977fff640 1 -- 172.21.15.148:0/190288561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997815db60 con 0x7f99780d10d0 2024-03-29T04:12:35.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.507+0000 7f997ccad640 1 --2- 172.21.15.148:0/190288561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99780ba030 0x7f997815a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:35.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.507+0000 7f9977fff640 1 --2- 172.21.15.148:0/190288561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99780d10d0 0x7f99780b4490 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9968009890 tx=0x7f99680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:35.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.507+0000 7f99757fa640 1 -- 172.21.15.148:0/190288561 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9968046020 con 0x7f99780d10d0 2024-03-29T04:12:35.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.507+0000 7f99757fa640 1 -- 172.21.15.148:0/190288561 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9968039a10 con 0x7f99780d10d0 2024-03-29T04:12:35.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.507+0000 7f997ef11640 1 -- 172.21.15.148:0/190288561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f997815d5a0 con 0x7f99780d10d0 2024-03-29T04:12:35.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.507+0000 7f99757fa640 1 -- 172.21.15.148:0/190288561 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9968034040 con 0x7f99780d10d0 2024-03-29T04:12:35.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.507+0000 7f997ef11640 1 -- 172.21.15.148:0/190288561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f997806c5f0 con 0x7f99780d10d0 2024-03-29T04:12:35.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.508+0000 7f997ef11640 1 -- 172.21.15.148:0/190288561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99780687e0 con 0x7f99780d10d0 2024-03-29T04:12:35.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.509+0000 7f99757fa640 1 -- 172.21.15.148:0/190288561 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9968037070 con 0x7f99780d10d0 2024-03-29T04:12:35.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.509+0000 7f99757fa640 1 --2- 172.21.15.148:0/190288561 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f995806c530 0x7f995806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:35.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.510+0000 7f99757fa640 1 -- 172.21.15.148:0/190288561 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f99680ad6b0 con 0x7f99780d10d0 2024-03-29T04:12:35.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.510+0000 7f997ccad640 1 --2- 172.21.15.148:0/190288561 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f995806c530 0x7f995806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:35.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.510+0000 7f997ccad640 1 --2- 172.21.15.148:0/190288561 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f995806c530 0x7f995806e9f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9970009a80 tx=0x7f99700039d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:35.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.514+0000 7f99757fa640 1 -- 172.21.15.148:0/190288561 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9968050020 con 0x7f99780d10d0 2024-03-29T04:12:35.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.732+0000 7f997ef11640 1 -- 172.21.15.148:0/190288561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99780c7a10 con 0x7f99780d10d0 2024-03-29T04:12:35.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.733+0000 7f99757fa640 1 -- 172.21.15.148:0/190288561 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5190 (secure 0 0 0) 0x7f9968039e20 con 0x7f99780d10d0 2024-03-29T04:12:35.712 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:12:35.713 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2618617956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2618617956},{"type":"v1","addr":"172.21.15.157:6834","nonce":2618617956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5260,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1177855619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1177855619},{"type":"v1","addr":"172.21.15.157:6835","nonce":1177855619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5562,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3610463670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3610463670},{"type":"v1","addr":"172.21.15.148:6835","nonce":3610463670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:11:32.950391+0000","modified":"2024-03-29T04:11:53.076916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5583},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5583":{"gid":5583,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/473448013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":473448013},{"type":"v1","addr":"172.21.15.148:6837","nonce":473448013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5583,"qdb_cluster":[5583]},"id":2}]} 2024-03-29T04:12:35.715 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 19 2024-03-29T04:12:35.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.735+0000 7f997ef11640 1 -- 172.21.15.148:0/190288561 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f995806c530 msgr2=0x7f995806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:35.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.735+0000 7f997ef11640 1 --2- 172.21.15.148:0/190288561 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f995806c530 0x7f995806e9f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9970009a80 tx=0x7f99700039d0 comp rx=0 tx=0).stop 2024-03-29T04:12:35.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.735+0000 7f997ef11640 1 -- 172.21.15.148:0/190288561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99780d10d0 msgr2=0x7f99780b4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:35.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.735+0000 7f997ef11640 1 --2- 172.21.15.148:0/190288561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99780d10d0 0x7f99780b4490 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9968009890 tx=0x7f99680046c0 comp rx=0 tx=0).stop 2024-03-29T04:12:35.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.735+0000 7f997ef11640 1 -- 172.21.15.148:0/190288561 shutdown_connections 2024-03-29T04:12:35.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.735+0000 7f997ef11640 1 --2- 172.21.15.148:0/190288561 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f995806c530 0x7f995806e9f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:35.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.735+0000 7f997ef11640 1 --2- 172.21.15.148:0/190288561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997815e3e0 0x7f997806c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:35.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.735+0000 7f997ef11640 1 --2- 172.21.15.148:0/190288561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99780d10d0 0x7f99780b4490 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:35.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.735+0000 7f997ef11640 1 --2- 172.21.15.148:0/190288561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99780ba030 0x7f997815a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:35.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.735+0000 7f997ef11640 1 -- 172.21.15.148:0/190288561 >> 172.21.15.148:0/190288561 conn(0x7f99780b6210 msgr2=0x7f99780b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:35.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.735+0000 7f997ef11640 1 -- 172.21.15.148:0/190288561 shutdown_connections 2024-03-29T04:12:35.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.735+0000 7f997ef11640 1 -- 172.21.15.148:0/190288561 wait complete. 2024-03-29T04:12:35.726 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:12:35.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.848+0000 7fb6639a8640 1 Processor -- start 2024-03-29T04:12:35.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.848+0000 7fb6639a8640 1 -- start start 2024-03-29T04:12:35.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.849+0000 7fb6639a8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb65c15e3e0 0x7fb65c0d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:35.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.849+0000 7fb6639a8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb65c0ba030 0x7fb65c0d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:35.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.849+0000 7fb6639a8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb65c15e7c0 con 0x7fb65c0d10d0 2024-03-29T04:12:35.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.849+0000 7fb6639a8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb65c0d3840 con 0x7fb65c15e3e0 2024-03-29T04:12:35.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.849+0000 7fb6639a8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb65c0d2c30 con 0x7fb65c0ba030 2024-03-29T04:12:35.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.849+0000 7fb661f45640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb65c0d10d0 0x7fb65c0b72d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39762/0 (socket says 172.21.15.148:39762) 2024-03-29T04:12:35.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.849+0000 7fb661f45640 1 -- 172.21.15.148:0/1405215268 learned_addr learned my addr 172.21.15.148:0/1405215268 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:35.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.849+0000 7fb660f43640 1 --2- 172.21.15.148:0/1405215268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb65c0ba030 0x7fb65c0d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:35.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.849+0000 7fb661744640 1 --2- 172.21.15.148:0/1405215268 >> v2:172.21.15.157:3300/0 conn(0x7fb65c15e3e0 0x7fb65c0d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:35.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.849+0000 7fb653fff640 1 -- 172.21.15.148:0/1405215268 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 15104577 0 0) 0x7fb65c15e7c0 con 0x7fb65c0d10d0 2024-03-29T04:12:35.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.849+0000 7fb653fff640 1 -- 172.21.15.148:0/1405215268 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb648003520 con 0x7fb65c0d10d0 2024-03-29T04:12:35.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.849+0000 7fb653fff640 1 -- 172.21.15.148:0/1405215268 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 668340969 0 0) 0x7fb648003520 con 0x7fb65c0d10d0 2024-03-29T04:12:35.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.849+0000 7fb661744640 1 -- 172.21.15.148:0/1405215268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb65c0ba030 msgr2=0x7fb65c0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:35.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.849+0000 7fb661744640 1 --2- 172.21.15.148:0/1405215268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb65c0ba030 0x7fb65c0d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:35.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.849+0000 7fb661744640 1 -- 172.21.15.148:0/1405215268 >> v1:172.21.15.148:6789/0 conn(0x7fb65c0d10d0 legacy=0x7fb65c0b72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:35.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.849+0000 7fb661744640 1 -- 172.21.15.148:0/1405215268 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65c0d2f10 con 0x7fb65c15e3e0 2024-03-29T04:12:35.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.849+0000 7fb660f43640 1 --2- 172.21.15.148:0/1405215268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb65c0ba030 0x7fb65c0d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:12:35.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.850+0000 7fb661744640 1 --2- 172.21.15.148:0/1405215268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb65c15e3e0 0x7fb65c0d3cd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb6580097a0 tx=0x7fb65802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bafc15558bcbb834 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:35.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.850+0000 7fb653fff640 1 -- 172.21.15.148:0/1405215268 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb658038070 con 0x7fb65c15e3e0 2024-03-29T04:12:35.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.850+0000 7fb653fff640 1 -- 172.21.15.148:0/1405215268 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb658031500 con 0x7fb65c15e3e0 2024-03-29T04:12:35.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.850+0000 7fb653fff640 1 -- 172.21.15.148:0/1405215268 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6540019f0 con 0x7fb65c15e3e0 2024-03-29T04:12:35.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.851+0000 7fb6639a8640 1 -- 172.21.15.148:0/1405215268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb65c15e3e0 msgr2=0x7fb65c0d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:35.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.851+0000 7fb6639a8640 1 --2- 172.21.15.148:0/1405215268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb65c15e3e0 0x7fb65c0d3cd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb6580097a0 tx=0x7fb65802b820 comp rx=0 tx=0).stop 2024-03-29T04:12:35.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.851+0000 7fb6639a8640 1 -- 172.21.15.148:0/1405215268 shutdown_connections 2024-03-29T04:12:35.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.851+0000 7fb6639a8640 1 --2- 172.21.15.148:0/1405215268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb65c0ba030 0x7fb65c0d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:35.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.851+0000 7fb6639a8640 1 --2- 172.21.15.148:0/1405215268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb65c15e3e0 0x7fb65c0d3cd0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:35.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.851+0000 7fb6639a8640 1 -- 172.21.15.148:0/1405215268 >> 172.21.15.148:0/1405215268 conn(0x7fb65c0b6210 msgr2=0x7fb65c0b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:35.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.851+0000 7fb6639a8640 1 -- 172.21.15.148:0/1405215268 shutdown_connections 2024-03-29T04:12:35.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.851+0000 7fb6639a8640 1 -- 172.21.15.148:0/1405215268 wait complete. 2024-03-29T04:12:35.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.852+0000 7fb6639a8640 1 Processor -- start 2024-03-29T04:12:35.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.852+0000 7fb6639a8640 1 -- start start 2024-03-29T04:12:35.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.853+0000 7fb6639a8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb65c0ba030 0x7fb65c15ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:35.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.853+0000 7fb6639a8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb65c0d10d0 0x7fb65c1f6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:35.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.853+0000 7fb6639a8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb65c15e3e0 0x7fb65c1456e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:35.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.853+0000 7fb6639a8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c0d2870 con 0x7fb65c0ba030 2024-03-29T04:12:35.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.853+0000 7fb6639a8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb65c0d34d0 con 0x7fb65c0d10d0 2024-03-29T04:12:35.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.853+0000 7fb6639a8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb65c0d29f0 con 0x7fb65c15e3e0 2024-03-29T04:12:35.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.853+0000 7fb661744640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb65c0ba030 0x7fb65c15ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:35.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.853+0000 7fb661744640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb65c0ba030 0x7fb65c15ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60534/0 (socket says 172.21.15.148:60534) 2024-03-29T04:12:35.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.853+0000 7fb661744640 1 -- 172.21.15.148:0/3333954100 learned_addr learned my addr 172.21.15.148:0/3333954100 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:35.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.853+0000 7fb661744640 1 -- 172.21.15.148:0/3333954100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb65c15e3e0 msgr2=0x7fb65c1456e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:35.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.853+0000 7fb661744640 1 --2- 172.21.15.148:0/3333954100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb65c15e3e0 0x7fb65c1456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:35.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.853+0000 7fb661744640 1 -- 172.21.15.148:0/3333954100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb65c0d10d0 msgr2=0x7fb65c1f6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:35.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.853+0000 7fb660f43640 1 --2- 172.21.15.148:0/3333954100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb65c0d10d0 0x7fb65c1f6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:35.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.853+0000 7fb661744640 1 --2- 172.21.15.148:0/3333954100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb65c0d10d0 0x7fb65c1f6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:35.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.853+0000 7fb661744640 1 -- 172.21.15.148:0/3333954100 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65c0cccb0 con 0x7fb65c0ba030 2024-03-29T04:12:35.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.854+0000 7fb661744640 1 --2- 172.21.15.148:0/3333954100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb65c0ba030 0x7fb65c15ca00 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb65802b860 tx=0x7fb6580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:35.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.854+0000 7fb651ffb640 1 -- 172.21.15.148:0/3333954100 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb658038050 con 0x7fb65c0ba030 2024-03-29T04:12:35.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.854+0000 7fb651ffb640 1 -- 172.21.15.148:0/3333954100 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb658044070 con 0x7fb65c0ba030 2024-03-29T04:12:35.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.854+0000 7fb6639a8640 1 -- 172.21.15.148:0/3333954100 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb65c15d040 con 0x7fb65c0ba030 2024-03-29T04:12:35.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.854+0000 7fb651ffb640 1 -- 172.21.15.148:0/3333954100 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb658003a30 con 0x7fb65c0ba030 2024-03-29T04:12:35.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.854+0000 7fb6639a8640 1 -- 172.21.15.148:0/3333954100 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb65c161650 con 0x7fb65c0ba030 2024-03-29T04:12:35.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.855+0000 7fb6639a8640 1 -- 172.21.15.148:0/3333954100 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb65c0687e0 con 0x7fb65c0ba030 2024-03-29T04:12:35.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.855+0000 7fb651ffb640 1 -- 172.21.15.148:0/3333954100 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb658003bd0 con 0x7fb65c0ba030 2024-03-29T04:12:35.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.856+0000 7fb651ffb640 1 --2- 172.21.15.148:0/3333954100 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb64006c530 0x7fb64006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:35.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.856+0000 7fb660f43640 1 --2- 172.21.15.148:0/3333954100 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb64006c530 0x7fb64006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:35.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.856+0000 7fb651ffb640 1 -- 172.21.15.148:0/3333954100 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7fb6580ad6d0 con 0x7fb65c0ba030 2024-03-29T04:12:35.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.856+0000 7fb660f43640 1 --2- 172.21.15.148:0/3333954100 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb64006c530 0x7fb64006e9f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb65c06d560 tx=0x7fb64c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:35.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:35.860+0000 7fb651ffb640 1 -- 172.21.15.148:0/3333954100 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb658030070 con 0x7fb65c0ba030 2024-03-29T04:12:36.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.041+0000 7fb6639a8640 1 -- 172.21.15.148:0/3333954100 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb65c0b6690 con 0x7fb65c0ba030 2024-03-29T04:12:36.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.042+0000 7fb651ffb640 1 -- 172.21.15.148:0/3333954100 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+17003 (secure 0 0 0) 0x7fb65807bb40 con 0x7fb65c0ba030 2024-03-29T04:12:36.021 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:12:36.021 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":41,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:11:42.632417+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:11:30.794790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T04:11:31.907100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:12:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.045+0000 7fb6639a8640 1 -- 172.21.15.148:0/3333954100 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb64006c530 msgr2=0x7fb64006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.045+0000 7fb6639a8640 1 --2- 172.21.15.148:0/3333954100 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb64006c530 0x7fb64006e9f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb65c06d560 tx=0x7fb64c008040 comp rx=0 tx=0).stop 2024-03-29T04:12:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.045+0000 7fb6639a8640 1 -- 172.21.15.148:0/3333954100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb65c0ba030 msgr2=0x7fb65c15ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.045+0000 7fb6639a8640 1 --2- 172.21.15.148:0/3333954100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb65c0ba030 0x7fb65c15ca00 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb65802b860 tx=0x7fb6580046c0 comp rx=0 tx=0).stop 2024-03-29T04:12:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.045+0000 7fb6639a8640 1 -- 172.21.15.148:0/3333954100 shutdown_connections 2024-03-29T04:12:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.045+0000 7fb6639a8640 1 --2- 172.21.15.148:0/3333954100 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb64006c530 0x7fb64006e9f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:36.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.045+0000 7fb6639a8640 1 --2- 172.21.15.148:0/3333954100 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb65c15e3e0 0x7fb65c1456e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:36.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.045+0000 7fb6639a8640 1 --2- 172.21.15.148:0/3333954100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb65c0d10d0 0x7fb65c1f6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:36.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.045+0000 7fb6639a8640 1 --2- 172.21.15.148:0/3333954100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb65c0ba030 0x7fb65c15ca00 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:36.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.045+0000 7fb6639a8640 1 -- 172.21.15.148:0/3333954100 >> 172.21.15.148:0/3333954100 conn(0x7fb65c0b6210 msgr2=0x7fb65c0b8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:36.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.045+0000 7fb6639a8640 1 -- 172.21.15.148:0/3333954100 shutdown_connections 2024-03-29T04:12:36.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.045+0000 7fb6639a8640 1 -- 172.21.15.148:0/3333954100 wait complete. 2024-03-29T04:12:36.035 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:47.898351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:11:30.794790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:11:31.907100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:12:36.036 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:12:36.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.162+0000 7fbb2a17c640 1 Processor -- start 2024-03-29T04:12:36.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.162+0000 7fbb2a17c640 1 -- start start 2024-03-29T04:12:36.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.162+0000 7fbb2a17c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbb240b2f00 0x7fbb24162680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:36.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.162+0000 7fbb2a17c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb240cf330 con 0x7fbb2415e3c0 2024-03-29T04:12:36.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.162+0000 7fbb2a17c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb24162cf0 con 0x7fbb240b3430 2024-03-29T04:12:36.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.162+0000 7fbb2a17c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fbb240cf960 con 0x7fbb240b2f00 2024-03-29T04:12:36.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.163+0000 7fbb22ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbb240b2f00 0x7fbb24162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:36.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.163+0000 7fbb237fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbb2415e3c0 0x7fbb24161f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39770/0 (socket says 172.21.15.148:39770) 2024-03-29T04:12:36.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.163+0000 7fbb237fe640 1 -- 172.21.15.148:0/4127767526 learned_addr learned my addr 172.21.15.148:0/4127767526 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:36.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.163+0000 7fbb22ffd640 1 -- 172.21.15.148:0/4127767526 >> v1:172.21.15.157:6789/0 conn(0x7fbb240b3430 legacy=0x7fbb2406b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:36.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.163+0000 7fbb22ffd640 1 -- 172.21.15.148:0/4127767526 >> v1:172.21.15.148:6789/0 conn(0x7fbb2415e3c0 legacy=0x7fbb24161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:36.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.163+0000 7fbb22ffd640 1 -- 172.21.15.148:0/4127767526 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb240cf5f0 con 0x7fbb240b2f00 2024-03-29T04:12:36.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.163+0000 7fbb227fc640 1 -- 172.21.15.148:0/4127767526 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4126809335 0 0) 0x7fbb240cf330 con 0x7fbb2415e3c0 2024-03-29T04:12:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.164+0000 7fbb22ffd640 1 --2- 172.21.15.148:0/4127767526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb240b2f00 0x7fbb24162680 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fbb180097a0 tx=0x7fbb1802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3274853035d6c6ed server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.164+0000 7fbb227fc640 1 -- 172.21.15.148:0/4127767526 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb180315a0 con 0x7fbb240b2f00 2024-03-29T04:12:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.164+0000 7fbb227fc640 1 -- 172.21.15.148:0/4127767526 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb18037070 con 0x7fbb240b2f00 2024-03-29T04:12:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.164+0000 7fbb227fc640 1 -- 172.21.15.148:0/4127767526 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18031960 con 0x7fbb240b2f00 2024-03-29T04:12:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.165+0000 7fbb2a17c640 1 -- 172.21.15.148:0/4127767526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb240b2f00 msgr2=0x7fbb24162680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.165+0000 7fbb2a17c640 1 --2- 172.21.15.148:0/4127767526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb240b2f00 0x7fbb24162680 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fbb180097a0 tx=0x7fbb1802b840 comp rx=0 tx=0).stop 2024-03-29T04:12:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.165+0000 7fbb2a17c640 1 -- 172.21.15.148:0/4127767526 shutdown_connections 2024-03-29T04:12:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.165+0000 7fbb2a17c640 1 --2- 172.21.15.148:0/4127767526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb240b2f00 0x7fbb24162680 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.165+0000 7fbb2a17c640 1 -- 172.21.15.148:0/4127767526 >> 172.21.15.148:0/4127767526 conn(0x7fbb240b6430 msgr2=0x7fbb240d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.165+0000 7fbb2a17c640 1 -- 172.21.15.148:0/4127767526 shutdown_connections 2024-03-29T04:12:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.165+0000 7fbb2a17c640 1 -- 172.21.15.148:0/4127767526 wait complete. 2024-03-29T04:12:36.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.166+0000 7fbb2a17c640 1 Processor -- start 2024-03-29T04:12:36.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.166+0000 7fbb2a17c640 1 -- start start 2024-03-29T04:12:36.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.167+0000 7fbb2a17c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb240b3430 0x7fbb241f83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:36.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.167+0000 7fbb2a17c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb2415e3c0 0x7fbb240c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:36.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.167+0000 7fbb2a17c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb24159370 0x7fbb240d2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:36.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.167+0000 7fbb2a17c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbb240b3b20 con 0x7fbb240b3430 2024-03-29T04:12:36.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.167+0000 7fbb2a17c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbb240b39a0 con 0x7fbb24159370 2024-03-29T04:12:36.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.167+0000 7fbb2a17c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbb2406bf00 con 0x7fbb2415e3c0 2024-03-29T04:12:36.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.167+0000 7fbb237fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb240b3430 0x7fbb241f83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:36.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.167+0000 7fbb237fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb240b3430 0x7fbb241f83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60566/0 (socket says 172.21.15.148:60566) 2024-03-29T04:12:36.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.167+0000 7fbb237fe640 1 -- 172.21.15.148:0/2925219186 learned_addr learned my addr 172.21.15.148:0/2925219186 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:36.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.167+0000 7fbb23fff640 1 --2- 172.21.15.148:0/2925219186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb24159370 0x7fbb240d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:36.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.167+0000 7fbb22ffd640 1 --2- 172.21.15.148:0/2925219186 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb2415e3c0 0x7fbb240c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:36.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.167+0000 7fbb237fe640 1 -- 172.21.15.148:0/2925219186 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb2415e3c0 msgr2=0x7fbb240c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:36.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.167+0000 7fbb237fe640 1 --2- 172.21.15.148:0/2925219186 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb2415e3c0 0x7fbb240c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:36.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.167+0000 7fbb237fe640 1 -- 172.21.15.148:0/2925219186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb24159370 msgr2=0x7fbb240d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:36.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.167+0000 7fbb237fe640 1 --2- 172.21.15.148:0/2925219186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb24159370 0x7fbb240d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:36.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.167+0000 7fbb237fe640 1 -- 172.21.15.148:0/2925219186 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb240ca1f0 con 0x7fbb240b3430 2024-03-29T04:12:36.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.167+0000 7fbb23fff640 1 --2- 172.21.15.148:0/2925219186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb24159370 0x7fbb240d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:36.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.167+0000 7fbb237fe640 1 --2- 172.21.15.148:0/2925219186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb240b3430 0x7fbb241f83a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400c460 tx=0x7fbb1400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:36.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.168+0000 7fbb0bfff640 1 -- 172.21.15.148:0/2925219186 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14018070 con 0x7fbb240b3430 2024-03-29T04:12:36.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.168+0000 7fbb0bfff640 1 -- 172.21.15.148:0/2925219186 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb14007500 con 0x7fbb240b3430 2024-03-29T04:12:36.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.168+0000 7fbb0bfff640 1 -- 172.21.15.148:0/2925219186 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14004a20 con 0x7fbb240b3430 2024-03-29T04:12:36.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.168+0000 7fbb2a17c640 1 -- 172.21.15.148:0/2925219186 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb240ca4d0 con 0x7fbb240b3430 2024-03-29T04:12:36.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.168+0000 7fbb2a17c640 1 -- 172.21.15.148:0/2925219186 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb240caa90 con 0x7fbb240b3430 2024-03-29T04:12:36.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.169+0000 7fbb2a17c640 1 -- 172.21.15.148:0/2925219186 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb24068850 con 0x7fbb240b3430 2024-03-29T04:12:36.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.169+0000 7fbb0bfff640 1 -- 172.21.15.148:0/2925219186 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbb1401b070 con 0x7fbb240b3430 2024-03-29T04:12:36.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.169+0000 7fbb0bfff640 1 --2- 172.21.15.148:0/2925219186 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbb1006c530 0x7fbb1006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:36.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.169+0000 7fbb0bfff640 1 -- 172.21.15.148:0/2925219186 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7fbb1408ab40 con 0x7fbb240b3430 2024-03-29T04:12:36.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.169+0000 7fbb22ffd640 1 --2- 172.21.15.148:0/2925219186 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbb1006c530 0x7fbb1006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:36.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.170+0000 7fbb22ffd640 1 --2- 172.21.15.148:0/2925219186 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbb1006c530 0x7fbb1006e9f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbb18036000 tx=0x7fbb18008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:36.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.178+0000 7fbb0bfff640 1 -- 172.21.15.148:0/2925219186 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbb14058fb0 con 0x7fbb240b3430 2024-03-29T04:12:36.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.351+0000 7fbb2a17c640 1 -- 172.21.15.148:0/2925219186 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbb2415d2a0 con 0x7fbb240b3430 2024-03-29T04:12:36.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.352+0000 7fbb0bfff640 1 -- 172.21.15.148:0/2925219186 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7fbb140588b0 con 0x7fbb240b3430 2024-03-29T04:12:36.330 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-03-29T04:12:36.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.353+0000 7fbb2a17c640 1 -- 172.21.15.148:0/2925219186 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbb1006c530 msgr2=0x7fbb1006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.353+0000 7fbb2a17c640 1 --2- 172.21.15.148:0/2925219186 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbb1006c530 0x7fbb1006e9f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbb18036000 tx=0x7fbb18008030 comp rx=0 tx=0).stop 2024-03-29T04:12:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.353+0000 7fbb2a17c640 1 -- 172.21.15.148:0/2925219186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb240b3430 msgr2=0x7fbb241f83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.353+0000 7fbb2a17c640 1 --2- 172.21.15.148:0/2925219186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb240b3430 0x7fbb241f83a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400c460 tx=0x7fbb1400c920 comp rx=0 tx=0).stop 2024-03-29T04:12:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.353+0000 7fbb2a17c640 1 -- 172.21.15.148:0/2925219186 shutdown_connections 2024-03-29T04:12:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.353+0000 7fbb2a17c640 1 --2- 172.21.15.148:0/2925219186 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbb1006c530 0x7fbb1006e9f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.353+0000 7fbb2a17c640 1 --2- 172.21.15.148:0/2925219186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb24159370 0x7fbb240d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.353+0000 7fbb2a17c640 1 --2- 172.21.15.148:0/2925219186 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb2415e3c0 0x7fbb240c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.353+0000 7fbb2a17c640 1 --2- 172.21.15.148:0/2925219186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb240b3430 0x7fbb241f83a0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.354+0000 7fbb2a17c640 1 -- 172.21.15.148:0/2925219186 >> 172.21.15.148:0/2925219186 conn(0x7fbb240b6430 msgr2=0x7fbb240b54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.354+0000 7fbb2a17c640 1 -- 172.21.15.148:0/2925219186 shutdown_connections 2024-03-29T04:12:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.354+0000 7fbb2a17c640 1 -- 172.21.15.148:0/2925219186 wait complete. 2024-03-29T04:12:36.344 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:12:36.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.469+0000 7f28fb952640 1 Processor -- start 2024-03-29T04:12:36.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.470+0000 7f28fb952640 1 -- start start 2024-03-29T04:12:36.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.470+0000 7f28fb952640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f28f415e3c0 0x7f28f4161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:36.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.470+0000 7f28fb952640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f28f40b3430 0x7f28f406b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:36.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.470+0000 7f28fb952640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28f40cf330 con 0x7f28f40b2f00 2024-03-29T04:12:36.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.470+0000 7f28fb952640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f28f40cf960 con 0x7f28f415e3c0 2024-03-29T04:12:36.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.470+0000 7f28fb952640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f28f4162cf0 con 0x7f28f40b3430 2024-03-29T04:12:36.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.470+0000 7f28f96ee640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f28f415e3c0 0x7f28f4161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:36.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.470+0000 7f28f96ee640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f28f415e3c0 0x7f28f4161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60568/0 (socket says 172.21.15.148:60568) 2024-03-29T04:12:36.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.470+0000 7f28f96ee640 1 -- 172.21.15.148:0/2240160916 learned_addr learned my addr 172.21.15.148:0/2240160916 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:36.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.471+0000 7f28f9eef640 1 --2- 172.21.15.148:0/2240160916 >> v2:172.21.15.157:3300/0 conn(0x7f28f40b3430 0x7f28f406b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:36.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.471+0000 7f28f96ee640 1 -- 172.21.15.148:0/2240160916 >> v2:172.21.15.157:3300/0 conn(0x7f28f40b3430 msgr2=0x7f28f406b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:36.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.471+0000 7f28f96ee640 1 --2- 172.21.15.148:0/2240160916 >> v2:172.21.15.157:3300/0 conn(0x7f28f40b3430 0x7f28f406b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:36.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.471+0000 7f28f96ee640 1 -- 172.21.15.148:0/2240160916 >> v1:172.21.15.157:6789/0 conn(0x7f28f40b2f00 legacy=0x7f28f41624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:36.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.471+0000 7f28f96ee640 1 -- 172.21.15.148:0/2240160916 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f40cf5f0 con 0x7f28f415e3c0 2024-03-29T04:12:36.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.471+0000 7f28f9eef640 1 --2- 172.21.15.148:0/2240160916 >> v2:172.21.15.157:3300/0 conn(0x7f28f40b3430 0x7f28f406b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:36.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.471+0000 7f28f96ee640 1 --2- 172.21.15.148:0/2240160916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28f415e3c0 0x7f28f4161f70 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f28f40cf8d0 tx=0x7f28f002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3f927dc858eb27 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:36.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.472+0000 7f28ebfff640 1 -- 172.21.15.148:0/2240160916 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f0035020 con 0x7f28f415e3c0 2024-03-29T04:12:36.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.472+0000 7f28ebfff640 1 -- 172.21.15.148:0/2240160916 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28f0031870 con 0x7f28f415e3c0 2024-03-29T04:12:36.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.472+0000 7f28ebfff640 1 -- 172.21.15.148:0/2240160916 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f0031c40 con 0x7f28f415e3c0 2024-03-29T04:12:36.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.472+0000 7f28fb952640 1 -- 172.21.15.148:0/2240160916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28f415e3c0 msgr2=0x7f28f4161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:36.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.472+0000 7f28fb952640 1 --2- 172.21.15.148:0/2240160916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28f415e3c0 0x7f28f4161f70 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f28f40cf8d0 tx=0x7f28f002b910 comp rx=0 tx=0).stop 2024-03-29T04:12:36.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.472+0000 7f28fb952640 1 -- 172.21.15.148:0/2240160916 shutdown_connections 2024-03-29T04:12:36.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.472+0000 7f28fb952640 1 --2- 172.21.15.148:0/2240160916 >> v2:172.21.15.157:3300/0 conn(0x7f28f40b3430 0x7f28f406b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:36.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.472+0000 7f28fb952640 1 --2- 172.21.15.148:0/2240160916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28f415e3c0 0x7f28f4161f70 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:36.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.472+0000 7f28fb952640 1 -- 172.21.15.148:0/2240160916 >> 172.21.15.148:0/2240160916 conn(0x7f28f40b6430 msgr2=0x7f28f40d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:36.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.473+0000 7f28fb952640 1 -- 172.21.15.148:0/2240160916 shutdown_connections 2024-03-29T04:12:36.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.473+0000 7f28fb952640 1 -- 172.21.15.148:0/2240160916 wait complete. 2024-03-29T04:12:36.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.474+0000 7f28fb952640 1 Processor -- start 2024-03-29T04:12:36.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.474+0000 7f28fb952640 1 -- start start 2024-03-29T04:12:36.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.476+0000 7f28fb952640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28f40b3430 0x7f28f4161500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:36.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.476+0000 7f28fb952640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28f415e3c0 0x7f28f4160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:36.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.476+0000 7f28fb952640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28f40c7e00 0x7f28f41f3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:36.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.476+0000 7f28fb952640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f28f40b3b20 con 0x7f28f40c7e00 2024-03-29T04:12:36.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.476+0000 7f28fb952640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f28f40b39a0 con 0x7f28f415e3c0 2024-03-29T04:12:36.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.476+0000 7f28fb952640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f28f406bd30 con 0x7f28f40b3430 2024-03-29T04:12:36.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.476+0000 7f28f9eef640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28f40c7e00 0x7f28f41f3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:36.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.476+0000 7f28f9eef640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28f40c7e00 0x7f28f41f3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60584/0 (socket says 172.21.15.148:60584) 2024-03-29T04:12:36.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.476+0000 7f28f9eef640 1 -- 172.21.15.148:0/363502062 learned_addr learned my addr 172.21.15.148:0/363502062 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:36.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.476+0000 7f28f96ee640 1 --2- 172.21.15.148:0/363502062 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28f40b3430 0x7f28f4161500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:36.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.476+0000 7f28f8eed640 1 --2- 172.21.15.148:0/363502062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28f415e3c0 0x7f28f4160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:36.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.476+0000 7f28f9eef640 1 -- 172.21.15.148:0/363502062 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28f40b3430 msgr2=0x7f28f4161500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:36.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.476+0000 7f28f9eef640 1 --2- 172.21.15.148:0/363502062 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28f40b3430 0x7f28f4161500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:36.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.476+0000 7f28f9eef640 1 -- 172.21.15.148:0/363502062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28f415e3c0 msgr2=0x7f28f4160eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:36.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.476+0000 7f28f9eef640 1 --2- 172.21.15.148:0/363502062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28f415e3c0 0x7f28f4160eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:36.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.476+0000 7f28f9eef640 1 -- 172.21.15.148:0/363502062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f415eb50 con 0x7f28f40c7e00 2024-03-29T04:12:36.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.476+0000 7f28f8eed640 1 --2- 172.21.15.148:0/363502062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28f415e3c0 0x7f28f4160eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:36.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.476+0000 7f28f9eef640 1 --2- 172.21.15.148:0/363502062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28f40c7e00 0x7f28f41f3a60 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f28ec007ef0 tx=0x7f28ec00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:36.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.476+0000 7f28e9ffb640 1 -- 172.21.15.148:0/363502062 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec01a070 con 0x7f28f40c7e00 2024-03-29T04:12:36.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.476+0000 7f28e9ffb640 1 -- 172.21.15.148:0/363502062 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28ec003b90 con 0x7f28f40c7e00 2024-03-29T04:12:36.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.476+0000 7f28fb952640 1 -- 172.21.15.148:0/363502062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f415ee30 con 0x7f28f40c7e00 2024-03-29T04:12:36.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.476+0000 7f28e9ffb640 1 -- 172.21.15.148:0/363502062 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec004030 con 0x7f28f40c7e00 2024-03-29T04:12:36.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.477+0000 7f28fb952640 1 -- 172.21.15.148:0/363502062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f415f3f0 con 0x7f28f40c7e00 2024-03-29T04:12:36.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.478+0000 7f28fb952640 1 -- 172.21.15.148:0/363502062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28f4068850 con 0x7f28f40c7e00 2024-03-29T04:12:36.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.479+0000 7f28e9ffb640 1 -- 172.21.15.148:0/363502062 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f28ec01d070 con 0x7f28f40c7e00 2024-03-29T04:12:36.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.479+0000 7f28e9ffb640 1 --2- 172.21.15.148:0/363502062 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f28d806c530 0x7f28d806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:36.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.479+0000 7f28e9ffb640 1 -- 172.21.15.148:0/363502062 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f28ec08ba00 con 0x7f28f40c7e00 2024-03-29T04:12:36.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.479+0000 7f28f96ee640 1 --2- 172.21.15.148:0/363502062 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f28d806c530 0x7f28d806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:36.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.480+0000 7f28f96ee640 1 --2- 172.21.15.148:0/363502062 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f28d806c530 0x7f28d806e9f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f28f0036000 tx=0x7f28f003c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:36.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.483+0000 7f28e9ffb640 1 -- 172.21.15.148:0/363502062 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f28ec010070 con 0x7f28f40c7e00 2024-03-29T04:12:36.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.724+0000 7f28fb952640 1 -- 172.21.15.148:0/363502062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f28f40cf110 con 0x7f28f40c7e00 2024-03-29T04:12:36.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.724+0000 7f28e9ffb640 1 -- 172.21.15.148:0/363502062 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) ==== 104+0+11 (secure 0 0 0) 0x7f28ec059820 con 0x7f28f40c7e00 2024-03-29T04:12:36.703 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-03-29T04:12:36.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.726+0000 7f28fb952640 1 -- 172.21.15.148:0/363502062 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f28d806c530 msgr2=0x7f28d806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:36.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.726+0000 7f28fb952640 1 --2- 172.21.15.148:0/363502062 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f28d806c530 0x7f28d806e9f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f28f0036000 tx=0x7f28f003c040 comp rx=0 tx=0).stop 2024-03-29T04:12:36.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.726+0000 7f28fb952640 1 -- 172.21.15.148:0/363502062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28f40c7e00 msgr2=0x7f28f41f3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:36.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.726+0000 7f28fb952640 1 --2- 172.21.15.148:0/363502062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28f40c7e00 0x7f28f41f3a60 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f28ec007ef0 tx=0x7f28ec00d4a0 comp rx=0 tx=0).stop 2024-03-29T04:12:36.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.726+0000 7f28fb952640 1 -- 172.21.15.148:0/363502062 shutdown_connections 2024-03-29T04:12:36.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.726+0000 7f28fb952640 1 --2- 172.21.15.148:0/363502062 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f28d806c530 0x7f28d806e9f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:36.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.726+0000 7f28fb952640 1 --2- 172.21.15.148:0/363502062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28f40c7e00 0x7f28f41f3a60 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:36.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.726+0000 7f28fb952640 1 --2- 172.21.15.148:0/363502062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28f415e3c0 0x7f28f4160eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:36.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.726+0000 7f28fb952640 1 --2- 172.21.15.148:0/363502062 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28f40b3430 0x7f28f4161500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:36.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.726+0000 7f28fb952640 1 -- 172.21.15.148:0/363502062 >> 172.21.15.148:0/363502062 conn(0x7f28f40b6430 msgr2=0x7f28f40b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:36.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.726+0000 7f28fb952640 1 -- 172.21.15.148:0/363502062 shutdown_connections 2024-03-29T04:12:36.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.726+0000 7f28fb952640 1 -- 172.21.15.148:0/363502062 wait complete. 2024-03-29T04:12:36.720 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:12:36.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.850+0000 7f02560b7640 1 Processor -- start 2024-03-29T04:12:36.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.850+0000 7f02560b7640 1 -- start start 2024-03-29T04:12:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.850+0000 7f02560b7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f025015a120 0x7f02500700d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.850+0000 7f02560b7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f02500ce310 0x7f025006e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.850+0000 7f02560b7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f025006f220 con 0x7f025006ee10 2024-03-29T04:12:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.850+0000 7f02560b7640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f02500d1b20 con 0x7f025015a120 2024-03-29T04:12:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.850+0000 7f02560b7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f025015d460 con 0x7f02500ce310 2024-03-29T04:12:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.851+0000 7f024f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f025015a120 0x7f02500700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.851+0000 7f024f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f025015a120 0x7f02500700d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60598/0 (socket says 172.21.15.148:60598) 2024-03-29T04:12:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.851+0000 7f024f7fe640 1 -- 172.21.15.148:0/2341626352 learned_addr learned my addr 172.21.15.148:0/2341626352 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.851+0000 7f024effd640 1 --2- 172.21.15.148:0/2341626352 >> v2:172.21.15.157:3300/0 conn(0x7f02500ce310 0x7f025006e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.851+0000 7f024f7fe640 1 -- 172.21.15.148:0/2341626352 >> v2:172.21.15.157:3300/0 conn(0x7f02500ce310 msgr2=0x7f025006e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.851+0000 7f024f7fe640 1 --2- 172.21.15.148:0/2341626352 >> v2:172.21.15.157:3300/0 conn(0x7f02500ce310 0x7f025006e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:36.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.851+0000 7f024f7fe640 1 -- 172.21.15.148:0/2341626352 >> v1:172.21.15.157:6789/0 conn(0x7f025006ee10 legacy=0x7f02500b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:36.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.851+0000 7f024f7fe640 1 -- 172.21.15.148:0/2341626352 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02500d0f60 con 0x7f025015a120 2024-03-29T04:12:36.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.852+0000 7f024f7fe640 1 --2- 172.21.15.148:0/2341626352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f025015a120 0x7f02500700d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f025006fe70 tx=0x7f024002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68bfd612f2014e2f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:36.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.852+0000 7f024e7fc640 1 -- 172.21.15.148:0/2341626352 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0240034070 con 0x7f025015a120 2024-03-29T04:12:36.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.852+0000 7f024e7fc640 1 -- 172.21.15.148:0/2341626352 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0240030860 con 0x7f025015a120 2024-03-29T04:12:36.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.852+0000 7f024e7fc640 1 -- 172.21.15.148:0/2341626352 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0240030c30 con 0x7f025015a120 2024-03-29T04:12:36.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.852+0000 7f02560b7640 1 -- 172.21.15.148:0/2341626352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f025015a120 msgr2=0x7f02500700d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:36.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.852+0000 7f02560b7640 1 --2- 172.21.15.148:0/2341626352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f025015a120 0x7f02500700d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f025006fe70 tx=0x7f024002b9a0 comp rx=0 tx=0).stop 2024-03-29T04:12:36.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.852+0000 7f02560b7640 1 -- 172.21.15.148:0/2341626352 shutdown_connections 2024-03-29T04:12:36.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.852+0000 7f02560b7640 1 --2- 172.21.15.148:0/2341626352 >> v2:172.21.15.157:3300/0 conn(0x7f02500ce310 0x7f025006e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:36.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.852+0000 7f02560b7640 1 --2- 172.21.15.148:0/2341626352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f025015a120 0x7f02500700d0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:36.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.852+0000 7f02560b7640 1 -- 172.21.15.148:0/2341626352 >> 172.21.15.148:0/2341626352 conn(0x7f02500cead0 msgr2=0x7f02500ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:36.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.852+0000 7f02560b7640 1 -- 172.21.15.148:0/2341626352 shutdown_connections 2024-03-29T04:12:36.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.853+0000 7f02560b7640 1 -- 172.21.15.148:0/2341626352 wait complete. 2024-03-29T04:12:36.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.854+0000 7f02560b7640 1 Processor -- start 2024-03-29T04:12:36.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.855+0000 7f02560b7640 1 -- start start 2024-03-29T04:12:36.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.856+0000 7f02560b7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f025006ee10 0x7f02500c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:36.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.856+0000 7f02560b7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02500ce310 0x7f0250147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:36.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.856+0000 7f02560b7640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f025015a120 0x7f025015bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:36.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.856+0000 7f02560b7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f02500ce7d0 con 0x7f025006ee10 2024-03-29T04:12:36.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.856+0000 7f02560b7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f02500cc820 con 0x7f02500ce310 2024-03-29T04:12:36.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.856+0000 7f02560b7640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f025006f5c0 con 0x7f025015a120 2024-03-29T04:12:36.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.856+0000 7f024f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f025006ee10 0x7f02500c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:36.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.856+0000 7f024f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f025006ee10 0x7f02500c9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60608/0 (socket says 172.21.15.148:60608) 2024-03-29T04:12:36.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.856+0000 7f024f7fe640 1 -- 172.21.15.148:0/1255989707 learned_addr learned my addr 172.21.15.148:0/1255989707 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:36.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.856+0000 7f024f7fe640 1 -- 172.21.15.148:0/1255989707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f025015a120 msgr2=0x7f025015bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:36.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.856+0000 7f024f7fe640 1 --2- 172.21.15.148:0/1255989707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f025015a120 0x7f025015bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:36.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.856+0000 7f024f7fe640 1 -- 172.21.15.148:0/1255989707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02500ce310 msgr2=0x7f0250147210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:36.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.856+0000 7f024f7fe640 1 --2- 172.21.15.148:0/1255989707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02500ce310 0x7f0250147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:36.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.856+0000 7f024f7fe640 1 -- 172.21.15.148:0/1255989707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02500b56e0 con 0x7f025006ee10 2024-03-29T04:12:36.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.856+0000 7f024f7fe640 1 --2- 172.21.15.148:0/1255989707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f025006ee10 0x7f02500c9b30 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f024002b9e0 tx=0x7f02400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:36.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.856+0000 7f0237fff640 1 -- 172.21.15.148:0/1255989707 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0240034040 con 0x7f025006ee10 2024-03-29T04:12:36.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.856+0000 7f0237fff640 1 -- 172.21.15.148:0/1255989707 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0240039d90 con 0x7f025006ee10 2024-03-29T04:12:36.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.856+0000 7f02560b7640 1 -- 172.21.15.148:0/1255989707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02500b59c0 con 0x7f025006ee10 2024-03-29T04:12:36.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.856+0000 7f0237fff640 1 -- 172.21.15.148:0/1255989707 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0240043610 con 0x7f025006ee10 2024-03-29T04:12:36.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.856+0000 7f02560b7640 1 -- 172.21.15.148:0/1255989707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02500b6260 con 0x7f025006ee10 2024-03-29T04:12:36.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.858+0000 7f02560b7640 1 -- 172.21.15.148:0/1255989707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02500b5b90 con 0x7f025006ee10 2024-03-29T04:12:36.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.858+0000 7f0237fff640 1 -- 172.21.15.148:0/1255989707 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0240037070 con 0x7f025006ee10 2024-03-29T04:12:36.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.859+0000 7f0237fff640 1 --2- 172.21.15.148:0/1255989707 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f023c06c530 0x7f023c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:36.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.859+0000 7f024effd640 1 --2- 172.21.15.148:0/1255989707 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f023c06c530 0x7f023c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:36.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.859+0000 7f0237fff640 1 -- 172.21.15.148:0/1255989707 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f02400ad940 con 0x7f025006ee10 2024-03-29T04:12:36.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.859+0000 7f024effd640 1 --2- 172.21.15.148:0/1255989707 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f023c06c530 0x7f023c06e9f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0244006a90 tx=0x7f0244006a20 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:36.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:36.863+0000 7f0237fff640 1 -- 172.21.15.148:0/1255989707 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f024003e070 con 0x7f025006ee10 2024-03-29T04:12:37.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.176+0000 7f02560b7640 1 -- 172.21.15.148:0/1255989707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0250068850 con 0x7f025006ee10 2024-03-29T04:12:37.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.176+0000 7f0237fff640 1 -- 172.21.15.148:0/1255989707 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) ==== 100+0+11 (secure 0 0 0) 0x7f024007bdb0 con 0x7f025006ee10 2024-03-29T04:12:37.155 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-03-29T04:12:37.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.179+0000 7f02560b7640 1 -- 172.21.15.148:0/1255989707 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f023c06c530 msgr2=0x7f023c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:37.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.179+0000 7f02560b7640 1 --2- 172.21.15.148:0/1255989707 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f023c06c530 0x7f023c06e9f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0244006a90 tx=0x7f0244006a20 comp rx=0 tx=0).stop 2024-03-29T04:12:37.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.179+0000 7f02560b7640 1 -- 172.21.15.148:0/1255989707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f025006ee10 msgr2=0x7f02500c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:37.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.179+0000 7f02560b7640 1 --2- 172.21.15.148:0/1255989707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f025006ee10 0x7f02500c9b30 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f024002b9e0 tx=0x7f02400046c0 comp rx=0 tx=0).stop 2024-03-29T04:12:37.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.179+0000 7f02560b7640 1 -- 172.21.15.148:0/1255989707 shutdown_connections 2024-03-29T04:12:37.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.179+0000 7f02560b7640 1 --2- 172.21.15.148:0/1255989707 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f023c06c530 0x7f023c06e9f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:37.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.179+0000 7f02560b7640 1 --2- 172.21.15.148:0/1255989707 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f025015a120 0x7f025015bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:37.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.179+0000 7f02560b7640 1 --2- 172.21.15.148:0/1255989707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02500ce310 0x7f0250147210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:37.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.179+0000 7f02560b7640 1 --2- 172.21.15.148:0/1255989707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f025006ee10 0x7f02500c9b30 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:37.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.179+0000 7f02560b7640 1 -- 172.21.15.148:0/1255989707 >> 172.21.15.148:0/1255989707 conn(0x7f02500cead0 msgr2=0x7f025015dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:37.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.179+0000 7f02560b7640 1 -- 172.21.15.148:0/1255989707 shutdown_connections 2024-03-29T04:12:37.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.179+0000 7f02560b7640 1 -- 172.21.15.148:0/1255989707 wait complete. 2024-03-29T04:12:37.170 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:12:37.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.329+0000 7f3b6c574640 1 Processor -- start 2024-03-29T04:12:37.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.329+0000 7f3b6c574640 1 -- start start 2024-03-29T04:12:37.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.330+0000 7f3b6c574640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3b640d17e0 0x7f3b640d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:37.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.330+0000 7f3b6c574640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b6415e7c0 con 0x7f3b640ba030 2024-03-29T04:12:37.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.330+0000 7f3b6c574640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b640d2c30 con 0x7f3b6415e3e0 2024-03-29T04:12:37.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.330+0000 7f3b6c574640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f3b640d3840 con 0x7f3b640d17e0 2024-03-29T04:12:37.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.330+0000 7f3b69b0f640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3b640ba030 0x7f3b640d10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39780/0 (socket says 172.21.15.148:39780) 2024-03-29T04:12:37.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.330+0000 7f3b69b0f640 1 -- 172.21.15.148:0/3737951491 learned_addr learned my addr 172.21.15.148:0/3737951491 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:37.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.330+0000 7f3b6930e640 1 -- 172.21.15.148:0/3737951491 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1740587458 0 0) 0x7f3b6415e7c0 con 0x7f3b640ba030 2024-03-29T04:12:37.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.330+0000 7f3b6930e640 1 -- 172.21.15.148:0/3737951491 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b50003520 con 0x7f3b640ba030 2024-03-29T04:12:37.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.330+0000 7f3b6ab11640 1 --2- 172.21.15.148:0/3737951491 >> v2:172.21.15.157:3300/0 conn(0x7f3b640d17e0 0x7f3b640d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:37.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.331+0000 7f3b6ab11640 1 -- 172.21.15.148:0/3737951491 >> v1:172.21.15.157:6789/0 conn(0x7f3b6415e3e0 legacy=0x7f3b640d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:37.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.331+0000 7f3b6930e640 1 -- 172.21.15.148:0/3737951491 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 140254162 0 0) 0x7f3b640d2c30 con 0x7f3b6415e3e0 2024-03-29T04:12:37.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.331+0000 7f3b6ab11640 1 -- 172.21.15.148:0/3737951491 >> v1:172.21.15.148:6789/0 conn(0x7f3b640ba030 legacy=0x7f3b640d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:37.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.331+0000 7f3b6ab11640 1 -- 172.21.15.148:0/3737951491 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b640d2f10 con 0x7f3b640d17e0 2024-03-29T04:12:37.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.332+0000 7f3b6ab11640 1 --2- 172.21.15.148:0/3737951491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b640d17e0 0x7f3b640d3cd0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c009d90 tx=0x7f3b5c02bdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85894ca25be96158 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:37.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.332+0000 7f3b6930e640 1 -- 172.21.15.148:0/3737951491 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c039070 con 0x7f3b640d17e0 2024-03-29T04:12:37.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.332+0000 7f3b6930e640 1 -- 172.21.15.148:0/3737951491 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b5c02c8f0 con 0x7f3b640d17e0 2024-03-29T04:12:37.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.332+0000 7f3b6930e640 1 -- 172.21.15.148:0/3737951491 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b60001670 con 0x7f3b640d17e0 2024-03-29T04:12:37.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.333+0000 7f3b6c574640 1 -- 172.21.15.148:0/3737951491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b640d17e0 msgr2=0x7f3b640d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:37.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.333+0000 7f3b6c574640 1 --2- 172.21.15.148:0/3737951491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b640d17e0 0x7f3b640d3cd0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c009d90 tx=0x7f3b5c02bdc0 comp rx=0 tx=0).stop 2024-03-29T04:12:37.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.333+0000 7f3b6c574640 1 -- 172.21.15.148:0/3737951491 shutdown_connections 2024-03-29T04:12:37.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.333+0000 7f3b6c574640 1 --2- 172.21.15.148:0/3737951491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b640d17e0 0x7f3b640d3cd0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:37.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.333+0000 7f3b6c574640 1 -- 172.21.15.148:0/3737951491 >> 172.21.15.148:0/3737951491 conn(0x7f3b640b6210 msgr2=0x7f3b640b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:37.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.333+0000 7f3b6c574640 1 -- 172.21.15.148:0/3737951491 shutdown_connections 2024-03-29T04:12:37.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.333+0000 7f3b6c574640 1 -- 172.21.15.148:0/3737951491 wait complete. 2024-03-29T04:12:37.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.334+0000 7f3b6c574640 1 Processor -- start 2024-03-29T04:12:37.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.334+0000 7f3b6c574640 1 -- start start 2024-03-29T04:12:37.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.335+0000 7f3b6c574640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b640ba030 0x7f3b6414b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:37.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.335+0000 7f3b6c574640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b640d17e0 0x7f3b641479f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:37.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.335+0000 7f3b6c574640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b6415e3e0 0x7f3b6414a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:37.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.335+0000 7f3b6c574640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3b64168f60 con 0x7f3b640d17e0 2024-03-29T04:12:37.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.335+0000 7f3b6c574640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3b640d34d0 con 0x7f3b640ba030 2024-03-29T04:12:37.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.335+0000 7f3b6c574640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3b641690e0 con 0x7f3b6415e3e0 2024-03-29T04:12:37.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.335+0000 7f3b69b0f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b640d17e0 0x7f3b641479f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:37.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.335+0000 7f3b69b0f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b640d17e0 0x7f3b641479f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60620/0 (socket says 172.21.15.148:60620) 2024-03-29T04:12:37.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.335+0000 7f3b69b0f640 1 -- 172.21.15.148:0/1324513561 learned_addr learned my addr 172.21.15.148:0/1324513561 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:37.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.335+0000 7f3b6a310640 1 --2- 172.21.15.148:0/1324513561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b640ba030 0x7f3b6414b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:37.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.335+0000 7f3b6ab11640 1 --2- 172.21.15.148:0/1324513561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b6415e3e0 0x7f3b6414a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:37.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.335+0000 7f3b69b0f640 1 -- 172.21.15.148:0/1324513561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b6415e3e0 msgr2=0x7f3b6414a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:37.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.335+0000 7f3b69b0f640 1 --2- 172.21.15.148:0/1324513561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b6415e3e0 0x7f3b6414a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:37.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.335+0000 7f3b69b0f640 1 -- 172.21.15.148:0/1324513561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b640ba030 msgr2=0x7f3b6414b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:37.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.335+0000 7f3b69b0f640 1 --2- 172.21.15.148:0/1324513561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b640ba030 0x7f3b6414b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:37.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.335+0000 7f3b69b0f640 1 -- 172.21.15.148:0/1324513561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b6414eeb0 con 0x7f3b640d17e0 2024-03-29T04:12:37.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.335+0000 7f3b6a310640 1 --2- 172.21.15.148:0/1324513561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b640ba030 0x7f3b6414b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:12:37.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.335+0000 7f3b69b0f640 1 --2- 172.21.15.148:0/1324513561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b640d17e0 0x7f3b641479f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000c7d0 tx=0x7f3b6000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:37.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.335+0000 7f3b5affd640 1 -- 172.21.15.148:0/1324513561 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b60018070 con 0x7f3b640d17e0 2024-03-29T04:12:37.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.335+0000 7f3b5affd640 1 -- 172.21.15.148:0/1324513561 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b60007500 con 0x7f3b640d17e0 2024-03-29T04:12:37.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.335+0000 7f3b6c574640 1 -- 172.21.15.148:0/1324513561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b6414f690 con 0x7f3b640d17e0 2024-03-29T04:12:37.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.335+0000 7f3b5affd640 1 -- 172.21.15.148:0/1324513561 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b60004a20 con 0x7f3b640d17e0 2024-03-29T04:12:37.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.335+0000 7f3b6c574640 1 -- 172.21.15.148:0/1324513561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b6414fc50 con 0x7f3b640d17e0 2024-03-29T04:12:37.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.336+0000 7f3b6c574640 1 -- 172.21.15.148:0/1324513561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b6406cf80 con 0x7f3b640d17e0 2024-03-29T04:12:37.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.337+0000 7f3b5affd640 1 -- 172.21.15.148:0/1324513561 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3b6001b070 con 0x7f3b640d17e0 2024-03-29T04:12:37.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.337+0000 7f3b5affd640 1 --2- 172.21.15.148:0/1324513561 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3b4806c530 0x7f3b4806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:37.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.337+0000 7f3b5affd640 1 -- 172.21.15.148:0/1324513561 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f3b6008af00 con 0x7f3b640d17e0 2024-03-29T04:12:37.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.337+0000 7f3b6a310640 1 --2- 172.21.15.148:0/1324513561 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3b4806c530 0x7f3b4806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:37.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.338+0000 7f3b6a310640 1 --2- 172.21.15.148:0/1324513561 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3b4806c530 0x7f3b4806e9f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3b54000f60 tx=0x7f3b54009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:37.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.342+0000 7f3b5affd640 1 -- 172.21.15.148:0/1324513561 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3b6406cf80 con 0x7f3b640d17e0 2024-03-29T04:12:37.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.560+0000 7f3b6c574640 1 -- 172.21.15.148:0/1324513561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b640d1460 con 0x7f3b640d17e0 2024-03-29T04:12:37.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.561+0000 7f3b5affd640 1 -- 172.21.15.148:0/1324513561 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5190 (secure 0 0 0) 0x7f3b640d1460 con 0x7f3b640d17e0 2024-03-29T04:12:37.540 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:12:37.540 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2618617956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2618617956},{"type":"v1","addr":"172.21.15.157:6834","nonce":2618617956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5260,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1177855619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1177855619},{"type":"v1","addr":"172.21.15.157:6835","nonce":1177855619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5562,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3610463670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3610463670},{"type":"v1","addr":"172.21.15.148:6835","nonce":3610463670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:11:32.950391+0000","modified":"2024-03-29T04:11:53.076916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5583},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5583":{"gid":5583,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/473448013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":473448013},{"type":"v1","addr":"172.21.15.148:6837","nonce":473448013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5583,"qdb_cluster":[5583]},"id":2}]} 2024-03-29T04:12:37.540 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 19 2024-03-29T04:12:37.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.563+0000 7f3b6c574640 1 -- 172.21.15.148:0/1324513561 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3b4806c530 msgr2=0x7f3b4806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:37.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.563+0000 7f3b6c574640 1 --2- 172.21.15.148:0/1324513561 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3b4806c530 0x7f3b4806e9f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3b54000f60 tx=0x7f3b54009040 comp rx=0 tx=0).stop 2024-03-29T04:12:37.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.563+0000 7f3b6c574640 1 -- 172.21.15.148:0/1324513561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b640d17e0 msgr2=0x7f3b641479f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:37.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.563+0000 7f3b6c574640 1 --2- 172.21.15.148:0/1324513561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b640d17e0 0x7f3b641479f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000c7d0 tx=0x7f3b6000cc90 comp rx=0 tx=0).stop 2024-03-29T04:12:37.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.563+0000 7f3b6c574640 1 -- 172.21.15.148:0/1324513561 shutdown_connections 2024-03-29T04:12:37.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.563+0000 7f3b6c574640 1 --2- 172.21.15.148:0/1324513561 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3b4806c530 0x7f3b4806e9f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:37.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.563+0000 7f3b6c574640 1 --2- 172.21.15.148:0/1324513561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b6415e3e0 0x7f3b6414a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:37.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.563+0000 7f3b6c574640 1 --2- 172.21.15.148:0/1324513561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b640d17e0 0x7f3b641479f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:37.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.563+0000 7f3b6c574640 1 --2- 172.21.15.148:0/1324513561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b640ba030 0x7f3b6414b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:37.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.563+0000 7f3b6c574640 1 -- 172.21.15.148:0/1324513561 >> 172.21.15.148:0/1324513561 conn(0x7f3b640b6210 msgr2=0x7f3b640b6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:37.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.563+0000 7f3b6c574640 1 -- 172.21.15.148:0/1324513561 shutdown_connections 2024-03-29T04:12:37.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.563+0000 7f3b6c574640 1 -- 172.21.15.148:0/1324513561 wait complete. 2024-03-29T04:12:37.556 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:12:37.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.683+0000 7fb0bcba8640 1 Processor -- start 2024-03-29T04:12:37.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.683+0000 7fb0bcba8640 1 -- start start 2024-03-29T04:12:37.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.684+0000 7fb0bcba8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb0b80d17e0 0x7fb0b80d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:37.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.684+0000 7fb0bcba8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0b815e7c0 con 0x7fb0b815e3e0 2024-03-29T04:12:37.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.684+0000 7fb0bcba8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0b80d2c30 con 0x7fb0b80ba030 2024-03-29T04:12:37.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.684+0000 7fb0bcba8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fb0b80d3840 con 0x7fb0b80d17e0 2024-03-29T04:12:37.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.684+0000 7fb0b6d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb0b80d17e0 0x7fb0b80d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:37.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.684+0000 7fb0b659c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb0b815e3e0 0x7fb0b80d2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39788/0 (socket says 172.21.15.148:39788) 2024-03-29T04:12:37.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.684+0000 7fb0b659c640 1 -- 172.21.15.148:0/943259705 learned_addr learned my addr 172.21.15.148:0/943259705 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:37.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.684+0000 7fb0b6d9d640 1 -- 172.21.15.148:0/943259705 >> v1:172.21.15.157:6789/0 conn(0x7fb0b80ba030 legacy=0x7fb0b80d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:37.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.685+0000 7fb0b6d9d640 1 -- 172.21.15.148:0/943259705 >> v1:172.21.15.148:6789/0 conn(0x7fb0b815e3e0 legacy=0x7fb0b80d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:37.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.685+0000 7fb0b5d9b640 1 -- 172.21.15.148:0/943259705 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 266730334 0 0) 0x7fb0b815e7c0 con 0x7fb0b815e3e0 2024-03-29T04:12:37.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.685+0000 7fb0b6d9d640 1 -- 172.21.15.148:0/943259705 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b80d2f10 con 0x7fb0b80d17e0 2024-03-29T04:12:37.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.685+0000 7fb0b6d9d640 1 --2- 172.21.15.148:0/943259705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b80d17e0 0x7fb0b80d3cd0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0009d90 tx=0x7fb0b002bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1520cee4655b12e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:37.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.685+0000 7fb0b5d9b640 1 -- 172.21.15.148:0/943259705 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0039070 con 0x7fb0b80d17e0 2024-03-29T04:12:37.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.685+0000 7fb0b5d9b640 1 -- 172.21.15.148:0/943259705 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0b002cd80 con 0x7fb0b80d17e0 2024-03-29T04:12:37.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.686+0000 7fb0b5d9b640 1 -- 172.21.15.148:0/943259705 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0034660 con 0x7fb0b80d17e0 2024-03-29T04:12:37.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.686+0000 7fb0bcba8640 1 -- 172.21.15.148:0/943259705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b80d17e0 msgr2=0x7fb0b80d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:37.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.686+0000 7fb0bcba8640 1 --2- 172.21.15.148:0/943259705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b80d17e0 0x7fb0b80d3cd0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0009d90 tx=0x7fb0b002bfd0 comp rx=0 tx=0).stop 2024-03-29T04:12:37.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.686+0000 7fb0bcba8640 1 -- 172.21.15.148:0/943259705 shutdown_connections 2024-03-29T04:12:37.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.686+0000 7fb0bcba8640 1 --2- 172.21.15.148:0/943259705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b80d17e0 0x7fb0b80d3cd0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:37.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.686+0000 7fb0bcba8640 1 -- 172.21.15.148:0/943259705 >> 172.21.15.148:0/943259705 conn(0x7fb0b80b6210 msgr2=0x7fb0b80b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:37.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.686+0000 7fb0bcba8640 1 -- 172.21.15.148:0/943259705 shutdown_connections 2024-03-29T04:12:37.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.686+0000 7fb0bcba8640 1 -- 172.21.15.148:0/943259705 wait complete. 2024-03-29T04:12:37.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.687+0000 7fb0bcba8640 1 Processor -- start 2024-03-29T04:12:37.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.687+0000 7fb0bcba8640 1 -- start start 2024-03-29T04:12:37.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.688+0000 7fb0bcba8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b80ba030 0x7fb0b8154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:37.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.688+0000 7fb0bcba8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0b80d17e0 0x7fb0b8159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:37.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.688+0000 7fb0bcba8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0b815e3e0 0x7fb0b8159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:37.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.688+0000 7fb0bcba8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b8168f60 con 0x7fb0b80ba030 2024-03-29T04:12:37.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.688+0000 7fb0bcba8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b80d34d0 con 0x7fb0b80d17e0 2024-03-29T04:12:37.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.688+0000 7fb0bcba8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb0b81690e0 con 0x7fb0b815e3e0 2024-03-29T04:12:37.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.688+0000 7fb0b659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b80ba030 0x7fb0b8154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:37.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.688+0000 7fb0b659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b80ba030 0x7fb0b8154700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60642/0 (socket says 172.21.15.148:60642) 2024-03-29T04:12:37.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.688+0000 7fb0b659c640 1 -- 172.21.15.148:0/1862717286 learned_addr learned my addr 172.21.15.148:0/1862717286 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:37.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.689+0000 7fb0b659c640 1 -- 172.21.15.148:0/1862717286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0b815e3e0 msgr2=0x7fb0b8159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:37.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.689+0000 7fb0b659c640 1 --2- 172.21.15.148:0/1862717286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0b815e3e0 0x7fb0b8159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:37.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.689+0000 7fb0b659c640 1 -- 172.21.15.148:0/1862717286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0b80d17e0 msgr2=0x7fb0b8159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:37.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.689+0000 7fb0affff640 1 --2- 172.21.15.148:0/1862717286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0b80d17e0 0x7fb0b8159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:37.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.689+0000 7fb0b659c640 1 --2- 172.21.15.148:0/1862717286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0b80d17e0 0x7fb0b8159760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:37.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.689+0000 7fb0b659c640 1 -- 172.21.15.148:0/1862717286 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b80cb510 con 0x7fb0b80ba030 2024-03-29T04:12:37.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.689+0000 7fb0b659c640 1 --2- 172.21.15.148:0/1862717286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b80ba030 0x7fb0b8154700 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800c3f0 tx=0x7fb0a800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:37.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.689+0000 7fb0aeffd640 1 -- 172.21.15.148:0/1862717286 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a80074e0 con 0x7fb0b80ba030 2024-03-29T04:12:37.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.689+0000 7fb0aeffd640 1 -- 172.21.15.148:0/1862717286 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0a80047e0 con 0x7fb0b80ba030 2024-03-29T04:12:37.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.689+0000 7fb0bcba8640 1 -- 172.21.15.148:0/1862717286 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b80cb7f0 con 0x7fb0b80ba030 2024-03-29T04:12:37.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.689+0000 7fb0aeffd640 1 -- 172.21.15.148:0/1862717286 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8004c10 con 0x7fb0b80ba030 2024-03-29T04:12:37.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.689+0000 7fb0bcba8640 1 -- 172.21.15.148:0/1862717286 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b80cbdb0 con 0x7fb0b80ba030 2024-03-29T04:12:37.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.690+0000 7fb0bcba8640 1 -- 172.21.15.148:0/1862717286 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0b80ce940 con 0x7fb0b80ba030 2024-03-29T04:12:37.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.691+0000 7fb0aeffd640 1 -- 172.21.15.148:0/1862717286 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb0a8011070 con 0x7fb0b80ba030 2024-03-29T04:12:37.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.691+0000 7fb0aeffd640 1 --2- 172.21.15.148:0/1862717286 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb09806c560 0x7fb09806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:37.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.691+0000 7fb0aeffd640 1 -- 172.21.15.148:0/1862717286 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7fb0a808b440 con 0x7fb0b80ba030 2024-03-29T04:12:37.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.691+0000 7fb0affff640 1 --2- 172.21.15.148:0/1862717286 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb09806c560 0x7fb09806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:37.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.692+0000 7fb0affff640 1 --2- 172.21.15.148:0/1862717286 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb09806c560 0x7fb09806ea20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8161850 tx=0x7fb0a0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:37.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.696+0000 7fb0aeffd640 1 -- 172.21.15.148:0/1862717286 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb0b80ce940 con 0x7fb0b80ba030 2024-03-29T04:12:37.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.877+0000 7fb0bcba8640 1 -- 172.21.15.148:0/1862717286 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb0b8161880 con 0x7fb0b80ba030 2024-03-29T04:12:37.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.878+0000 7fb0aeffd640 1 -- 172.21.15.148:0/1862717286 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+17003 (secure 0 0 0) 0x7fb0a8088280 con 0x7fb0b80ba030 2024-03-29T04:12:37.858 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:12:37.858 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":41,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:11:42.632417+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:11:30.794790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T04:11:31.907100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:12:37.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.880+0000 7fb0bcba8640 1 -- 172.21.15.148:0/1862717286 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb09806c560 msgr2=0x7fb09806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:37.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.880+0000 7fb0bcba8640 1 --2- 172.21.15.148:0/1862717286 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb09806c560 0x7fb09806ea20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8161850 tx=0x7fb0a0009040 comp rx=0 tx=0).stop 2024-03-29T04:12:37.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.880+0000 7fb0bcba8640 1 -- 172.21.15.148:0/1862717286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b80ba030 msgr2=0x7fb0b8154700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:37.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.880+0000 7fb0bcba8640 1 --2- 172.21.15.148:0/1862717286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b80ba030 0x7fb0b8154700 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800c3f0 tx=0x7fb0a800c8b0 comp rx=0 tx=0).stop 2024-03-29T04:12:37.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.880+0000 7fb0bcba8640 1 -- 172.21.15.148:0/1862717286 shutdown_connections 2024-03-29T04:12:37.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.880+0000 7fb0bcba8640 1 --2- 172.21.15.148:0/1862717286 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb09806c560 0x7fb09806ea20 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:37.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.880+0000 7fb0bcba8640 1 --2- 172.21.15.148:0/1862717286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb0b815e3e0 0x7fb0b8159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:37.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.880+0000 7fb0bcba8640 1 --2- 172.21.15.148:0/1862717286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0b80d17e0 0x7fb0b8159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:37.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.880+0000 7fb0bcba8640 1 --2- 172.21.15.148:0/1862717286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b80ba030 0x7fb0b8154700 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:37.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.880+0000 7fb0bcba8640 1 -- 172.21.15.148:0/1862717286 >> 172.21.15.148:0/1862717286 conn(0x7fb0b80b6210 msgr2=0x7fb0b80d22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:37.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.880+0000 7fb0bcba8640 1 -- 172.21.15.148:0/1862717286 shutdown_connections 2024-03-29T04:12:37.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.880+0000 7fb0bcba8640 1 -- 172.21.15.148:0/1862717286 wait complete. 2024-03-29T04:12:37.871 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-29T04:12:37.871 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:12:37.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.996+0000 7f54ea3cd640 1 Processor -- start 2024-03-29T04:12:37.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.996+0000 7f54ea3cd640 1 -- start start 2024-03-29T04:12:37.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.996+0000 7f54ea3cd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f54e40ba030 0x7f54e40d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:37.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.996+0000 7f54ea3cd640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54e415e7c0 con 0x7f54e40d10d0 2024-03-29T04:12:37.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.996+0000 7f54ea3cd640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54e40d2c30 con 0x7f54e415e3e0 2024-03-29T04:12:37.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.996+0000 7f54ea3cd640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f54e40d3840 con 0x7f54e40ba030 2024-03-29T04:12:37.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.996+0000 7f54e37fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f54e40ba030 0x7f54e40d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:37.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.997+0000 7f54e37fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f54e40ba030 0x7f54e40d3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60644/0 (socket says 172.21.15.148:60644) 2024-03-29T04:12:37.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.997+0000 7f54e896a640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f54e40d10d0 0x7f54e4167f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39800/0 (socket says 172.21.15.148:39800) 2024-03-29T04:12:37.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.997+0000 7f54e37fe640 1 -- 172.21.15.148:0/873371046 learned_addr learned my addr 172.21.15.148:0/873371046 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:37.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.997+0000 7f54e37fe640 1 -- 172.21.15.148:0/873371046 >> v1:172.21.15.157:6789/0 conn(0x7f54e415e3e0 legacy=0x7f54e40d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:37.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.997+0000 7f54e37fe640 1 -- 172.21.15.148:0/873371046 >> v1:172.21.15.148:6789/0 conn(0x7f54e40d10d0 legacy=0x7f54e4167f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:37.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.997+0000 7f54e37fe640 1 -- 172.21.15.148:0/873371046 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e40d2f10 con 0x7f54e40ba030 2024-03-29T04:12:37.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.997+0000 7f54e2ffd640 1 -- 172.21.15.148:0/873371046 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 280914850 0 0) 0x7f54e415e7c0 con 0x7f54e40d10d0 2024-03-29T04:12:37.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.998+0000 7f54e37fe640 1 --2- 172.21.15.148:0/873371046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e40ba030 0x7f54e40d3cd0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f54d8009870 tx=0x7f54d802bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde01ac4907e627f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:37.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.998+0000 7f54e2ffd640 1 -- 172.21.15.148:0/873371046 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d802d870 con 0x7f54e40ba030 2024-03-29T04:12:37.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.998+0000 7f54e2ffd640 1 -- 172.21.15.148:0/873371046 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54d802da10 con 0x7f54e40ba030 2024-03-29T04:12:37.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.998+0000 7f54e2ffd640 1 -- 172.21.15.148:0/873371046 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d802dde0 con 0x7f54e40ba030 2024-03-29T04:12:37.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.998+0000 7f54ea3cd640 1 -- 172.21.15.148:0/873371046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e40ba030 msgr2=0x7f54e40d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:37.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.998+0000 7f54ea3cd640 1 --2- 172.21.15.148:0/873371046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e40ba030 0x7f54e40d3cd0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f54d8009870 tx=0x7f54d802bab0 comp rx=0 tx=0).stop 2024-03-29T04:12:37.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.998+0000 7f54ea3cd640 1 -- 172.21.15.148:0/873371046 shutdown_connections 2024-03-29T04:12:37.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.998+0000 7f54ea3cd640 1 --2- 172.21.15.148:0/873371046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e40ba030 0x7f54e40d3cd0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:37.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.998+0000 7f54ea3cd640 1 -- 172.21.15.148:0/873371046 >> 172.21.15.148:0/873371046 conn(0x7f54e40b6210 msgr2=0x7f54e40b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:37.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.998+0000 7f54ea3cd640 1 -- 172.21.15.148:0/873371046 shutdown_connections 2024-03-29T04:12:37.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.998+0000 7f54ea3cd640 1 -- 172.21.15.148:0/873371046 wait complete. 2024-03-29T04:12:37.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:37.999+0000 7f54ea3cd640 1 Processor -- start 2024-03-29T04:12:37.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.000+0000 7f54ea3cd640 1 -- start start 2024-03-29T04:12:37.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.000+0000 7f54ea3cd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54e40ba030 0x7f54e4155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:37.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.000+0000 7f54ea3cd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e40d10d0 0x7f54e40cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:37.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.000+0000 7f54ea3cd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54e415e3e0 0x7f54e41f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:37.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.000+0000 7f54ea3cd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f54e40d1530 con 0x7f54e40d10d0 2024-03-29T04:12:37.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.000+0000 7f54ea3cd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f54e40d34d0 con 0x7f54e415e3e0 2024-03-29T04:12:37.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.000+0000 7f54ea3cd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f54e40d16b0 con 0x7f54e40ba030 2024-03-29T04:12:37.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.000+0000 7f54e37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e40d10d0 0x7f54e40cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:37.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.000+0000 7f54e37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e40d10d0 0x7f54e40cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60658/0 (socket says 172.21.15.148:60658) 2024-03-29T04:12:37.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.000+0000 7f54e37fe640 1 -- 172.21.15.148:0/64786742 learned_addr learned my addr 172.21.15.148:0/64786742 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:37.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.001+0000 7f54e896a640 1 --2- 172.21.15.148:0/64786742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54e415e3e0 0x7f54e41f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:37.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.001+0000 7f54e3fff640 1 --2- 172.21.15.148:0/64786742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54e40ba030 0x7f54e4155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:37.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.001+0000 7f54e37fe640 1 -- 172.21.15.148:0/64786742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54e40ba030 msgr2=0x7f54e4155060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:37.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.001+0000 7f54e37fe640 1 --2- 172.21.15.148:0/64786742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54e40ba030 0x7f54e4155060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:37.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.001+0000 7f54e37fe640 1 -- 172.21.15.148:0/64786742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54e415e3e0 msgr2=0x7f54e41f3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:37.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.001+0000 7f54e37fe640 1 --2- 172.21.15.148:0/64786742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54e415e3e0 0x7f54e41f3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:37.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.001+0000 7f54e37fe640 1 -- 172.21.15.148:0/64786742 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e41f4440 con 0x7f54e40d10d0 2024-03-29T04:12:37.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.001+0000 7f54e37fe640 1 --2- 172.21.15.148:0/64786742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e40d10d0 0x7f54e40cd9c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f54d8009990 tx=0x7f54d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:37.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.001+0000 7f54e0ff9640 1 -- 172.21.15.148:0/64786742 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d8047070 con 0x7f54e40d10d0 2024-03-29T04:12:37.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.001+0000 7f54e0ff9640 1 -- 172.21.15.148:0/64786742 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54d8037ac0 con 0x7f54e40d10d0 2024-03-29T04:12:37.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.001+0000 7f54ea3cd640 1 -- 172.21.15.148:0/64786742 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e4156170 con 0x7f54e40d10d0 2024-03-29T04:12:37.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.001+0000 7f54e0ff9640 1 -- 172.21.15.148:0/64786742 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d802e440 con 0x7f54e40d10d0 2024-03-29T04:12:37.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.001+0000 7f54ea3cd640 1 -- 172.21.15.148:0/64786742 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e40ccee0 con 0x7f54e40d10d0 2024-03-29T04:12:37.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.002+0000 7f54ea3cd640 1 -- 172.21.15.148:0/64786742 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54e40ce740 con 0x7f54e40d10d0 2024-03-29T04:12:37.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.003+0000 7f54e0ff9640 1 -- 172.21.15.148:0/64786742 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f54d804b430 con 0x7f54e40d10d0 2024-03-29T04:12:37.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.003+0000 7f54e0ff9640 1 --2- 172.21.15.148:0/64786742 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f54d006c530 0x7f54d006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:37.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.003+0000 7f54e0ff9640 1 -- 172.21.15.148:0/64786742 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f54d80ad450 con 0x7f54e40d10d0 2024-03-29T04:12:37.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.003+0000 7f54e3fff640 1 --2- 172.21.15.148:0/64786742 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f54d006c530 0x7f54d006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:37.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.003+0000 7f54e3fff640 1 --2- 172.21.15.148:0/64786742 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f54d006c530 0x7f54d006e9f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f54d4000f60 tx=0x7f54d4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:37.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.007+0000 7f54e0ff9640 1 -- 172.21.15.148:0/64786742 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f54e40ce740 con 0x7f54e40d10d0 2024-03-29T04:12:38.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.226+0000 7f54ea3cd640 1 -- 172.21.15.148:0/64786742 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54e40b9250 con 0x7f54e40d10d0 2024-03-29T04:12:38.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.227+0000 7f54e0ff9640 1 -- 172.21.15.148:0/64786742 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5190 (secure 0 0 0) 0x7f54e41f4440 con 0x7f54e40d10d0 2024-03-29T04:12:38.206 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:12:38.206 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2618617956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2618617956},{"type":"v1","addr":"172.21.15.157:6834","nonce":2618617956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5260,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1177855619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1177855619},{"type":"v1","addr":"172.21.15.157:6835","nonce":1177855619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5562,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3610463670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3610463670},{"type":"v1","addr":"172.21.15.148:6835","nonce":3610463670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:11:32.950391+0000","modified":"2024-03-29T04:11:53.076916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5583},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5583":{"gid":5583,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/473448013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":473448013},{"type":"v1","addr":"172.21.15.148:6837","nonce":473448013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5583,"qdb_cluster":[5583]},"id":2}]} 2024-03-29T04:12:38.206 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 19 2024-03-29T04:12:38.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.229+0000 7f54ea3cd640 1 -- 172.21.15.148:0/64786742 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f54d006c530 msgr2=0x7f54d006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:38.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.229+0000 7f54ea3cd640 1 --2- 172.21.15.148:0/64786742 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f54d006c530 0x7f54d006e9f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f54d4000f60 tx=0x7f54d4009040 comp rx=0 tx=0).stop 2024-03-29T04:12:38.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.229+0000 7f54ea3cd640 1 -- 172.21.15.148:0/64786742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e40d10d0 msgr2=0x7f54e40cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:38.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.229+0000 7f54ea3cd640 1 --2- 172.21.15.148:0/64786742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e40d10d0 0x7f54e40cd9c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f54d8009990 tx=0x7f54d8004730 comp rx=0 tx=0).stop 2024-03-29T04:12:38.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.229+0000 7f54ea3cd640 1 -- 172.21.15.148:0/64786742 shutdown_connections 2024-03-29T04:12:38.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.229+0000 7f54ea3cd640 1 --2- 172.21.15.148:0/64786742 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f54d006c530 0x7f54d006e9f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.229+0000 7f54ea3cd640 1 --2- 172.21.15.148:0/64786742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54e415e3e0 0x7f54e41f3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.229+0000 7f54ea3cd640 1 --2- 172.21.15.148:0/64786742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e40d10d0 0x7f54e40cd9c0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.229+0000 7f54ea3cd640 1 --2- 172.21.15.148:0/64786742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54e40ba030 0x7f54e4155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.229+0000 7f54ea3cd640 1 -- 172.21.15.148:0/64786742 >> 172.21.15.148:0/64786742 conn(0x7f54e40b6210 msgr2=0x7f54e40b6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:38.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.229+0000 7f54ea3cd640 1 -- 172.21.15.148:0/64786742 shutdown_connections 2024-03-29T04:12:38.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.229+0000 7f54ea3cd640 1 -- 172.21.15.148:0/64786742 wait complete. 2024-03-29T04:12:38.219 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:12:38.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.340+0000 7f4521d0e640 1 Processor -- start 2024-03-29T04:12:38.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.341+0000 7f4521d0e640 1 -- start start 2024-03-29T04:12:38.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.341+0000 7f4521d0e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f451c0ba030 0x7f451c0d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:38.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.341+0000 7f4521d0e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f451c0d10d0 0x7f451c0d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:38.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.341+0000 7f4521d0e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f451c15e7c0 con 0x7f451c15e3e0 2024-03-29T04:12:38.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.341+0000 7f4521d0e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f451c0d3840 con 0x7f451c0ba030 2024-03-29T04:12:38.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.341+0000 7f4521d0e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f451c0d2c30 con 0x7f451c0d10d0 2024-03-29T04:12:38.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.341+0000 7f451b7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f451c15e3e0 0x7f451c0d2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39816/0 (socket says 172.21.15.148:39816) 2024-03-29T04:12:38.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.341+0000 7f451b7fe640 1 -- 172.21.15.148:0/3774272734 learned_addr learned my addr 172.21.15.148:0/3774272734 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:38.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.342+0000 7f451affd640 1 --2- 172.21.15.148:0/3774272734 >> v2:172.21.15.157:3300/0 conn(0x7f451c0ba030 0x7f451c0d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:38.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.342+0000 7f451bfff640 1 --2- 172.21.15.148:0/3774272734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f451c0d10d0 0x7f451c0d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:38.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.342+0000 7f451a7fc640 1 -- 172.21.15.148:0/3774272734 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3504195482 0 0) 0x7f451c15e7c0 con 0x7f451c15e3e0 2024-03-29T04:12:38.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.342+0000 7f451a7fc640 1 -- 172.21.15.148:0/3774272734 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4508003520 con 0x7f451c15e3e0 2024-03-29T04:12:38.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.342+0000 7f451affd640 1 -- 172.21.15.148:0/3774272734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f451c0d10d0 msgr2=0x7f451c0d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:38.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.342+0000 7f451affd640 1 --2- 172.21.15.148:0/3774272734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f451c0d10d0 0x7f451c0d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.342+0000 7f451affd640 1 -- 172.21.15.148:0/3774272734 >> v1:172.21.15.148:6789/0 conn(0x7f451c15e3e0 legacy=0x7f451c0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:38.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.342+0000 7f451affd640 1 -- 172.21.15.148:0/3774272734 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451c0d2f10 con 0x7f451c0ba030 2024-03-29T04:12:38.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.343+0000 7f451affd640 1 --2- 172.21.15.148:0/3774272734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f451c0ba030 0x7f451c0d3cd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f45040097a0 tx=0x7f450402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=227839bcfc4d0e4a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:38.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.343+0000 7f451a7fc640 1 -- 172.21.15.148:0/3774272734 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4504034070 con 0x7f451c0ba030 2024-03-29T04:12:38.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.343+0000 7f451a7fc640 1 -- 172.21.15.148:0/3774272734 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4504030610 con 0x7f451c0ba030 2024-03-29T04:12:38.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.343+0000 7f451a7fc640 1 -- 172.21.15.148:0/3774272734 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45040309e0 con 0x7f451c0ba030 2024-03-29T04:12:38.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.344+0000 7f4521d0e640 1 -- 172.21.15.148:0/3774272734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f451c0ba030 msgr2=0x7f451c0d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:38.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.344+0000 7f4521d0e640 1 --2- 172.21.15.148:0/3774272734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f451c0ba030 0x7f451c0d3cd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f45040097a0 tx=0x7f450402b960 comp rx=0 tx=0).stop 2024-03-29T04:12:38.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.344+0000 7f4521d0e640 1 -- 172.21.15.148:0/3774272734 shutdown_connections 2024-03-29T04:12:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.344+0000 7f4521d0e640 1 --2- 172.21.15.148:0/3774272734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f451c0d10d0 0x7f451c0d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.344+0000 7f4521d0e640 1 --2- 172.21.15.148:0/3774272734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f451c0ba030 0x7f451c0d3cd0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.344+0000 7f4521d0e640 1 -- 172.21.15.148:0/3774272734 >> 172.21.15.148:0/3774272734 conn(0x7f451c0b6210 msgr2=0x7f451c0b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.344+0000 7f4521d0e640 1 -- 172.21.15.148:0/3774272734 shutdown_connections 2024-03-29T04:12:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.344+0000 7f4521d0e640 1 -- 172.21.15.148:0/3774272734 wait complete. 2024-03-29T04:12:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.345+0000 7f4521d0e640 1 Processor -- start 2024-03-29T04:12:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.345+0000 7f4521d0e640 1 -- start start 2024-03-29T04:12:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.346+0000 7f4521d0e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f451c0ba030 0x7f451c156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.346+0000 7f4521d0e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f451c0d10d0 0x7f451c155890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.346+0000 7f4521d0e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f451c15e3e0 0x7f451c06c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.346+0000 7f4521d0e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f451c167f50 con 0x7f451c0d10d0 2024-03-29T04:12:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.346+0000 7f4521d0e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f451c0d34d0 con 0x7f451c0ba030 2024-03-29T04:12:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.346+0000 7f4521d0e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f451c1680d0 con 0x7f451c15e3e0 2024-03-29T04:12:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.346+0000 7f451affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f451c0d10d0 0x7f451c155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:38.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.346+0000 7f451affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f451c0d10d0 0x7f451c155890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60664/0 (socket says 172.21.15.148:60664) 2024-03-29T04:12:38.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.346+0000 7f451affd640 1 -- 172.21.15.148:0/1963531058 learned_addr learned my addr 172.21.15.148:0/1963531058 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:38.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.346+0000 7f451bfff640 1 --2- 172.21.15.148:0/1963531058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f451c15e3e0 0x7f451c06c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:38.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.346+0000 7f451b7fe640 1 --2- 172.21.15.148:0/1963531058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f451c0ba030 0x7f451c156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:38.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.346+0000 7f451affd640 1 -- 172.21.15.148:0/1963531058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f451c15e3e0 msgr2=0x7f451c06c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:38.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.346+0000 7f451affd640 1 --2- 172.21.15.148:0/1963531058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f451c15e3e0 0x7f451c06c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.346+0000 7f451affd640 1 -- 172.21.15.148:0/1963531058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f451c0ba030 msgr2=0x7f451c156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:38.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.346+0000 7f451affd640 1 --2- 172.21.15.148:0/1963531058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f451c0ba030 0x7f451c156f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.346+0000 7f451affd640 1 -- 172.21.15.148:0/1963531058 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451c144f20 con 0x7f451c0d10d0 2024-03-29T04:12:38.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.346+0000 7f451bfff640 1 --2- 172.21.15.148:0/1963531058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f451c15e3e0 0x7f451c06c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:38.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.346+0000 7f451affd640 1 --2- 172.21.15.148:0/1963531058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f451c0d10d0 0x7f451c155890 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f450402bf70 tx=0x7f45040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:38.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.346+0000 7f4513fff640 1 -- 172.21.15.148:0/1963531058 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4504034040 con 0x7f451c0d10d0 2024-03-29T04:12:38.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.346+0000 7f4513fff640 1 -- 172.21.15.148:0/1963531058 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f450403aaa0 con 0x7f451c0d10d0 2024-03-29T04:12:38.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.346+0000 7f4521d0e640 1 -- 172.21.15.148:0/1963531058 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f451c156180 con 0x7f451c0d10d0 2024-03-29T04:12:38.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.346+0000 7f4513fff640 1 -- 172.21.15.148:0/1963531058 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4504006e10 con 0x7f451c0d10d0 2024-03-29T04:12:38.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.346+0000 7f4521d0e640 1 -- 172.21.15.148:0/1963531058 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f451c156740 con 0x7f451c0d10d0 2024-03-29T04:12:38.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.348+0000 7f4521d0e640 1 -- 172.21.15.148:0/1963531058 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f451c0687e0 con 0x7f451c0d10d0 2024-03-29T04:12:38.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.348+0000 7f4513fff640 1 -- 172.21.15.148:0/1963531058 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f450404b060 con 0x7f451c0d10d0 2024-03-29T04:12:38.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.348+0000 7f4513fff640 1 --2- 172.21.15.148:0/1963531058 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f44fc06c530 0x7f44fc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:38.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.348+0000 7f4513fff640 1 -- 172.21.15.148:0/1963531058 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f45040ad510 con 0x7f451c0d10d0 2024-03-29T04:12:38.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.348+0000 7f451b7fe640 1 --2- 172.21.15.148:0/1963531058 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f44fc06c530 0x7f44fc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:38.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.349+0000 7f451b7fe640 1 --2- 172.21.15.148:0/1963531058 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f44fc06c530 0x7f44fc06e9f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f450c001580 tx=0x7f450c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:38.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.352+0000 7f4513fff640 1 -- 172.21.15.148:0/1963531058 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f450403f070 con 0x7f451c0d10d0 2024-03-29T04:12:38.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.566+0000 7f4521d0e640 1 -- 172.21.15.148:0/1963531058 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f451c074a20 con 0x7f451c0d10d0 2024-03-29T04:12:38.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.566+0000 7f4513fff640 1 -- 172.21.15.148:0/1963531058 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5190 (secure 0 0 0) 0x7f450404b350 con 0x7f451c0d10d0 2024-03-29T04:12:38.546 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:12:38.547 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2618617956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2618617956},{"type":"v1","addr":"172.21.15.157:6834","nonce":2618617956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5260,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1177855619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1177855619},{"type":"v1","addr":"172.21.15.157:6835","nonce":1177855619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5562,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3610463670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3610463670},{"type":"v1","addr":"172.21.15.148:6835","nonce":3610463670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:11:32.950391+0000","modified":"2024-03-29T04:11:53.076916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5583},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5583":{"gid":5583,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/473448013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":473448013},{"type":"v1","addr":"172.21.15.148:6837","nonce":473448013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5583,"qdb_cluster":[5583]},"id":2}]} 2024-03-29T04:12:38.547 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 19 2024-03-29T04:12:38.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.568+0000 7f4521d0e640 1 -- 172.21.15.148:0/1963531058 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f44fc06c530 msgr2=0x7f44fc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:38.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.568+0000 7f4521d0e640 1 --2- 172.21.15.148:0/1963531058 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f44fc06c530 0x7f44fc06e9f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f450c001580 tx=0x7f450c009040 comp rx=0 tx=0).stop 2024-03-29T04:12:38.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.568+0000 7f4521d0e640 1 -- 172.21.15.148:0/1963531058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f451c0d10d0 msgr2=0x7f451c155890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:38.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.568+0000 7f4521d0e640 1 --2- 172.21.15.148:0/1963531058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f451c0d10d0 0x7f451c155890 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f450402bf70 tx=0x7f45040046c0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.568+0000 7f4521d0e640 1 -- 172.21.15.148:0/1963531058 shutdown_connections 2024-03-29T04:12:38.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.568+0000 7f4521d0e640 1 --2- 172.21.15.148:0/1963531058 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f44fc06c530 0x7f44fc06e9f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.568+0000 7f4521d0e640 1 --2- 172.21.15.148:0/1963531058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f451c15e3e0 0x7f451c06c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.568+0000 7f4521d0e640 1 --2- 172.21.15.148:0/1963531058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f451c0d10d0 0x7f451c155890 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.568+0000 7f4521d0e640 1 --2- 172.21.15.148:0/1963531058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f451c0ba030 0x7f451c156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.568+0000 7f4521d0e640 1 -- 172.21.15.148:0/1963531058 >> 172.21.15.148:0/1963531058 conn(0x7f451c0b6210 msgr2=0x7f451c0b87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:38.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.569+0000 7f4521d0e640 1 -- 172.21.15.148:0/1963531058 shutdown_connections 2024-03-29T04:12:38.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.569+0000 7f4521d0e640 1 -- 172.21.15.148:0/1963531058 wait complete. 2024-03-29T04:12:38.560 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:12:38.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.682+0000 7fadbd9fb640 1 Processor -- start 2024-03-29T04:12:38.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.682+0000 7fadbd9fb640 1 -- start start 2024-03-29T04:12:38.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.683+0000 7fadbd9fb640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fadb815c000 0x7fadb8162920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:38.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.683+0000 7fadbd9fb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fadb80b5000 0x7fadb80cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:38.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.683+0000 7fadbd9fb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadb80ceb50 0x7fadb80cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:38.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.683+0000 7fadbd9fb640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fadb8162e60 con 0x7fadb815c000 2024-03-29T04:12:38.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.683+0000 7fadbd9fb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fadb815c3e0 con 0x7fadb80b5000 2024-03-29T04:12:38.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.683+0000 7fadbd9fb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fadb80d0610 con 0x7fadb80ceb50 2024-03-29T04:12:38.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.683+0000 7fadb6ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fadb815c000 0x7fadb8162920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:38.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.683+0000 7fadb6ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fadb815c000 0x7fadb8162920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60670/0 (socket says 172.21.15.148:60670) 2024-03-29T04:12:38.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.684+0000 7fadb6ffd640 1 -- 172.21.15.148:0/1755902094 learned_addr learned my addr 172.21.15.148:0/1755902094 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:38.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.684+0000 7fadb77fe640 1 --2- 172.21.15.148:0/1755902094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadb80ceb50 0x7fadb80cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:38.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.684+0000 7fadb67fc640 1 --2- 172.21.15.148:0/1755902094 >> v2:172.21.15.157:3300/0 conn(0x7fadb80b5000 0x7fadb80cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:38.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.684+0000 7fadb6ffd640 1 -- 172.21.15.148:0/1755902094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadb80ceb50 msgr2=0x7fadb80cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:38.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.684+0000 7fadb6ffd640 1 --2- 172.21.15.148:0/1755902094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadb80ceb50 0x7fadb80cefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.684+0000 7fadb6ffd640 1 -- 172.21.15.148:0/1755902094 >> v2:172.21.15.157:3300/0 conn(0x7fadb80b5000 msgr2=0x7fadb80cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:38.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.684+0000 7fadb6ffd640 1 --2- 172.21.15.148:0/1755902094 >> v2:172.21.15.157:3300/0 conn(0x7fadb80b5000 0x7fadb80cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.684+0000 7fadb6ffd640 1 -- 172.21.15.148:0/1755902094 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb80d08f0 con 0x7fadb815c000 2024-03-29T04:12:38.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.685+0000 7fadb6ffd640 1 --2- 172.21.15.148:0/1755902094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadb815c000 0x7fadb8162920 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fadb80c91a0 tx=0x7fada802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd90a7a6119e1d64 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:38.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.685+0000 7fadb5ffb640 1 -- 172.21.15.148:0/1755902094 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada8035020 con 0x7fadb815c000 2024-03-29T04:12:38.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.685+0000 7fadb5ffb640 1 -- 172.21.15.148:0/1755902094 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fada8031870 con 0x7fadb815c000 2024-03-29T04:12:38.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.685+0000 7fadb5ffb640 1 -- 172.21.15.148:0/1755902094 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada8031c40 con 0x7fadb815c000 2024-03-29T04:12:38.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.685+0000 7fadbd9fb640 1 -- 172.21.15.148:0/1755902094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadb815c000 msgr2=0x7fadb8162920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:38.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.685+0000 7fadbd9fb640 1 --2- 172.21.15.148:0/1755902094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadb815c000 0x7fadb8162920 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fadb80c91a0 tx=0x7fada802b910 comp rx=0 tx=0).stop 2024-03-29T04:12:38.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.685+0000 7fadbd9fb640 1 -- 172.21.15.148:0/1755902094 shutdown_connections 2024-03-29T04:12:38.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.685+0000 7fadbd9fb640 1 --2- 172.21.15.148:0/1755902094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadb80ceb50 0x7fadb80cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.685+0000 7fadbd9fb640 1 --2- 172.21.15.148:0/1755902094 >> v2:172.21.15.157:3300/0 conn(0x7fadb80b5000 0x7fadb80cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.685+0000 7fadbd9fb640 1 --2- 172.21.15.148:0/1755902094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadb815c000 0x7fadb8162920 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.685+0000 7fadbd9fb640 1 -- 172.21.15.148:0/1755902094 >> 172.21.15.148:0/1755902094 conn(0x7fadb80b3c70 msgr2=0x7fadb80b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:38.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.685+0000 7fadbd9fb640 1 -- 172.21.15.148:0/1755902094 shutdown_connections 2024-03-29T04:12:38.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.685+0000 7fadbd9fb640 1 -- 172.21.15.148:0/1755902094 wait complete. 2024-03-29T04:12:38.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.686+0000 7fadbd9fb640 1 Processor -- start 2024-03-29T04:12:38.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.687+0000 7fadbd9fb640 1 -- start start 2024-03-29T04:12:38.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.687+0000 7fadbd9fb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadb80b5000 0x7fadb806fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:38.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.687+0000 7fadbd9fb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadb80ceb50 0x7fadb815d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:38.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.687+0000 7fadbd9fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadb815c000 0x7fadb81f4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:38.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.687+0000 7fadbd9fb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fadb80d0250 con 0x7fadb80ceb50 2024-03-29T04:12:38.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.687+0000 7fadbd9fb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fadb8162400 con 0x7fadb815c000 2024-03-29T04:12:38.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.687+0000 7fadbd9fb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fadb80d03d0 con 0x7fadb80b5000 2024-03-29T04:12:38.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.687+0000 7fadb67fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadb80ceb50 0x7fadb815d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:38.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.688+0000 7fadb67fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadb80ceb50 0x7fadb815d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60684/0 (socket says 172.21.15.148:60684) 2024-03-29T04:12:38.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.688+0000 7fadb67fc640 1 -- 172.21.15.148:0/1935589314 learned_addr learned my addr 172.21.15.148:0/1935589314 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:38.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.688+0000 7fadb6ffd640 1 --2- 172.21.15.148:0/1935589314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadb80b5000 0x7fadb806fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:38.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.688+0000 7fadb77fe640 1 --2- 172.21.15.148:0/1935589314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadb815c000 0x7fadb81f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:38.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.688+0000 7fadb67fc640 1 -- 172.21.15.148:0/1935589314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadb80b5000 msgr2=0x7fadb806fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:38.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.688+0000 7fadb67fc640 1 --2- 172.21.15.148:0/1935589314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadb80b5000 0x7fadb806fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.688+0000 7fadb67fc640 1 -- 172.21.15.148:0/1935589314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadb815c000 msgr2=0x7fadb81f4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:38.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.688+0000 7fadb67fc640 1 --2- 172.21.15.148:0/1935589314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadb815c000 0x7fadb81f4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.688+0000 7fadb67fc640 1 -- 172.21.15.148:0/1935589314 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb81f7620 con 0x7fadb80ceb50 2024-03-29T04:12:38.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.688+0000 7fadb77fe640 1 --2- 172.21.15.148:0/1935589314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadb815c000 0x7fadb81f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:38.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.688+0000 7fadb67fc640 1 --2- 172.21.15.148:0/1935589314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadb80ceb50 0x7fadb815d820 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fadac00d800 tx=0x7fadac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:38.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.688+0000 7fad9ffff640 1 -- 172.21.15.148:0/1935589314 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac018020 con 0x7fadb80ceb50 2024-03-29T04:12:38.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.688+0000 7fad9ffff640 1 -- 172.21.15.148:0/1935589314 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadac0096a0 con 0x7fadb80ceb50 2024-03-29T04:12:38.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.688+0000 7fadbd9fb640 1 -- 172.21.15.148:0/1935589314 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb81f7900 con 0x7fadb80ceb50 2024-03-29T04:12:38.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.688+0000 7fad9ffff640 1 -- 172.21.15.148:0/1935589314 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac009a70 con 0x7fadb80ceb50 2024-03-29T04:12:38.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.688+0000 7fadbd9fb640 1 -- 172.21.15.148:0/1935589314 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb81f7ec0 con 0x7fadb80ceb50 2024-03-29T04:12:38.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.689+0000 7fadbd9fb640 1 -- 172.21.15.148:0/1935589314 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadb80687e0 con 0x7fadb80ceb50 2024-03-29T04:12:38.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.690+0000 7fad9ffff640 1 -- 172.21.15.148:0/1935589314 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fadac01b020 con 0x7fadb80ceb50 2024-03-29T04:12:38.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.690+0000 7fad9ffff640 1 --2- 172.21.15.148:0/1935589314 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fada406c530 0x7fada406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:38.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.690+0000 7fad9ffff640 1 -- 172.21.15.148:0/1935589314 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7fadac08ba40 con 0x7fadb80ceb50 2024-03-29T04:12:38.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.690+0000 7fadb6ffd640 1 --2- 172.21.15.148:0/1935589314 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fada406c530 0x7fada406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:38.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.691+0000 7fadb6ffd640 1 --2- 172.21.15.148:0/1935589314 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fada406c530 0x7fada406e9f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fada8036000 tx=0x7fada803c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:38.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.694+0000 7fad9ffff640 1 -- 172.21.15.148:0/1935589314 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fadac022030 con 0x7fadb80ceb50 2024-03-29T04:12:38.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.871+0000 7fadbd9fb640 1 -- 172.21.15.148:0/1935589314 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fadb80d1610 con 0x7fadb80ceb50 2024-03-29T04:12:38.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.872+0000 7fad9ffff640 1 -- 172.21.15.148:0/1935589314 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+17003 (secure 0 0 0) 0x7fadac059eb0 con 0x7fadb80ceb50 2024-03-29T04:12:38.852 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:12:38.852 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":41,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:11:42.632417+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:11:30.794790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T04:11:31.907100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:12:38.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.874+0000 7fadbd9fb640 1 -- 172.21.15.148:0/1935589314 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fada406c530 msgr2=0x7fada406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:38.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.874+0000 7fadbd9fb640 1 --2- 172.21.15.148:0/1935589314 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fada406c530 0x7fada406e9f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fada8036000 tx=0x7fada803c040 comp rx=0 tx=0).stop 2024-03-29T04:12:38.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.874+0000 7fadbd9fb640 1 -- 172.21.15.148:0/1935589314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadb80ceb50 msgr2=0x7fadb815d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:38.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.874+0000 7fadbd9fb640 1 --2- 172.21.15.148:0/1935589314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadb80ceb50 0x7fadb815d820 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fadac00d800 tx=0x7fadac00dcc0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.874+0000 7fadbd9fb640 1 -- 172.21.15.148:0/1935589314 shutdown_connections 2024-03-29T04:12:38.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.874+0000 7fadbd9fb640 1 --2- 172.21.15.148:0/1935589314 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fada406c530 0x7fada406e9f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.874+0000 7fadbd9fb640 1 --2- 172.21.15.148:0/1935589314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadb815c000 0x7fadb81f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.874+0000 7fadbd9fb640 1 --2- 172.21.15.148:0/1935589314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadb80ceb50 0x7fadb815d820 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.874+0000 7fadbd9fb640 1 --2- 172.21.15.148:0/1935589314 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fadb80b5000 0x7fadb806fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.874+0000 7fadbd9fb640 1 -- 172.21.15.148:0/1935589314 >> 172.21.15.148:0/1935589314 conn(0x7fadb80b3c70 msgr2=0x7fadb80d2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:38.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.874+0000 7fadbd9fb640 1 -- 172.21.15.148:0/1935589314 shutdown_connections 2024-03-29T04:12:38.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.874+0000 7fadbd9fb640 1 -- 172.21.15.148:0/1935589314 wait complete. 2024-03-29T04:12:38.865 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-29T04:12:38.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.989+0000 7f56821bd640 1 Processor -- start 2024-03-29T04:12:38.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.989+0000 7f56821bd640 1 -- start start 2024-03-29T04:12:38.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.989+0000 7f56821bd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f567c15e3e0 0x7f567c0d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:38.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.989+0000 7f56821bd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f567c0ba030 0x7f567c0d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:38.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.989+0000 7f56821bd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f567c0d10d0 0x7f567c0d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:38.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.989+0000 7f56821bd640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f567c0d3840 con 0x7f567c0ba030 2024-03-29T04:12:38.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.989+0000 7f56821bd640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f567c15e7c0 con 0x7f567c15e3e0 2024-03-29T04:12:38.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.989+0000 7f56821bd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f567c0d2c30 con 0x7f567c0d10d0 2024-03-29T04:12:38.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.990+0000 7f567affd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f567c0ba030 0x7f567c0d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:38.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.990+0000 7f567affd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f567c0ba030 0x7f567c0d2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60692/0 (socket says 172.21.15.148:60692) 2024-03-29T04:12:38.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.990+0000 7f567affd640 1 -- 172.21.15.148:0/426107344 learned_addr learned my addr 172.21.15.148:0/426107344 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:38.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.990+0000 7f567b7fe640 1 --2- 172.21.15.148:0/426107344 >> v2:172.21.15.157:3300/0 conn(0x7f567c15e3e0 0x7f567c0d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:38.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.990+0000 7f567bfff640 1 --2- 172.21.15.148:0/426107344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f567c0d10d0 0x7f567c0d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:38.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.990+0000 7f567affd640 1 -- 172.21.15.148:0/426107344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f567c0d10d0 msgr2=0x7f567c0d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:38.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.990+0000 7f567affd640 1 --2- 172.21.15.148:0/426107344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f567c0d10d0 0x7f567c0d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.990+0000 7f567affd640 1 -- 172.21.15.148:0/426107344 >> v2:172.21.15.157:3300/0 conn(0x7f567c15e3e0 msgr2=0x7f567c0d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:38.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.990+0000 7f567affd640 1 --2- 172.21.15.148:0/426107344 >> v2:172.21.15.157:3300/0 conn(0x7f567c15e3e0 0x7f567c0d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.990+0000 7f567affd640 1 -- 172.21.15.148:0/426107344 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f567c0d2f10 con 0x7f567c0ba030 2024-03-29T04:12:38.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.991+0000 7f567affd640 1 --2- 172.21.15.148:0/426107344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f567c0ba030 0x7f567c0d2330 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f567c0b4eb0 tx=0x7f566402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0315a80cfc9c51 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:38.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.991+0000 7f567a7fc640 1 -- 172.21.15.148:0/426107344 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664034070 con 0x7f567c0ba030 2024-03-29T04:12:38.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.991+0000 7f567a7fc640 1 -- 172.21.15.148:0/426107344 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5664030860 con 0x7f567c0ba030 2024-03-29T04:12:38.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.991+0000 7f567a7fc640 1 -- 172.21.15.148:0/426107344 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664030c30 con 0x7f567c0ba030 2024-03-29T04:12:38.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.991+0000 7f56821bd640 1 -- 172.21.15.148:0/426107344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f567c0ba030 msgr2=0x7f567c0d2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:38.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.991+0000 7f56821bd640 1 --2- 172.21.15.148:0/426107344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f567c0ba030 0x7f567c0d2330 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f567c0b4eb0 tx=0x7f566402b9a0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.991+0000 7f56821bd640 1 -- 172.21.15.148:0/426107344 shutdown_connections 2024-03-29T04:12:38.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.991+0000 7f56821bd640 1 --2- 172.21.15.148:0/426107344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f567c0ba030 0x7f567c0d2330 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.991+0000 7f56821bd640 1 --2- 172.21.15.148:0/426107344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f567c0d10d0 0x7f567c0d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.992+0000 7f56821bd640 1 --2- 172.21.15.148:0/426107344 >> v2:172.21.15.157:3300/0 conn(0x7f567c15e3e0 0x7f567c0d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.992+0000 7f56821bd640 1 -- 172.21.15.148:0/426107344 >> 172.21.15.148:0/426107344 conn(0x7f567c0b6210 msgr2=0x7f567c0b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:38.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.992+0000 7f56821bd640 1 -- 172.21.15.148:0/426107344 shutdown_connections 2024-03-29T04:12:38.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.992+0000 7f56821bd640 1 -- 172.21.15.148:0/426107344 wait complete. 2024-03-29T04:12:38.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.993+0000 7f56821bd640 1 Processor -- start 2024-03-29T04:12:38.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.993+0000 7f56821bd640 1 -- start start 2024-03-29T04:12:38.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.994+0000 7f56821bd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f567c0ba030 0x7f567c15d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:38.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.994+0000 7f56821bd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f567c0d10d0 0x7f567c158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:38.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.994+0000 7f56821bd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f567c15e3e0 0x7f567c0ac430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:38.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.994+0000 7f56821bd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f567c0d2870 con 0x7f567c15e3e0 2024-03-29T04:12:38.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.994+0000 7f56821bd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f567c0d34d0 con 0x7f567c0d10d0 2024-03-29T04:12:38.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.994+0000 7f56821bd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f567c0d29f0 con 0x7f567c0ba030 2024-03-29T04:12:38.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.994+0000 7f567bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f567c15e3e0 0x7f567c0ac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:38.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.994+0000 7f567bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f567c15e3e0 0x7f567c0ac430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60694/0 (socket says 172.21.15.148:60694) 2024-03-29T04:12:38.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.994+0000 7f567bfff640 1 -- 172.21.15.148:0/423450576 learned_addr learned my addr 172.21.15.148:0/423450576 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:38.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.994+0000 7f567affd640 1 --2- 172.21.15.148:0/423450576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f567c0d10d0 0x7f567c158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:38.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.994+0000 7f567b7fe640 1 --2- 172.21.15.148:0/423450576 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f567c0ba030 0x7f567c15d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:38.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.995+0000 7f567bfff640 1 -- 172.21.15.148:0/423450576 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f567c0ba030 msgr2=0x7f567c15d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:38.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.995+0000 7f567bfff640 1 --2- 172.21.15.148:0/423450576 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f567c0ba030 0x7f567c15d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.995+0000 7f567bfff640 1 -- 172.21.15.148:0/423450576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f567c0d10d0 msgr2=0x7f567c158db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:38.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.995+0000 7f567bfff640 1 --2- 172.21.15.148:0/423450576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f567c0d10d0 0x7f567c158db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:38.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.995+0000 7f567bfff640 1 -- 172.21.15.148:0/423450576 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f567c15d930 con 0x7f567c15e3e0 2024-03-29T04:12:38.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.995+0000 7f567bfff640 1 --2- 172.21.15.148:0/423450576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f567c15e3e0 0x7f567c0ac430 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f566c00c570 tx=0x7f566c00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:38.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.995+0000 7f5673fff640 1 -- 172.21.15.148:0/423450576 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c015070 con 0x7f567c15e3e0 2024-03-29T04:12:38.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.995+0000 7f5673fff640 1 -- 172.21.15.148:0/423450576 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f566c004220 con 0x7f567c15e3e0 2024-03-29T04:12:38.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.995+0000 7f56821bd640 1 -- 172.21.15.148:0/423450576 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f567c0aaff0 con 0x7f567c15e3e0 2024-03-29T04:12:38.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.995+0000 7f5673fff640 1 -- 172.21.15.148:0/423450576 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c003610 con 0x7f567c15e3e0 2024-03-29T04:12:38.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.995+0000 7f56821bd640 1 -- 172.21.15.148:0/423450576 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f567c0ab5b0 con 0x7f567c15e3e0 2024-03-29T04:12:38.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.996+0000 7f56821bd640 1 -- 172.21.15.148:0/423450576 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f567c0687e0 con 0x7f567c15e3e0 2024-03-29T04:12:38.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.997+0000 7f5673fff640 1 -- 172.21.15.148:0/423450576 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f566c0037b0 con 0x7f567c15e3e0 2024-03-29T04:12:38.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.997+0000 7f5673fff640 1 --2- 172.21.15.148:0/423450576 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f565c06c530 0x7f565c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:38.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.997+0000 7f5673fff640 1 -- 172.21.15.148:0/423450576 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f566c08b590 con 0x7f567c15e3e0 2024-03-29T04:12:38.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.997+0000 7f567b7fe640 1 --2- 172.21.15.148:0/423450576 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f565c06c530 0x7f565c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:38.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:38.997+0000 7f567b7fe640 1 --2- 172.21.15.148:0/423450576 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f565c06c530 0x7f565c06e9f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5674003aa0 tx=0x7f5674009830 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:38.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.001+0000 7f5673fff640 1 -- 172.21.15.148:0/423450576 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f566c0593b0 con 0x7f567c15e3e0 2024-03-29T04:12:39.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.221+0000 7f56821bd640 1 -- 172.21.15.148:0/423450576 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f567c0cf2c0 con 0x7f567c15e3e0 2024-03-29T04:12:39.202 INFO:tasks.ceph.mon.a.smithi148.stderr:2024-03-29T04:12:39.222+0000 7fa221fb2640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T04:12:39.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.223+0000 7f5673fff640 1 -- 172.21.15.148:0/423450576 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v20) ==== 161+0+0 (secure 0 0 0) 0x7f566c05d6b0 con 0x7f567c15e3e0 2024-03-29T04:12:39.203 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-29T04:12:39.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.226+0000 7f56821bd640 1 -- 172.21.15.148:0/423450576 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f565c06c530 msgr2=0x7f565c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:39.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.226+0000 7f56821bd640 1 --2- 172.21.15.148:0/423450576 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f565c06c530 0x7f565c06e9f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5674003aa0 tx=0x7f5674009830 comp rx=0 tx=0).stop 2024-03-29T04:12:39.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.227+0000 7f56821bd640 1 -- 172.21.15.148:0/423450576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f567c15e3e0 msgr2=0x7f567c0ac430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:39.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.227+0000 7f56821bd640 1 --2- 172.21.15.148:0/423450576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f567c15e3e0 0x7f567c0ac430 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f566c00c570 tx=0x7f566c00ca30 comp rx=0 tx=0).stop 2024-03-29T04:12:39.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.227+0000 7f56821bd640 1 -- 172.21.15.148:0/423450576 shutdown_connections 2024-03-29T04:12:39.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.227+0000 7f56821bd640 1 --2- 172.21.15.148:0/423450576 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f565c06c530 0x7f565c06e9f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:39.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.227+0000 7f56821bd640 1 --2- 172.21.15.148:0/423450576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f567c15e3e0 0x7f567c0ac430 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:39.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.227+0000 7f56821bd640 1 --2- 172.21.15.148:0/423450576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f567c0d10d0 0x7f567c158db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:39.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.227+0000 7f56821bd640 1 --2- 172.21.15.148:0/423450576 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f567c0ba030 0x7f567c15d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:39.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.227+0000 7f56821bd640 1 -- 172.21.15.148:0/423450576 >> 172.21.15.148:0/423450576 conn(0x7f567c0b6210 msgr2=0x7f567c0cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:39.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.228+0000 7f56821bd640 1 -- 172.21.15.148:0/423450576 shutdown_connections 2024-03-29T04:12:39.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.228+0000 7f56821bd640 1 -- 172.21.15.148:0/423450576 wait complete. 2024-03-29T04:12:39.219 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-29T04:12:39.243 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2024-03-29T04:12:39.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.343+0000 7f596bfba640 1 Processor -- start 2024-03-29T04:12:39.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.343+0000 7f596bfba640 1 -- start start 2024-03-29T04:12:39.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.344+0000 7f596bfba640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f59640ca100 0x7f5964162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:39.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.344+0000 7f596bfba640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f59640cf290 0x7f5964162690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:39.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.344+0000 7f596bfba640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5964162ec0 con 0x7f596415e3c0 2024-03-29T04:12:39.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.344+0000 7f596bfba640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f59640cee00 con 0x7f59640ca100 2024-03-29T04:12:39.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.344+0000 7f596bfba640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f59640ce4d0 con 0x7f59640cf290 2024-03-29T04:12:39.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.344+0000 7f5969555640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f59640ca100 0x7f5964162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:39.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.344+0000 7f5969555640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f59640ca100 0x7f5964162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60710/0 (socket says 172.21.15.148:60710) 2024-03-29T04:12:39.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.344+0000 7f5969555640 1 -- 172.21.15.148:0/3503244257 learned_addr learned my addr 172.21.15.148:0/3503244257 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:39.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.344+0000 7f596a557640 1 --2- 172.21.15.148:0/3503244257 >> v2:172.21.15.157:3300/0 conn(0x7f59640cf290 0x7f5964162690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:39.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.344+0000 7f5969555640 1 -- 172.21.15.148:0/3503244257 >> v2:172.21.15.157:3300/0 conn(0x7f59640cf290 msgr2=0x7f5964162690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:39.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.344+0000 7f5969555640 1 --2- 172.21.15.148:0/3503244257 >> v2:172.21.15.157:3300/0 conn(0x7f59640cf290 0x7f5964162690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:39.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.344+0000 7f5969555640 1 -- 172.21.15.148:0/3503244257 >> v1:172.21.15.157:6789/0 conn(0x7f596415e3c0 legacy=0x7f59640c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:39.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.344+0000 7f5969555640 1 -- 172.21.15.148:0/3503244257 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59640ce7b0 con 0x7f59640ca100 2024-03-29T04:12:39.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.344+0000 7f596a557640 1 --2- 172.21.15.148:0/3503244257 >> v2:172.21.15.157:3300/0 conn(0x7f59640cf290 0x7f5964162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:39.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.345+0000 7f5969555640 1 --2- 172.21.15.148:0/3503244257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59640ca100 0x7f5964162150 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f59640cef80 tx=0x7f595402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1dd9942ef7eac11 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:39.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.345+0000 7f5968d54640 1 -- 172.21.15.148:0/3503244257 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5954034070 con 0x7f59640ca100 2024-03-29T04:12:39.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.345+0000 7f5968d54640 1 -- 172.21.15.148:0/3503244257 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5954030860 con 0x7f59640ca100 2024-03-29T04:12:39.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.345+0000 7f5968d54640 1 -- 172.21.15.148:0/3503244257 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5954030c30 con 0x7f59640ca100 2024-03-29T04:12:39.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.346+0000 7f596bfba640 1 -- 172.21.15.148:0/3503244257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59640ca100 msgr2=0x7f5964162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:39.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.346+0000 7f596bfba640 1 --2- 172.21.15.148:0/3503244257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59640ca100 0x7f5964162150 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f59640cef80 tx=0x7f595402b9a0 comp rx=0 tx=0).stop 2024-03-29T04:12:39.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.346+0000 7f596bfba640 1 -- 172.21.15.148:0/3503244257 shutdown_connections 2024-03-29T04:12:39.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.346+0000 7f596bfba640 1 --2- 172.21.15.148:0/3503244257 >> v2:172.21.15.157:3300/0 conn(0x7f59640cf290 0x7f5964162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:39.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.346+0000 7f596bfba640 1 --2- 172.21.15.148:0/3503244257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59640ca100 0x7f5964162150 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:39.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.346+0000 7f596bfba640 1 -- 172.21.15.148:0/3503244257 >> 172.21.15.148:0/3503244257 conn(0x7f59640cd160 msgr2=0x7f59640cd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:39.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.346+0000 7f596bfba640 1 -- 172.21.15.148:0/3503244257 shutdown_connections 2024-03-29T04:12:39.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.346+0000 7f596bfba640 1 -- 172.21.15.148:0/3503244257 wait complete. 2024-03-29T04:12:39.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.347+0000 7f596bfba640 1 Processor -- start 2024-03-29T04:12:39.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.347+0000 7f596bfba640 1 -- start start 2024-03-29T04:12:39.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.348+0000 7f596bfba640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59640ca100 0x7f5964160570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:39.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.348+0000 7f596bfba640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59640cf290 0x7f5964151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:39.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.348+0000 7f596bfba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f596415e3c0 0x7f5964153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:39.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.348+0000 7f596bfba640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f596415e7a0 con 0x7f59640cf290 2024-03-29T04:12:39.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.348+0000 7f596bfba640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5964162bd0 con 0x7f596415e3c0 2024-03-29T04:12:39.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.348+0000 7f596bfba640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f596415e920 con 0x7f59640ca100 2024-03-29T04:12:39.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.348+0000 7f5969555640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59640cf290 0x7f5964151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:39.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.348+0000 7f5969555640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59640cf290 0x7f5964151db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60726/0 (socket says 172.21.15.148:60726) 2024-03-29T04:12:39.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.348+0000 7f5969555640 1 -- 172.21.15.148:0/1397729287 learned_addr learned my addr 172.21.15.148:0/1397729287 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:39.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.348+0000 7f596a557640 1 --2- 172.21.15.148:0/1397729287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f596415e3c0 0x7f5964153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:39.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.348+0000 7f5969d56640 1 --2- 172.21.15.148:0/1397729287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59640ca100 0x7f5964160570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:39.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.348+0000 7f5969555640 1 -- 172.21.15.148:0/1397729287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59640ca100 msgr2=0x7f5964160570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:39.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.348+0000 7f5969555640 1 --2- 172.21.15.148:0/1397729287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59640ca100 0x7f5964160570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:39.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.348+0000 7f5969555640 1 -- 172.21.15.148:0/1397729287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f596415e3c0 msgr2=0x7f5964153370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:39.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.348+0000 7f5969555640 1 --2- 172.21.15.148:0/1397729287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f596415e3c0 0x7f5964153370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:39.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.348+0000 7f5969555640 1 -- 172.21.15.148:0/1397729287 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59640b3670 con 0x7f59640cf290 2024-03-29T04:12:39.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.348+0000 7f5969d56640 1 --2- 172.21.15.148:0/1397729287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59640ca100 0x7f5964160570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:12:39.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.348+0000 7f596a557640 1 --2- 172.21.15.148:0/1397729287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f596415e3c0 0x7f5964153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:12:39.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.348+0000 7f5969555640 1 --2- 172.21.15.148:0/1397729287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59640cf290 0x7f5964151db0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f595402b9e0 tx=0x7f59540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:39.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.348+0000 7f595a7fc640 1 -- 172.21.15.148:0/1397729287 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5954034040 con 0x7f59640cf290 2024-03-29T04:12:39.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.349+0000 7f595a7fc640 1 -- 172.21.15.148:0/1397729287 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5954039d90 con 0x7f59640cf290 2024-03-29T04:12:39.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.349+0000 7f596bfba640 1 -- 172.21.15.148:0/1397729287 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59640b3950 con 0x7f59640cf290 2024-03-29T04:12:39.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.349+0000 7f595a7fc640 1 -- 172.21.15.148:0/1397729287 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5954043610 con 0x7f59640cf290 2024-03-29T04:12:39.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.349+0000 7f596bfba640 1 -- 172.21.15.148:0/1397729287 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5964160c60 con 0x7f59640cf290 2024-03-29T04:12:39.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.349+0000 7f596bfba640 1 -- 172.21.15.148:0/1397729287 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5964068850 con 0x7f59640cf290 2024-03-29T04:12:39.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.350+0000 7f595a7fc640 1 -- 172.21.15.148:0/1397729287 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5954037070 con 0x7f59640cf290 2024-03-29T04:12:39.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.350+0000 7f595a7fc640 1 --2- 172.21.15.148:0/1397729287 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f594806c530 0x7f594806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:39.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.350+0000 7f595a7fc640 1 -- 172.21.15.148:0/1397729287 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6216+0+0 (secure 0 0 0) 0x7f59540ada60 con 0x7f59640cf290 2024-03-29T04:12:39.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.350+0000 7f5969d56640 1 --2- 172.21.15.148:0/1397729287 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f594806c530 0x7f594806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:39.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.351+0000 7f5969d56640 1 --2- 172.21.15.148:0/1397729287 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f594806c530 0x7f594806e9f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5960001d70 tx=0x7f5960009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:39.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.355+0000 7f595a7fc640 1 -- 172.21.15.148:0/1397729287 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f595403e070 con 0x7f59640cf290 2024-03-29T04:12:39.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:39.568+0000 7f596bfba640 1 -- 172.21.15.148:0/1397729287 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5964067c90 con 0x7f59640cf290 2024-03-29T04:12:40.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.227+0000 7f595a7fc640 1 -- 172.21.15.148:0/1397729287 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7f595407be80 con 0x7f59640cf290 2024-03-29T04:12:40.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.230+0000 7f596bfba640 1 -- 172.21.15.148:0/1397729287 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f594806c530 msgr2=0x7f594806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:40.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.230+0000 7f596bfba640 1 --2- 172.21.15.148:0/1397729287 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f594806c530 0x7f594806e9f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5960001d70 tx=0x7f5960009040 comp rx=0 tx=0).stop 2024-03-29T04:12:40.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.230+0000 7f596bfba640 1 -- 172.21.15.148:0/1397729287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59640cf290 msgr2=0x7f5964151db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:40.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.230+0000 7f596bfba640 1 --2- 172.21.15.148:0/1397729287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59640cf290 0x7f5964151db0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f595402b9e0 tx=0x7f59540046c0 comp rx=0 tx=0).stop 2024-03-29T04:12:40.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.230+0000 7f596bfba640 1 -- 172.21.15.148:0/1397729287 shutdown_connections 2024-03-29T04:12:40.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.230+0000 7f596bfba640 1 --2- 172.21.15.148:0/1397729287 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f594806c530 0x7f594806e9f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:40.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.230+0000 7f596bfba640 1 --2- 172.21.15.148:0/1397729287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f596415e3c0 0x7f5964153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:40.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.230+0000 7f596bfba640 1 --2- 172.21.15.148:0/1397729287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59640cf290 0x7f5964151db0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:40.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.230+0000 7f596bfba640 1 --2- 172.21.15.148:0/1397729287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59640ca100 0x7f5964160570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:40.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.230+0000 7f596bfba640 1 -- 172.21.15.148:0/1397729287 >> 172.21.15.148:0/1397729287 conn(0x7f59640cd160 msgr2=0x7f59640cd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:40.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.230+0000 7f596bfba640 1 -- 172.21.15.148:0/1397729287 shutdown_connections 2024-03-29T04:12:40.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.230+0000 7f596bfba640 1 -- 172.21.15.148:0/1397729287 wait complete. 2024-03-29T04:12:40.223 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-29T04:12:40.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.394+0000 7fb747663640 1 Processor -- start 2024-03-29T04:12:40.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.394+0000 7fb747663640 1 -- start start 2024-03-29T04:12:40.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.395+0000 7fb747663640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb740161ce0 0x7fb7401694e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:40.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.395+0000 7fb747663640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7400d2a50 con 0x7fb7400d3840 2024-03-29T04:12:40.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.395+0000 7fb747663640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7400d2d30 con 0x7fb7400b36e0 2024-03-29T04:12:40.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.395+0000 7fb747663640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb7400d3010 con 0x7fb740161ce0 2024-03-29T04:12:40.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.395+0000 7fb7453ff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb7400d3840 0x7fb7400d2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39846/0 (socket says 172.21.15.148:39846) 2024-03-29T04:12:40.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.395+0000 7fb7453ff640 1 -- 172.21.15.148:0/956057086 learned_addr learned my addr 172.21.15.148:0/956057086 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:40.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.395+0000 7fb73ffff640 1 -- 172.21.15.148:0/956057086 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 809211496 0 0) 0x7fb7400d2a50 con 0x7fb7400d3840 2024-03-29T04:12:40.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.395+0000 7fb745c00640 1 --2- 172.21.15.148:0/956057086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb740161ce0 0x7fb7401694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:40.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.395+0000 7fb73ffff640 1 -- 172.21.15.148:0/956057086 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb728003520 con 0x7fb7400d3840 2024-03-29T04:12:40.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.395+0000 7fb73ffff640 1 -- 172.21.15.148:0/956057086 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2796811523 0 0) 0x7fb728003520 con 0x7fb7400d3840 2024-03-29T04:12:40.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.396+0000 7fb73ffff640 1 -- 172.21.15.148:0/956057086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb740161ce0 msgr2=0x7fb7401694e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:40.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.396+0000 7fb73ffff640 1 --2- 172.21.15.148:0/956057086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb740161ce0 0x7fb7401694e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:40.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.396+0000 7fb73ffff640 1 -- 172.21.15.148:0/956057086 >> v1:172.21.15.157:6789/0 conn(0x7fb7400b36e0 legacy=0x7fb7401615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:40.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.396+0000 7fb73ffff640 1 -- 172.21.15.148:0/956057086 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb74006b7c0 con 0x7fb7400d3840 2024-03-29T04:12:40.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.396+0000 7fb73ffff640 1 -- 172.21.15.148:0/956057086 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7fb7300019f0 con 0x7fb7400d3840 2024-03-29T04:12:40.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.396+0000 7fb73ffff640 1 -- 172.21.15.148:0/956057086 >> v1:172.21.15.148:6789/0 conn(0x7fb7400d3840 legacy=0x7fb7400d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:40.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.396+0000 7fb73ffff640 1 --2- 172.21.15.148:0/956057086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb740161ce0 0x7fb7280039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:40.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.396+0000 7fb73ffff640 1 --2- 172.21.15.148:0/956057086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb728003f60 0x7fb728004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:40.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.396+0000 7fb73ffff640 1 --2- 172.21.15.148:0/956057086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb728004d40 0x7fb7280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:40.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.396+0000 7fb73ffff640 1 -- 172.21.15.148:0/956057086 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb7400d3010 con 0x7fb728003f60 2024-03-29T04:12:40.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.396+0000 7fb73ffff640 1 -- 172.21.15.148:0/956057086 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb72800baa0 con 0x7fb728004d40 2024-03-29T04:12:40.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.396+0000 7fb73ffff640 1 -- 172.21.15.148:0/956057086 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb72800bca0 con 0x7fb740161ce0 2024-03-29T04:12:40.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.396+0000 7fb744bfe640 1 --2- 172.21.15.148:0/956057086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb728003f60 0x7fb728004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:40.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.396+0000 7fb7453ff640 1 --2- 172.21.15.148:0/956057086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb740161ce0 0x7fb7280039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:40.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.396+0000 7fb745c00640 1 --2- 172.21.15.148:0/956057086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb728004d40 0x7fb7280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:40.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.396+0000 7fb744bfe640 1 -- 172.21.15.148:0/956057086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb740161ce0 msgr2=0x7fb7280039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:40.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.396+0000 7fb744bfe640 1 --2- 172.21.15.148:0/956057086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb740161ce0 0x7fb7280039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:40.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.396+0000 7fb744bfe640 1 -- 172.21.15.148:0/956057086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb728004d40 msgr2=0x7fb7280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:40.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.396+0000 7fb744bfe640 1 --2- 172.21.15.148:0/956057086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb728004d40 0x7fb7280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:40.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.396+0000 7fb744bfe640 1 -- 172.21.15.148:0/956057086 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb728003520 con 0x7fb728003f60 2024-03-29T04:12:40.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.397+0000 7fb744bfe640 1 --2- 172.21.15.148:0/956057086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb728003f60 0x7fb728004800 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb734001750 tx=0x7fb728005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21fb2049d41d97e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:40.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.397+0000 7fb73ffff640 1 -- 172.21.15.148:0/956057086 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb734001370 con 0x7fb728003f60 2024-03-29T04:12:40.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.397+0000 7fb73ffff640 1 -- 172.21.15.148:0/956057086 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7340052d0 con 0x7fb728003f60 2024-03-29T04:12:40.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.397+0000 7fb73ffff640 1 -- 172.21.15.148:0/956057086 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb734003830 con 0x7fb728003f60 2024-03-29T04:12:40.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.398+0000 7fb747663640 1 -- 172.21.15.148:0/956057086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb728003f60 msgr2=0x7fb728004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:40.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.398+0000 7fb747663640 1 --2- 172.21.15.148:0/956057086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb728003f60 0x7fb728004800 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb734001750 tx=0x7fb728005830 comp rx=0 tx=0).stop 2024-03-29T04:12:40.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.398+0000 7fb745c00640 1 -- 172.21.15.148:0/956057086 reap_dead start 2024-03-29T04:12:40.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.398+0000 7fb747663640 1 -- 172.21.15.148:0/956057086 shutdown_connections 2024-03-29T04:12:40.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.398+0000 7fb747663640 1 -- 172.21.15.148:0/956057086 >> 172.21.15.148:0/956057086 conn(0x7fb7400d42d0 msgr2=0x7fb7400d46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:40.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.398+0000 7fb747663640 1 -- 172.21.15.148:0/956057086 shutdown_connections 2024-03-29T04:12:40.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.398+0000 7fb747663640 1 -- 172.21.15.148:0/956057086 wait complete. 2024-03-29T04:12:40.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.399+0000 7fb747663640 1 Processor -- start 2024-03-29T04:12:40.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.399+0000 7fb747663640 1 -- start start 2024-03-29T04:12:40.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.400+0000 7fb747663640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb728003f60 0x7fb7400b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:40.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.400+0000 7fb747663640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7400b5230 0x7fb74014f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:40.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.400+0000 7fb747663640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb74015b370 0x7fb74015b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:40.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.400+0000 7fb747663640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb7400d3c20 con 0x7fb74015b370 2024-03-29T04:12:40.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.400+0000 7fb747663640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb74006cf40 con 0x7fb728003f60 2024-03-29T04:12:40.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.400+0000 7fb747663640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb7400d3da0 con 0x7fb7400b5230 2024-03-29T04:12:40.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.400+0000 7fb745c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb74015b370 0x7fb74015b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:40.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.400+0000 7fb745c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb74015b370 0x7fb74015b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60730/0 (socket says 172.21.15.148:60730) 2024-03-29T04:12:40.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.400+0000 7fb745c00640 1 -- 172.21.15.148:0/4182914540 learned_addr learned my addr 172.21.15.148:0/4182914540 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:40.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.400+0000 7fb744bfe640 1 --2- 172.21.15.148:0/4182914540 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7400b5230 0x7fb74014f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:40.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.400+0000 7fb7453ff640 1 --2- 172.21.15.148:0/4182914540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb728003f60 0x7fb7400b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:40.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.400+0000 7fb745c00640 1 -- 172.21.15.148:0/4182914540 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7400b5230 msgr2=0x7fb74014f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:40.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.400+0000 7fb745c00640 1 --2- 172.21.15.148:0/4182914540 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7400b5230 0x7fb74014f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:40.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.400+0000 7fb745c00640 1 -- 172.21.15.148:0/4182914540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb728003f60 msgr2=0x7fb7400b67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:40.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.400+0000 7fb745c00640 1 --2- 172.21.15.148:0/4182914540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb728003f60 0x7fb7400b67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:40.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.400+0000 7fb745c00640 1 -- 172.21.15.148:0/4182914540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb74014fb30 con 0x7fb74015b370 2024-03-29T04:12:40.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.400+0000 7fb7453ff640 1 --2- 172.21.15.148:0/4182914540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb728003f60 0x7fb7400b67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:40.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.400+0000 7fb745c00640 1 --2- 172.21.15.148:0/4182914540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb74015b370 0x7fb74015b7f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb740162110 tx=0x7fb738011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:40.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.400+0000 7fb73dffb640 1 -- 172.21.15.148:0/4182914540 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb738004240 con 0x7fb74015b370 2024-03-29T04:12:40.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.400+0000 7fb73dffb640 1 -- 172.21.15.148:0/4182914540 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb73801b070 con 0x7fb74015b370 2024-03-29T04:12:40.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.400+0000 7fb73dffb640 1 -- 172.21.15.148:0/4182914540 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73800f7a0 con 0x7fb74015b370 2024-03-29T04:12:40.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.400+0000 7fb747663640 1 -- 172.21.15.148:0/4182914540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb74015a5a0 con 0x7fb74015b370 2024-03-29T04:12:40.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.401+0000 7fb747663640 1 -- 172.21.15.148:0/4182914540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7400b4a30 con 0x7fb74015b370 2024-03-29T04:12:40.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.402+0000 7fb747663640 1 -- 172.21.15.148:0/4182914540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7401588d0 con 0x7fb74015b370 2024-03-29T04:12:40.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.402+0000 7fb73dffb640 1 -- 172.21.15.148:0/4182914540 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb738012040 con 0x7fb74015b370 2024-03-29T04:12:40.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.402+0000 7fb73dffb640 1 --2- 172.21.15.148:0/4182914540 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb72c06c530 0x7fb72c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:40.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.402+0000 7fb73dffb640 1 -- 172.21.15.148:0/4182914540 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6216+0+0 (secure 0 0 0) 0x7fb73808b820 con 0x7fb74015b370 2024-03-29T04:12:40.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.402+0000 7fb7453ff640 1 --2- 172.21.15.148:0/4182914540 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb72c06c530 0x7fb72c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:40.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.403+0000 7fb7453ff640 1 --2- 172.21.15.148:0/4182914540 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb72c06c530 0x7fb72c06e9f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb730004d00 tx=0x7fb730006040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:40.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.407+0000 7fb73dffb640 1 -- 172.21.15.148:0/4182914540 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb738059bb0 con 0x7fb74015b370 2024-03-29T04:12:40.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:40.585+0000 7fb747663640 1 -- 172.21.15.148:0/4182914540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb740068740 con 0x7fb74015b370 2024-03-29T04:12:41.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.231+0000 7fb73dffb640 1 -- 172.21.15.148:0/4182914540 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v43) ==== 181+0+0 (secure 0 0 0) 0x7fb7380594b0 con 0x7fb74015b370 2024-03-29T04:12:41.211 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' removed 2024-03-29T04:12:41.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.234+0000 7fb747663640 1 -- 172.21.15.148:0/4182914540 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb72c06c530 msgr2=0x7fb72c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:41.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.234+0000 7fb747663640 1 --2- 172.21.15.148:0/4182914540 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb72c06c530 0x7fb72c06e9f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb730004d00 tx=0x7fb730006040 comp rx=0 tx=0).stop 2024-03-29T04:12:41.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.234+0000 7fb747663640 1 -- 172.21.15.148:0/4182914540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb74015b370 msgr2=0x7fb74015b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:41.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.234+0000 7fb747663640 1 --2- 172.21.15.148:0/4182914540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb74015b370 0x7fb74015b7f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb740162110 tx=0x7fb738011cf0 comp rx=0 tx=0).stop 2024-03-29T04:12:41.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.234+0000 7fb747663640 1 -- 172.21.15.148:0/4182914540 shutdown_connections 2024-03-29T04:12:41.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.234+0000 7fb747663640 1 --2- 172.21.15.148:0/4182914540 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb72c06c530 0x7fb72c06e9f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:41.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.234+0000 7fb747663640 1 --2- 172.21.15.148:0/4182914540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb74015b370 0x7fb74015b7f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:41.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.234+0000 7fb747663640 1 --2- 172.21.15.148:0/4182914540 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7400b5230 0x7fb74014f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:41.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.234+0000 7fb747663640 1 --2- 172.21.15.148:0/4182914540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb728003f60 0x7fb7400b67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:41.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.234+0000 7fb747663640 1 -- 172.21.15.148:0/4182914540 >> 172.21.15.148:0/4182914540 conn(0x7fb7400d42d0 msgr2=0x7fb7401615b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:41.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.234+0000 7fb747663640 1 -- 172.21.15.148:0/4182914540 shutdown_connections 2024-03-29T04:12:41.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.234+0000 7fb747663640 1 -- 172.21.15.148:0/4182914540 wait complete. 2024-03-29T04:12:41.227 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-29T04:12:41.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.354+0000 7f3a92849640 1 Processor -- start 2024-03-29T04:12:41.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.354+0000 7f3a92849640 1 -- start start 2024-03-29T04:12:41.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.354+0000 7f3a92849640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8c0d3840 0x7f3a8c0d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:41.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.354+0000 7f3a92849640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3a8c0b36e0 0x7f3a8c1615d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:41.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.354+0000 7f3a92849640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a8c0d27b0 con 0x7f3a8c0d3c20 2024-03-29T04:12:41.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.354+0000 7f3a92849640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f3a8c0d2a50 con 0x7f3a8c0b36e0 2024-03-29T04:12:41.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.354+0000 7f3a92849640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3a8c0d2d30 con 0x7f3a8c0d3840 2024-03-29T04:12:41.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.355+0000 7f3a8b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3a8c0b36e0 0x7f3a8c1615d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:41.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.355+0000 7f3a8b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3a8c0b36e0 0x7f3a8c1615d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60738/0 (socket says 172.21.15.148:60738) 2024-03-29T04:12:41.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.355+0000 7f3a8b7fe640 1 -- 172.21.15.148:0/1103294086 learned_addr learned my addr 172.21.15.148:0/1103294086 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:41.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.355+0000 7f3a8bfff640 1 --2- 172.21.15.148:0/1103294086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8c0d3840 0x7f3a8c0d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:41.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.355+0000 7f3a8b7fe640 1 -- 172.21.15.148:0/1103294086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8c0d3840 msgr2=0x7f3a8c0d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:41.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.355+0000 7f3a8b7fe640 1 --2- 172.21.15.148:0/1103294086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8c0d3840 0x7f3a8c0d2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:41.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.355+0000 7f3a8b7fe640 1 -- 172.21.15.148:0/1103294086 >> v1:172.21.15.157:6789/0 conn(0x7f3a8c0d3c20 legacy=0x7f3a8c1621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:41.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.355+0000 7f3a8b7fe640 1 -- 172.21.15.148:0/1103294086 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a8c0d3010 con 0x7f3a8c0b36e0 2024-03-29T04:12:41.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.356+0000 7f3a8b7fe640 1 --2- 172.21.15.148:0/1103294086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a8c0b36e0 0x7f3a8c1615d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3a80009870 tx=0x7f3a8002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a264621e4f5715e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:41.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.356+0000 7f3a8affd640 1 -- 172.21.15.148:0/1103294086 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a80034070 con 0x7f3a8c0b36e0 2024-03-29T04:12:41.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.356+0000 7f3a8affd640 1 -- 172.21.15.148:0/1103294086 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a80030840 con 0x7f3a8c0b36e0 2024-03-29T04:12:41.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.356+0000 7f3a8affd640 1 -- 172.21.15.148:0/1103294086 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a80030c10 con 0x7f3a8c0b36e0 2024-03-29T04:12:41.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.356+0000 7f3a92849640 1 -- 172.21.15.148:0/1103294086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a8c0b36e0 msgr2=0x7f3a8c1615d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:41.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.356+0000 7f3a92849640 1 --2- 172.21.15.148:0/1103294086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a8c0b36e0 0x7f3a8c1615d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3a80009870 tx=0x7f3a8002b980 comp rx=0 tx=0).stop 2024-03-29T04:12:41.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.357+0000 7f3a92849640 1 -- 172.21.15.148:0/1103294086 shutdown_connections 2024-03-29T04:12:41.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.357+0000 7f3a92849640 1 --2- 172.21.15.148:0/1103294086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a8c0b36e0 0x7f3a8c1615d0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:41.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.357+0000 7f3a92849640 1 --2- 172.21.15.148:0/1103294086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8c0d3840 0x7f3a8c0d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:41.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.357+0000 7f3a92849640 1 -- 172.21.15.148:0/1103294086 >> 172.21.15.148:0/1103294086 conn(0x7f3a8c0d42d0 msgr2=0x7f3a8c0d46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:41.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.357+0000 7f3a92849640 1 -- 172.21.15.148:0/1103294086 shutdown_connections 2024-03-29T04:12:41.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.357+0000 7f3a92849640 1 -- 172.21.15.148:0/1103294086 wait complete. 2024-03-29T04:12:41.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.358+0000 7f3a92849640 1 Processor -- start 2024-03-29T04:12:41.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.358+0000 7f3a92849640 1 -- start start 2024-03-29T04:12:41.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.358+0000 7f3a92849640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a8c0b36e0 0x7f3a8c15e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:41.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.358+0000 7f3a92849640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8c0d3840 0x7f3a8c15da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:41.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.359+0000 7f3a92849640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a8c0d3c20 0x7f3a8c0b7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:41.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.359+0000 7f3a92849640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c06ce70 con 0x7f3a8c0d3c20 2024-03-29T04:12:41.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.359+0000 7f3a92849640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c06ccf0 con 0x7f3a8c0b36e0 2024-03-29T04:12:41.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.359+0000 7f3a92849640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3a8c06cff0 con 0x7f3a8c0d3840 2024-03-29T04:12:41.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.359+0000 7f3a90de6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a8c0d3c20 0x7f3a8c0b7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:41.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.359+0000 7f3a90de6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a8c0d3c20 0x7f3a8c0b7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60744/0 (socket says 172.21.15.148:60744) 2024-03-29T04:12:41.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.359+0000 7f3a90de6640 1 -- 172.21.15.148:0/2134222657 learned_addr learned my addr 172.21.15.148:0/2134222657 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:41.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.359+0000 7f3a8b7fe640 1 --2- 172.21.15.148:0/2134222657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8c0d3840 0x7f3a8c15da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:41.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.359+0000 7f3a8bfff640 1 --2- 172.21.15.148:0/2134222657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a8c0b36e0 0x7f3a8c15e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:41.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.359+0000 7f3a90de6640 1 -- 172.21.15.148:0/2134222657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8c0d3840 msgr2=0x7f3a8c15da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:41.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.359+0000 7f3a90de6640 1 --2- 172.21.15.148:0/2134222657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8c0d3840 0x7f3a8c15da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:41.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.359+0000 7f3a90de6640 1 -- 172.21.15.148:0/2134222657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a8c0b36e0 msgr2=0x7f3a8c15e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:41.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.359+0000 7f3a90de6640 1 --2- 172.21.15.148:0/2134222657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a8c0b36e0 0x7f3a8c15e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:41.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.359+0000 7f3a90de6640 1 -- 172.21.15.148:0/2134222657 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a8c1e65b0 con 0x7f3a8c0d3c20 2024-03-29T04:12:41.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.359+0000 7f3a90de6640 1 --2- 172.21.15.148:0/2134222657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a8c0d3c20 0x7f3a8c0b7e50 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400c730 tx=0x7f3a8400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:41.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.359+0000 7f3a88ff9640 1 -- 172.21.15.148:0/2134222657 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84003b90 con 0x7f3a8c0d3c20 2024-03-29T04:12:41.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.359+0000 7f3a88ff9640 1 -- 172.21.15.148:0/2134222657 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a84003d30 con 0x7f3a8c0d3c20 2024-03-29T04:12:41.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.359+0000 7f3a92849640 1 -- 172.21.15.148:0/2134222657 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a8c1e6890 con 0x7f3a8c0d3c20 2024-03-29T04:12:41.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.359+0000 7f3a88ff9640 1 -- 172.21.15.148:0/2134222657 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84005310 con 0x7f3a8c0d3c20 2024-03-29T04:12:41.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.359+0000 7f3a92849640 1 -- 172.21.15.148:0/2134222657 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a8c1e7130 con 0x7f3a8c0d3c20 2024-03-29T04:12:41.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.360+0000 7f3a92849640 1 -- 172.21.15.148:0/2134222657 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a8c1e6a60 con 0x7f3a8c0d3c20 2024-03-29T04:12:41.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.361+0000 7f3a88ff9640 1 -- 172.21.15.148:0/2134222657 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3a840056c0 con 0x7f3a8c0d3c20 2024-03-29T04:12:41.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.361+0000 7f3a88ff9640 1 --2- 172.21.15.148:0/2134222657 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3a7806c530 0x7f3a7806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:41.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.361+0000 7f3a88ff9640 1 -- 172.21.15.148:0/2134222657 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5778+0+0 (secure 0 0 0) 0x7f3a8408b620 con 0x7f3a8c0d3c20 2024-03-29T04:12:41.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.361+0000 7f3a8bfff640 1 --2- 172.21.15.148:0/2134222657 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3a7806c530 0x7f3a7806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:41.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.362+0000 7f3a8bfff640 1 --2- 172.21.15.148:0/2134222657 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3a7806c530 0x7f3a7806e9f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c009b40 tx=0x7f3a7c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:41.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.366+0000 7f3a88ff9640 1 -- 172.21.15.148:0/2134222657 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3a840595a0 con 0x7f3a8c0d3c20 2024-03-29T04:12:41.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:41.540+0000 7f3a92849640 1 -- 172.21.15.148:0/2134222657 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3a8c1e7330 con 0x7f3a8c0d3c20 2024-03-29T04:12:42.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.235+0000 7f3a88ff9640 1 -- 172.21.15.148:0/2134222657 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v44) ==== 169+0+0 (secure 0 0 0) 0x7f3a8c1e7330 con 0x7f3a8c0d3c20 2024-03-29T04:12:42.215 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' removed 2024-03-29T04:12:42.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.239+0000 7f3a92849640 1 -- 172.21.15.148:0/2134222657 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3a7806c530 msgr2=0x7f3a7806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:42.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.239+0000 7f3a92849640 1 --2- 172.21.15.148:0/2134222657 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3a7806c530 0x7f3a7806e9f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c009b40 tx=0x7f3a7c009190 comp rx=0 tx=0).stop 2024-03-29T04:12:42.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.239+0000 7f3a92849640 1 -- 172.21.15.148:0/2134222657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a8c0d3c20 msgr2=0x7f3a8c0b7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:42.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.239+0000 7f3a92849640 1 --2- 172.21.15.148:0/2134222657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a8c0d3c20 0x7f3a8c0b7e50 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400c730 tx=0x7f3a8400cbf0 comp rx=0 tx=0).stop 2024-03-29T04:12:42.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.239+0000 7f3a92849640 1 -- 172.21.15.148:0/2134222657 shutdown_connections 2024-03-29T04:12:42.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.239+0000 7f3a92849640 1 --2- 172.21.15.148:0/2134222657 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f3a7806c530 0x7f3a7806e9f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:42.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.239+0000 7f3a92849640 1 --2- 172.21.15.148:0/2134222657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a8c0d3c20 0x7f3a8c0b7e50 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:42.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.239+0000 7f3a92849640 1 --2- 172.21.15.148:0/2134222657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a8c0d3840 0x7f3a8c15da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:42.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.239+0000 7f3a92849640 1 --2- 172.21.15.148:0/2134222657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a8c0b36e0 0x7f3a8c15e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:42.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.239+0000 7f3a92849640 1 -- 172.21.15.148:0/2134222657 >> 172.21.15.148:0/2134222657 conn(0x7f3a8c0d42d0 msgr2=0x7f3a8c0b2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:42.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.239+0000 7f3a92849640 1 -- 172.21.15.148:0/2134222657 shutdown_connections 2024-03-29T04:12:42.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.239+0000 7f3a92849640 1 -- 172.21.15.148:0/2134222657 wait complete. 2024-03-29T04:12:42.234 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-29T04:12:42.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.360+0000 7fe4d26c3640 1 Processor -- start 2024-03-29T04:12:42.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.360+0000 7fe4d26c3640 1 -- start start 2024-03-29T04:12:42.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.361+0000 7fe4d26c3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4cc06dfd0 0x7fe4cc06c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:42.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.361+0000 7fe4d26c3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe4cc06cb00 0x7fe4cc165e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:42.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.361+0000 7fe4d26c3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4cc0ceb00 con 0x7fe4cc06dbf0 2024-03-29T04:12:42.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.361+0000 7fe4d26c3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe4cc13f5d0 con 0x7fe4cc06cb00 2024-03-29T04:12:42.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.361+0000 7fe4d26c3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe4cc0ce8d0 con 0x7fe4cc06dfd0 2024-03-29T04:12:42.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.361+0000 7fe4cbfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe4cc06dbf0 0x7fe4cc15d120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39858/0 (socket says 172.21.15.148:39858) 2024-03-29T04:12:42.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.361+0000 7fe4cbfff640 1 -- 172.21.15.148:0/2541341499 learned_addr learned my addr 172.21.15.148:0/2541341499 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:42.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.361+0000 7fe4cb7fe640 1 --2- 172.21.15.148:0/2541341499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4cc06dfd0 0x7fe4cc06c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:42.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.361+0000 7fe4d0c60640 1 --2- 172.21.15.148:0/2541341499 >> v2:172.21.15.157:3300/0 conn(0x7fe4cc06cb00 0x7fe4cc165e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:42.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.361+0000 7fe4caffd640 1 -- 172.21.15.148:0/2541341499 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1446038849 0 0) 0x7fe4cc0ceb00 con 0x7fe4cc06dbf0 2024-03-29T04:12:42.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.362+0000 7fe4caffd640 1 -- 172.21.15.148:0/2541341499 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4b4003520 con 0x7fe4cc06dbf0 2024-03-29T04:12:42.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.362+0000 7fe4caffd640 1 -- 172.21.15.148:0/2541341499 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4281552315 0 0) 0x7fe4b4003520 con 0x7fe4cc06dbf0 2024-03-29T04:12:42.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.362+0000 7fe4caffd640 1 -- 172.21.15.148:0/2541341499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4cc06dfd0 msgr2=0x7fe4cc06c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:42.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.362+0000 7fe4caffd640 1 --2- 172.21.15.148:0/2541341499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4cc06dfd0 0x7fe4cc06c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:42.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.362+0000 7fe4caffd640 1 -- 172.21.15.148:0/2541341499 >> v2:172.21.15.157:3300/0 conn(0x7fe4cc06cb00 msgr2=0x7fe4cc165e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:42.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.362+0000 7fe4caffd640 1 --2- 172.21.15.148:0/2541341499 >> v2:172.21.15.157:3300/0 conn(0x7fe4cc06cb00 0x7fe4cc165e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:42.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.362+0000 7fe4caffd640 1 -- 172.21.15.148:0/2541341499 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4cc0cede0 con 0x7fe4cc06dbf0 2024-03-29T04:12:42.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.362+0000 7fe4caffd640 1 -- 172.21.15.148:0/2541341499 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7fe4bc001670 con 0x7fe4cc06dbf0 2024-03-29T04:12:42.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.362+0000 7fe4caffd640 1 -- 172.21.15.148:0/2541341499 >> v1:172.21.15.148:6789/0 conn(0x7fe4cc06dbf0 legacy=0x7fe4cc15d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:42.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.362+0000 7fe4caffd640 1 --2- 172.21.15.148:0/2541341499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4b40039b0 0x7fe4b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:42.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.362+0000 7fe4caffd640 1 --2- 172.21.15.148:0/2541341499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4b4004800 0x7fe4b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:42.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.362+0000 7fe4caffd640 1 --2- 172.21.15.148:0/2541341499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4cc06dfd0 0x7fe4b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:42.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.362+0000 7fe4caffd640 1 -- 172.21.15.148:0/2541341499 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b400b9c0 con 0x7fe4b4004800 2024-03-29T04:12:42.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.362+0000 7fe4caffd640 1 -- 172.21.15.148:0/2541341499 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b400bb40 con 0x7fe4b40039b0 2024-03-29T04:12:42.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.362+0000 7fe4caffd640 1 -- 172.21.15.148:0/2541341499 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b400bcc0 con 0x7fe4cc06dfd0 2024-03-29T04:12:42.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.362+0000 7fe4cb7fe640 1 --2- 172.21.15.148:0/2541341499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4b4004800 0x7fe4b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:42.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.362+0000 7fe4cbfff640 1 --2- 172.21.15.148:0/2541341499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4b40039b0 0x7fe4b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:42.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.362+0000 7fe4d0c60640 1 --2- 172.21.15.148:0/2541341499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4cc06dfd0 0x7fe4b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:42.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.362+0000 7fe4cb7fe640 1 -- 172.21.15.148:0/2541341499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4cc06dfd0 msgr2=0x7fe4b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:42.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.362+0000 7fe4cb7fe640 1 --2- 172.21.15.148:0/2541341499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4cc06dfd0 0x7fe4b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:42.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.362+0000 7fe4cb7fe640 1 -- 172.21.15.148:0/2541341499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4b40039b0 msgr2=0x7fe4b4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:42.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.362+0000 7fe4cb7fe640 1 --2- 172.21.15.148:0/2541341499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4b40039b0 0x7fe4b4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:42.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.362+0000 7fe4cb7fe640 1 -- 172.21.15.148:0/2541341499 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b4003520 con 0x7fe4b4004800 2024-03-29T04:12:42.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.363+0000 7fe4cb7fe640 1 --2- 172.21.15.148:0/2541341499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4b4004800 0x7fe4b4004c80 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009ac0 tx=0x7fe4b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c90031a7337d41c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:42.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.363+0000 7fe4caffd640 1 -- 172.21.15.148:0/2541341499 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0039070 con 0x7fe4b4004800 2024-03-29T04:12:42.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.363+0000 7fe4caffd640 1 -- 172.21.15.148:0/2541341499 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c002c6b0 con 0x7fe4b4004800 2024-03-29T04:12:42.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.363+0000 7fe4caffd640 1 -- 172.21.15.148:0/2541341499 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0009690 con 0x7fe4b4004800 2024-03-29T04:12:42.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.364+0000 7fe4d26c3640 1 -- 172.21.15.148:0/2541341499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4b4004800 msgr2=0x7fe4b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:42.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.364+0000 7fe4d26c3640 1 --2- 172.21.15.148:0/2541341499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4b4004800 0x7fe4b4004c80 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009ac0 tx=0x7fe4b4005830 comp rx=0 tx=0).stop 2024-03-29T04:12:42.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.364+0000 7fe4d0c60640 1 -- 172.21.15.148:0/2541341499 reap_dead start 2024-03-29T04:12:42.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.364+0000 7fe4d26c3640 1 -- 172.21.15.148:0/2541341499 shutdown_connections 2024-03-29T04:12:42.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.364+0000 7fe4d26c3640 1 -- 172.21.15.148:0/2541341499 >> 172.21.15.148:0/2541341499 conn(0x7fe4cc0d00a0 msgr2=0x7fe4cc0d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:42.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.364+0000 7fe4d26c3640 1 -- 172.21.15.148:0/2541341499 shutdown_connections 2024-03-29T04:12:42.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.364+0000 7fe4d26c3640 1 -- 172.21.15.148:0/2541341499 wait complete. 2024-03-29T04:12:42.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.365+0000 7fe4d26c3640 1 Processor -- start 2024-03-29T04:12:42.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.365+0000 7fe4d26c3640 1 -- start start 2024-03-29T04:12:42.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.366+0000 7fe4d26c3640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4b4004800 0x7fe4cc155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:42.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.366+0000 7fe4d26c3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4cc13bad0 0x7fe4cc142a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:42.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.366+0000 7fe4d26c3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4cc1456b0 0x7fe4cc15d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:42.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.366+0000 7fe4d26c3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc0cf680 con 0x7fe4cc1456b0 2024-03-29T04:12:42.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.366+0000 7fe4d26c3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc0cf3a0 con 0x7fe4cc13bad0 2024-03-29T04:12:42.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.366+0000 7fe4d26c3640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe4cc06e450 con 0x7fe4b4004800 2024-03-29T04:12:42.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.366+0000 7fe4d0c60640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4cc1456b0 0x7fe4cc15d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:42.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.366+0000 7fe4d0c60640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4cc1456b0 0x7fe4cc15d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60756/0 (socket says 172.21.15.148:60756) 2024-03-29T04:12:42.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.366+0000 7fe4d0c60640 1 -- 172.21.15.148:0/3968217295 learned_addr learned my addr 172.21.15.148:0/3968217295 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:42.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.366+0000 7fe4cbfff640 1 --2- 172.21.15.148:0/3968217295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4b4004800 0x7fe4cc155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:42.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.366+0000 7fe4cb7fe640 1 --2- 172.21.15.148:0/3968217295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4cc13bad0 0x7fe4cc142a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:42.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.366+0000 7fe4d0c60640 1 -- 172.21.15.148:0/3968217295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4b4004800 msgr2=0x7fe4cc155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:42.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.366+0000 7fe4d0c60640 1 --2- 172.21.15.148:0/3968217295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4b4004800 0x7fe4cc155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:42.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.366+0000 7fe4d0c60640 1 -- 172.21.15.148:0/3968217295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4cc13bad0 msgr2=0x7fe4cc142a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:42.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.366+0000 7fe4d0c60640 1 --2- 172.21.15.148:0/3968217295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4cc13bad0 0x7fe4cc142a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:42.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.366+0000 7fe4d0c60640 1 -- 172.21.15.148:0/3968217295 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4cc156050 con 0x7fe4cc1456b0 2024-03-29T04:12:42.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.366+0000 7fe4cb7fe640 1 --2- 172.21.15.148:0/3968217295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4cc13bad0 0x7fe4cc142a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:42.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.366+0000 7fe4d0c60640 1 --2- 172.21.15.148:0/3968217295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4cc1456b0 0x7fe4cc15d960 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc142880 tx=0x7fe4c400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:42.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.367+0000 7fe4c8ff9640 1 -- 172.21.15.148:0/3968217295 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c400fdb0 con 0x7fe4cc1456b0 2024-03-29T04:12:42.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.367+0000 7fe4c8ff9640 1 -- 172.21.15.148:0/3968217295 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c4015430 con 0x7fe4cc1456b0 2024-03-29T04:12:42.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.367+0000 7fe4d26c3640 1 -- 172.21.15.148:0/3968217295 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4cc142f60 con 0x7fe4cc1456b0 2024-03-29T04:12:42.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.367+0000 7fe4c8ff9640 1 -- 172.21.15.148:0/3968217295 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4015820 con 0x7fe4cc1456b0 2024-03-29T04:12:42.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.367+0000 7fe4d26c3640 1 -- 172.21.15.148:0/3968217295 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4cc15b950 con 0x7fe4cc1456b0 2024-03-29T04:12:42.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.368+0000 7fe4d26c3640 1 -- 172.21.15.148:0/3968217295 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4cc068850 con 0x7fe4cc1456b0 2024-03-29T04:12:42.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.368+0000 7fe4c8ff9640 1 -- 172.21.15.148:0/3968217295 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe4c40159c0 con 0x7fe4cc1456b0 2024-03-29T04:12:42.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.368+0000 7fe4c8ff9640 1 --2- 172.21.15.148:0/3968217295 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe4b806c560 0x7fe4b806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:42.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.368+0000 7fe4c8ff9640 1 -- 172.21.15.148:0/3968217295 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7fe4c408c260 con 0x7fe4cc1456b0 2024-03-29T04:12:42.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.368+0000 7fe4cbfff640 1 --2- 172.21.15.148:0/3968217295 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe4b806c560 0x7fe4b806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:42.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.369+0000 7fe4cbfff640 1 --2- 172.21.15.148:0/3968217295 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe4b806c560 0x7fe4b806ea20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc001050 tx=0x7fe4bc007970 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:42.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.373+0000 7fe4c8ff9640 1 -- 172.21.15.148:0/3968217295 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe4cc156050 con 0x7fe4cc1456b0 2024-03-29T04:12:42.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.548+0000 7fe4d26c3640 1 -- 172.21.15.148:0/3968217295 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fe4cc0cd390 con 0x7fe4cc1456b0 2024-03-29T04:12:42.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.552+0000 7fe4c8ff9640 1 -- 172.21.15.148:0/3968217295 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fe4c4019030 con 0x7fe4cc1456b0 2024-03-29T04:12:42.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.554+0000 7fe4d26c3640 1 -- 172.21.15.148:0/3968217295 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe4b806c560 msgr2=0x7fe4b806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:42.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.554+0000 7fe4d26c3640 1 --2- 172.21.15.148:0/3968217295 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe4b806c560 0x7fe4b806ea20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc001050 tx=0x7fe4bc007970 comp rx=0 tx=0).stop 2024-03-29T04:12:42.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.554+0000 7fe4d26c3640 1 -- 172.21.15.148:0/3968217295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4cc1456b0 msgr2=0x7fe4cc15d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:42.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.554+0000 7fe4d26c3640 1 --2- 172.21.15.148:0/3968217295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4cc1456b0 0x7fe4cc15d960 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc142880 tx=0x7fe4c400f580 comp rx=0 tx=0).stop 2024-03-29T04:12:42.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.554+0000 7fe4d26c3640 1 -- 172.21.15.148:0/3968217295 shutdown_connections 2024-03-29T04:12:42.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.554+0000 7fe4d26c3640 1 --2- 172.21.15.148:0/3968217295 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fe4b806c560 0x7fe4b806ea20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:42.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.554+0000 7fe4d26c3640 1 --2- 172.21.15.148:0/3968217295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4cc1456b0 0x7fe4cc15d960 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:42.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.554+0000 7fe4d26c3640 1 --2- 172.21.15.148:0/3968217295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4cc13bad0 0x7fe4cc142a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:42.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.554+0000 7fe4d26c3640 1 --2- 172.21.15.148:0/3968217295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4b4004800 0x7fe4cc155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:42.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.554+0000 7fe4d26c3640 1 -- 172.21.15.148:0/3968217295 >> 172.21.15.148:0/3968217295 conn(0x7fe4cc0d00a0 msgr2=0x7fe4cc15d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:42.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.554+0000 7fe4d26c3640 1 -- 172.21.15.148:0/3968217295 shutdown_connections 2024-03-29T04:12:42.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.554+0000 7fe4d26c3640 1 -- 172.21.15.148:0/3968217295 wait complete. 2024-03-29T04:12:42.545 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining' 2024-03-29T04:12:42.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.665+0000 7f1a54047640 1 Processor -- start 2024-03-29T04:12:42.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.666+0000 7f1a54047640 1 -- start start 2024-03-29T04:12:42.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.666+0000 7f1a54047640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a4c0d17e0 0x7f1a4c0d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:42.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.666+0000 7f1a54047640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a4c15e7c0 con 0x7f1a4c15e3e0 2024-03-29T04:12:42.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.666+0000 7f1a54047640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a4c0d2c30 con 0x7f1a4c0ba030 2024-03-29T04:12:42.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.666+0000 7f1a54047640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1a4c0d3840 con 0x7f1a4c0d17e0 2024-03-29T04:12:42.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.667+0000 7f1a51de3640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f1a4c15e3e0 0x7f1a4c0d2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39866/0 (socket says 172.21.15.148:39866) 2024-03-29T04:12:42.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.667+0000 7f1a51de3640 1 -- 172.21.15.148:0/3395199574 learned_addr learned my addr 172.21.15.148:0/3395199574 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:42.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.667+0000 7f1a50de1640 1 -- 172.21.15.148:0/3395199574 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1196748777 0 0) 0x7f1a4c15e7c0 con 0x7f1a4c15e3e0 2024-03-29T04:12:42.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.667+0000 7f1a525e4640 1 --2- 172.21.15.148:0/3395199574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a4c0d17e0 0x7f1a4c0d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:42.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.667+0000 7f1a50de1640 1 -- 172.21.15.148:0/3395199574 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a38003520 con 0x7f1a4c15e3e0 2024-03-29T04:12:42.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.667+0000 7f1a50de1640 1 -- 172.21.15.148:0/3395199574 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3264433236 0 0) 0x7f1a38003520 con 0x7f1a4c15e3e0 2024-03-29T04:12:42.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.667+0000 7f1a50de1640 1 -- 172.21.15.148:0/3395199574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a4c0d17e0 msgr2=0x7f1a4c0d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:42.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.667+0000 7f1a50de1640 1 --2- 172.21.15.148:0/3395199574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a4c0d17e0 0x7f1a4c0d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:42.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.667+0000 7f1a50de1640 1 -- 172.21.15.148:0/3395199574 >> v1:172.21.15.157:6789/0 conn(0x7f1a4c0ba030 legacy=0x7f1a4c0d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:42.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.667+0000 7f1a50de1640 1 -- 172.21.15.148:0/3395199574 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4c0d2f10 con 0x7f1a4c15e3e0 2024-03-29T04:12:42.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.667+0000 7f1a50de1640 1 -- 172.21.15.148:0/3395199574 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f1a48001670 con 0x7f1a4c15e3e0 2024-03-29T04:12:42.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.667+0000 7f1a50de1640 1 -- 172.21.15.148:0/3395199574 >> v1:172.21.15.148:6789/0 conn(0x7f1a4c15e3e0 legacy=0x7f1a4c0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:42.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.667+0000 7f1a50de1640 1 --2- 172.21.15.148:0/3395199574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a380039e0 0x7f1a38003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:42.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.667+0000 7f1a50de1640 1 --2- 172.21.15.148:0/3395199574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a4c0d17e0 0x7f1a38004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:42.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.667+0000 7f1a50de1640 1 --2- 172.21.15.148:0/3395199574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a38004d40 0x7f1a380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:42.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.667+0000 7f1a50de1640 1 -- 172.21.15.148:0/3395199574 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c0d3840 con 0x7f1a38004d40 2024-03-29T04:12:42.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.667+0000 7f1a50de1640 1 -- 172.21.15.148:0/3395199574 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3800ba70 con 0x7f1a380039e0 2024-03-29T04:12:42.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.667+0000 7f1a50de1640 1 -- 172.21.15.148:0/3395199574 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1a3800bc70 con 0x7f1a4c0d17e0 2024-03-29T04:12:42.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.667+0000 7f1a525e4640 1 --2- 172.21.15.148:0/3395199574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a38004d40 0x7f1a380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:42.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.667+0000 7f1a51de3640 1 --2- 172.21.15.148:0/3395199574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a380039e0 0x7f1a38003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:42.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.667+0000 7f1a515e2640 1 --2- 172.21.15.148:0/3395199574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a4c0d17e0 0x7f1a38004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:42.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.668+0000 7f1a525e4640 1 -- 172.21.15.148:0/3395199574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a4c0d17e0 msgr2=0x7f1a38004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:42.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.668+0000 7f1a525e4640 1 --2- 172.21.15.148:0/3395199574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a4c0d17e0 0x7f1a38004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:42.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.668+0000 7f1a525e4640 1 -- 172.21.15.148:0/3395199574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a380039e0 msgr2=0x7f1a38003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:42.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.668+0000 7f1a525e4640 1 --2- 172.21.15.148:0/3395199574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a380039e0 0x7f1a38003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:42.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.668+0000 7f1a525e4640 1 -- 172.21.15.148:0/3395199574 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a38003520 con 0x7f1a38004d40 2024-03-29T04:12:42.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.668+0000 7f1a515e2640 1 --2- 172.21.15.148:0/3395199574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a4c0d17e0 0x7f1a38004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:42.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.668+0000 7f1a525e4640 1 --2- 172.21.15.148:0/3395199574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a38004d40 0x7f1a380051c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009f90 tx=0x7f1a38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45028daa413c3e86 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:42.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.668+0000 7f1a50de1640 1 -- 172.21.15.148:0/3395199574 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44039070 con 0x7f1a38004d40 2024-03-29T04:12:42.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.668+0000 7f1a50de1640 1 -- 172.21.15.148:0/3395199574 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a4402cbd0 con 0x7f1a38004d40 2024-03-29T04:12:42.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.668+0000 7f1a50de1640 1 -- 172.21.15.148:0/3395199574 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44009c10 con 0x7f1a38004d40 2024-03-29T04:12:42.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.669+0000 7f1a54047640 1 -- 172.21.15.148:0/3395199574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a38004d40 msgr2=0x7f1a380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:42.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.669+0000 7f1a54047640 1 --2- 172.21.15.148:0/3395199574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a38004d40 0x7f1a380051c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009f90 tx=0x7f1a38005830 comp rx=0 tx=0).stop 2024-03-29T04:12:42.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.669+0000 7f1a525e4640 1 -- 172.21.15.148:0/3395199574 reap_dead start 2024-03-29T04:12:42.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.669+0000 7f1a54047640 1 -- 172.21.15.148:0/3395199574 shutdown_connections 2024-03-29T04:12:42.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.669+0000 7f1a54047640 1 -- 172.21.15.148:0/3395199574 >> 172.21.15.148:0/3395199574 conn(0x7f1a4c0b6210 msgr2=0x7f1a4c0b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:42.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.669+0000 7f1a54047640 1 -- 172.21.15.148:0/3395199574 shutdown_connections 2024-03-29T04:12:42.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.669+0000 7f1a54047640 1 -- 172.21.15.148:0/3395199574 wait complete. 2024-03-29T04:12:42.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.670+0000 7f1a54047640 1 Processor -- start 2024-03-29T04:12:42.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.670+0000 7f1a54047640 1 -- start start 2024-03-29T04:12:42.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.671+0000 7f1a54047640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a4c1519a0 0x7f1a4c156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:42.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.671+0000 7f1a54047640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a4c151d80 0x7f1a4c153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:42.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.671+0000 7f1a54047640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a4c0ca340 0x7f1a4c160bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:42.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.671+0000 7f1a54047640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c0d3cd0 con 0x7f1a4c151d80 2024-03-29T04:12:42.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.671+0000 7f1a54047640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c0d34d0 con 0x7f1a4c1519a0 2024-03-29T04:12:42.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.671+0000 7f1a54047640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1a4c0d3e50 con 0x7f1a4c0ca340 2024-03-29T04:12:42.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.671+0000 7f1a515e2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a4c151d80 0x7f1a4c153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:42.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.671+0000 7f1a515e2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a4c151d80 0x7f1a4c153fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60764/0 (socket says 172.21.15.148:60764) 2024-03-29T04:12:42.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.671+0000 7f1a515e2640 1 -- 172.21.15.148:0/3639189638 learned_addr learned my addr 172.21.15.148:0/3639189638 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:42.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.671+0000 7f1a51de3640 1 --2- 172.21.15.148:0/3639189638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a4c1519a0 0x7f1a4c156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:42.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.671+0000 7f1a525e4640 1 --2- 172.21.15.148:0/3639189638 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a4c0ca340 0x7f1a4c160bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:42.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.671+0000 7f1a515e2640 1 -- 172.21.15.148:0/3639189638 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a4c0ca340 msgr2=0x7f1a4c160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:42.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.671+0000 7f1a515e2640 1 --2- 172.21.15.148:0/3639189638 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a4c0ca340 0x7f1a4c160bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:42.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.671+0000 7f1a515e2640 1 -- 172.21.15.148:0/3639189638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a4c1519a0 msgr2=0x7f1a4c156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:42.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.671+0000 7f1a515e2640 1 --2- 172.21.15.148:0/3639189638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a4c1519a0 0x7f1a4c156fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:42.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.671+0000 7f1a515e2640 1 -- 172.21.15.148:0/3639189638 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4c15fe10 con 0x7f1a4c151d80 2024-03-29T04:12:42.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.671+0000 7f1a51de3640 1 --2- 172.21.15.148:0/3639189638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a4c1519a0 0x7f1a4c156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:42.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.671+0000 7f1a515e2640 1 --2- 172.21.15.148:0/3639189638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a4c151d80 0x7f1a4c153fe0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00e7d0 tx=0x7f1a3c00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:42.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.672+0000 7f1a427fc640 1 -- 172.21.15.148:0/3639189638 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c00ee80 con 0x7f1a4c151d80 2024-03-29T04:12:42.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.672+0000 7f1a427fc640 1 -- 172.21.15.148:0/3639189638 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a3c018070 con 0x7f1a4c151d80 2024-03-29T04:12:42.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.672+0000 7f1a54047640 1 -- 172.21.15.148:0/3639189638 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a4c1600f0 con 0x7f1a4c151d80 2024-03-29T04:12:42.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.672+0000 7f1a427fc640 1 -- 172.21.15.148:0/3639189638 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c00c6c0 con 0x7f1a4c151d80 2024-03-29T04:12:42.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.672+0000 7f1a54047640 1 -- 172.21.15.148:0/3639189638 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a4c0ca7c0 con 0x7f1a4c151d80 2024-03-29T04:12:42.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.673+0000 7f1a54047640 1 -- 172.21.15.148:0/3639189638 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a4c0687e0 con 0x7f1a4c151d80 2024-03-29T04:12:42.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.673+0000 7f1a427fc640 1 -- 172.21.15.148:0/3639189638 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1a3c00c860 con 0x7f1a4c151d80 2024-03-29T04:12:42.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.674+0000 7f1a427fc640 1 --2- 172.21.15.148:0/3639189638 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1a3006c530 0x7f1a3006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:42.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.674+0000 7f1a427fc640 1 -- 172.21.15.148:0/3639189638 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f1a3c08b350 con 0x7f1a4c151d80 2024-03-29T04:12:42.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.674+0000 7f1a51de3640 1 --2- 172.21.15.148:0/3639189638 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1a3006c530 0x7f1a3006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:42.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.674+0000 7f1a51de3640 1 --2- 172.21.15.148:0/3639189638 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1a3006c530 0x7f1a3006e9f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1a48004840 tx=0x7f1a48009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:42.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.678+0000 7f1a427fc640 1 -- 172.21.15.148:0/3639189638 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1a3c01a310 con 0x7f1a4c151d80 2024-03-29T04:12:42.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:42.918+0000 7f1a54047640 1 -- 172.21.15.148:0/3639189638 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining"]} v 0) -- 0x7f1a4c06cdd0 con 0x7f1a4c151d80 2024-03-29T04:12:43.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:43.236+0000 7f1a427fc640 1 -- 172.21.15.148:0/3639189638 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining"]}]=0 v159) ==== 147+0+0 (secure 0 0 0) 0x7f1a3c059b70 con 0x7f1a4c151d80 2024-03-29T04:12:43.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:43.238+0000 7f1a54047640 1 -- 172.21.15.148:0/3639189638 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1a3006c530 msgr2=0x7f1a3006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:43.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:43.238+0000 7f1a54047640 1 --2- 172.21.15.148:0/3639189638 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1a3006c530 0x7f1a3006e9f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1a48004840 tx=0x7f1a48009040 comp rx=0 tx=0).stop 2024-03-29T04:12:43.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:43.239+0000 7f1a54047640 1 -- 172.21.15.148:0/3639189638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a4c151d80 msgr2=0x7f1a4c153fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:43.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:43.239+0000 7f1a54047640 1 --2- 172.21.15.148:0/3639189638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a4c151d80 0x7f1a4c153fe0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00e7d0 tx=0x7f1a3c00ec90 comp rx=0 tx=0).stop 2024-03-29T04:12:43.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:43.239+0000 7f1a54047640 1 -- 172.21.15.148:0/3639189638 shutdown_connections 2024-03-29T04:12:43.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:43.239+0000 7f1a54047640 1 --2- 172.21.15.148:0/3639189638 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f1a3006c530 0x7f1a3006e9f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:43.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:43.239+0000 7f1a54047640 1 --2- 172.21.15.148:0/3639189638 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1a4c0ca340 0x7f1a4c160bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:43.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:43.239+0000 7f1a54047640 1 --2- 172.21.15.148:0/3639189638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a4c151d80 0x7f1a4c153fe0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:43.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:43.239+0000 7f1a54047640 1 --2- 172.21.15.148:0/3639189638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a4c1519a0 0x7f1a4c156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:43.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:43.239+0000 7f1a54047640 1 -- 172.21.15.148:0/3639189638 >> 172.21.15.148:0/3639189638 conn(0x7f1a4c0b6210 msgr2=0x7f1a4c0b8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:43.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:43.239+0000 7f1a54047640 1 -- 172.21.15.148:0/3639189638 shutdown_connections 2024-03-29T04:12:43.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:43.239+0000 7f1a54047640 1 -- 172.21.15.148:0/3639189638 wait complete. 2024-03-29T04:12:43.232 INFO:tasks.cephfs_test_runner:test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) ... ERROR 2024-03-29T04:12:43.232 INFO:tasks.cephfs_test_runner: 2024-03-29T04:12:43.232 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-29T04:12:43.232 INFO:tasks.cephfs_test_runner:ERROR: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-29T04:12:43.232 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-29T04:12:43.232 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-29T04:12:43.233 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 322, in test_data_uninlining 2024-03-29T04:12:43.233 INFO:tasks.cephfs_test_runner: self.run_test_worker(opt_clear_cache, 2024-03-29T04:12:43.233 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 259, in run_test_worker 2024-03-29T04:12:43.233 INFO:tasks.cephfs_test_runner: info = self.get_inline_data_info(inodes, files, dir_pins, num_mds) 2024-03-29T04:12:43.233 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 116, in get_inline_data_info 2024-03-29T04:12:43.233 INFO:tasks.cephfs_test_runner: json_out = get_inode_dump(inode, 2024-03-29T04:12:43.233 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 101, in get_inode_dump 2024-03-29T04:12:43.233 INFO:tasks.cephfs_test_runner: json_out = self.fs.rank_tell(['dump', 'inode', inode], rank) 2024-03-29T04:12:43.233 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/filesystem.py", line 1332, in rank_tell 2024-03-29T04:12:43.233 INFO:tasks.cephfs_test_runner: return self.mds_tell(*args, mds_id=f"{self.id}:{rank}", **kwargs) 2024-03-29T04:12:43.233 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/filesystem.py", line 1319, in mds_tell 2024-03-29T04:12:43.233 INFO:tasks.cephfs_test_runner: out = self.get_ceph_cmd_stdout("tell", f"mds.{mds_id}", *args, **kwargs) 2024-03-29T04:12:43.233 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-03-29T04:12:43.233 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2024-03-29T04:12:43.233 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-03-29T04:12:43.233 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-03-29T04:12:43.233 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd 2024-03-29T04:12:43.233 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-03-29T04:12:43.234 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run 2024-03-29T04:12:43.234 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-03-29T04:12:43.234 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 447, in run 2024-03-29T04:12:43.234 INFO:tasks.cephfs_test_runner: r = RemoteProcess(client, args, check_status=check_status, hostname=name, 2024-03-29T04:12:43.234 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 65, in __init__ 2024-03-29T04:12:43.234 INFO:tasks.cephfs_test_runner: self.command = quote(args) 2024-03-29T04:12:43.234 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 257, in quote 2024-03-29T04:12:43.234 INFO:tasks.cephfs_test_runner: return ' '.join(_quote(args)) 2024-03-29T04:12:43.234 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 255, in _quote 2024-03-29T04:12:43.234 INFO:tasks.cephfs_test_runner: yield pipes.quote(a) 2024-03-29T04:12:43.234 INFO:tasks.cephfs_test_runner: File "/usr/lib/python3.8/shlex.py", line 325, in quote 2024-03-29T04:12:43.234 INFO:tasks.cephfs_test_runner: if _find_unsafe(s) is None: 2024-03-29T04:12:43.234 INFO:tasks.cephfs_test_runner:TypeError: expected string or bytes-like object 2024-03-29T04:12:43.234 INFO:tasks.cephfs_test_runner: 2024-03-29T04:12:43.234 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-29T04:12:43.234 INFO:tasks.cephfs_test_runner:Ran 1 test in 112.435s 2024-03-29T04:12:43.235 INFO:tasks.cephfs_test_runner: 2024-03-29T04:12:43.235 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-03-29T04:12:43.235 INFO:tasks.cephfs_test_runner: 2024-03-29T04:12:43.235 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-29T04:12:43.235 INFO:tasks.cephfs_test_runner:ERROR: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-29T04:12:43.235 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-29T04:12:43.235 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-29T04:12:43.235 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 322, in test_data_uninlining 2024-03-29T04:12:43.235 INFO:tasks.cephfs_test_runner: self.run_test_worker(opt_clear_cache, 2024-03-29T04:12:43.235 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 259, in run_test_worker 2024-03-29T04:12:43.235 INFO:tasks.cephfs_test_runner: info = self.get_inline_data_info(inodes, files, dir_pins, num_mds) 2024-03-29T04:12:43.235 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 116, in get_inline_data_info 2024-03-29T04:12:43.235 INFO:tasks.cephfs_test_runner: json_out = get_inode_dump(inode, 2024-03-29T04:12:43.235 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 101, in get_inode_dump 2024-03-29T04:12:43.236 INFO:tasks.cephfs_test_runner: json_out = self.fs.rank_tell(['dump', 'inode', inode], rank) 2024-03-29T04:12:43.236 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/filesystem.py", line 1332, in rank_tell 2024-03-29T04:12:43.236 INFO:tasks.cephfs_test_runner: return self.mds_tell(*args, mds_id=f"{self.id}:{rank}", **kwargs) 2024-03-29T04:12:43.236 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/filesystem.py", line 1319, in mds_tell 2024-03-29T04:12:43.236 INFO:tasks.cephfs_test_runner: out = self.get_ceph_cmd_stdout("tell", f"mds.{mds_id}", *args, **kwargs) 2024-03-29T04:12:43.236 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-03-29T04:12:43.236 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2024-03-29T04:12:43.236 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-03-29T04:12:43.236 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-03-29T04:12:43.236 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd 2024-03-29T04:12:43.236 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-03-29T04:12:43.236 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run 2024-03-29T04:12:43.236 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-03-29T04:12:43.236 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 447, in run 2024-03-29T04:12:43.236 INFO:tasks.cephfs_test_runner: r = RemoteProcess(client, args, check_status=check_status, hostname=name, 2024-03-29T04:12:43.236 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 65, in __init__ 2024-03-29T04:12:43.236 INFO:tasks.cephfs_test_runner: self.command = quote(args) 2024-03-29T04:12:43.236 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 257, in quote 2024-03-29T04:12:43.237 INFO:tasks.cephfs_test_runner: return ' '.join(_quote(args)) 2024-03-29T04:12:43.237 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 255, in _quote 2024-03-29T04:12:43.237 INFO:tasks.cephfs_test_runner: yield pipes.quote(a) 2024-03-29T04:12:43.237 INFO:tasks.cephfs_test_runner: File "/usr/lib/python3.8/shlex.py", line 325, in quote 2024-03-29T04:12:43.237 INFO:tasks.cephfs_test_runner: if _find_unsafe(s) is None: 2024-03-29T04:12:43.237 INFO:tasks.cephfs_test_runner:TypeError: expected string or bytes-like object 2024-03-29T04:12:43.237 INFO:tasks.cephfs_test_runner: 2024-03-29T04:12:43.237 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-29T04:12:43.493 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=c1f9a4edd6f3472b8a05cc4bf58637a0 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-29T04:12:43.494 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-03-29T04:12:43.504 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-29T04:12:43.543 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-29T04:12:43.544 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:12:43.544 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:12:43.562 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18560 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T04:12:43.562 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T04:12:43.563 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:12:43.618 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056370,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18560 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T04:12:43.618 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:12:43.618 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:12:43.637 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16940 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T04:12:43.637 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-29T04:12:43.637 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:12:43.695 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16940 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T04:12:43.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:12:43.695 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-03-29T04:12:43.753 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-29T04:12:43.753 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-29T04:12:43.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:12:43.754 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:12:43.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:12:43.824 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:12:43.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:12:43.896 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2024-03-29T04:12:43.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:12:43.995 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-03-29T04:12:44.011 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T04:12:44.011 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T04:12:44.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:12:44.011 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:12:44.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:12:44.079 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:12:44.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:12:44.149 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link delete ceph-brx 2024-03-29T04:12:44.255 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-29T04:12:44.265 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-29T04:12:44.265 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:12:44.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.387+0000 7f9eb7172640 1 Processor -- start 2024-03-29T04:12:44.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.387+0000 7f9eb7172640 1 -- start start 2024-03-29T04:12:44.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.388+0000 7f9eb7172640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9eb00cf780 0x7f9eb00ce610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:44.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.388+0000 7f9eb7172640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9eb015d6a0 con 0x7f9eb015d210 2024-03-29T04:12:44.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.388+0000 7f9eb7172640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9eb006da30 con 0x7f9eb006ccd0 2024-03-29T04:12:44.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.388+0000 7f9eb7172640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f9eb006dd10 con 0x7f9eb00cf780 2024-03-29T04:12:44.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.388+0000 7f9eb4f0e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9eb00cf780 0x7f9eb00ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:44.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.388+0000 7f9eaffff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9eb015d210 0x7f9eb006c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39874/0 (socket says 172.21.15.148:39874) 2024-03-29T04:12:44.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.388+0000 7f9eaffff640 1 -- 172.21.15.148:0/3887309107 learned_addr learned my addr 172.21.15.148:0/3887309107 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:44.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.389+0000 7f9eb4f0e640 1 -- 172.21.15.148:0/3887309107 >> v1:172.21.15.157:6789/0 conn(0x7f9eb006ccd0 legacy=0x7f9eb00b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:44.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.389+0000 7f9eb4f0e640 1 -- 172.21.15.148:0/3887309107 >> v1:172.21.15.148:6789/0 conn(0x7f9eb015d210 legacy=0x7f9eb006c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:44.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.389+0000 7f9eaf7fe640 1 -- 172.21.15.148:0/3887309107 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2479194817 0 0) 0x7f9eb015d6a0 con 0x7f9eb015d210 2024-03-29T04:12:44.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.389+0000 7f9eb4f0e640 1 -- 172.21.15.148:0/3887309107 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb006dff0 con 0x7f9eb00cf780 2024-03-29T04:12:44.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.389+0000 7f9eb4f0e640 1 --2- 172.21.15.148:0/3887309107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb00cf780 0x7f9eb00ce610 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00097a0 tx=0x7f9ea002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf2bc0c746616a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:44.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.389+0000 7f9eaf7fe640 1 -- 172.21.15.148:0/3887309107 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea00315a0 con 0x7f9eb00cf780 2024-03-29T04:12:44.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.390+0000 7f9eaf7fe640 1 -- 172.21.15.148:0/3887309107 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ea0037070 con 0x7f9eb00cf780 2024-03-29T04:12:44.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.390+0000 7f9eaf7fe640 1 -- 172.21.15.148:0/3887309107 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0031960 con 0x7f9eb00cf780 2024-03-29T04:12:44.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.390+0000 7f9eb7172640 1 -- 172.21.15.148:0/3887309107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb00cf780 msgr2=0x7f9eb00ce610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:44.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.390+0000 7f9eb7172640 1 --2- 172.21.15.148:0/3887309107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb00cf780 0x7f9eb00ce610 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00097a0 tx=0x7f9ea002b840 comp rx=0 tx=0).stop 2024-03-29T04:12:44.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.390+0000 7f9eb7172640 1 -- 172.21.15.148:0/3887309107 shutdown_connections 2024-03-29T04:12:44.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.390+0000 7f9eb7172640 1 --2- 172.21.15.148:0/3887309107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb00cf780 0x7f9eb00ce610 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:44.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.390+0000 7f9eb7172640 1 -- 172.21.15.148:0/3887309107 >> 172.21.15.148:0/3887309107 conn(0x7f9eb00cfed0 msgr2=0x7f9eb00d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:44.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.390+0000 7f9eb7172640 1 -- 172.21.15.148:0/3887309107 shutdown_connections 2024-03-29T04:12:44.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.390+0000 7f9eb7172640 1 -- 172.21.15.148:0/3887309107 wait complete. 2024-03-29T04:12:44.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.391+0000 7f9eb7172640 1 Processor -- start 2024-03-29T04:12:44.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.391+0000 7f9eb7172640 1 -- start start 2024-03-29T04:12:44.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.392+0000 7f9eb7172640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb006ccd0 0x7f9eb00b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:44.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.392+0000 7f9eb7172640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9eb00cf780 0x7f9eb01f1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:44.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.392+0000 7f9eb7172640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb015d210 0x7f9eb01f2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:44.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.392+0000 7f9eb7172640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb006d5c0 con 0x7f9eb006ccd0 2024-03-29T04:12:44.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.392+0000 7f9eb7172640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb006f340 con 0x7f9eb015d210 2024-03-29T04:12:44.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.392+0000 7f9eb7172640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9eb006d740 con 0x7f9eb00cf780 2024-03-29T04:12:44.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.392+0000 7f9eb4f0e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb006ccd0 0x7f9eb00b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:44.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.392+0000 7f9eb4f0e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb006ccd0 0x7f9eb00b4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60782/0 (socket says 172.21.15.148:60782) 2024-03-29T04:12:44.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.392+0000 7f9eb4f0e640 1 -- 172.21.15.148:0/4197822339 learned_addr learned my addr 172.21.15.148:0/4197822339 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:44.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.392+0000 7f9eb570f640 1 --2- 172.21.15.148:0/4197822339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb015d210 0x7f9eb01f2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:44.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.392+0000 7f9eaffff640 1 --2- 172.21.15.148:0/4197822339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9eb00cf780 0x7f9eb01f1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:44.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.393+0000 7f9eb4f0e640 1 -- 172.21.15.148:0/4197822339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9eb00cf780 msgr2=0x7f9eb01f1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:44.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.393+0000 7f9eb4f0e640 1 --2- 172.21.15.148:0/4197822339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9eb00cf780 0x7f9eb01f1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:44.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.393+0000 7f9eb4f0e640 1 -- 172.21.15.148:0/4197822339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb015d210 msgr2=0x7f9eb01f2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:44.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.393+0000 7f9eb4f0e640 1 --2- 172.21.15.148:0/4197822339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb015d210 0x7f9eb01f2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:44.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.393+0000 7f9eb4f0e640 1 -- 172.21.15.148:0/4197822339 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb01f2c70 con 0x7f9eb006ccd0 2024-03-29T04:12:44.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.393+0000 7f9eb570f640 1 --2- 172.21.15.148:0/4197822339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb015d210 0x7f9eb01f2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:44.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.393+0000 7f9eb4f0e640 1 --2- 172.21.15.148:0/4197822339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb006ccd0 0x7f9eb00b4c80 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00097a0 tx=0x7f9ea00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:44.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.393+0000 7f9ead7fa640 1 -- 172.21.15.148:0/4197822339 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0032840 con 0x7f9eb006ccd0 2024-03-29T04:12:44.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.393+0000 7f9ead7fa640 1 -- 172.21.15.148:0/4197822339 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ea00329e0 con 0x7f9eb006ccd0 2024-03-29T04:12:44.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.393+0000 7f9eb7172640 1 -- 172.21.15.148:0/4197822339 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb01f2990 con 0x7f9eb006ccd0 2024-03-29T04:12:44.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.393+0000 7f9ead7fa640 1 -- 172.21.15.148:0/4197822339 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0032dd0 con 0x7f9eb006ccd0 2024-03-29T04:12:44.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.393+0000 7f9eb7172640 1 -- 172.21.15.148:0/4197822339 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb01f5720 con 0x7f9eb006ccd0 2024-03-29T04:12:44.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.394+0000 7f9eb7172640 1 -- 172.21.15.148:0/4197822339 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eb00687e0 con 0x7f9eb006ccd0 2024-03-29T04:12:44.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.395+0000 7f9ead7fa640 1 -- 172.21.15.148:0/4197822339 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9ea003b3f0 con 0x7f9eb006ccd0 2024-03-29T04:12:44.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.395+0000 7f9ead7fa640 1 --2- 172.21.15.148:0/4197822339 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9e9c06c530 0x7f9e9c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:44.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.395+0000 7f9ead7fa640 1 -- 172.21.15.148:0/4197822339 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f9ea002f030 con 0x7f9eb006ccd0 2024-03-29T04:12:44.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.395+0000 7f9eaffff640 1 --2- 172.21.15.148:0/4197822339 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9e9c06c530 0x7f9e9c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:44.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.395+0000 7f9eaffff640 1 --2- 172.21.15.148:0/4197822339 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9e9c06c530 0x7f9e9c06e9f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9eb015b540 tx=0x7f9ea4007b50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:44.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.399+0000 7f9ead7fa640 1 -- 172.21.15.148:0/4197822339 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9eb01f2c70 con 0x7f9eb006ccd0 2024-03-29T04:12:44.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.572+0000 7f9eb7172640 1 -- 172.21.15.148:0/4197822339 --> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9eb006b930 con 0x7f9e9c06c530 2024-03-29T04:12:44.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.573+0000 7f9ead7fa640 1 -- 172.21.15.148:0/4197822339 <== mgr.4113 v2:172.21.15.148:6832/3271445075 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13521 (secure 0 0 0) 0x7f9eb006b930 con 0x7f9e9c06c530 2024-03-29T04:12:44.552 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:12:44.553 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-03-29T04:12:44.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.575+0000 7f9eb7172640 1 -- 172.21.15.148:0/4197822339 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9e9c06c530 msgr2=0x7f9e9c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:44.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.575+0000 7f9eb7172640 1 --2- 172.21.15.148:0/4197822339 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9e9c06c530 0x7f9e9c06e9f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9eb015b540 tx=0x7f9ea4007b50 comp rx=0 tx=0).stop 2024-03-29T04:12:44.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.575+0000 7f9eb7172640 1 -- 172.21.15.148:0/4197822339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb006ccd0 msgr2=0x7f9eb00b4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:44.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.575+0000 7f9eb7172640 1 --2- 172.21.15.148:0/4197822339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb006ccd0 0x7f9eb00b4c80 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00097a0 tx=0x7f9ea00046c0 comp rx=0 tx=0).stop 2024-03-29T04:12:44.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.575+0000 7f9eb7172640 1 -- 172.21.15.148:0/4197822339 shutdown_connections 2024-03-29T04:12:44.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.575+0000 7f9eb7172640 1 --2- 172.21.15.148:0/4197822339 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9e9c06c530 0x7f9e9c06e9f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:44.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.575+0000 7f9eb7172640 1 --2- 172.21.15.148:0/4197822339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb015d210 0x7f9eb01f2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:44.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.575+0000 7f9eb7172640 1 --2- 172.21.15.148:0/4197822339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9eb00cf780 0x7f9eb01f1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:44.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.575+0000 7f9eb7172640 1 --2- 172.21.15.148:0/4197822339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb006ccd0 0x7f9eb00b4c80 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:44.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.575+0000 7f9eb7172640 1 -- 172.21.15.148:0/4197822339 >> 172.21.15.148:0/4197822339 conn(0x7f9eb00cfed0 msgr2=0x7f9eb015de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:44.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.575+0000 7f9eb7172640 1 -- 172.21.15.148:0/4197822339 shutdown_connections 2024-03-29T04:12:44.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.575+0000 7f9eb7172640 1 -- 172.21.15.148:0/4197822339 wait complete. 2024-03-29T04:12:44.565 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2024-03-29T04:12:43.912975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":578632,"kb_used_data":345832,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754396088,"statfs":{"total":773094113280,"available":772501594112,"internally_reserved":0,"allocated":354131968,"data_stored":351525108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-174309572,"num_objects":-138,"num_object_clones":0,"num_object_copies":-276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-138,"num_whiteouts":0,"num_read":-43,"num_read_kb":-2,"num_write":-334,"num_write_kb":-170317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001125"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":213,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-29T04:12:39.522742+0000","last_change":"2024-03-29T04:09:49.002570+0000","last_active":"2024-03-29T04:12:39.522742+0000","last_peered":"2024-03-29T04:12:39.522742+0000","last_clean":"2024-03-29T04:12:39.522742+0000","last_became_active":"2024-03-29T04:09:49.002235+0000","last_became_peered":"2024-03-29T04:09:49.002235+0000","last_unstale":"2024-03-29T04:12:39.522742+0000","last_undegraded":"2024-03-29T04:12:39.522742+0000","last_fullsized":"2024-03-29T04:12:39.522742+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:47.900733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:47.900733+0000","last_clean_scrub_stamp":"2024-03-29T04:09:47.900733+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:44:04.608265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94404,"kb_used_data":63224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94277436,"statfs":{"total":96636764160,"available":96540094464,"internally_reserved":0,"allocated":64741376,"data_stored":64417814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73612,"kb_used_data":42432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94298228,"statfs":{"total":96636764160,"available":96561385472,"internally_reserved":0,"allocated":43450368,"data_stored":43120765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":101708,"kb_used_data":70528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94270132,"statfs":{"total":96636764160,"available":96532615168,"internally_reserved":0,"allocated":72220672,"data_stored":71880611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48488,"kb_used_data":17308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323352,"statfs":{"total":96636764160,"available":96587112448,"internally_reserved":0,"allocated":17723392,"data_stored":17400625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82088,"kb_used_data":55068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94289752,"statfs":{"total":96636764160,"available":96552706048,"internally_reserved":0,"allocated":56389632,"data_stored":56041135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77168,"kb_used_data":50148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94294672,"statfs":{"total":96636764160,"available":96557744128,"internally_reserved":0,"allocated":51351552,"data_stored":51036602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40008,"kb_used_data":12988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331832,"statfs":{"total":96636764160,"available":96595795968,"internally_reserved":0,"allocated":13299712,"data_stored":13000093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61156,"kb_used_data":34136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94310684,"statfs":{"total":96636764160,"available":96574140416,"internally_reserved":0,"allocated":34955264,"data_stored":34627463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:12:44.566 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:12:44.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.687+0000 7f364c24e640 1 Processor -- start 2024-03-29T04:12:44.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.688+0000 7f364c24e640 1 -- start start 2024-03-29T04:12:44.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.688+0000 7f364c24e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36440ba030 0x7f36440d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:44.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.688+0000 7f364c24e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f364415e7c0 con 0x7f36440d10d0 2024-03-29T04:12:44.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.688+0000 7f364c24e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36440d2c30 con 0x7f364415e3e0 2024-03-29T04:12:44.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.688+0000 7f364c24e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f36440d3840 con 0x7f36440ba030 2024-03-29T04:12:44.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.689+0000 7f364a7eb640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f36440d10d0 0x7f3644167f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39890/0 (socket says 172.21.15.148:39890) 2024-03-29T04:12:44.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.689+0000 7f364a7eb640 1 -- 172.21.15.148:0/1081969772 learned_addr learned my addr 172.21.15.148:0/1081969772 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:44.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.689+0000 7f36497e9640 1 --2- 172.21.15.148:0/1081969772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36440ba030 0x7f36440d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:44.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.689+0000 7f3648fe8640 1 -- 172.21.15.148:0/1081969772 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1362287521 0 0) 0x7f364415e7c0 con 0x7f36440d10d0 2024-03-29T04:12:44.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.689+0000 7f3648fe8640 1 -- 172.21.15.148:0/1081969772 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3630003520 con 0x7f36440d10d0 2024-03-29T04:12:44.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.689+0000 7f3648fe8640 1 -- 172.21.15.148:0/1081969772 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1511470131 0 0) 0x7f36440d2c30 con 0x7f364415e3e0 2024-03-29T04:12:44.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.689+0000 7f3648fe8640 1 -- 172.21.15.148:0/1081969772 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f364415e7c0 con 0x7f364415e3e0 2024-03-29T04:12:44.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.689+0000 7f3648fe8640 1 -- 172.21.15.148:0/1081969772 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3103922750 0 0) 0x7f3630003520 con 0x7f36440d10d0 2024-03-29T04:12:44.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.689+0000 7f3648fe8640 1 -- 172.21.15.148:0/1081969772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36440ba030 msgr2=0x7f36440d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:44.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.689+0000 7f3648fe8640 1 --2- 172.21.15.148:0/1081969772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36440ba030 0x7f36440d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:44.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.689+0000 7f3648fe8640 1 -- 172.21.15.148:0/1081969772 >> v1:172.21.15.157:6789/0 conn(0x7f364415e3e0 legacy=0x7f36440d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:44.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.689+0000 7f3648fe8640 1 -- 172.21.15.148:0/1081969772 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36440d2f10 con 0x7f36440d10d0 2024-03-29T04:12:44.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.689+0000 7f3648fe8640 1 -- 172.21.15.148:0/1081969772 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f363c001670 con 0x7f36440d10d0 2024-03-29T04:12:44.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.689+0000 7f3648fe8640 1 -- 172.21.15.148:0/1081969772 >> v1:172.21.15.148:6789/0 conn(0x7f36440d10d0 legacy=0x7f3644167f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:44.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.689+0000 7f3648fe8640 1 --2- 172.21.15.148:0/1081969772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36440ba030 0x7f3630003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:44.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.689+0000 7f3648fe8640 1 --2- 172.21.15.148:0/1081969772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36300040d0 0x7f3630004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:44.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.689+0000 7f3648fe8640 1 --2- 172.21.15.148:0/1081969772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3630004d40 0x7f36300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:44.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.689+0000 7f3648fe8640 1 -- 172.21.15.148:0/1081969772 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f36440d3840 con 0x7f36300040d0 2024-03-29T04:12:44.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.689+0000 7f3648fe8640 1 -- 172.21.15.148:0/1081969772 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f363000baa0 con 0x7f3630004d40 2024-03-29T04:12:44.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.689+0000 7f3648fe8640 1 -- 172.21.15.148:0/1081969772 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f363000bca0 con 0x7f36440ba030 2024-03-29T04:12:44.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.689+0000 7f36497e9640 1 --2- 172.21.15.148:0/1081969772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36300040d0 0x7f3630004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:44.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.689+0000 7f3649fea640 1 --2- 172.21.15.148:0/1081969772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36440ba030 0x7f3630003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:44.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.689+0000 7f364a7eb640 1 --2- 172.21.15.148:0/1081969772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3630004d40 0x7f36300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:44.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.690+0000 7f36497e9640 1 -- 172.21.15.148:0/1081969772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36440ba030 msgr2=0x7f3630003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:44.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.690+0000 7f36497e9640 1 --2- 172.21.15.148:0/1081969772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36440ba030 0x7f3630003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:44.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.690+0000 7f36497e9640 1 -- 172.21.15.148:0/1081969772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3630004d40 msgr2=0x7f36300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:44.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.690+0000 7f36497e9640 1 --2- 172.21.15.148:0/1081969772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3630004d40 0x7f36300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:44.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.690+0000 7f36497e9640 1 -- 172.21.15.148:0/1081969772 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3630003520 con 0x7f36300040d0 2024-03-29T04:12:44.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.690+0000 7f36497e9640 1 --2- 172.21.15.148:0/1081969772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36300040d0 0x7f3630004800 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3634009ac0 tx=0x7f3630005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e3a54ba76620a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:44.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.690+0000 7f3648fe8640 1 -- 172.21.15.148:0/1081969772 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3634039070 con 0x7f36300040d0 2024-03-29T04:12:44.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.690+0000 7f3648fe8640 1 -- 172.21.15.148:0/1081969772 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f363402c6b0 con 0x7f36300040d0 2024-03-29T04:12:44.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.690+0000 7f3648fe8640 1 -- 172.21.15.148:0/1081969772 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3634009690 con 0x7f36300040d0 2024-03-29T04:12:44.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.691+0000 7f364c24e640 1 -- 172.21.15.148:0/1081969772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36300040d0 msgr2=0x7f3630004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:44.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.691+0000 7f364c24e640 1 --2- 172.21.15.148:0/1081969772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36300040d0 0x7f3630004800 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3634009ac0 tx=0x7f3630005830 comp rx=0 tx=0).stop 2024-03-29T04:12:44.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.691+0000 7f364a7eb640 1 -- 172.21.15.148:0/1081969772 reap_dead start 2024-03-29T04:12:44.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.691+0000 7f364c24e640 1 -- 172.21.15.148:0/1081969772 shutdown_connections 2024-03-29T04:12:44.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.691+0000 7f364c24e640 1 -- 172.21.15.148:0/1081969772 >> 172.21.15.148:0/1081969772 conn(0x7f36440b6210 msgr2=0x7f36440b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:44.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.691+0000 7f364c24e640 1 -- 172.21.15.148:0/1081969772 shutdown_connections 2024-03-29T04:12:44.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.691+0000 7f364c24e640 1 -- 172.21.15.148:0/1081969772 wait complete. 2024-03-29T04:12:44.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.692+0000 7f364c24e640 1 Processor -- start 2024-03-29T04:12:44.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.692+0000 7f364c24e640 1 -- start start 2024-03-29T04:12:44.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.693+0000 7f364c24e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36300040d0 0x7f36440cd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:44.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.693+0000 7f364c24e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36440c7d70 0x7f36440c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:44.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.693+0000 7f364c24e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36440cb3e0 0x7f36440cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:44.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.693+0000 7f364c24e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f36440d3cd0 con 0x7f36300040d0 2024-03-29T04:12:44.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.693+0000 7f364c24e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f36440d34d0 con 0x7f36440c7d70 2024-03-29T04:12:44.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.693+0000 7f364c24e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f36440d3e50 con 0x7f36440cb3e0 2024-03-29T04:12:44.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.693+0000 7f3649fea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36300040d0 0x7f36440cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:44.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.693+0000 7f3649fea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36300040d0 0x7f36440cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60798/0 (socket says 172.21.15.148:60798) 2024-03-29T04:12:44.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.693+0000 7f3649fea640 1 -- 172.21.15.148:0/3481701106 learned_addr learned my addr 172.21.15.148:0/3481701106 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:44.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.693+0000 7f36497e9640 1 --2- 172.21.15.148:0/3481701106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36440c7d70 0x7f36440c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:44.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.693+0000 7f364a7eb640 1 --2- 172.21.15.148:0/3481701106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36440cb3e0 0x7f36440cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:44.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.693+0000 7f3649fea640 1 -- 172.21.15.148:0/3481701106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36440cb3e0 msgr2=0x7f36440cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:44.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.693+0000 7f3649fea640 1 --2- 172.21.15.148:0/3481701106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36440cb3e0 0x7f36440cb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:44.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.693+0000 7f3649fea640 1 -- 172.21.15.148:0/3481701106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36440c7d70 msgr2=0x7f36440c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:44.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.693+0000 7f3649fea640 1 --2- 172.21.15.148:0/3481701106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36440c7d70 0x7f36440c81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:44.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.693+0000 7f3649fea640 1 -- 172.21.15.148:0/3481701106 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36440c8ae0 con 0x7f36300040d0 2024-03-29T04:12:44.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.694+0000 7f3649fea640 1 --2- 172.21.15.148:0/3481701106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36300040d0 0x7f36440cd240 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f364000fb40 tx=0x7f36400021e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:44.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.694+0000 7f363a7fc640 1 -- 172.21.15.148:0/3481701106 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3640003b80 con 0x7f36300040d0 2024-03-29T04:12:44.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.694+0000 7f363a7fc640 1 -- 172.21.15.148:0/3481701106 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3640018070 con 0x7f36300040d0 2024-03-29T04:12:44.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.694+0000 7f363a7fc640 1 -- 172.21.15.148:0/3481701106 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364000d910 con 0x7f36300040d0 2024-03-29T04:12:44.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.694+0000 7f364c24e640 1 -- 172.21.15.148:0/3481701106 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36440c8dc0 con 0x7f36300040d0 2024-03-29T04:12:44.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.694+0000 7f364c24e640 1 -- 172.21.15.148:0/3481701106 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36440c9660 con 0x7f36300040d0 2024-03-29T04:12:44.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.695+0000 7f364c24e640 1 -- 172.21.15.148:0/3481701106 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36440687e0 con 0x7f36300040d0 2024-03-29T04:12:44.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.695+0000 7f363a7fc640 1 -- 172.21.15.148:0/3481701106 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f364000dab0 con 0x7f36300040d0 2024-03-29T04:12:44.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.696+0000 7f363a7fc640 1 --2- 172.21.15.148:0/3481701106 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f362806c530 0x7f362806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:44.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.696+0000 7f363a7fc640 1 -- 172.21.15.148:0/3481701106 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f364008b870 con 0x7f36300040d0 2024-03-29T04:12:44.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.696+0000 7f36497e9640 1 --2- 172.21.15.148:0/3481701106 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f362806c530 0x7f362806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:44.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.696+0000 7f36497e9640 1 --2- 172.21.15.148:0/3481701106 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f362806c530 0x7f362806e9f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3634036000 tx=0x7f3634030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:44.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.700+0000 7f363a7fc640 1 -- 172.21.15.148:0/3481701106 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f364005a000 con 0x7f36300040d0 2024-03-29T04:12:44.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.871+0000 7f364c24e640 1 -- 172.21.15.148:0/3481701106 --> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f36440c9830 con 0x7f362806c530 2024-03-29T04:12:44.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.871+0000 7f363a7fc640 1 -- 172.21.15.148:0/3481701106 <== mgr.4113 v2:172.21.15.148:6832/3271445075 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13521 (secure 0 0 0) 0x7f36440c9830 con 0x7f362806c530 2024-03-29T04:12:44.851 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:12:44.852 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-03-29T04:12:44.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.873+0000 7f364c24e640 1 -- 172.21.15.148:0/3481701106 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f362806c530 msgr2=0x7f362806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:44.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.873+0000 7f364c24e640 1 --2- 172.21.15.148:0/3481701106 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f362806c530 0x7f362806e9f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3634036000 tx=0x7f3634030000 comp rx=0 tx=0).stop 2024-03-29T04:12:44.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.873+0000 7f364c24e640 1 -- 172.21.15.148:0/3481701106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36300040d0 msgr2=0x7f36440cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:44.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.873+0000 7f364c24e640 1 --2- 172.21.15.148:0/3481701106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36300040d0 0x7f36440cd240 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f364000fb40 tx=0x7f36400021e0 comp rx=0 tx=0).stop 2024-03-29T04:12:44.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.873+0000 7f364c24e640 1 -- 172.21.15.148:0/3481701106 shutdown_connections 2024-03-29T04:12:44.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.873+0000 7f364c24e640 1 --2- 172.21.15.148:0/3481701106 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f362806c530 0x7f362806e9f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:44.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.873+0000 7f364c24e640 1 --2- 172.21.15.148:0/3481701106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f36440cb3e0 0x7f36440cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:44.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.873+0000 7f364c24e640 1 --2- 172.21.15.148:0/3481701106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36440c7d70 0x7f36440c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:44.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.873+0000 7f364c24e640 1 --2- 172.21.15.148:0/3481701106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36300040d0 0x7f36440cd240 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:44.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.873+0000 7f364c24e640 1 -- 172.21.15.148:0/3481701106 >> 172.21.15.148:0/3481701106 conn(0x7f36440b6210 msgr2=0x7f36440b8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:44.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.873+0000 7f364c24e640 1 -- 172.21.15.148:0/3481701106 shutdown_connections 2024-03-29T04:12:44.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.873+0000 7f364c24e640 1 -- 172.21.15.148:0/3481701106 wait complete. 2024-03-29T04:12:44.864 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2024-03-29T04:12:43.912975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":578632,"kb_used_data":345832,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754396088,"statfs":{"total":773094113280,"available":772501594112,"internally_reserved":0,"allocated":354131968,"data_stored":351525108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-174309572,"num_objects":-138,"num_object_clones":0,"num_object_copies":-276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-138,"num_whiteouts":0,"num_read":-43,"num_read_kb":-2,"num_write":-334,"num_write_kb":-170317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001125"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":213,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-29T04:12:39.522742+0000","last_change":"2024-03-29T04:09:49.002570+0000","last_active":"2024-03-29T04:12:39.522742+0000","last_peered":"2024-03-29T04:12:39.522742+0000","last_clean":"2024-03-29T04:12:39.522742+0000","last_became_active":"2024-03-29T04:09:49.002235+0000","last_became_peered":"2024-03-29T04:09:49.002235+0000","last_unstale":"2024-03-29T04:12:39.522742+0000","last_undegraded":"2024-03-29T04:12:39.522742+0000","last_fullsized":"2024-03-29T04:12:39.522742+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:47.900733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:47.900733+0000","last_clean_scrub_stamp":"2024-03-29T04:09:47.900733+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:44:04.608265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94404,"kb_used_data":63224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94277436,"statfs":{"total":96636764160,"available":96540094464,"internally_reserved":0,"allocated":64741376,"data_stored":64417814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73612,"kb_used_data":42432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94298228,"statfs":{"total":96636764160,"available":96561385472,"internally_reserved":0,"allocated":43450368,"data_stored":43120765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":101708,"kb_used_data":70528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94270132,"statfs":{"total":96636764160,"available":96532615168,"internally_reserved":0,"allocated":72220672,"data_stored":71880611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48488,"kb_used_data":17308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323352,"statfs":{"total":96636764160,"available":96587112448,"internally_reserved":0,"allocated":17723392,"data_stored":17400625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82088,"kb_used_data":55068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94289752,"statfs":{"total":96636764160,"available":96552706048,"internally_reserved":0,"allocated":56389632,"data_stored":56041135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77168,"kb_used_data":50148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94294672,"statfs":{"total":96636764160,"available":96557744128,"internally_reserved":0,"allocated":51351552,"data_stored":51036602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40008,"kb_used_data":12988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331832,"statfs":{"total":96636764160,"available":96595795968,"internally_reserved":0,"allocated":13299712,"data_stored":13000093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61156,"kb_used_data":34136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94310684,"statfs":{"total":96636764160,"available":96574140416,"internally_reserved":0,"allocated":34955264,"data_stored":34627463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:12:44.864 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-29T04:12:44.865 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:12:44.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.985+0000 7fb3482e4640 1 Processor -- start 2024-03-29T04:12:44.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.985+0000 7fb3482e4640 1 -- start start 2024-03-29T04:12:44.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.986+0000 7fb3482e4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3400cf780 0x7fb3400ce610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:44.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.986+0000 7fb3482e4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb34015d210 0x7fb34006c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:44.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.986+0000 7fb3482e4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb34006cb00 0x7fb3400b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:44.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.986+0000 7fb3482e4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fb34006da30 con 0x7fb34006cb00 2024-03-29T04:12:44.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.986+0000 7fb3482e4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb34006dd10 con 0x7fb34015d210 2024-03-29T04:12:44.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.986+0000 7fb3482e4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb34006dff0 con 0x7fb3400cf780 2024-03-29T04:12:44.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.986+0000 7fb346881640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb34006cb00 0x7fb3400b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:44.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.986+0000 7fb346881640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb34006cb00 0x7fb3400b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60814/0 (socket says 172.21.15.148:60814) 2024-03-29T04:12:44.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.986+0000 7fb346881640 1 -- 172.21.15.148:0/2880235614 learned_addr learned my addr 172.21.15.148:0/2880235614 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:44.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.986+0000 7fb346080640 1 --2- 172.21.15.148:0/2880235614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3400cf780 0x7fb3400ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:44.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.986+0000 7fb34587f640 1 --2- 172.21.15.148:0/2880235614 >> v2:172.21.15.157:3300/0 conn(0x7fb34015d210 0x7fb34006c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:44.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.987+0000 7fb346881640 1 -- 172.21.15.148:0/2880235614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3400cf780 msgr2=0x7fb3400ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:44.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.987+0000 7fb346881640 1 --2- 172.21.15.148:0/2880235614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3400cf780 0x7fb3400ce610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:44.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.987+0000 7fb346881640 1 -- 172.21.15.148:0/2880235614 >> v2:172.21.15.157:3300/0 conn(0x7fb34015d210 msgr2=0x7fb34006c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:44.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.987+0000 7fb346881640 1 --2- 172.21.15.148:0/2880235614 >> v2:172.21.15.157:3300/0 conn(0x7fb34015d210 0x7fb34006c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:44.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.987+0000 7fb346881640 1 -- 172.21.15.148:0/2880235614 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34006e2d0 con 0x7fb34006cb00 2024-03-29T04:12:44.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.987+0000 7fb346881640 1 --2- 172.21.15.148:0/2880235614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb34006cb00 0x7fb3400b3820 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fb3400d0f10 tx=0x7fb33802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f86e05ba9cbfe85f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:44.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.987+0000 7fb34507e640 1 -- 172.21.15.148:0/2880235614 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33802e070 con 0x7fb34006cb00 2024-03-29T04:12:44.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.988+0000 7fb34507e640 1 -- 172.21.15.148:0/2880235614 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb338037070 con 0x7fb34006cb00 2024-03-29T04:12:44.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.988+0000 7fb34507e640 1 -- 172.21.15.148:0/2880235614 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb338031a00 con 0x7fb34006cb00 2024-03-29T04:12:44.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.988+0000 7fb3482e4640 1 -- 172.21.15.148:0/2880235614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb34006cb00 msgr2=0x7fb3400b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:44.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.988+0000 7fb3482e4640 1 --2- 172.21.15.148:0/2880235614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb34006cb00 0x7fb3400b3820 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fb3400d0f10 tx=0x7fb33802b880 comp rx=0 tx=0).stop 2024-03-29T04:12:44.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.988+0000 7fb3482e4640 1 -- 172.21.15.148:0/2880235614 shutdown_connections 2024-03-29T04:12:44.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.988+0000 7fb3482e4640 1 --2- 172.21.15.148:0/2880235614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb34006cb00 0x7fb3400b3820 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:44.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.988+0000 7fb3482e4640 1 --2- 172.21.15.148:0/2880235614 >> v2:172.21.15.157:3300/0 conn(0x7fb34015d210 0x7fb34006c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:44.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.988+0000 7fb3482e4640 1 --2- 172.21.15.148:0/2880235614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3400cf780 0x7fb3400ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:44.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.988+0000 7fb3482e4640 1 -- 172.21.15.148:0/2880235614 >> 172.21.15.148:0/2880235614 conn(0x7fb3400cfed0 msgr2=0x7fb3400d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:44.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.988+0000 7fb3482e4640 1 -- 172.21.15.148:0/2880235614 shutdown_connections 2024-03-29T04:12:44.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.988+0000 7fb3482e4640 1 -- 172.21.15.148:0/2880235614 wait complete. 2024-03-29T04:12:44.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.989+0000 7fb3482e4640 1 Processor -- start 2024-03-29T04:12:44.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.989+0000 7fb3482e4640 1 -- start start 2024-03-29T04:12:44.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.990+0000 7fb3482e4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb34006cb00 0x7fb34014e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:44.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.990+0000 7fb3482e4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3400cf780 0x7fb34014c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:44.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.990+0000 7fb3482e4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb34015d210 0x7fb34014bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:44.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.990+0000 7fb3482e4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb34006d5c0 con 0x7fb3400cf780 2024-03-29T04:12:44.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.990+0000 7fb3482e4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb34006f340 con 0x7fb34006cb00 2024-03-29T04:12:44.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.990+0000 7fb3482e4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb34006d740 con 0x7fb34015d210 2024-03-29T04:12:44.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.990+0000 7fb34587f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3400cf780 0x7fb34014c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:44.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.990+0000 7fb34587f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3400cf780 0x7fb34014c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60830/0 (socket says 172.21.15.148:60830) 2024-03-29T04:12:44.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.990+0000 7fb34587f640 1 -- 172.21.15.148:0/658168049 learned_addr learned my addr 172.21.15.148:0/658168049 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:44.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.990+0000 7fb346080640 1 --2- 172.21.15.148:0/658168049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb34006cb00 0x7fb34014e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:44.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.990+0000 7fb346881640 1 --2- 172.21.15.148:0/658168049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb34015d210 0x7fb34014bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:44.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.990+0000 7fb34587f640 1 -- 172.21.15.148:0/658168049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb34015d210 msgr2=0x7fb34014bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:44.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.990+0000 7fb34587f640 1 --2- 172.21.15.148:0/658168049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb34015d210 0x7fb34014bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:44.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.990+0000 7fb34587f640 1 -- 172.21.15.148:0/658168049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb34006cb00 msgr2=0x7fb34014e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:44.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.990+0000 7fb34587f640 1 --2- 172.21.15.148:0/658168049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb34006cb00 0x7fb34014e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:44.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.990+0000 7fb34587f640 1 -- 172.21.15.148:0/658168049 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34014c490 con 0x7fb3400cf780 2024-03-29T04:12:44.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.990+0000 7fb346080640 1 --2- 172.21.15.148:0/658168049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb34006cb00 0x7fb34014e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:44.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.991+0000 7fb34587f640 1 --2- 172.21.15.148:0/658168049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3400cf780 0x7fb34014c6c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb33000e9b0 tx=0x7fb33000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:44.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.991+0000 7fb336ffd640 1 -- 172.21.15.148:0/658168049 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb330017070 con 0x7fb3400cf780 2024-03-29T04:12:44.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.991+0000 7fb336ffd640 1 -- 172.21.15.148:0/658168049 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb33000c9a0 con 0x7fb3400cf780 2024-03-29T04:12:44.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.991+0000 7fb3482e4640 1 -- 172.21.15.148:0/658168049 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb34014cdc0 con 0x7fb3400cf780 2024-03-29T04:12:44.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.991+0000 7fb336ffd640 1 -- 172.21.15.148:0/658168049 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33000cd90 con 0x7fb3400cf780 2024-03-29T04:12:44.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.991+0000 7fb3482e4640 1 -- 172.21.15.148:0/658168049 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb34014d380 con 0x7fb3400cf780 2024-03-29T04:12:44.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.992+0000 7fb3482e4640 1 -- 172.21.15.148:0/658168049 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3400687c0 con 0x7fb3400cf780 2024-03-29T04:12:44.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.992+0000 7fb336ffd640 1 -- 172.21.15.148:0/658168049 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb33001a070 con 0x7fb3400cf780 2024-03-29T04:12:44.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.993+0000 7fb336ffd640 1 --2- 172.21.15.148:0/658168049 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb32406c530 0x7fb32406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:44.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.993+0000 7fb336ffd640 1 -- 172.21.15.148:0/658168049 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7fb33008b960 con 0x7fb3400cf780 2024-03-29T04:12:44.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.993+0000 7fb346080640 1 --2- 172.21.15.148:0/658168049 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb32406c530 0x7fb32406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:44.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.993+0000 7fb346080640 1 --2- 172.21.15.148:0/658168049 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb32406c530 0x7fb32406e9f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009fd0 tx=0x7fb33c00d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:44.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:44.997+0000 7fb336ffd640 1 -- 172.21.15.148:0/658168049 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb33005a0f0 con 0x7fb3400cf780 2024-03-29T04:12:45.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.168+0000 7fb3482e4640 1 -- 172.21.15.148:0/658168049 --> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb3400cfc90 con 0x7fb32406c530 2024-03-29T04:12:45.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.169+0000 7fb336ffd640 1 -- 172.21.15.148:0/658168049 <== mgr.4113 v2:172.21.15.148:6832/3271445075 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13521 (secure 0 0 0) 0x7fb33c0204f0 con 0x7fb32406c530 2024-03-29T04:12:45.149 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:12:45.150 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-03-29T04:12:45.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.171+0000 7fb3482e4640 1 -- 172.21.15.148:0/658168049 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb32406c530 msgr2=0x7fb32406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:45.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.171+0000 7fb3482e4640 1 --2- 172.21.15.148:0/658168049 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb32406c530 0x7fb32406e9f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009fd0 tx=0x7fb33c00d040 comp rx=0 tx=0).stop 2024-03-29T04:12:45.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.171+0000 7fb3482e4640 1 -- 172.21.15.148:0/658168049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3400cf780 msgr2=0x7fb34014c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:45.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.171+0000 7fb3482e4640 1 --2- 172.21.15.148:0/658168049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3400cf780 0x7fb34014c6c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb33000e9b0 tx=0x7fb33000ee70 comp rx=0 tx=0).stop 2024-03-29T04:12:45.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.171+0000 7fb3482e4640 1 -- 172.21.15.148:0/658168049 shutdown_connections 2024-03-29T04:12:45.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.171+0000 7fb3482e4640 1 --2- 172.21.15.148:0/658168049 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb32406c530 0x7fb32406e9f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:45.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.171+0000 7fb3482e4640 1 --2- 172.21.15.148:0/658168049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb34015d210 0x7fb34014bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:45.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.171+0000 7fb3482e4640 1 --2- 172.21.15.148:0/658168049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3400cf780 0x7fb34014c6c0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:45.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.171+0000 7fb3482e4640 1 --2- 172.21.15.148:0/658168049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb34006cb00 0x7fb34014e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:45.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.171+0000 7fb3482e4640 1 -- 172.21.15.148:0/658168049 >> 172.21.15.148:0/658168049 conn(0x7fb3400cfed0 msgr2=0x7fb340159a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:45.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.171+0000 7fb3482e4640 1 -- 172.21.15.148:0/658168049 shutdown_connections 2024-03-29T04:12:45.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.171+0000 7fb3482e4640 1 -- 172.21.15.148:0/658168049 wait complete. 2024-03-29T04:12:45.162 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2024-03-29T04:12:43.912975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":578632,"kb_used_data":345832,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754396088,"statfs":{"total":773094113280,"available":772501594112,"internally_reserved":0,"allocated":354131968,"data_stored":351525108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-174309572,"num_objects":-138,"num_object_clones":0,"num_object_copies":-276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-138,"num_whiteouts":0,"num_read":-43,"num_read_kb":-2,"num_write":-334,"num_write_kb":-170317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001125"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":213,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-29T04:12:39.522742+0000","last_change":"2024-03-29T04:09:49.002570+0000","last_active":"2024-03-29T04:12:39.522742+0000","last_peered":"2024-03-29T04:12:39.522742+0000","last_clean":"2024-03-29T04:12:39.522742+0000","last_became_active":"2024-03-29T04:09:49.002235+0000","last_became_peered":"2024-03-29T04:09:49.002235+0000","last_unstale":"2024-03-29T04:12:39.522742+0000","last_undegraded":"2024-03-29T04:12:39.522742+0000","last_fullsized":"2024-03-29T04:12:39.522742+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:47.900733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:47.900733+0000","last_clean_scrub_stamp":"2024-03-29T04:09:47.900733+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:44:04.608265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94404,"kb_used_data":63224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94277436,"statfs":{"total":96636764160,"available":96540094464,"internally_reserved":0,"allocated":64741376,"data_stored":64417814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73612,"kb_used_data":42432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94298228,"statfs":{"total":96636764160,"available":96561385472,"internally_reserved":0,"allocated":43450368,"data_stored":43120765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":101708,"kb_used_data":70528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94270132,"statfs":{"total":96636764160,"available":96532615168,"internally_reserved":0,"allocated":72220672,"data_stored":71880611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48488,"kb_used_data":17308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323352,"statfs":{"total":96636764160,"available":96587112448,"internally_reserved":0,"allocated":17723392,"data_stored":17400625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82088,"kb_used_data":55068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94289752,"statfs":{"total":96636764160,"available":96552706048,"internally_reserved":0,"allocated":56389632,"data_stored":56041135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77168,"kb_used_data":50148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94294672,"statfs":{"total":96636764160,"available":96557744128,"internally_reserved":0,"allocated":51351552,"data_stored":51036602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40008,"kb_used_data":12988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331832,"statfs":{"total":96636764160,"available":96595795968,"internally_reserved":0,"allocated":13299712,"data_stored":13000093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61156,"kb_used_data":34136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94310684,"statfs":{"total":96636764160,"available":96574140416,"internally_reserved":0,"allocated":34955264,"data_stored":34627463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:12:45.162 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:12:45.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.284+0000 7fc73950e640 1 Processor -- start 2024-03-29T04:12:45.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.284+0000 7fc73950e640 1 -- start start 2024-03-29T04:12:45.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.285+0000 7fc73950e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc73415b060 0x7fc7340b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:45.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.285+0000 7fc73950e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc73406c5c0 0x7fc73406ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:45.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.285+0000 7fc73950e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7340b3630 con 0x7fc73415f6d0 2024-03-29T04:12:45.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.285+0000 7fc73950e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc7340b3910 con 0x7fc73415b060 2024-03-29T04:12:45.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.285+0000 7fc73950e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc7340b3bf0 con 0x7fc73406c5c0 2024-03-29T04:12:45.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.285+0000 7fc7337fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc73406c5c0 0x7fc73406ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:45.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.285+0000 7fc732ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc73415b060 0x7fc7340b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:45.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.285+0000 7fc7337fe640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc73406c5c0 0x7fc73406ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.148:41584/0 (socket says 172.21.15.148:41584) 2024-03-29T04:12:45.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.285+0000 7fc7337fe640 1 -- 172.21.15.148:0/2480742444 learned_addr learned my addr 172.21.15.148:0/2480742444 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:45.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.285+0000 7fc7327fc640 1 --1- 172.21.15.148:0/2480742444 >> v1:172.21.15.157:6789/0 conn(0x7fc73415f6d0 0x7fc7340b46d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.148:38958/0 (socket says 172.21.15.148:38958) 2024-03-29T04:12:45.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.285+0000 7fc732ffd640 1 -- 172.21.15.148:0/2480742444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc73406c5c0 msgr2=0x7fc73406ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:45.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.285+0000 7fc732ffd640 1 --2- 172.21.15.148:0/2480742444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc73406c5c0 0x7fc73406ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:45.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.285+0000 7fc732ffd640 1 -- 172.21.15.148:0/2480742444 >> v1:172.21.15.157:6789/0 conn(0x7fc73415f6d0 legacy=0x7fc7340b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:45.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.285+0000 7fc732ffd640 1 -- 172.21.15.148:0/2480742444 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7340b3ed0 con 0x7fc73415b060 2024-03-29T04:12:45.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.285+0000 7fc7337fe640 1 --2- 172.21.15.148:0/2480742444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc73406c5c0 0x7fc73406ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:12:45.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.286+0000 7fc732ffd640 1 --2- 172.21.15.148:0/2480742444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc73415b060 0x7fc7340b2ee0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc724009d90 tx=0x7fc72402df90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a070f8e6302301b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:45.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.286+0000 7fc731ffb640 1 -- 172.21.15.148:0/2480742444 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72403b070 con 0x7fc73415b060 2024-03-29T04:12:45.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.286+0000 7fc731ffb640 1 -- 172.21.15.148:0/2480742444 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc72402ed40 con 0x7fc73415b060 2024-03-29T04:12:45.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.286+0000 7fc731ffb640 1 -- 172.21.15.148:0/2480742444 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724036660 con 0x7fc73415b060 2024-03-29T04:12:45.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.287+0000 7fc73950e640 1 -- 172.21.15.148:0/2480742444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc73415b060 msgr2=0x7fc7340b2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:45.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.287+0000 7fc73950e640 1 --2- 172.21.15.148:0/2480742444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc73415b060 0x7fc7340b2ee0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc724009d90 tx=0x7fc72402df90 comp rx=0 tx=0).stop 2024-03-29T04:12:45.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.287+0000 7fc73950e640 1 -- 172.21.15.148:0/2480742444 shutdown_connections 2024-03-29T04:12:45.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.287+0000 7fc73950e640 1 --2- 172.21.15.148:0/2480742444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc73406c5c0 0x7fc73406ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:45.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.287+0000 7fc73950e640 1 --2- 172.21.15.148:0/2480742444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc73415b060 0x7fc7340b2ee0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:45.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.287+0000 7fc73950e640 1 -- 172.21.15.148:0/2480742444 >> 172.21.15.148:0/2480742444 conn(0x7fc7340cfed0 msgr2=0x7fc7340d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:45.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.287+0000 7fc73950e640 1 -- 172.21.15.148:0/2480742444 shutdown_connections 2024-03-29T04:12:45.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.287+0000 7fc73950e640 1 -- 172.21.15.148:0/2480742444 wait complete. 2024-03-29T04:12:45.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.288+0000 7fc73950e640 1 Processor -- start 2024-03-29T04:12:45.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.288+0000 7fc73950e640 1 -- start start 2024-03-29T04:12:45.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.289+0000 7fc73950e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc73406c5c0 0x7fc73414ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:45.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.289+0000 7fc73950e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc73415b060 0x7fc73415d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:45.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.289+0000 7fc73950e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc73415f6d0 0x7fc73415dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:45.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.289+0000 7fc73950e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc73415b440 con 0x7fc73415f6d0 2024-03-29T04:12:45.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.289+0000 7fc73950e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc73415fb30 con 0x7fc73406c5c0 2024-03-29T04:12:45.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.289+0000 7fc73950e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc7340cd530 con 0x7fc73415b060 2024-03-29T04:12:45.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.289+0000 7fc7337fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc73415f6d0 0x7fc73415dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:45.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.289+0000 7fc7337fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc73415f6d0 0x7fc73415dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60846/0 (socket says 172.21.15.148:60846) 2024-03-29T04:12:45.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.289+0000 7fc7337fe640 1 -- 172.21.15.148:0/2989891924 learned_addr learned my addr 172.21.15.148:0/2989891924 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:45.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.289+0000 7fc732ffd640 1 --2- 172.21.15.148:0/2989891924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc73406c5c0 0x7fc73414ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:45.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.289+0000 7fc7327fc640 1 --2- 172.21.15.148:0/2989891924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc73415b060 0x7fc73415d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:45.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.289+0000 7fc7337fe640 1 -- 172.21.15.148:0/2989891924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc73415b060 msgr2=0x7fc73415d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:45.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.289+0000 7fc7337fe640 1 --2- 172.21.15.148:0/2989891924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc73415b060 0x7fc73415d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:45.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.289+0000 7fc7337fe640 1 -- 172.21.15.148:0/2989891924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc73406c5c0 msgr2=0x7fc73414ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:45.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.289+0000 7fc7337fe640 1 --2- 172.21.15.148:0/2989891924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc73406c5c0 0x7fc73414ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:45.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.289+0000 7fc7337fe640 1 -- 172.21.15.148:0/2989891924 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc73414d550 con 0x7fc73415f6d0 2024-03-29T04:12:45.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.289+0000 7fc7327fc640 1 --2- 172.21.15.148:0/2989891924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc73415b060 0x7fc73415d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:45.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.289+0000 7fc732ffd640 1 --2- 172.21.15.148:0/2989891924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc73406c5c0 0x7fc73414ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:45.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.290+0000 7fc7337fe640 1 --2- 172.21.15.148:0/2989891924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc73415f6d0 0x7fc73415dac0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00e960 tx=0x7fc72c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:45.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.290+0000 7fc71b7fe640 1 -- 172.21.15.148:0/2989891924 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c017070 con 0x7fc73415f6d0 2024-03-29T04:12:45.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.290+0000 7fc71b7fe640 1 -- 172.21.15.148:0/2989891924 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc72c004020 con 0x7fc73415f6d0 2024-03-29T04:12:45.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.290+0000 7fc71b7fe640 1 -- 172.21.15.148:0/2989891924 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c0043f0 con 0x7fc73415f6d0 2024-03-29T04:12:45.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.290+0000 7fc73950e640 1 -- 172.21.15.148:0/2989891924 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc73414d830 con 0x7fc73415f6d0 2024-03-29T04:12:45.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.290+0000 7fc73950e640 1 -- 172.21.15.148:0/2989891924 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc73414b280 con 0x7fc73415f6d0 2024-03-29T04:12:45.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.291+0000 7fc73950e640 1 -- 172.21.15.148:0/2989891924 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc73406cc30 con 0x7fc73415f6d0 2024-03-29T04:12:45.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.291+0000 7fc71b7fe640 1 -- 172.21.15.148:0/2989891924 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc72c00ca00 con 0x7fc73415f6d0 2024-03-29T04:12:45.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.292+0000 7fc71b7fe640 1 --2- 172.21.15.148:0/2989891924 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc72006c530 0x7fc72006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:45.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.292+0000 7fc71b7fe640 1 -- 172.21.15.148:0/2989891924 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7fc72c08b370 con 0x7fc73415f6d0 2024-03-29T04:12:45.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.292+0000 7fc732ffd640 1 --2- 172.21.15.148:0/2989891924 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc72006c530 0x7fc72006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:45.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.292+0000 7fc732ffd640 1 --2- 172.21.15.148:0/2989891924 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc72006c530 0x7fc72006e9f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc724038000 tx=0x7fc724032000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:45.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.296+0000 7fc71b7fe640 1 -- 172.21.15.148:0/2989891924 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc73406cc30 con 0x7fc73415f6d0 2024-03-29T04:12:45.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.474+0000 7fc73950e640 1 -- 172.21.15.148:0/2989891924 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc734155bb0 con 0x7fc73415f6d0 2024-03-29T04:12:45.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.474+0000 7fc71b7fe640 1 -- 172.21.15.148:0/2989891924 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+13305 (secure 0 0 0) 0x7fc72c059b90 con 0x7fc73415f6d0 2024-03-29T04:12:45.454 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:12:45.454 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":44,"fsid":"56b659bb-6db8-45e8-b612-135349c3fd6b","created":"2024-03-29T04:09:35.655243+0000","modified":"2024-03-29T04:12:42.233024+0000","last_up_change":"2024-03-29T04:09:46.294752+0000","last_in_change":"2024-03-29T04:09:37.971465+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:47.898351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"443b9c39-6bbf-4ed3-92a1-914d9f9cc53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6825","nonce":1406601759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6827","nonce":1406601759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6831","nonce":1406601759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1406601759},{"type":"v1","addr":"172.21.15.148:6829","nonce":1406601759}]},"public_addr":"172.21.15.148:6825/1406601759","cluster_addr":"172.21.15.148:6827/1406601759","heartbeat_back_addr":"172.21.15.148:6831/1406601759","heartbeat_front_addr":"172.21.15.148:6829/1406601759","state":["exists","up"]},{"osd":1,"uuid":"aa9f99b2-59d6-4e0f-bb8a-c4eae3be1f42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6801","nonce":686342976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6803","nonce":686342976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6807","nonce":686342976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":686342976},{"type":"v1","addr":"172.21.15.148:6805","nonce":686342976}]},"public_addr":"172.21.15.148:6801/686342976","cluster_addr":"172.21.15.148:6803/686342976","heartbeat_back_addr":"172.21.15.148:6807/686342976","heartbeat_front_addr":"172.21.15.148:6805/686342976","state":["exists","up"]},{"osd":2,"uuid":"63302fb8-9a90-4da0-a37a-a6470547e394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6809","nonce":90026237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6811","nonce":90026237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6815","nonce":90026237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":90026237},{"type":"v1","addr":"172.21.15.148:6813","nonce":90026237}]},"public_addr":"172.21.15.148:6809/90026237","cluster_addr":"172.21.15.148:6811/90026237","heartbeat_back_addr":"172.21.15.148:6815/90026237","heartbeat_front_addr":"172.21.15.148:6813/90026237","state":["exists","up"]},{"osd":3,"uuid":"a2cc77fe-acf1-4f34-8cdb-b9eb007c93fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6817","nonce":466300755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6819","nonce":466300755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6823","nonce":466300755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":466300755},{"type":"v1","addr":"172.21.15.148:6821","nonce":466300755}]},"public_addr":"172.21.15.148:6817/466300755","cluster_addr":"172.21.15.148:6819/466300755","heartbeat_back_addr":"172.21.15.148:6823/466300755","heartbeat_front_addr":"172.21.15.148:6821/466300755","state":["exists","up"]},{"osd":4,"uuid":"96c44823-5bd3-44a9-a145-f59514696617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6801","nonce":1932472006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6803","nonce":1932472006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6807","nonce":1932472006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1932472006},{"type":"v1","addr":"172.21.15.157:6805","nonce":1932472006}]},"public_addr":"172.21.15.157:6801/1932472006","cluster_addr":"172.21.15.157:6803/1932472006","heartbeat_back_addr":"172.21.15.157:6807/1932472006","heartbeat_front_addr":"172.21.15.157:6805/1932472006","state":["exists","up"]},{"osd":5,"uuid":"00de4401-55e6-4463-8a34-f9dadcc8aff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6817","nonce":1608291780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6819","nonce":1608291780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6823","nonce":1608291780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":1608291780},{"type":"v1","addr":"172.21.15.157:6821","nonce":1608291780}]},"public_addr":"172.21.15.157:6817/1608291780","cluster_addr":"172.21.15.157:6819/1608291780","heartbeat_back_addr":"172.21.15.157:6823/1608291780","heartbeat_front_addr":"172.21.15.157:6821/1608291780","state":["exists","up"]},{"osd":6,"uuid":"cc1e32e2-e5ff-4f61-bf48-37e4ea7aa37a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6809","nonce":3576931649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6811","nonce":3576931649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6815","nonce":3576931649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":3576931649},{"type":"v1","addr":"172.21.15.157:6813","nonce":3576931649}]},"public_addr":"172.21.15.157:6809/3576931649","cluster_addr":"172.21.15.157:6811/3576931649","heartbeat_back_addr":"172.21.15.157:6815/3576931649","heartbeat_front_addr":"172.21.15.157:6813/3576931649","state":["exists","up"]},{"osd":7,"uuid":"95766452-2e6e-4091-8373-a3a7a0f22315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6825","nonce":2669171853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6827","nonce":2669171853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6831","nonce":2669171853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2669171853},{"type":"v1","addr":"172.21.15.157:6829","nonce":2669171853}]},"public_addr":"172.21.15.157:6825/2669171853","cluster_addr":"172.21.15.157:6827/2669171853","heartbeat_back_addr":"172.21.15.157:6831/2669171853","heartbeat_front_addr":"172.21.15.157:6829/2669171853","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.136559+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.160484+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.108856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.221939+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.127078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.124336+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.174597+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:09:43.153118+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6837/473448013":"2024-03-30T04:12:39.223757+0000","172.21.15.148:6836/473448013":"2024-03-30T04:12:39.223757+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:12:45.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.477+0000 7fc73950e640 1 -- 172.21.15.148:0/2989891924 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc72006c530 msgr2=0x7fc72006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:45.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.477+0000 7fc73950e640 1 --2- 172.21.15.148:0/2989891924 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc72006c530 0x7fc72006e9f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc724038000 tx=0x7fc724032000 comp rx=0 tx=0).stop 2024-03-29T04:12:45.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.477+0000 7fc73950e640 1 -- 172.21.15.148:0/2989891924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc73415f6d0 msgr2=0x7fc73415dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:45.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.477+0000 7fc73950e640 1 --2- 172.21.15.148:0/2989891924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc73415f6d0 0x7fc73415dac0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00e960 tx=0x7fc72c00ee20 comp rx=0 tx=0).stop 2024-03-29T04:12:45.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.477+0000 7fc73950e640 1 -- 172.21.15.148:0/2989891924 shutdown_connections 2024-03-29T04:12:45.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.477+0000 7fc73950e640 1 --2- 172.21.15.148:0/2989891924 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc72006c530 0x7fc72006e9f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:45.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.477+0000 7fc73950e640 1 --2- 172.21.15.148:0/2989891924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc73415f6d0 0x7fc73415dac0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:45.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.477+0000 7fc73950e640 1 --2- 172.21.15.148:0/2989891924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc73415b060 0x7fc73415d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:45.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.477+0000 7fc73950e640 1 --2- 172.21.15.148:0/2989891924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc73406c5c0 0x7fc73414ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:45.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.477+0000 7fc73950e640 1 -- 172.21.15.148:0/2989891924 >> 172.21.15.148:0/2989891924 conn(0x7fc7340cfed0 msgr2=0x7fc734071480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:45.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.477+0000 7fc73950e640 1 -- 172.21.15.148:0/2989891924 shutdown_connections 2024-03-29T04:12:45.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:45.477+0000 7fc73950e640 1 -- 172.21.15.148:0/2989891924 wait complete. 2024-03-29T04:12:46.468 INFO:tasks.ceph:Scrubbing osd.0 2024-03-29T04:12:46.469 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:12:46.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.606+0000 7f9cb6982640 1 Processor -- start 2024-03-29T04:12:46.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.606+0000 7f9cb6982640 1 -- start start 2024-03-29T04:12:46.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.607+0000 7f9cb6982640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9cb0158bb0 0x7f9cb00ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:46.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.607+0000 7f9cb6982640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cb015ea60 0x7f9cb015eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:46.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.607+0000 7f9cb6982640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cb00681c0 con 0x7f9cb00c99f0 2024-03-29T04:12:46.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.607+0000 7f9cb6982640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9cb015f600 con 0x7f9cb0158bb0 2024-03-29T04:12:46.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.607+0000 7f9cb6982640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb015f8e0 con 0x7f9cb015ea60 2024-03-29T04:12:46.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.607+0000 7f9cb4f1f640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9cb00c99f0 0x7f9cb00ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43996/0 (socket says 172.21.15.148:43996) 2024-03-29T04:12:46.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.607+0000 7f9cb4f1f640 1 -- 172.21.15.148:0/3458755242 learned_addr learned my addr 172.21.15.148:0/3458755242 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:46.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.607+0000 7f9caeffd640 1 -- 172.21.15.148:0/3458755242 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4155999303 0 0) 0x7f9cb00681c0 con 0x7f9cb00c99f0 2024-03-29T04:12:46.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.607+0000 7f9caffff640 1 --2- 172.21.15.148:0/3458755242 >> v2:172.21.15.157:3300/0 conn(0x7f9cb0158bb0 0x7f9cb00ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:46.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.607+0000 7f9caf7fe640 1 --2- 172.21.15.148:0/3458755242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cb015ea60 0x7f9cb015eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:46.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.607+0000 7f9caeffd640 1 -- 172.21.15.148:0/3458755242 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c98003520 con 0x7f9cb00c99f0 2024-03-29T04:12:46.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.608+0000 7f9caeffd640 1 -- 172.21.15.148:0/3458755242 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 15930764 0 0) 0x7f9c98003520 con 0x7f9cb00c99f0 2024-03-29T04:12:46.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.608+0000 7f9caeffd640 1 -- 172.21.15.148:0/3458755242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cb015ea60 msgr2=0x7f9cb015eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:46.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.608+0000 7f9caeffd640 1 --2- 172.21.15.148:0/3458755242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cb015ea60 0x7f9cb015eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:46.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.608+0000 7f9caeffd640 1 -- 172.21.15.148:0/3458755242 >> v2:172.21.15.157:3300/0 conn(0x7f9cb0158bb0 msgr2=0x7f9cb00ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:46.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.608+0000 7f9caeffd640 1 --2- 172.21.15.148:0/3458755242 >> v2:172.21.15.157:3300/0 conn(0x7f9cb0158bb0 0x7f9cb00ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:46.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.608+0000 7f9caeffd640 1 -- 172.21.15.148:0/3458755242 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb00cc2e0 con 0x7f9cb00c99f0 2024-03-29T04:12:46.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.608+0000 7f9caeffd640 1 -- 172.21.15.148:0/3458755242 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f9ca80019f0 con 0x7f9cb00c99f0 2024-03-29T04:12:46.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.608+0000 7f9caeffd640 1 -- 172.21.15.148:0/3458755242 >> v1:172.21.15.148:6789/0 conn(0x7f9cb00c99f0 legacy=0x7f9cb00ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:46.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.608+0000 7f9caeffd640 1 --2- 172.21.15.148:0/3458755242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c980039b0 0x7f9c98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:46.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.608+0000 7f9caeffd640 1 --2- 172.21.15.148:0/3458755242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cb015ea60 0x7f9c98004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:46.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.608+0000 7f9caeffd640 1 --2- 172.21.15.148:0/3458755242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c98004d40 0x7f9c980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:46.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.608+0000 7f9caeffd640 1 -- 172.21.15.148:0/3458755242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb015f600 con 0x7f9c98004d40 2024-03-29T04:12:46.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.608+0000 7f9caeffd640 1 -- 172.21.15.148:0/3458755242 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb015f8e0 con 0x7f9c980039b0 2024-03-29T04:12:46.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.608+0000 7f9caeffd640 1 -- 172.21.15.148:0/3458755242 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9c9800b9c0 con 0x7f9cb015ea60 2024-03-29T04:12:46.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.608+0000 7f9cb4f1f640 1 --2- 172.21.15.148:0/3458755242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c98004d40 0x7f9c980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:46.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.608+0000 7f9caffff640 1 --2- 172.21.15.148:0/3458755242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c980039b0 0x7f9c98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:46.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.608+0000 7f9caf7fe640 1 --2- 172.21.15.148:0/3458755242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cb015ea60 0x7f9c98004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:46.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.608+0000 7f9caffff640 1 -- 172.21.15.148:0/3458755242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cb015ea60 msgr2=0x7f9c98004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:46.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.608+0000 7f9caffff640 1 --2- 172.21.15.148:0/3458755242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cb015ea60 0x7f9c98004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:46.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.608+0000 7f9caffff640 1 -- 172.21.15.148:0/3458755242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c98004d40 msgr2=0x7f9c980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:46.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.608+0000 7f9caffff640 1 --2- 172.21.15.148:0/3458755242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c98004d40 0x7f9c980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:46.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.608+0000 7f9caffff640 1 -- 172.21.15.148:0/3458755242 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c98003520 con 0x7f9c980039b0 2024-03-29T04:12:46.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.608+0000 7f9cb4f1f640 1 --2- 172.21.15.148:0/3458755242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c98004d40 0x7f9c980051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:12:46.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.609+0000 7f9caffff640 1 --2- 172.21.15.148:0/3458755242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c980039b0 0x7f9c98003dd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0009af0 tx=0x7f9c98005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=616f07f7fb30b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:46.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.609+0000 7f9caeffd640 1 -- 172.21.15.148:0/3458755242 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0039070 con 0x7f9c980039b0 2024-03-29T04:12:46.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.609+0000 7f9caeffd640 1 -- 172.21.15.148:0/3458755242 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ca002c710 con 0x7f9c980039b0 2024-03-29T04:12:46.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.609+0000 7f9caeffd640 1 -- 172.21.15.148:0/3458755242 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0009690 con 0x7f9c980039b0 2024-03-29T04:12:46.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.610+0000 7f9cb6982640 1 -- 172.21.15.148:0/3458755242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c980039b0 msgr2=0x7f9c98003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:46.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.610+0000 7f9cb6982640 1 --2- 172.21.15.148:0/3458755242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c980039b0 0x7f9c98003dd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0009af0 tx=0x7f9c98005830 comp rx=0 tx=0).stop 2024-03-29T04:12:46.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.610+0000 7f9cb4f1f640 1 -- 172.21.15.148:0/3458755242 reap_dead start 2024-03-29T04:12:46.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.610+0000 7f9cb6982640 1 -- 172.21.15.148:0/3458755242 shutdown_connections 2024-03-29T04:12:46.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.610+0000 7f9cb6982640 1 -- 172.21.15.148:0/3458755242 >> 172.21.15.148:0/3458755242 conn(0x7f9cb00cd140 msgr2=0x7f9cb00cd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:46.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.610+0000 7f9cb6982640 1 -- 172.21.15.148:0/3458755242 shutdown_connections 2024-03-29T04:12:46.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.610+0000 7f9cb6982640 1 -- 172.21.15.148:0/3458755242 wait complete. 2024-03-29T04:12:46.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.611+0000 7f9cb6982640 1 Processor -- start 2024-03-29T04:12:46.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.611+0000 7f9cb6982640 1 -- start start 2024-03-29T04:12:46.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.612+0000 7f9cb6982640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c980039b0 0x7f9cb01576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:46.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.612+0000 7f9cb6982640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cb00b3e70 0x7f9cb00b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:46.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.612+0000 7f9cb6982640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cb00b59a0 0x7f9cb006eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:46.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.612+0000 7f9cb6982640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb00cc080 con 0x7f9c980039b0 2024-03-29T04:12:46.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.612+0000 7f9cb6982640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb00cbf00 con 0x7f9cb00b59a0 2024-03-29T04:12:46.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.612+0000 7f9cb6982640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb015eee0 con 0x7f9cb00b3e70 2024-03-29T04:12:46.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.612+0000 7f9caffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c980039b0 0x7f9cb01576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:46.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.612+0000 7f9caffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c980039b0 0x7f9cb01576a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56484/0 (socket says 172.21.15.148:56484) 2024-03-29T04:12:46.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.612+0000 7f9caffff640 1 -- 172.21.15.148:0/2105237378 learned_addr learned my addr 172.21.15.148:0/2105237378 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:46.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.612+0000 7f9caf7fe640 1 --2- 172.21.15.148:0/2105237378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cb00b3e70 0x7f9cb00b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:46.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.612+0000 7f9cb4f1f640 1 --2- 172.21.15.148:0/2105237378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cb00b59a0 0x7f9cb006eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:46.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.612+0000 7f9caffff640 1 -- 172.21.15.148:0/2105237378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cb00b3e70 msgr2=0x7f9cb00b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:46.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.612+0000 7f9caffff640 1 --2- 172.21.15.148:0/2105237378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cb00b3e70 0x7f9cb00b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:46.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.612+0000 7f9caffff640 1 -- 172.21.15.148:0/2105237378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cb00b59a0 msgr2=0x7f9cb006eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:46.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.612+0000 7f9caffff640 1 --2- 172.21.15.148:0/2105237378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cb00b59a0 0x7f9cb006eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:46.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.612+0000 7f9caffff640 1 -- 172.21.15.148:0/2105237378 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb00b43c0 con 0x7f9c980039b0 2024-03-29T04:12:46.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.612+0000 7f9cb4f1f640 1 --2- 172.21.15.148:0/2105237378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cb00b59a0 0x7f9cb006eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:46.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.612+0000 7f9caffff640 1 --2- 172.21.15.148:0/2105237378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c980039b0 0x7f9cb01576a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0000c00 tx=0x7f9ca0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:46.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.612+0000 7f9cacff9640 1 -- 172.21.15.148:0/2105237378 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0039050 con 0x7f9c980039b0 2024-03-29T04:12:46.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.612+0000 7f9cacff9640 1 -- 172.21.15.148:0/2105237378 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ca0045070 con 0x7f9c980039b0 2024-03-29T04:12:46.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.612+0000 7f9cb6982640 1 -- 172.21.15.148:0/2105237378 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb00b46a0 con 0x7f9c980039b0 2024-03-29T04:12:46.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.613+0000 7f9cacff9640 1 -- 172.21.15.148:0/2105237378 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca00044e0 con 0x7f9c980039b0 2024-03-29T04:12:46.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.613+0000 7f9cb6982640 1 -- 172.21.15.148:0/2105237378 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb00b4c60 con 0x7f9c980039b0 2024-03-29T04:12:46.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.613+0000 7f9cb6982640 1 -- 172.21.15.148:0/2105237378 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9cb0068740 con 0x7f9c980039b0 2024-03-29T04:12:46.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.614+0000 7f9cacff9640 1 -- 172.21.15.148:0/2105237378 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9ca0003a50 con 0x7f9c980039b0 2024-03-29T04:12:46.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.614+0000 7f9cacff9640 1 --2- 172.21.15.148:0/2105237378 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9c9c06c530 0x7f9c9c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:46.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.614+0000 7f9caf7fe640 1 --2- 172.21.15.148:0/2105237378 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9c9c06c530 0x7f9c9c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:46.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.614+0000 7f9cacff9640 1 -- 172.21.15.148:0/2105237378 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f9ca00acf40 con 0x7f9c980039b0 2024-03-29T04:12:46.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.615+0000 7f9cacff9640 1 --2- 172.21.15.148:0/2105237378 >> [v2:172.21.15.148:6824/1406601759,v1:172.21.15.148:6825/1406601759] conn(0x7f9c9c0729d0 0x7f9c9c074e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:46.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.615+0000 7f9cacff9640 1 -- 172.21.15.148:0/2105237378 --> [v2:172.21.15.148:6824/1406601759,v1:172.21.15.148:6825/1406601759] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9ca002eea0 con 0x7f9c9c0729d0 2024-03-29T04:12:46.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.615+0000 7f9cacff9640 1 -- 172.21.15.148:0/2105237378 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9ca00af5e0 con 0x7f9c980039b0 2024-03-29T04:12:46.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.615+0000 7f9cb4f1f640 1 --2- 172.21.15.148:0/2105237378 >> [v2:172.21.15.148:6824/1406601759,v1:172.21.15.148:6825/1406601759] conn(0x7f9c9c0729d0 0x7f9c9c074e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:46.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.615+0000 7f9caf7fe640 1 --2- 172.21.15.148:0/2105237378 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9c9c06c530 0x7f9c9c06e9f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0070390 tx=0x7f9ca4009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:46.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.615+0000 7f9cb4f1f640 1 --2- 172.21.15.148:0/2105237378 >> [v2:172.21.15.148:6824/1406601759,v1:172.21.15.148:6825/1406601759] conn(0x7f9c9c0729d0 0x7f9c9c074e10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:46.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.616+0000 7f9cacff9640 1 -- 172.21.15.148:0/2105237378 <== osd.0 v2:172.21.15.148:6824/1406601759 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9ca002eea0 con 0x7f9c9c0729d0 2024-03-29T04:12:46.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.628+0000 7f9cb6982640 1 -- 172.21.15.148:0/2105237378 --> [v2:172.21.15.148:6824/1406601759,v1:172.21.15.148:6825/1406601759] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9cb015f340 con 0x7f9c9c0729d0 2024-03-29T04:12:46.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.629+0000 7f9cacff9640 1 -- 172.21.15.148:0/2105237378 <== osd.0 v2:172.21.15.148:6824/1406601759 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9cb015f340 con 0x7f9c9c0729d0 2024-03-29T04:12:46.608 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-03-29T04:12:46.609 INFO:teuthology.orchestra.run.smithi148.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:12:46.609 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-03-29T04:12:46.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.629+0000 7f9cb6982640 1 -- 172.21.15.148:0/2105237378 >> [v2:172.21.15.148:6824/1406601759,v1:172.21.15.148:6825/1406601759] conn(0x7f9c9c0729d0 msgr2=0x7f9c9c074e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:46.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.629+0000 7f9cb6982640 1 --2- 172.21.15.148:0/2105237378 >> [v2:172.21.15.148:6824/1406601759,v1:172.21.15.148:6825/1406601759] conn(0x7f9c9c0729d0 0x7f9c9c074e10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:46.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.629+0000 7f9cb6982640 1 -- 172.21.15.148:0/2105237378 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9c9c06c530 msgr2=0x7f9c9c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:46.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.629+0000 7f9cb6982640 1 --2- 172.21.15.148:0/2105237378 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9c9c06c530 0x7f9c9c06e9f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0070390 tx=0x7f9ca4009190 comp rx=0 tx=0).stop 2024-03-29T04:12:46.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.629+0000 7f9cb6982640 1 -- 172.21.15.148:0/2105237378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c980039b0 msgr2=0x7f9cb01576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:46.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.629+0000 7f9cb6982640 1 --2- 172.21.15.148:0/2105237378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c980039b0 0x7f9cb01576a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0000c00 tx=0x7f9ca0004730 comp rx=0 tx=0).stop 2024-03-29T04:12:46.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.629+0000 7f9cb4f1f640 1 -- 172.21.15.148:0/2105237378 reap_dead start 2024-03-29T04:12:46.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.629+0000 7f9cb6982640 1 -- 172.21.15.148:0/2105237378 shutdown_connections 2024-03-29T04:12:46.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.629+0000 7f9cb6982640 1 -- 172.21.15.148:0/2105237378 >> 172.21.15.148:0/2105237378 conn(0x7f9cb00cd140 msgr2=0x7f9cb0071b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:46.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.629+0000 7f9cb6982640 1 -- 172.21.15.148:0/2105237378 shutdown_connections 2024-03-29T04:12:46.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.629+0000 7f9cb6982640 1 -- 172.21.15.148:0/2105237378 wait complete. 2024-03-29T04:12:46.620 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-29T04:12:46.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.742+0000 7fb859aeb640 1 Processor -- start 2024-03-29T04:12:46.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.742+0000 7fb859aeb640 1 -- start start 2024-03-29T04:12:46.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.742+0000 7fb859aeb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb85415a740 0x7fb85415e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:46.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.742+0000 7fb859aeb640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb85415e910 0x7fb85415ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:46.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.742+0000 7fb859aeb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb85406d6a0 0x7fb85406dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:46.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.743+0000 7fb859aeb640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fb8540cf790 con 0x7fb85415e910 2024-03-29T04:12:46.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.743+0000 7fb859aeb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb8540728d0 con 0x7fb85406d6a0 2024-03-29T04:12:46.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.743+0000 7fb859aeb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb854072bb0 con 0x7fb85415a740 2024-03-29T04:12:46.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.743+0000 7fb852ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb85415e910 0x7fb85415ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:46.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.743+0000 7fb852ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb85415e910 0x7fb85415ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56490/0 (socket says 172.21.15.148:56490) 2024-03-29T04:12:46.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.743+0000 7fb852ffd640 1 -- 172.21.15.148:0/1391915163 learned_addr learned my addr 172.21.15.148:0/1391915163 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:46.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.743+0000 7fb853fff640 1 --2- 172.21.15.148:0/1391915163 >> v2:172.21.15.157:3300/0 conn(0x7fb85406d6a0 0x7fb85406dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:46.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.743+0000 7fb8537fe640 1 --2- 172.21.15.148:0/1391915163 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb85415a740 0x7fb85415e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:46.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.743+0000 7fb853fff640 1 -- 172.21.15.148:0/1391915163 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb85415a740 msgr2=0x7fb85415e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:46.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.743+0000 7fb853fff640 1 --2- 172.21.15.148:0/1391915163 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb85415a740 0x7fb85415e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:46.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.743+0000 7fb853fff640 1 -- 172.21.15.148:0/1391915163 >> v2:172.21.15.148:3300/0 conn(0x7fb85415e910 msgr2=0x7fb85415ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:46.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.743+0000 7fb853fff640 1 --2- 172.21.15.148:0/1391915163 >> v2:172.21.15.148:3300/0 conn(0x7fb85415e910 0x7fb85415ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:46.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.743+0000 7fb853fff640 1 -- 172.21.15.148:0/1391915163 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb854072e90 con 0x7fb85406d6a0 2024-03-29T04:12:46.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.743+0000 7fb852ffd640 1 --2- 172.21.15.148:0/1391915163 >> v2:172.21.15.148:3300/0 conn(0x7fb85415e910 0x7fb85415ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:12:46.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.744+0000 7fb853fff640 1 --2- 172.21.15.148:0/1391915163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb85406d6a0 0x7fb85406dae0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb854071900 tx=0x7fb84c02bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=732fc185bbd74122 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:46.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.744+0000 7fb8527fc640 1 -- 172.21.15.148:0/1391915163 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c039070 con 0x7fb85406d6a0 2024-03-29T04:12:46.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.744+0000 7fb8527fc640 1 -- 172.21.15.148:0/1391915163 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb84c02cb40 con 0x7fb85406d6a0 2024-03-29T04:12:46.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.744+0000 7fb8527fc640 1 -- 172.21.15.148:0/1391915163 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c034440 con 0x7fb85406d6a0 2024-03-29T04:12:46.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.744+0000 7fb859aeb640 1 -- 172.21.15.148:0/1391915163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb85406d6a0 msgr2=0x7fb85406dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:46.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.744+0000 7fb859aeb640 1 --2- 172.21.15.148:0/1391915163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb85406d6a0 0x7fb85406dae0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb854071900 tx=0x7fb84c02bd90 comp rx=0 tx=0).stop 2024-03-29T04:12:46.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.745+0000 7fb859aeb640 1 -- 172.21.15.148:0/1391915163 shutdown_connections 2024-03-29T04:12:46.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.745+0000 7fb859aeb640 1 --2- 172.21.15.148:0/1391915163 >> v2:172.21.15.148:3300/0 conn(0x7fb85415e910 0x7fb85415ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:46.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.745+0000 7fb859aeb640 1 --2- 172.21.15.148:0/1391915163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb85406d6a0 0x7fb85406dae0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:46.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.745+0000 7fb859aeb640 1 --2- 172.21.15.148:0/1391915163 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb85415a740 0x7fb85415e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:46.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.745+0000 7fb859aeb640 1 -- 172.21.15.148:0/1391915163 >> 172.21.15.148:0/1391915163 conn(0x7fb8540cfed0 msgr2=0x7fb8540d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:46.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.745+0000 7fb859aeb640 1 -- 172.21.15.148:0/1391915163 shutdown_connections 2024-03-29T04:12:46.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.745+0000 7fb859aeb640 1 -- 172.21.15.148:0/1391915163 wait complete. 2024-03-29T04:12:46.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.746+0000 7fb859aeb640 1 Processor -- start 2024-03-29T04:12:46.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.746+0000 7fb859aeb640 1 -- start start 2024-03-29T04:12:46.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.747+0000 7fb859aeb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb85406d6a0 0x7fb854141b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:46.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.747+0000 7fb859aeb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb85415a740 0x7fb85415afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:46.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.747+0000 7fb859aeb640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb85415e910 0x7fb8541519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:46.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.747+0000 7fb859aeb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb85415aca0 con 0x7fb85415a740 2024-03-29T04:12:46.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.747+0000 7fb859aeb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb85415ab20 con 0x7fb85406d6a0 2024-03-29T04:12:46.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.747+0000 7fb859aeb640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb85406c5c0 con 0x7fb85415e910 2024-03-29T04:12:46.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.747+0000 7fb852ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb85415a740 0x7fb85415afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:46.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.747+0000 7fb852ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb85415a740 0x7fb85415afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56506/0 (socket says 172.21.15.148:56506) 2024-03-29T04:12:46.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.747+0000 7fb852ffd640 1 -- 172.21.15.148:0/4095285237 learned_addr learned my addr 172.21.15.148:0/4095285237 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:46.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.747+0000 7fb8537fe640 1 --2- 172.21.15.148:0/4095285237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb85406d6a0 0x7fb854141b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:46.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.747+0000 7fb853fff640 1 --2- 172.21.15.148:0/4095285237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb85415e910 0x7fb8541519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:46.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.747+0000 7fb852ffd640 1 -- 172.21.15.148:0/4095285237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb85415e910 msgr2=0x7fb8541519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:46.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.747+0000 7fb852ffd640 1 --2- 172.21.15.148:0/4095285237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb85415e910 0x7fb8541519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:46.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.747+0000 7fb852ffd640 1 -- 172.21.15.148:0/4095285237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb85406d6a0 msgr2=0x7fb854141b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:46.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.747+0000 7fb852ffd640 1 --2- 172.21.15.148:0/4095285237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb85406d6a0 0x7fb854141b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:46.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.747+0000 7fb852ffd640 1 -- 172.21.15.148:0/4095285237 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb854142060 con 0x7fb85415a740 2024-03-29T04:12:46.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.747+0000 7fb852ffd640 1 --2- 172.21.15.148:0/4095285237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb85415a740 0x7fb85415afe0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb84800dd50 tx=0x7fb8480094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:46.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.747+0000 7fb83bfff640 1 -- 172.21.15.148:0/4095285237 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84801c070 con 0x7fb85415a740 2024-03-29T04:12:46.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.747+0000 7fb83bfff640 1 -- 172.21.15.148:0/4095285237 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb84800de40 con 0x7fb85415a740 2024-03-29T04:12:46.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.747+0000 7fb83bfff640 1 -- 172.21.15.148:0/4095285237 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb848017410 con 0x7fb85415a740 2024-03-29T04:12:46.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.747+0000 7fb859aeb640 1 -- 172.21.15.148:0/4095285237 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb85415bdd0 con 0x7fb85415a740 2024-03-29T04:12:46.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.747+0000 7fb859aeb640 1 -- 172.21.15.148:0/4095285237 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb854145a10 con 0x7fb85415a740 2024-03-29T04:12:46.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.749+0000 7fb859aeb640 1 -- 172.21.15.148:0/4095285237 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8540cfcb0 con 0x7fb85415a740 2024-03-29T04:12:46.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.749+0000 7fb83bfff640 1 -- 172.21.15.148:0/4095285237 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb84801f070 con 0x7fb85415a740 2024-03-29T04:12:46.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.749+0000 7fb83bfff640 1 --2- 172.21.15.148:0/4095285237 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb84006c530 0x7fb84006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:46.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.749+0000 7fb83bfff640 1 -- 172.21.15.148:0/4095285237 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7fb84808b1d0 con 0x7fb85415a740 2024-03-29T04:12:46.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.750+0000 7fb8537fe640 1 --2- 172.21.15.148:0/4095285237 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb84006c530 0x7fb84006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:46.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.750+0000 7fb8537fe640 1 --2- 172.21.15.148:0/4095285237 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb84006c530 0x7fb84006e9f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb844009ac0 tx=0x7fb844009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:46.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.754+0000 7fb83bfff640 1 -- 172.21.15.148:0/4095285237 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb854142060 con 0x7fb85415a740 2024-03-29T04:12:46.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.928+0000 7fb859aeb640 1 -- 172.21.15.148:0/4095285237 --> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fb854071960 con 0x7fb84006c530 2024-03-29T04:12:46.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.929+0000 7fb83bfff640 1 -- 172.21.15.148:0/4095285237 <== mgr.4113 v2:172.21.15.148:6832/3271445075 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb854071960 con 0x7fb84006c530 2024-03-29T04:12:46.909 INFO:teuthology.orchestra.run.smithi148.stderr:instructed osd(s) 0 to deep-scrub 2024-03-29T04:12:46.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.931+0000 7fb859aeb640 1 -- 172.21.15.148:0/4095285237 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb84006c530 msgr2=0x7fb84006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:46.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.931+0000 7fb859aeb640 1 --2- 172.21.15.148:0/4095285237 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb84006c530 0x7fb84006e9f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb844009ac0 tx=0x7fb844009190 comp rx=0 tx=0).stop 2024-03-29T04:12:46.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.931+0000 7fb859aeb640 1 -- 172.21.15.148:0/4095285237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb85415a740 msgr2=0x7fb85415afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:46.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.931+0000 7fb859aeb640 1 --2- 172.21.15.148:0/4095285237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb85415a740 0x7fb85415afe0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb84800dd50 tx=0x7fb8480094f0 comp rx=0 tx=0).stop 2024-03-29T04:12:46.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.931+0000 7fb859aeb640 1 -- 172.21.15.148:0/4095285237 shutdown_connections 2024-03-29T04:12:46.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.931+0000 7fb859aeb640 1 --2- 172.21.15.148:0/4095285237 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fb84006c530 0x7fb84006e9f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:46.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.931+0000 7fb859aeb640 1 --2- 172.21.15.148:0/4095285237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb85415e910 0x7fb8541519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:46.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.931+0000 7fb859aeb640 1 --2- 172.21.15.148:0/4095285237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb85415a740 0x7fb85415afe0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:46.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.931+0000 7fb859aeb640 1 --2- 172.21.15.148:0/4095285237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb85406d6a0 0x7fb854141b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:46.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.931+0000 7fb859aeb640 1 -- 172.21.15.148:0/4095285237 >> 172.21.15.148:0/4095285237 conn(0x7fb8540cfed0 msgr2=0x7fb8540b2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:46.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.931+0000 7fb859aeb640 1 -- 172.21.15.148:0/4095285237 shutdown_connections 2024-03-29T04:12:46.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:46.931+0000 7fb859aeb640 1 -- 172.21.15.148:0/4095285237 wait complete. 2024-03-29T04:12:46.921 INFO:tasks.ceph:Scrubbing osd.1 2024-03-29T04:12:46.922 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:12:47.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.047+0000 7f4405924640 1 Processor -- start 2024-03-29T04:12:47.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.047+0000 7f4405924640 1 -- start start 2024-03-29T04:12:47.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.048+0000 7f4405924640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f44000d3840 0x7f44000d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.048+0000 7f4405924640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44000b36e0 0x7f44001615d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.048+0000 7f4405924640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f44000d3c20 0x7f4400161b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.048+0000 7f4405924640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f44000d27b0 con 0x7f44000d3840 2024-03-29T04:12:47.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.048+0000 7f4405924640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f44000d2a50 con 0x7f44000d3c20 2024-03-29T04:12:47.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.048+0000 7f4405924640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f44000d2d30 con 0x7f44000b36e0 2024-03-29T04:12:47.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.048+0000 7f43feffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f44000d3840 0x7f44000d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.048+0000 7f43feffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f44000d3840 0x7f44000d2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56510/0 (socket says 172.21.15.148:56510) 2024-03-29T04:12:47.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.048+0000 7f43feffd640 1 -- 172.21.15.148:0/1776831344 learned_addr learned my addr 172.21.15.148:0/1776831344 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:47.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.048+0000 7f43ff7fe640 1 --2- 172.21.15.148:0/1776831344 >> v2:172.21.15.157:3300/0 conn(0x7f44000d3c20 0x7f4400161b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.048+0000 7f43f7fff640 1 --2- 172.21.15.148:0/1776831344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44000b36e0 0x7f44001615d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.048+0000 7f43feffd640 1 -- 172.21.15.148:0/1776831344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44000b36e0 msgr2=0x7f44001615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:47.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.048+0000 7f43feffd640 1 --2- 172.21.15.148:0/1776831344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44000b36e0 0x7f44001615d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.048+0000 7f43feffd640 1 -- 172.21.15.148:0/1776831344 >> v2:172.21.15.157:3300/0 conn(0x7f44000d3c20 msgr2=0x7f4400161b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:47.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.048+0000 7f43feffd640 1 --2- 172.21.15.148:0/1776831344 >> v2:172.21.15.157:3300/0 conn(0x7f44000d3c20 0x7f4400161b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.048+0000 7f43feffd640 1 -- 172.21.15.148:0/1776831344 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44000d3010 con 0x7f44000d3840 2024-03-29T04:12:47.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.048+0000 7f43ff7fe640 1 --2- 172.21.15.148:0/1776831344 >> v2:172.21.15.157:3300/0 conn(0x7f44000d3c20 0x7f4400161b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:12:47.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.048+0000 7f43f7fff640 1 --2- 172.21.15.148:0/1776831344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44000b36e0 0x7f44001615d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:12:47.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.049+0000 7f43feffd640 1 --2- 172.21.15.148:0/1776831344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44000d3840 0x7f44000d2270 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f44000d0430 tx=0x7f43f002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac907b5bd0eca26 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:47.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.049+0000 7f43fe7fc640 1 -- 172.21.15.148:0/1776831344 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f0035020 con 0x7f44000d3840 2024-03-29T04:12:47.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.049+0000 7f43fe7fc640 1 -- 172.21.15.148:0/1776831344 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43f0031870 con 0x7f44000d3840 2024-03-29T04:12:47.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.049+0000 7f43fe7fc640 1 -- 172.21.15.148:0/1776831344 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f0031c40 con 0x7f44000d3840 2024-03-29T04:12:47.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.050+0000 7f4405924640 1 -- 172.21.15.148:0/1776831344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44000d3840 msgr2=0x7f44000d2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.050+0000 7f4405924640 1 --2- 172.21.15.148:0/1776831344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44000d3840 0x7f44000d2270 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f44000d0430 tx=0x7f43f002b910 comp rx=0 tx=0).stop 2024-03-29T04:12:47.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.050+0000 7f4405924640 1 -- 172.21.15.148:0/1776831344 shutdown_connections 2024-03-29T04:12:47.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.050+0000 7f4405924640 1 --2- 172.21.15.148:0/1776831344 >> v2:172.21.15.157:3300/0 conn(0x7f44000d3c20 0x7f4400161b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.050+0000 7f4405924640 1 --2- 172.21.15.148:0/1776831344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44000b36e0 0x7f44001615d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.050+0000 7f4405924640 1 --2- 172.21.15.148:0/1776831344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44000d3840 0x7f44000d2270 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.050+0000 7f4405924640 1 -- 172.21.15.148:0/1776831344 >> 172.21.15.148:0/1776831344 conn(0x7f44000d42d0 msgr2=0x7f44000d46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:47.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.050+0000 7f4405924640 1 -- 172.21.15.148:0/1776831344 shutdown_connections 2024-03-29T04:12:47.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.050+0000 7f4405924640 1 -- 172.21.15.148:0/1776831344 wait complete. 2024-03-29T04:12:47.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.051+0000 7f4405924640 1 Processor -- start 2024-03-29T04:12:47.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.051+0000 7f4405924640 1 -- start start 2024-03-29T04:12:47.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.052+0000 7f4405924640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44000b36e0 0x7f4400158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.052+0000 7f4405924640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44000d3840 0x7f44000c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.052+0000 7f4405924640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44000d3c20 0x7f440015e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.052+0000 7f4405924640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4400162330 con 0x7f44000b36e0 2024-03-29T04:12:47.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.052+0000 7f4405924640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f44001621b0 con 0x7f44000d3c20 2024-03-29T04:12:47.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.052+0000 7f4405924640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f44001624b0 con 0x7f44000d3840 2024-03-29T04:12:47.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.052+0000 7f43feffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44000b36e0 0x7f4400158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.052+0000 7f43feffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44000b36e0 0x7f4400158d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56526/0 (socket says 172.21.15.148:56526) 2024-03-29T04:12:47.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.052+0000 7f43feffd640 1 -- 172.21.15.148:0/1365202571 learned_addr learned my addr 172.21.15.148:0/1365202571 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:47.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.052+0000 7f43ff7fe640 1 --2- 172.21.15.148:0/1365202571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44000d3c20 0x7f440015e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.052+0000 7f43f7fff640 1 --2- 172.21.15.148:0/1365202571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44000d3840 0x7f44000c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.052+0000 7f43feffd640 1 -- 172.21.15.148:0/1365202571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44000d3840 msgr2=0x7f44000c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.052+0000 7f43feffd640 1 --2- 172.21.15.148:0/1365202571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44000d3840 0x7f44000c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.052+0000 7f43feffd640 1 -- 172.21.15.148:0/1365202571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44000d3c20 msgr2=0x7f440015e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.052+0000 7f43feffd640 1 --2- 172.21.15.148:0/1365202571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44000d3c20 0x7f440015e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.052+0000 7f43feffd640 1 -- 172.21.15.148:0/1365202571 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44000ca700 con 0x7f44000b36e0 2024-03-29T04:12:47.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.052+0000 7f43ff7fe640 1 --2- 172.21.15.148:0/1365202571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44000d3c20 0x7f440015e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:12:47.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.052+0000 7f43f7fff640 1 --2- 172.21.15.148:0/1365202571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44000d3840 0x7f44000c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:12:47.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.052+0000 7f43feffd640 1 --2- 172.21.15.148:0/1365202571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44000b36e0 0x7f4400158d80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f43f002b950 tx=0x7f43f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:47.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.052+0000 7f43fcff9640 1 -- 172.21.15.148:0/1365202571 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f0040070 con 0x7f44000b36e0 2024-03-29T04:12:47.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.052+0000 7f43fcff9640 1 -- 172.21.15.148:0/1365202571 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43f003a420 con 0x7f44000b36e0 2024-03-29T04:12:47.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.052+0000 7f4405924640 1 -- 172.21.15.148:0/1365202571 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44000cd060 con 0x7f44000b36e0 2024-03-29T04:12:47.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.052+0000 7f43fcff9640 1 -- 172.21.15.148:0/1365202571 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f0035040 con 0x7f44000b36e0 2024-03-29T04:12:47.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.052+0000 7f4405924640 1 -- 172.21.15.148:0/1365202571 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44000cd900 con 0x7f44000b36e0 2024-03-29T04:12:47.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.053+0000 7f4405924640 1 -- 172.21.15.148:0/1365202571 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4400068740 con 0x7f44000b36e0 2024-03-29T04:12:47.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.054+0000 7f43fcff9640 1 -- 172.21.15.148:0/1365202571 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f43f004b060 con 0x7f44000b36e0 2024-03-29T04:12:47.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.054+0000 7f43fcff9640 1 --2- 172.21.15.148:0/1365202571 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f43e006c530 0x7f43e006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.054+0000 7f43fcff9640 1 -- 172.21.15.148:0/1365202571 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f43f00ad550 con 0x7f44000b36e0 2024-03-29T04:12:47.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.054+0000 7f43f7fff640 1 --2- 172.21.15.148:0/1365202571 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f43e006c530 0x7f43e006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.054+0000 7f43fcff9640 1 --2- 172.21.15.148:0/1365202571 >> [v2:172.21.15.148:6800/686342976,v1:172.21.15.148:6801/686342976] conn(0x7f43e00729d0 0x7f43e0074e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.054+0000 7f43fcff9640 1 -- 172.21.15.148:0/1365202571 --> [v2:172.21.15.148:6800/686342976,v1:172.21.15.148:6801/686342976] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f43f00067f0 con 0x7f43e00729d0 2024-03-29T04:12:47.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.054+0000 7f43fcff9640 1 -- 172.21.15.148:0/1365202571 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f43f00b4050 con 0x7f44000b36e0 2024-03-29T04:12:47.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.054+0000 7f43ff7fe640 1 --2- 172.21.15.148:0/1365202571 >> [v2:172.21.15.148:6800/686342976,v1:172.21.15.148:6801/686342976] conn(0x7f43e00729d0 0x7f43e0074e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.055+0000 7f43f7fff640 1 --2- 172.21.15.148:0/1365202571 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f43e006c530 0x7f43e006e9f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f440006d240 tx=0x7f43e8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:47.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.055+0000 7f43ff7fe640 1 --2- 172.21.15.148:0/1365202571 >> [v2:172.21.15.148:6800/686342976,v1:172.21.15.148:6801/686342976] conn(0x7f43e00729d0 0x7f43e0074e10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:47.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.055+0000 7f43fcff9640 1 -- 172.21.15.148:0/1365202571 <== osd.1 v2:172.21.15.148:6800/686342976 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f43f00067f0 con 0x7f43e00729d0 2024-03-29T04:12:47.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.068+0000 7f4405924640 1 -- 172.21.15.148:0/1365202571 --> [v2:172.21.15.148:6800/686342976,v1:172.21.15.148:6801/686342976] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f44000d2050 con 0x7f43e00729d0 2024-03-29T04:12:47.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.068+0000 7f43fcff9640 1 -- 172.21.15.148:0/1365202571 <== osd.1 v2:172.21.15.148:6800/686342976 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f44000d2050 con 0x7f43e00729d0 2024-03-29T04:12:47.048 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-03-29T04:12:47.048 INFO:teuthology.orchestra.run.smithi148.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:12:47.049 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-03-29T04:12:47.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.069+0000 7f4405924640 1 -- 172.21.15.148:0/1365202571 >> [v2:172.21.15.148:6800/686342976,v1:172.21.15.148:6801/686342976] conn(0x7f43e00729d0 msgr2=0x7f43e0074e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.069+0000 7f4405924640 1 --2- 172.21.15.148:0/1365202571 >> [v2:172.21.15.148:6800/686342976,v1:172.21.15.148:6801/686342976] conn(0x7f43e00729d0 0x7f43e0074e10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.069+0000 7f4405924640 1 -- 172.21.15.148:0/1365202571 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f43e006c530 msgr2=0x7f43e006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.069+0000 7f4405924640 1 --2- 172.21.15.148:0/1365202571 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f43e006c530 0x7f43e006e9f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f440006d240 tx=0x7f43e8008040 comp rx=0 tx=0).stop 2024-03-29T04:12:47.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.069+0000 7f4405924640 1 -- 172.21.15.148:0/1365202571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44000b36e0 msgr2=0x7f4400158d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.069+0000 7f4405924640 1 --2- 172.21.15.148:0/1365202571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44000b36e0 0x7f4400158d80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f43f002b950 tx=0x7f43f00046c0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.069+0000 7f43ff7fe640 1 -- 172.21.15.148:0/1365202571 reap_dead start 2024-03-29T04:12:47.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.069+0000 7f4405924640 1 -- 172.21.15.148:0/1365202571 shutdown_connections 2024-03-29T04:12:47.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.069+0000 7f4405924640 1 -- 172.21.15.148:0/1365202571 >> 172.21.15.148:0/1365202571 conn(0x7f44000d42d0 msgr2=0x7f44000b55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:47.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.069+0000 7f4405924640 1 -- 172.21.15.148:0/1365202571 shutdown_connections 2024-03-29T04:12:47.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.069+0000 7f4405924640 1 -- 172.21.15.148:0/1365202571 wait complete. 2024-03-29T04:12:47.059 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-29T04:12:47.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.180+0000 7fa2c82d9640 1 Processor -- start 2024-03-29T04:12:47.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.180+0000 7fa2c82d9640 1 -- start start 2024-03-29T04:12:47.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.181+0000 7fa2c82d9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa2c015b060 0x7fa2c00b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.181+0000 7fa2c82d9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2c015f6d0 0x7fa2c00b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.181+0000 7fa2c82d9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa2c006c5c0 0x7fa2c006ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.181+0000 7fa2c82d9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fa2c00b3630 con 0x7fa2c006c5c0 2024-03-29T04:12:47.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.181+0000 7fa2c82d9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa2c00b3910 con 0x7fa2c015b060 2024-03-29T04:12:47.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.181+0000 7fa2c82d9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa2c00b3bf0 con 0x7fa2c015f6d0 2024-03-29T04:12:47.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.181+0000 7fa2c6876640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa2c006c5c0 0x7fa2c006ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.181+0000 7fa2c6876640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa2c006c5c0 0x7fa2c006ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56538/0 (socket says 172.21.15.148:56538) 2024-03-29T04:12:47.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.181+0000 7fa2c6876640 1 -- 172.21.15.148:0/3427498716 learned_addr learned my addr 172.21.15.148:0/3427498716 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:47.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.181+0000 7fa2c5874640 1 --2- 172.21.15.148:0/3427498716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2c015f6d0 0x7fa2c00b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.181+0000 7fa2c6075640 1 --2- 172.21.15.148:0/3427498716 >> v2:172.21.15.157:3300/0 conn(0x7fa2c015b060 0x7fa2c00b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.181+0000 7fa2c6876640 1 -- 172.21.15.148:0/3427498716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2c015f6d0 msgr2=0x7fa2c00b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:47.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.181+0000 7fa2c6876640 1 --2- 172.21.15.148:0/3427498716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2c015f6d0 0x7fa2c00b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.181+0000 7fa2c6876640 1 -- 172.21.15.148:0/3427498716 >> v2:172.21.15.157:3300/0 conn(0x7fa2c015b060 msgr2=0x7fa2c00b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:47.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.181+0000 7fa2c6876640 1 --2- 172.21.15.148:0/3427498716 >> v2:172.21.15.157:3300/0 conn(0x7fa2c015b060 0x7fa2c00b2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.181+0000 7fa2c6876640 1 -- 172.21.15.148:0/3427498716 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c00b3ed0 con 0x7fa2c006c5c0 2024-03-29T04:12:47.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.182+0000 7fa2c6876640 1 --2- 172.21.15.148:0/3427498716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2c006c5c0 0x7fa2c006ca50 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00d0db0 tx=0x7fa2b802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b341cd04fcf19782 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:47.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.182+0000 7fa2c5073640 1 -- 172.21.15.148:0/3427498716 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b802c9e0 con 0x7fa2c006c5c0 2024-03-29T04:12:47.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.182+0000 7fa2c5073640 1 -- 172.21.15.148:0/3427498716 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2b802cb80 con 0x7fa2c006c5c0 2024-03-29T04:12:47.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.182+0000 7fa2c5073640 1 -- 172.21.15.148:0/3427498716 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b80343f0 con 0x7fa2c006c5c0 2024-03-29T04:12:47.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.182+0000 7fa2c82d9640 1 -- 172.21.15.148:0/3427498716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2c006c5c0 msgr2=0x7fa2c006ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.182+0000 7fa2c82d9640 1 --2- 172.21.15.148:0/3427498716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2c006c5c0 0x7fa2c006ca50 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00d0db0 tx=0x7fa2b802be40 comp rx=0 tx=0).stop 2024-03-29T04:12:47.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.183+0000 7fa2c82d9640 1 -- 172.21.15.148:0/3427498716 shutdown_connections 2024-03-29T04:12:47.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.183+0000 7fa2c82d9640 1 --2- 172.21.15.148:0/3427498716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2c006c5c0 0x7fa2c006ca50 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.183+0000 7fa2c82d9640 1 --2- 172.21.15.148:0/3427498716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2c015f6d0 0x7fa2c00b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.183+0000 7fa2c82d9640 1 --2- 172.21.15.148:0/3427498716 >> v2:172.21.15.157:3300/0 conn(0x7fa2c015b060 0x7fa2c00b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.183+0000 7fa2c82d9640 1 -- 172.21.15.148:0/3427498716 >> 172.21.15.148:0/3427498716 conn(0x7fa2c00cfed0 msgr2=0x7fa2c00d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:47.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.183+0000 7fa2c82d9640 1 -- 172.21.15.148:0/3427498716 shutdown_connections 2024-03-29T04:12:47.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.183+0000 7fa2c82d9640 1 -- 172.21.15.148:0/3427498716 wait complete. 2024-03-29T04:12:47.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.184+0000 7fa2c82d9640 1 Processor -- start 2024-03-29T04:12:47.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.184+0000 7fa2c82d9640 1 -- start start 2024-03-29T04:12:47.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.185+0000 7fa2c82d9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2c006c5c0 0x7fa2c0146b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.185+0000 7fa2c82d9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c015b060 0x7fa2c01e6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.185+0000 7fa2c82d9640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2c015f6d0 0x7fa2c01e6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.185+0000 7fa2c82d9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c00b4c10 con 0x7fa2c006c5c0 2024-03-29T04:12:47.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.185+0000 7fa2c82d9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c015fb30 con 0x7fa2c015b060 2024-03-29T04:12:47.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.185+0000 7fa2c82d9640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa2c015b440 con 0x7fa2c015f6d0 2024-03-29T04:12:47.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.185+0000 7fa2c6075640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2c006c5c0 0x7fa2c0146b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.185+0000 7fa2c6075640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2c006c5c0 0x7fa2c0146b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56548/0 (socket says 172.21.15.148:56548) 2024-03-29T04:12:47.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.185+0000 7fa2c6075640 1 -- 172.21.15.148:0/2503039522 learned_addr learned my addr 172.21.15.148:0/2503039522 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:47.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.185+0000 7fa2c5874640 1 --2- 172.21.15.148:0/2503039522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c015b060 0x7fa2c01e6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.185+0000 7fa2c6876640 1 --2- 172.21.15.148:0/2503039522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2c015f6d0 0x7fa2c01e6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.185+0000 7fa2c6075640 1 -- 172.21.15.148:0/2503039522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2c015f6d0 msgr2=0x7fa2c01e6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.185+0000 7fa2c6075640 1 --2- 172.21.15.148:0/2503039522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2c015f6d0 0x7fa2c01e6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.185+0000 7fa2c6075640 1 -- 172.21.15.148:0/2503039522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c015b060 msgr2=0x7fa2c01e6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.185+0000 7fa2c6075640 1 --2- 172.21.15.148:0/2503039522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c015b060 0x7fa2c01e6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.185+0000 7fa2c6075640 1 -- 172.21.15.148:0/2503039522 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c01e7390 con 0x7fa2c006c5c0 2024-03-29T04:12:47.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.185+0000 7fa2c6075640 1 --2- 172.21.15.148:0/2503039522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2c006c5c0 0x7fa2c0146b00 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00d5f0 tx=0x7fa2bc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.185+0000 7fa2b6ffd640 1 -- 172.21.15.148:0/2503039522 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc0125f0 con 0x7fa2c006c5c0 2024-03-29T04:12:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.185+0000 7fa2b6ffd640 1 -- 172.21.15.148:0/2503039522 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2bc019070 con 0x7fa2c006c5c0 2024-03-29T04:12:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.185+0000 7fa2c82d9640 1 -- 172.21.15.148:0/2503039522 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c01e70b0 con 0x7fa2c006c5c0 2024-03-29T04:12:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.185+0000 7fa2b6ffd640 1 -- 172.21.15.148:0/2503039522 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc0129d0 con 0x7fa2c006c5c0 2024-03-29T04:12:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.185+0000 7fa2c82d9640 1 -- 172.21.15.148:0/2503039522 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c01f26f0 con 0x7fa2c006c5c0 2024-03-29T04:12:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.186+0000 7fa2c82d9640 1 -- 172.21.15.148:0/2503039522 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c00687e0 con 0x7fa2c006c5c0 2024-03-29T04:12:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.187+0000 7fa2b6ffd640 1 -- 172.21.15.148:0/2503039522 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa2bc012b70 con 0x7fa2c006c5c0 2024-03-29T04:12:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.187+0000 7fa2b6ffd640 1 --2- 172.21.15.148:0/2503039522 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa2a406c560 0x7fa2a406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.187+0000 7fa2c5874640 1 --2- 172.21.15.148:0/2503039522 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa2a406c560 0x7fa2a406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.187+0000 7fa2b6ffd640 1 -- 172.21.15.148:0/2503039522 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7fa2bc08b740 con 0x7fa2c006c5c0 2024-03-29T04:12:47.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.188+0000 7fa2c5874640 1 --2- 172.21.15.148:0/2503039522 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa2a406c560 0x7fa2a406ea20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00d06d0 tx=0x7fa2b0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:47.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.192+0000 7fa2b6ffd640 1 -- 172.21.15.148:0/2503039522 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa2bc059910 con 0x7fa2c006c5c0 2024-03-29T04:12:47.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.366+0000 7fa2c82d9640 1 -- 172.21.15.148:0/2503039522 --> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fa2c00721e0 con 0x7fa2a406c560 2024-03-29T04:12:47.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.366+0000 7fa2b6ffd640 1 -- 172.21.15.148:0/2503039522 <== mgr.4113 v2:172.21.15.148:6832/3271445075 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa2c00721e0 con 0x7fa2a406c560 2024-03-29T04:12:47.346 INFO:teuthology.orchestra.run.smithi148.stderr:instructed osd(s) 1 to deep-scrub 2024-03-29T04:12:47.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.368+0000 7fa2c82d9640 1 -- 172.21.15.148:0/2503039522 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa2a406c560 msgr2=0x7fa2a406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.368+0000 7fa2c82d9640 1 --2- 172.21.15.148:0/2503039522 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa2a406c560 0x7fa2a406ea20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00d06d0 tx=0x7fa2b0008040 comp rx=0 tx=0).stop 2024-03-29T04:12:47.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.368+0000 7fa2c82d9640 1 -- 172.21.15.148:0/2503039522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2c006c5c0 msgr2=0x7fa2c0146b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.368+0000 7fa2c82d9640 1 --2- 172.21.15.148:0/2503039522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2c006c5c0 0x7fa2c0146b00 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00d5f0 tx=0x7fa2bc00dab0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.368+0000 7fa2c82d9640 1 -- 172.21.15.148:0/2503039522 shutdown_connections 2024-03-29T04:12:47.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.368+0000 7fa2c82d9640 1 --2- 172.21.15.148:0/2503039522 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa2a406c560 0x7fa2a406ea20 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.368+0000 7fa2c82d9640 1 --2- 172.21.15.148:0/2503039522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2c015f6d0 0x7fa2c01e6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.368+0000 7fa2c82d9640 1 --2- 172.21.15.148:0/2503039522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c015b060 0x7fa2c01e6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.368+0000 7fa2c82d9640 1 --2- 172.21.15.148:0/2503039522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2c006c5c0 0x7fa2c0146b00 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.368+0000 7fa2c82d9640 1 -- 172.21.15.148:0/2503039522 >> 172.21.15.148:0/2503039522 conn(0x7fa2c00cfed0 msgr2=0x7fa2c0156fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:47.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.368+0000 7fa2c82d9640 1 -- 172.21.15.148:0/2503039522 shutdown_connections 2024-03-29T04:12:47.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.368+0000 7fa2c82d9640 1 -- 172.21.15.148:0/2503039522 wait complete. 2024-03-29T04:12:47.359 INFO:tasks.ceph:Scrubbing osd.2 2024-03-29T04:12:47.359 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:12:47.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.484+0000 7f73e3675640 1 Processor -- start 2024-03-29T04:12:47.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.484+0000 7f73e3675640 1 -- start start 2024-03-29T04:12:47.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.485+0000 7f73e3675640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f73dc145b40 0x7f73dc06ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.485+0000 7f73e3675640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f73dc06c9c0 0x7f73dc15fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.485+0000 7f73e3675640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73dc06b920 con 0x7f73dc06f200 2024-03-29T04:12:47.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.485+0000 7f73e3675640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f73dc145f20 con 0x7f73dc06c9c0 2024-03-29T04:12:47.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.485+0000 7f73e3675640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f73dc06bc00 con 0x7f73dc145b40 2024-03-29T04:12:47.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.485+0000 7f73e0c10640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f73dc06c9c0 0x7f73dc15fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.485+0000 7f73e0c10640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f73dc06c9c0 0x7f73dc15fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56560/0 (socket says 172.21.15.148:56560) 2024-03-29T04:12:47.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.485+0000 7f73e0c10640 1 -- 172.21.15.148:0/3136026978 learned_addr learned my addr 172.21.15.148:0/3136026978 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:47.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.486+0000 7f73e1411640 1 --2- 172.21.15.148:0/3136026978 >> v2:172.21.15.157:3300/0 conn(0x7f73dc145b40 0x7f73dc06ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.486+0000 7f73e0c10640 1 -- 172.21.15.148:0/3136026978 >> v2:172.21.15.157:3300/0 conn(0x7f73dc145b40 msgr2=0x7f73dc06ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:47.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.486+0000 7f73e0c10640 1 --2- 172.21.15.148:0/3136026978 >> v2:172.21.15.157:3300/0 conn(0x7f73dc145b40 0x7f73dc06ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.486+0000 7f73e0c10640 1 -- 172.21.15.148:0/3136026978 >> v1:172.21.15.157:6789/0 conn(0x7f73dc06f200 legacy=0x7f73dc0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.486+0000 7f73e0c10640 1 -- 172.21.15.148:0/3136026978 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73dc06bee0 con 0x7f73dc06c9c0 2024-03-29T04:12:47.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.486+0000 7f73e0c10640 1 --2- 172.21.15.148:0/3136026978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73dc06c9c0 0x7f73dc15fd80 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f73dc0b6360 tx=0x7f73cc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a30a2c1b5591111 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:47.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.487+0000 7f73d3fff640 1 -- 172.21.15.148:0/3136026978 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc034070 con 0x7f73dc06c9c0 2024-03-29T04:12:47.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.487+0000 7f73d3fff640 1 -- 172.21.15.148:0/3136026978 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73cc030860 con 0x7f73dc06c9c0 2024-03-29T04:12:47.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.487+0000 7f73d3fff640 1 -- 172.21.15.148:0/3136026978 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc030c30 con 0x7f73dc06c9c0 2024-03-29T04:12:47.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.487+0000 7f73e3675640 1 -- 172.21.15.148:0/3136026978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73dc06c9c0 msgr2=0x7f73dc15fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.487+0000 7f73e3675640 1 --2- 172.21.15.148:0/3136026978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73dc06c9c0 0x7f73dc15fd80 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f73dc0b6360 tx=0x7f73cc02b9a0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.487+0000 7f73e3675640 1 -- 172.21.15.148:0/3136026978 shutdown_connections 2024-03-29T04:12:47.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.487+0000 7f73e3675640 1 --2- 172.21.15.148:0/3136026978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73dc06c9c0 0x7f73dc15fd80 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.487+0000 7f73e3675640 1 --2- 172.21.15.148:0/3136026978 >> v2:172.21.15.157:3300/0 conn(0x7f73dc145b40 0x7f73dc06ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.487+0000 7f73e3675640 1 -- 172.21.15.148:0/3136026978 >> 172.21.15.148:0/3136026978 conn(0x7f73dc072490 msgr2=0x7f73dc0728a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:47.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.487+0000 7f73e3675640 1 -- 172.21.15.148:0/3136026978 shutdown_connections 2024-03-29T04:12:47.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.487+0000 7f73e3675640 1 -- 172.21.15.148:0/3136026978 wait complete. 2024-03-29T04:12:47.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.488+0000 7f73e3675640 1 Processor -- start 2024-03-29T04:12:47.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.489+0000 7f73e3675640 1 -- start start 2024-03-29T04:12:47.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.489+0000 7f73e3675640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73dc06c9c0 0x7f73dc142a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.489+0000 7f73e3675640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73dc06f200 0x7f73dc1f1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.489+0000 7f73e3675640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73dc145b40 0x7f73dc157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.489+0000 7f73e3675640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc06b5b0 con 0x7f73dc06f200 2024-03-29T04:12:47.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.489+0000 7f73e3675640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc06c4a0 con 0x7f73dc06c9c0 2024-03-29T04:12:47.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.489+0000 7f73e3675640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f73dc1602c0 con 0x7f73dc145b40 2024-03-29T04:12:47.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.489+0000 7f73e0c10640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73dc06f200 0x7f73dc1f1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.489+0000 7f73e0c10640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73dc06f200 0x7f73dc1f1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56576/0 (socket says 172.21.15.148:56576) 2024-03-29T04:12:47.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.489+0000 7f73e0c10640 1 -- 172.21.15.148:0/1171520766 learned_addr learned my addr 172.21.15.148:0/1171520766 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:47.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.490+0000 7f73e0c10640 1 -- 172.21.15.148:0/1171520766 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73dc145b40 msgr2=0x7f73dc157c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.490+0000 7f73e0c10640 1 --2- 172.21.15.148:0/1171520766 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73dc145b40 0x7f73dc157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.490+0000 7f73e0c10640 1 -- 172.21.15.148:0/1171520766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73dc06c9c0 msgr2=0x7f73dc142a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.490+0000 7f73e0c10640 1 --2- 172.21.15.148:0/1171520766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73dc06c9c0 0x7f73dc142a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.490+0000 7f73e0c10640 1 -- 172.21.15.148:0/1171520766 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73dc15b210 con 0x7f73dc06f200 2024-03-29T04:12:47.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.490+0000 7f73e0c10640 1 --2- 172.21.15.148:0/1171520766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73dc06f200 0x7f73dc1f1de0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f73cc02b9e0 tx=0x7f73cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:47.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.490+0000 7f73d1ffb640 1 -- 172.21.15.148:0/1171520766 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc048070 con 0x7f73dc06f200 2024-03-29T04:12:47.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.490+0000 7f73d1ffb640 1 -- 172.21.15.148:0/1171520766 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73cc030660 con 0x7f73dc06f200 2024-03-29T04:12:47.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.490+0000 7f73e3675640 1 -- 172.21.15.148:0/1171520766 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73dc155900 con 0x7f73dc06f200 2024-03-29T04:12:47.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.490+0000 7f73d1ffb640 1 -- 172.21.15.148:0/1171520766 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc034040 con 0x7f73dc06f200 2024-03-29T04:12:47.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.490+0000 7f73e3675640 1 -- 172.21.15.148:0/1171520766 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73dc155ec0 con 0x7f73dc06f200 2024-03-29T04:12:47.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.491+0000 7f73e3675640 1 -- 172.21.15.148:0/1171520766 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f73dc068740 con 0x7f73dc06f200 2024-03-29T04:12:47.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.491+0000 7f73d1ffb640 1 -- 172.21.15.148:0/1171520766 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f73cc04b070 con 0x7f73dc06f200 2024-03-29T04:12:47.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.492+0000 7f73d1ffb640 1 --2- 172.21.15.148:0/1171520766 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f73c006c560 0x7f73c006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.492+0000 7f73d1ffb640 1 -- 172.21.15.148:0/1171520766 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f73cc0ad530 con 0x7f73dc06f200 2024-03-29T04:12:47.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.492+0000 7f73e1411640 1 --2- 172.21.15.148:0/1171520766 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f73c006c560 0x7f73c006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.492+0000 7f73d1ffb640 1 --2- 172.21.15.148:0/1171520766 >> [v2:172.21.15.148:6808/90026237,v1:172.21.15.148:6809/90026237] conn(0x7f73c0072a00 0x7f73c0074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.492+0000 7f73d1ffb640 1 -- 172.21.15.148:0/1171520766 --> [v2:172.21.15.148:6808/90026237,v1:172.21.15.148:6809/90026237] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f73cc001d30 con 0x7f73c0072a00 2024-03-29T04:12:47.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.492+0000 7f73d1ffb640 1 -- 172.21.15.148:0/1171520766 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f73cc0b4050 con 0x7f73dc06f200 2024-03-29T04:12:47.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.492+0000 7f73e1c12640 1 --2- 172.21.15.148:0/1171520766 >> [v2:172.21.15.148:6808/90026237,v1:172.21.15.148:6809/90026237] conn(0x7f73c0072a00 0x7f73c0074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.492+0000 7f73e1411640 1 --2- 172.21.15.148:0/1171520766 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f73c006c560 0x7f73c006ea20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f73d80070b0 tx=0x7f73d8007040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:47.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.493+0000 7f73e1c12640 1 --2- 172.21.15.148:0/1171520766 >> [v2:172.21.15.148:6808/90026237,v1:172.21.15.148:6809/90026237] conn(0x7f73c0072a00 0x7f73c0074e40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:47.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.493+0000 7f73d1ffb640 1 -- 172.21.15.148:0/1171520766 <== osd.2 v2:172.21.15.148:6808/90026237 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f73cc001d30 con 0x7f73c0072a00 2024-03-29T04:12:47.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.506+0000 7f73e3675640 1 -- 172.21.15.148:0/1171520766 --> [v2:172.21.15.148:6808/90026237,v1:172.21.15.148:6809/90026237] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f73dc0736b0 con 0x7f73c0072a00 2024-03-29T04:12:47.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.506+0000 7f73d1ffb640 1 -- 172.21.15.148:0/1171520766 <== osd.2 v2:172.21.15.148:6808/90026237 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f73dc0736b0 con 0x7f73c0072a00 2024-03-29T04:12:47.486 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-03-29T04:12:47.487 INFO:teuthology.orchestra.run.smithi148.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:12:47.487 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-03-29T04:12:47.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.507+0000 7f73e3675640 1 -- 172.21.15.148:0/1171520766 >> [v2:172.21.15.148:6808/90026237,v1:172.21.15.148:6809/90026237] conn(0x7f73c0072a00 msgr2=0x7f73c0074e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.507+0000 7f73e3675640 1 --2- 172.21.15.148:0/1171520766 >> [v2:172.21.15.148:6808/90026237,v1:172.21.15.148:6809/90026237] conn(0x7f73c0072a00 0x7f73c0074e40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.507+0000 7f73e3675640 1 -- 172.21.15.148:0/1171520766 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f73c006c560 msgr2=0x7f73c006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.507+0000 7f73e3675640 1 --2- 172.21.15.148:0/1171520766 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f73c006c560 0x7f73c006ea20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f73d80070b0 tx=0x7f73d8007040 comp rx=0 tx=0).stop 2024-03-29T04:12:47.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.507+0000 7f73e3675640 1 -- 172.21.15.148:0/1171520766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73dc06f200 msgr2=0x7f73dc1f1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.507+0000 7f73e3675640 1 --2- 172.21.15.148:0/1171520766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73dc06f200 0x7f73dc1f1de0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f73cc02b9e0 tx=0x7f73cc004730 comp rx=0 tx=0).stop 2024-03-29T04:12:47.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.507+0000 7f73e1c12640 1 -- 172.21.15.148:0/1171520766 reap_dead start 2024-03-29T04:12:47.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.507+0000 7f73e3675640 1 -- 172.21.15.148:0/1171520766 shutdown_connections 2024-03-29T04:12:47.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.507+0000 7f73e3675640 1 -- 172.21.15.148:0/1171520766 >> 172.21.15.148:0/1171520766 conn(0x7f73dc072490 msgr2=0x7f73dc0cd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:47.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.507+0000 7f73e3675640 1 -- 172.21.15.148:0/1171520766 shutdown_connections 2024-03-29T04:12:47.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.507+0000 7f73e3675640 1 -- 172.21.15.148:0/1171520766 wait complete. 2024-03-29T04:12:47.497 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-29T04:12:47.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.618+0000 7f2d036b8640 1 Processor -- start 2024-03-29T04:12:47.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.619+0000 7f2d036b8640 1 -- start start 2024-03-29T04:12:47.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.619+0000 7f2d036b8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2cfc0d17e0 0x7f2cfc0d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.619+0000 7f2d036b8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cfc15e7c0 con 0x7f2cfc15e3e0 2024-03-29T04:12:47.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.619+0000 7f2d036b8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cfc0d2c30 con 0x7f2cfc0ba030 2024-03-29T04:12:47.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.619+0000 7f2d036b8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f2cfc0d3840 con 0x7f2cfc0d17e0 2024-03-29T04:12:47.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.620+0000 7f2d01454640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2cfc15e3e0 0x7f2cfc0d2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44012/0 (socket says 172.21.15.148:44012) 2024-03-29T04:12:47.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.620+0000 7f2d01454640 1 -- 172.21.15.148:0/4290174404 learned_addr learned my addr 172.21.15.148:0/4290174404 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:47.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.620+0000 7f2d01c55640 1 --2- 172.21.15.148:0/4290174404 >> v2:172.21.15.157:3300/0 conn(0x7f2cfc0d17e0 0x7f2cfc0d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.620+0000 7f2ceffff640 1 -- 172.21.15.148:0/4290174404 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2760067121 0 0) 0x7f2cfc15e7c0 con 0x7f2cfc15e3e0 2024-03-29T04:12:47.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.620+0000 7f2ceffff640 1 -- 172.21.15.148:0/4290174404 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ce8003520 con 0x7f2cfc15e3e0 2024-03-29T04:12:47.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.620+0000 7f2ceffff640 1 -- 172.21.15.148:0/4290174404 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3155498401 0 0) 0x7f2cfc0d2c30 con 0x7f2cfc0ba030 2024-03-29T04:12:47.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.620+0000 7f2ceffff640 1 -- 172.21.15.148:0/4290174404 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2cfc15e7c0 con 0x7f2cfc0ba030 2024-03-29T04:12:47.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.620+0000 7f2ceffff640 1 -- 172.21.15.148:0/4290174404 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4215846787 0 0) 0x7f2ce8003520 con 0x7f2cfc15e3e0 2024-03-29T04:12:47.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.620+0000 7f2ceffff640 1 -- 172.21.15.148:0/4290174404 >> v2:172.21.15.157:3300/0 conn(0x7f2cfc0d17e0 msgr2=0x7f2cfc0d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:47.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.620+0000 7f2ceffff640 1 --2- 172.21.15.148:0/4290174404 >> v2:172.21.15.157:3300/0 conn(0x7f2cfc0d17e0 0x7f2cfc0d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.620+0000 7f2ceffff640 1 -- 172.21.15.148:0/4290174404 >> v1:172.21.15.157:6789/0 conn(0x7f2cfc0ba030 legacy=0x7f2cfc0d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.620+0000 7f2ceffff640 1 -- 172.21.15.148:0/4290174404 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cfc0d2f10 con 0x7f2cfc15e3e0 2024-03-29T04:12:47.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.620+0000 7f2ceffff640 1 -- 172.21.15.148:0/4290174404 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f2cf8001670 con 0x7f2cfc15e3e0 2024-03-29T04:12:47.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.620+0000 7f2ceffff640 1 -- 172.21.15.148:0/4290174404 >> v1:172.21.15.148:6789/0 conn(0x7f2cfc15e3e0 legacy=0x7f2cfc0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.620+0000 7f2ceffff640 1 --2- 172.21.15.148:0/4290174404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ce8003b40 0x7f2ce8003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.620+0000 7f2ceffff640 1 --2- 172.21.15.148:0/4290174404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ce8004800 0x7f2ce8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.620+0000 7f2ceffff640 1 --2- 172.21.15.148:0/4290174404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce80051c0 0x7f2ce800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.620+0000 7f2ceffff640 1 -- 172.21.15.148:0/4290174404 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc0d3840 con 0x7f2ce8004800 2024-03-29T04:12:47.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.620+0000 7f2ceffff640 1 -- 172.21.15.148:0/4290174404 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce800bfb0 con 0x7f2ce80051c0 2024-03-29T04:12:47.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.620+0000 7f2ceffff640 1 -- 172.21.15.148:0/4290174404 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2ce800c1b0 con 0x7f2ce8003b40 2024-03-29T04:12:47.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.620+0000 7f2d00c53640 1 --2- 172.21.15.148:0/4290174404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ce8004800 0x7f2ce8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.620+0000 7f2d01454640 1 --2- 172.21.15.148:0/4290174404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ce8003b40 0x7f2ce8003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.620+0000 7f2d01c55640 1 --2- 172.21.15.148:0/4290174404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce80051c0 0x7f2ce800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.621+0000 7f2d00c53640 1 -- 172.21.15.148:0/4290174404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ce8003b40 msgr2=0x7f2ce8003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:47.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.621+0000 7f2d00c53640 1 --2- 172.21.15.148:0/4290174404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ce8003b40 0x7f2ce8003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.621+0000 7f2d00c53640 1 -- 172.21.15.148:0/4290174404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce80051c0 msgr2=0x7f2ce800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:47.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.621+0000 7f2d00c53640 1 --2- 172.21.15.148:0/4290174404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce80051c0 0x7f2ce800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.621+0000 7f2d00c53640 1 -- 172.21.15.148:0/4290174404 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce8003520 con 0x7f2ce8004800 2024-03-29T04:12:47.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.621+0000 7f2d01c55640 1 -- 172.21.15.148:0/4290174404 reap_dead start 2024-03-29T04:12:47.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.621+0000 7f2d00c53640 1 --2- 172.21.15.148:0/4290174404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ce8004800 0x7f2ce8004c80 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2cf00018a0 tx=0x7f2ce8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9f7cbfe0f20a3e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:47.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.621+0000 7f2ceffff640 1 -- 172.21.15.148:0/4290174404 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf002f050 con 0x7f2ce8004800 2024-03-29T04:12:47.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.621+0000 7f2ceffff640 1 -- 172.21.15.148:0/4290174404 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cf00052d0 con 0x7f2ce8004800 2024-03-29T04:12:47.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.622+0000 7f2ceffff640 1 -- 172.21.15.148:0/4290174404 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf00014f0 con 0x7f2ce8004800 2024-03-29T04:12:47.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.622+0000 7f2d036b8640 1 -- 172.21.15.148:0/4290174404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ce8004800 msgr2=0x7f2ce8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.622+0000 7f2d036b8640 1 --2- 172.21.15.148:0/4290174404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ce8004800 0x7f2ce8004c80 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2cf00018a0 tx=0x7f2ce8005830 comp rx=0 tx=0).stop 2024-03-29T04:12:47.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.622+0000 7f2d036b8640 1 -- 172.21.15.148:0/4290174404 shutdown_connections 2024-03-29T04:12:47.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.622+0000 7f2d036b8640 1 --2- 172.21.15.148:0/4290174404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ce8004800 0x7f2ce8004c80 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.622+0000 7f2d036b8640 1 -- 172.21.15.148:0/4290174404 >> 172.21.15.148:0/4290174404 conn(0x7f2cfc0b6210 msgr2=0x7f2cfc0b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:47.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.622+0000 7f2d036b8640 1 -- 172.21.15.148:0/4290174404 shutdown_connections 2024-03-29T04:12:47.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.622+0000 7f2d036b8640 1 -- 172.21.15.148:0/4290174404 wait complete. 2024-03-29T04:12:47.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.623+0000 7f2d036b8640 1 Processor -- start 2024-03-29T04:12:47.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.623+0000 7f2d036b8640 1 -- start start 2024-03-29T04:12:47.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.624+0000 7f2d036b8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ce8004800 0x7f2cfc153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.624+0000 7f2d036b8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cfc153a10 0x7f2cfc0cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.624+0000 7f2d036b8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cfc156b20 0x7f2cfc156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.624+0000 7f2d036b8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc0d3cd0 con 0x7f2ce8004800 2024-03-29T04:12:47.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.624+0000 7f2d036b8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc0d34d0 con 0x7f2cfc156b20 2024-03-29T04:12:47.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.624+0000 7f2d036b8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2cfc0d3e50 con 0x7f2cfc153a10 2024-03-29T04:12:47.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.624+0000 7f2d01454640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ce8004800 0x7f2cfc153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.624+0000 7f2d01454640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ce8004800 0x7f2cfc153350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56594/0 (socket says 172.21.15.148:56594) 2024-03-29T04:12:47.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.624+0000 7f2d01454640 1 -- 172.21.15.148:0/805392231 learned_addr learned my addr 172.21.15.148:0/805392231 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:47.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.624+0000 7f2d01c55640 1 --2- 172.21.15.148:0/805392231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cfc156b20 0x7f2cfc156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.624+0000 7f2d00c53640 1 --2- 172.21.15.148:0/805392231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cfc153a10 0x7f2cfc0cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.624+0000 7f2d01454640 1 -- 172.21.15.148:0/805392231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cfc153a10 msgr2=0x7f2cfc0cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.624+0000 7f2d01454640 1 --2- 172.21.15.148:0/805392231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cfc153a10 0x7f2cfc0cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.624+0000 7f2d01454640 1 -- 172.21.15.148:0/805392231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cfc156b20 msgr2=0x7f2cfc156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.624+0000 7f2d01454640 1 --2- 172.21.15.148:0/805392231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cfc156b20 0x7f2cfc156fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.624+0000 7f2d01454640 1 -- 172.21.15.148:0/805392231 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cfc1f45c0 con 0x7f2ce8004800 2024-03-29T04:12:47.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.624+0000 7f2d00c53640 1 --2- 172.21.15.148:0/805392231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cfc153a10 0x7f2cfc0cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:12:47.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.624+0000 7f2d01454640 1 --2- 172.21.15.148:0/805392231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ce8004800 0x7f2cfc153350 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8007340 tx=0x7f2cf8007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:47.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.625+0000 7f2cedffb640 1 -- 172.21.15.148:0/805392231 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf800f860 con 0x7f2ce8004800 2024-03-29T04:12:47.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.625+0000 7f2cedffb640 1 -- 172.21.15.148:0/805392231 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cf8018070 con 0x7f2ce8004800 2024-03-29T04:12:47.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.625+0000 7f2d036b8640 1 -- 172.21.15.148:0/805392231 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cfc1f48a0 con 0x7f2ce8004800 2024-03-29T04:12:47.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.625+0000 7f2cedffb640 1 -- 172.21.15.148:0/805392231 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf800fc40 con 0x7f2ce8004800 2024-03-29T04:12:47.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.625+0000 7f2d036b8640 1 -- 172.21.15.148:0/805392231 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cfc1f4e60 con 0x7f2ce8004800 2024-03-29T04:12:47.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.626+0000 7f2d036b8640 1 -- 172.21.15.148:0/805392231 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cfc0687e0 con 0x7f2ce8004800 2024-03-29T04:12:47.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.626+0000 7f2cedffb640 1 -- 172.21.15.148:0/805392231 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2cf801d460 con 0x7f2ce8004800 2024-03-29T04:12:47.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.627+0000 7f2cedffb640 1 --2- 172.21.15.148:0/805392231 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f2ce006c530 0x7f2ce006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.627+0000 7f2d00c53640 1 --2- 172.21.15.148:0/805392231 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f2ce006c530 0x7f2ce006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.627+0000 7f2cedffb640 1 -- 172.21.15.148:0/805392231 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f2cf808b540 con 0x7f2ce8004800 2024-03-29T04:12:47.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.627+0000 7f2d00c53640 1 --2- 172.21.15.148:0/805392231 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f2ce006c530 0x7f2ce006e9f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2cf00018a0 tx=0x7f2cf0031000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:47.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.631+0000 7f2cedffb640 1 -- 172.21.15.148:0/805392231 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2cf8059710 con 0x7f2ce8004800 2024-03-29T04:12:47.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.807+0000 7f2d036b8640 1 -- 172.21.15.148:0/805392231 --> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f2cfc06ca40 con 0x7f2ce006c530 2024-03-29T04:12:47.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.807+0000 7f2cedffb640 1 -- 172.21.15.148:0/805392231 <== mgr.4113 v2:172.21.15.148:6832/3271445075 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2cfc06ca40 con 0x7f2ce006c530 2024-03-29T04:12:47.787 INFO:teuthology.orchestra.run.smithi148.stderr:instructed osd(s) 2 to deep-scrub 2024-03-29T04:12:47.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.809+0000 7f2d036b8640 1 -- 172.21.15.148:0/805392231 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f2ce006c530 msgr2=0x7f2ce006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.809+0000 7f2d036b8640 1 --2- 172.21.15.148:0/805392231 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f2ce006c530 0x7f2ce006e9f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2cf00018a0 tx=0x7f2cf0031000 comp rx=0 tx=0).stop 2024-03-29T04:12:47.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.809+0000 7f2d036b8640 1 -- 172.21.15.148:0/805392231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ce8004800 msgr2=0x7f2cfc153350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.809+0000 7f2d036b8640 1 --2- 172.21.15.148:0/805392231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ce8004800 0x7f2cfc153350 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8007340 tx=0x7f2cf8007800 comp rx=0 tx=0).stop 2024-03-29T04:12:47.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.809+0000 7f2d036b8640 1 -- 172.21.15.148:0/805392231 shutdown_connections 2024-03-29T04:12:47.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.809+0000 7f2d036b8640 1 --2- 172.21.15.148:0/805392231 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f2ce006c530 0x7f2ce006e9f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.809+0000 7f2d036b8640 1 --2- 172.21.15.148:0/805392231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cfc156b20 0x7f2cfc156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.809+0000 7f2d036b8640 1 --2- 172.21.15.148:0/805392231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2cfc153a10 0x7f2cfc0cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.809+0000 7f2d036b8640 1 --2- 172.21.15.148:0/805392231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ce8004800 0x7f2cfc153350 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.809+0000 7f2d036b8640 1 -- 172.21.15.148:0/805392231 >> 172.21.15.148:0/805392231 conn(0x7f2cfc0b6210 msgr2=0x7f2cfc0b82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:47.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.809+0000 7f2d036b8640 1 -- 172.21.15.148:0/805392231 shutdown_connections 2024-03-29T04:12:47.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.809+0000 7f2d036b8640 1 -- 172.21.15.148:0/805392231 wait complete. 2024-03-29T04:12:47.800 INFO:tasks.ceph:Scrubbing osd.3 2024-03-29T04:12:47.800 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:12:47.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.925+0000 7fbbf7870640 1 Processor -- start 2024-03-29T04:12:47.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.925+0000 7fbbf7870640 1 -- start start 2024-03-29T04:12:47.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.926+0000 7fbbf7870640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbbf0145160 0x7fbbf0073470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.926+0000 7fbbf7870640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbf00cb950 con 0x7fbbf00cc9f0 2024-03-29T04:12:47.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.926+0000 7fbbf7870640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbf00cbc30 con 0x7fbbf00b5740 2024-03-29T04:12:47.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.926+0000 7fbbf7870640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fbbf00cbf10 con 0x7fbbf0145160 2024-03-29T04:12:47.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.926+0000 7fbbf4e0b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbbf0145160 0x7fbbf0073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.926+0000 7fbbf5e0d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbbf00cc9f0 0x7fbbf015c810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44020/0 (socket says 172.21.15.148:44020) 2024-03-29T04:12:47.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.926+0000 7fbbf5e0d640 1 -- 172.21.15.148:0/974628755 learned_addr learned my addr 172.21.15.148:0/974628755 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:47.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.926+0000 7fbbf4e0b640 1 -- 172.21.15.148:0/974628755 >> v1:172.21.15.157:6789/0 conn(0x7fbbf00b5740 legacy=0x7fbbf00b5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:47.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.926+0000 7fbbf4e0b640 1 -- 172.21.15.148:0/974628755 >> v1:172.21.15.148:6789/0 conn(0x7fbbf00cc9f0 legacy=0x7fbbf015c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.926+0000 7fbbf4e0b640 1 -- 172.21.15.148:0/974628755 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf00cc1f0 con 0x7fbbf0145160 2024-03-29T04:12:47.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.927+0000 7fbbf4e0b640 1 --2- 172.21.15.148:0/974628755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbf0145160 0x7fbbf0073470 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00097a0 tx=0x7fbbe002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4574d1b0db133c5c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:47.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.927+0000 7fbbe7fff640 1 -- 172.21.15.148:0/974628755 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0035020 con 0x7fbbf0145160 2024-03-29T04:12:47.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.927+0000 7fbbe7fff640 1 -- 172.21.15.148:0/974628755 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbe0031830 con 0x7fbbf0145160 2024-03-29T04:12:47.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.927+0000 7fbbe7fff640 1 -- 172.21.15.148:0/974628755 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0031c00 con 0x7fbbf0145160 2024-03-29T04:12:47.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.928+0000 7fbbf7870640 1 -- 172.21.15.148:0/974628755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbf0145160 msgr2=0x7fbbf0073470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.928+0000 7fbbf7870640 1 --2- 172.21.15.148:0/974628755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbf0145160 0x7fbbf0073470 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00097a0 tx=0x7fbbe002b8d0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.928+0000 7fbbf7870640 1 -- 172.21.15.148:0/974628755 shutdown_connections 2024-03-29T04:12:47.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.928+0000 7fbbf7870640 1 --2- 172.21.15.148:0/974628755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbf0145160 0x7fbbf0073470 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.928+0000 7fbbf7870640 1 -- 172.21.15.148:0/974628755 >> 172.21.15.148:0/974628755 conn(0x7fbbf0072490 msgr2=0x7fbbf0072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:47.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.928+0000 7fbbf7870640 1 -- 172.21.15.148:0/974628755 shutdown_connections 2024-03-29T04:12:47.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.928+0000 7fbbf7870640 1 -- 172.21.15.148:0/974628755 wait complete. 2024-03-29T04:12:47.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.929+0000 7fbbf7870640 1 Processor -- start 2024-03-29T04:12:47.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.929+0000 7fbbf7870640 1 -- start start 2024-03-29T04:12:47.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.930+0000 7fbbf7870640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbf00b5740 0x7fbbf013be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.930+0000 7fbbf7870640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbf00cc9f0 0x7fbbf0158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.930+0000 7fbbf7870640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbf015f3f0 0x7fbbf006bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.930+0000 7fbbf7870640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf00cb570 con 0x7fbbf00b5740 2024-03-29T04:12:47.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.930+0000 7fbbf7870640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf00cce20 con 0x7fbbf015f3f0 2024-03-29T04:12:47.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.930+0000 7fbbf7870640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbbf00cb6f0 con 0x7fbbf00cc9f0 2024-03-29T04:12:47.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.930+0000 7fbbf560c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbf00b5740 0x7fbbf013be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.930+0000 7fbbf560c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbf00b5740 0x7fbbf013be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56612/0 (socket says 172.21.15.148:56612) 2024-03-29T04:12:47.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.930+0000 7fbbf560c640 1 -- 172.21.15.148:0/4098874488 learned_addr learned my addr 172.21.15.148:0/4098874488 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:47.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.930+0000 7fbbf560c640 1 -- 172.21.15.148:0/4098874488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbf00cc9f0 msgr2=0x7fbbf0158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.930+0000 7fbbf560c640 1 --2- 172.21.15.148:0/4098874488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbf00cc9f0 0x7fbbf0158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.930+0000 7fbbf560c640 1 -- 172.21.15.148:0/4098874488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbf015f3f0 msgr2=0x7fbbf006bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.930+0000 7fbbf560c640 1 --2- 172.21.15.148:0/4098874488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbf015f3f0 0x7fbbf006bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.930+0000 7fbbf560c640 1 -- 172.21.15.148:0/4098874488 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf013c460 con 0x7fbbf00b5740 2024-03-29T04:12:47.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.930+0000 7fbbf560c640 1 --2- 172.21.15.148:0/4098874488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbf00b5740 0x7fbbf013be60 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00c7f0 tx=0x7fbbec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:47.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.931+0000 7fbbe5ffb640 1 -- 172.21.15.148:0/4098874488 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec016020 con 0x7fbbf00b5740 2024-03-29T04:12:47.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.931+0000 7fbbe5ffb640 1 -- 172.21.15.148:0/4098874488 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbec00aa50 con 0x7fbbf00b5740 2024-03-29T04:12:47.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.931+0000 7fbbe5ffb640 1 -- 172.21.15.148:0/4098874488 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec00ae40 con 0x7fbbf00b5740 2024-03-29T04:12:47.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.931+0000 7fbbf7870640 1 -- 172.21.15.148:0/4098874488 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf013c740 con 0x7fbbf00b5740 2024-03-29T04:12:47.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.931+0000 7fbbf7870640 1 -- 172.21.15.148:0/4098874488 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf013cd00 con 0x7fbbf00b5740 2024-03-29T04:12:47.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.932+0000 7fbbf7870640 1 -- 172.21.15.148:0/4098874488 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbbf0068740 con 0x7fbbf00b5740 2024-03-29T04:12:47.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.932+0000 7fbbe5ffb640 1 -- 172.21.15.148:0/4098874488 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbbec019020 con 0x7fbbf00b5740 2024-03-29T04:12:47.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.932+0000 7fbbe5ffb640 1 --2- 172.21.15.148:0/4098874488 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbbd406c560 0x7fbbd406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.932+0000 7fbbe5ffb640 1 -- 172.21.15.148:0/4098874488 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7fbbec08b2f0 con 0x7fbbf00b5740 2024-03-29T04:12:47.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.932+0000 7fbbf4e0b640 1 --2- 172.21.15.148:0/4098874488 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbbd406c560 0x7fbbd406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.933+0000 7fbbe5ffb640 1 --2- 172.21.15.148:0/4098874488 >> [v2:172.21.15.148:6816/466300755,v1:172.21.15.148:6817/466300755] conn(0x7fbbd4072a00 0x7fbbd4074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:47.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.933+0000 7fbbe5ffb640 1 -- 172.21.15.148:0/4098874488 --> [v2:172.21.15.148:6816/466300755,v1:172.21.15.148:6817/466300755] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbbec01ce50 con 0x7fbbd4072a00 2024-03-29T04:12:47.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.933+0000 7fbbe5ffb640 1 -- 172.21.15.148:0/4098874488 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fbbec092050 con 0x7fbbf00b5740 2024-03-29T04:12:47.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.933+0000 7fbbf5e0d640 1 --2- 172.21.15.148:0/4098874488 >> [v2:172.21.15.148:6816/466300755,v1:172.21.15.148:6817/466300755] conn(0x7fbbd4072a00 0x7fbbd4074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:47.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.933+0000 7fbbf4e0b640 1 --2- 172.21.15.148:0/4098874488 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbbd406c560 0x7fbbd406ea20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0036000 tx=0x7fbbe0030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:47.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.933+0000 7fbbf5e0d640 1 --2- 172.21.15.148:0/4098874488 >> [v2:172.21.15.148:6816/466300755,v1:172.21.15.148:6817/466300755] conn(0x7fbbd4072a00 0x7fbbd4074e40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:47.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.934+0000 7fbbe5ffb640 1 -- 172.21.15.148:0/4098874488 <== osd.3 v2:172.21.15.148:6816/466300755 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbbec01ce50 con 0x7fbbd4072a00 2024-03-29T04:12:47.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.947+0000 7fbbf7870640 1 -- 172.21.15.148:0/4098874488 --> [v2:172.21.15.148:6816/466300755,v1:172.21.15.148:6817/466300755] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbbf0143c50 con 0x7fbbd4072a00 2024-03-29T04:12:47.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.947+0000 7fbbe5ffb640 1 -- 172.21.15.148:0/4098874488 <== osd.3 v2:172.21.15.148:6816/466300755 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbbf0143c50 con 0x7fbbd4072a00 2024-03-29T04:12:47.927 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-03-29T04:12:47.927 INFO:teuthology.orchestra.run.smithi148.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:12:47.927 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-03-29T04:12:47.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.947+0000 7fbbf7870640 1 -- 172.21.15.148:0/4098874488 >> [v2:172.21.15.148:6816/466300755,v1:172.21.15.148:6817/466300755] conn(0x7fbbd4072a00 msgr2=0x7fbbd4074e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.947+0000 7fbbf7870640 1 --2- 172.21.15.148:0/4098874488 >> [v2:172.21.15.148:6816/466300755,v1:172.21.15.148:6817/466300755] conn(0x7fbbd4072a00 0x7fbbd4074e40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.947+0000 7fbbf7870640 1 -- 172.21.15.148:0/4098874488 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbbd406c560 msgr2=0x7fbbd406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.947+0000 7fbbf7870640 1 --2- 172.21.15.148:0/4098874488 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fbbd406c560 0x7fbbd406ea20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0036000 tx=0x7fbbe0030000 comp rx=0 tx=0).stop 2024-03-29T04:12:47.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.947+0000 7fbbf7870640 1 -- 172.21.15.148:0/4098874488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbf00b5740 msgr2=0x7fbbf013be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:47.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.947+0000 7fbbf7870640 1 --2- 172.21.15.148:0/4098874488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbf00b5740 0x7fbbf013be60 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00c7f0 tx=0x7fbbec00ccb0 comp rx=0 tx=0).stop 2024-03-29T04:12:47.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.947+0000 7fbbf5e0d640 1 -- 172.21.15.148:0/4098874488 reap_dead start 2024-03-29T04:12:47.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.947+0000 7fbbf7870640 1 -- 172.21.15.148:0/4098874488 shutdown_connections 2024-03-29T04:12:47.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.947+0000 7fbbf7870640 1 -- 172.21.15.148:0/4098874488 >> 172.21.15.148:0/4098874488 conn(0x7fbbf0072490 msgr2=0x7fbbf006f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:47.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.947+0000 7fbbf7870640 1 -- 172.21.15.148:0/4098874488 shutdown_connections 2024-03-29T04:12:47.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:47.947+0000 7fbbf7870640 1 -- 172.21.15.148:0/4098874488 wait complete. 2024-03-29T04:12:47.937 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-29T04:12:48.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.059+0000 7fa2710e4640 1 Processor -- start 2024-03-29T04:12:48.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.059+0000 7fa2710e4640 1 -- start start 2024-03-29T04:12:48.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.060+0000 7fa2710e4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa26c15a740 0x7fa26c15e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.060+0000 7fa2710e4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa26c15e910 0x7fa26c15ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.060+0000 7fa2710e4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa26c0cf790 con 0x7fa26c06d6a0 2024-03-29T04:12:48.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.060+0000 7fa2710e4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fa26c0728d0 con 0x7fa26c15e910 2024-03-29T04:12:48.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.060+0000 7fa2710e4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa26c072bb0 con 0x7fa26c15a740 2024-03-29T04:12:48.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.060+0000 7fa26a59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa26c15e910 0x7fa26c15ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.060+0000 7fa26b59e640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa26c06d6a0 0x7fa26c06dae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44034/0 (socket says 172.21.15.148:44034) 2024-03-29T04:12:48.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.060+0000 7fa26b59e640 1 -- 172.21.15.148:0/2464240762 learned_addr learned my addr 172.21.15.148:0/2464240762 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:48.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.060+0000 7fa26ad9d640 1 --2- 172.21.15.148:0/2464240762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa26c15a740 0x7fa26c15e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.060+0000 7fa26a59c640 1 -- 172.21.15.148:0/2464240762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa26c15a740 msgr2=0x7fa26c15e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:48.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.060+0000 7fa26a59c640 1 --2- 172.21.15.148:0/2464240762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa26c15a740 0x7fa26c15e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.060+0000 7fa269d9b640 1 -- 172.21.15.148:0/2464240762 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1709006501 0 0) 0x7fa264001110 con 0x7fa26c06d6a0 2024-03-29T04:12:48.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.060+0000 7fa26a59c640 1 -- 172.21.15.148:0/2464240762 >> v1:172.21.15.148:6789/0 conn(0x7fa26c06d6a0 legacy=0x7fa26c06dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.060+0000 7fa26a59c640 1 -- 172.21.15.148:0/2464240762 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa26c072e90 con 0x7fa26c15e910 2024-03-29T04:12:48.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.061+0000 7fa26a59c640 1 --2- 172.21.15.148:0/2464240762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa26c15e910 0x7fa26c15ed70 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa2600097a0 tx=0x7fa26002c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7092a563950774fa server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:48.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.061+0000 7fa269d9b640 1 -- 172.21.15.148:0/2464240762 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa260035070 con 0x7fa26c15e910 2024-03-29T04:12:48.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.061+0000 7fa269d9b640 1 -- 172.21.15.148:0/2464240762 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa260031610 con 0x7fa26c15e910 2024-03-29T04:12:48.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.061+0000 7fa269d9b640 1 -- 172.21.15.148:0/2464240762 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2600319e0 con 0x7fa26c15e910 2024-03-29T04:12:48.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.062+0000 7fa2710e4640 1 -- 172.21.15.148:0/2464240762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa26c15e910 msgr2=0x7fa26c15ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.062+0000 7fa2710e4640 1 --2- 172.21.15.148:0/2464240762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa26c15e910 0x7fa26c15ed70 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa2600097a0 tx=0x7fa26002c950 comp rx=0 tx=0).stop 2024-03-29T04:12:48.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.062+0000 7fa2710e4640 1 -- 172.21.15.148:0/2464240762 shutdown_connections 2024-03-29T04:12:48.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.062+0000 7fa2710e4640 1 --2- 172.21.15.148:0/2464240762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa26c15e910 0x7fa26c15ed70 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.062+0000 7fa2710e4640 1 --2- 172.21.15.148:0/2464240762 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa26c15a740 0x7fa26c15e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.062+0000 7fa2710e4640 1 -- 172.21.15.148:0/2464240762 >> 172.21.15.148:0/2464240762 conn(0x7fa26c0cfed0 msgr2=0x7fa26c0d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:48.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.062+0000 7fa2710e4640 1 -- 172.21.15.148:0/2464240762 shutdown_connections 2024-03-29T04:12:48.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.062+0000 7fa2710e4640 1 -- 172.21.15.148:0/2464240762 wait complete. 2024-03-29T04:12:48.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.063+0000 7fa2710e4640 1 Processor -- start 2024-03-29T04:12:48.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.063+0000 7fa2710e4640 1 -- start start 2024-03-29T04:12:48.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.064+0000 7fa2710e4640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa26c06d6a0 0x7fa26c14e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.064+0000 7fa2710e4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa26c15a740 0x7fa26c1e7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.064+0000 7fa2710e4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa26c15e910 0x7fa26c1e8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.064+0000 7fa2710e4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa26c15aca0 con 0x7fa26c15e910 2024-03-29T04:12:48.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.064+0000 7fa2710e4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa26c15ab20 con 0x7fa26c15a740 2024-03-29T04:12:48.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.064+0000 7fa2710e4640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa26c0b3ef0 con 0x7fa26c06d6a0 2024-03-29T04:12:48.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.064+0000 7fa26b59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa26c15e910 0x7fa26c1e8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.064+0000 7fa26b59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa26c15e910 0x7fa26c1e8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56630/0 (socket says 172.21.15.148:56630) 2024-03-29T04:12:48.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.064+0000 7fa26b59e640 1 -- 172.21.15.148:0/770237286 learned_addr learned my addr 172.21.15.148:0/770237286 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:48.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.064+0000 7fa26ad9d640 1 --2- 172.21.15.148:0/770237286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa26c06d6a0 0x7fa26c14e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.064+0000 7fa26a59c640 1 --2- 172.21.15.148:0/770237286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa26c15a740 0x7fa26c1e7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.064+0000 7fa26b59e640 1 -- 172.21.15.148:0/770237286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa26c06d6a0 msgr2=0x7fa26c14e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.064+0000 7fa26b59e640 1 --2- 172.21.15.148:0/770237286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa26c06d6a0 0x7fa26c14e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.064+0000 7fa26b59e640 1 -- 172.21.15.148:0/770237286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa26c15a740 msgr2=0x7fa26c1e7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.064+0000 7fa26b59e640 1 --2- 172.21.15.148:0/770237286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa26c15a740 0x7fa26c1e7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.064+0000 7fa26b59e640 1 -- 172.21.15.148:0/770237286 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa26c14f1e0 con 0x7fa26c15e910 2024-03-29T04:12:48.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.064+0000 7fa26a59c640 1 --2- 172.21.15.148:0/770237286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa26c15a740 0x7fa26c1e7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:48.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.065+0000 7fa26b59e640 1 --2- 172.21.15.148:0/770237286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa26c15e910 0x7fa26c1e8100 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa26400c780 tx=0x7fa26400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:48.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.065+0000 7fa2537fe640 1 -- 172.21.15.148:0/770237286 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa264018070 con 0x7fa26c15e910 2024-03-29T04:12:48.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.065+0000 7fa2537fe640 1 -- 172.21.15.148:0/770237286 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2640070d0 con 0x7fa26c15e910 2024-03-29T04:12:48.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.065+0000 7fa2710e4640 1 -- 172.21.15.148:0/770237286 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa26c15db00 con 0x7fa26c15e910 2024-03-29T04:12:48.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.065+0000 7fa2537fe640 1 -- 172.21.15.148:0/770237286 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa264007500 con 0x7fa26c15e910 2024-03-29T04:12:48.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.065+0000 7fa2710e4640 1 -- 172.21.15.148:0/770237286 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa26c15bd90 con 0x7fa26c15e910 2024-03-29T04:12:48.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.066+0000 7fa2710e4640 1 -- 172.21.15.148:0/770237286 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa26c0687e0 con 0x7fa26c15e910 2024-03-29T04:12:48.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.066+0000 7fa2537fe640 1 -- 172.21.15.148:0/770237286 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa2640047e0 con 0x7fa26c15e910 2024-03-29T04:12:48.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.067+0000 7fa2537fe640 1 --2- 172.21.15.148:0/770237286 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa25806c530 0x7fa25806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.067+0000 7fa26ad9d640 1 --2- 172.21.15.148:0/770237286 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa25806c530 0x7fa25806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.067+0000 7fa2537fe640 1 -- 172.21.15.148:0/770237286 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7fa26408a830 con 0x7fa26c15e910 2024-03-29T04:12:48.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.067+0000 7fa26ad9d640 1 --2- 172.21.15.148:0/770237286 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa25806c530 0x7fa25806e9f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa26c1e56f0 tx=0x7fa25c0077b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:48.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.071+0000 7fa2537fe640 1 -- 172.21.15.148:0/770237286 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa26c14f1e0 con 0x7fa26c15e910 2024-03-29T04:12:48.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.247+0000 7fa2710e4640 1 -- 172.21.15.148:0/770237286 --> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fa26c071960 con 0x7fa25806c530 2024-03-29T04:12:48.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.247+0000 7fa2537fe640 1 -- 172.21.15.148:0/770237286 <== mgr.4113 v2:172.21.15.148:6832/3271445075 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa26c071960 con 0x7fa25806c530 2024-03-29T04:12:48.227 INFO:teuthology.orchestra.run.smithi148.stderr:instructed osd(s) 3 to deep-scrub 2024-03-29T04:12:48.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.249+0000 7fa2710e4640 1 -- 172.21.15.148:0/770237286 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa25806c530 msgr2=0x7fa25806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.249+0000 7fa2710e4640 1 --2- 172.21.15.148:0/770237286 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa25806c530 0x7fa25806e9f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa26c1e56f0 tx=0x7fa25c0077b0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.249+0000 7fa2710e4640 1 -- 172.21.15.148:0/770237286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa26c15e910 msgr2=0x7fa26c1e8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.249+0000 7fa2710e4640 1 --2- 172.21.15.148:0/770237286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa26c15e910 0x7fa26c1e8100 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa26400c780 tx=0x7fa26400cc40 comp rx=0 tx=0).stop 2024-03-29T04:12:48.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.249+0000 7fa2710e4640 1 -- 172.21.15.148:0/770237286 shutdown_connections 2024-03-29T04:12:48.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.249+0000 7fa2710e4640 1 --2- 172.21.15.148:0/770237286 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fa25806c530 0x7fa25806e9f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.249+0000 7fa2710e4640 1 --2- 172.21.15.148:0/770237286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa26c15e910 0x7fa26c1e8100 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.249+0000 7fa2710e4640 1 --2- 172.21.15.148:0/770237286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa26c15a740 0x7fa26c1e7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.249+0000 7fa2710e4640 1 --2- 172.21.15.148:0/770237286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa26c06d6a0 0x7fa26c14e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.249+0000 7fa2710e4640 1 -- 172.21.15.148:0/770237286 >> 172.21.15.148:0/770237286 conn(0x7fa26c0cfed0 msgr2=0x7fa26c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:48.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.249+0000 7fa2710e4640 1 -- 172.21.15.148:0/770237286 shutdown_connections 2024-03-29T04:12:48.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.249+0000 7fa2710e4640 1 -- 172.21.15.148:0/770237286 wait complete. 2024-03-29T04:12:48.240 INFO:tasks.ceph:Scrubbing osd.4 2024-03-29T04:12:48.241 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:12:48.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.365+0000 7f5829f1c640 1 Processor -- start 2024-03-29T04:12:48.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.365+0000 7f5829f1c640 1 -- start start 2024-03-29T04:12:48.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.366+0000 7f5829f1c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5824158bb0 0x7f58240ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.366+0000 7f5829f1c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58240681c0 con 0x7f58240ca100 2024-03-29T04:12:48.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.366+0000 7f5829f1c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f582415f600 con 0x7f582415ea60 2024-03-29T04:12:48.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.366+0000 7f5829f1c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f582415f8e0 con 0x7f5824158bb0 2024-03-29T04:12:48.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.366+0000 7f5823fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f58240ca100 0x7f58240ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44038/0 (socket says 172.21.15.148:44038) 2024-03-29T04:12:48.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.366+0000 7f5823fff640 1 -- 172.21.15.148:0/3687424309 learned_addr learned my addr 172.21.15.148:0/3687424309 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:48.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.366+0000 7f58227fc640 1 -- 172.21.15.148:0/3687424309 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 435390095 0 0) 0x7f58240681c0 con 0x7f58240ca100 2024-03-29T04:12:48.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.366+0000 7f58227fc640 1 -- 172.21.15.148:0/3687424309 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f580c003520 con 0x7f58240ca100 2024-03-29T04:12:48.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.366+0000 7f58237fe640 1 --2- 172.21.15.148:0/3687424309 >> v2:172.21.15.157:3300/0 conn(0x7f5824158bb0 0x7f58240ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.367+0000 7f58227fc640 1 -- 172.21.15.148:0/3687424309 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2980023395 0 0) 0x7f580c003520 con 0x7f58240ca100 2024-03-29T04:12:48.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.367+0000 7f58227fc640 1 -- 172.21.15.148:0/3687424309 >> v2:172.21.15.157:3300/0 conn(0x7f5824158bb0 msgr2=0x7f58240ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:48.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.367+0000 7f58227fc640 1 --2- 172.21.15.148:0/3687424309 >> v2:172.21.15.157:3300/0 conn(0x7f5824158bb0 0x7f58240ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.367+0000 7f58227fc640 1 -- 172.21.15.148:0/3687424309 >> v1:172.21.15.157:6789/0 conn(0x7f582415ea60 legacy=0x7f58240c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.367+0000 7f58227fc640 1 -- 172.21.15.148:0/3687424309 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58240cc2e0 con 0x7f58240ca100 2024-03-29T04:12:48.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.367+0000 7f58227fc640 1 -- 172.21.15.148:0/3687424309 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f581c001a10 con 0x7f58240ca100 2024-03-29T04:12:48.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.367+0000 7f58227fc640 1 -- 172.21.15.148:0/3687424309 >> v1:172.21.15.148:6789/0 conn(0x7f58240ca100 legacy=0x7f58240ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.367+0000 7f58227fc640 1 --2- 172.21.15.148:0/3687424309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f580c003a00 0x7f580c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.367+0000 7f58227fc640 1 --2- 172.21.15.148:0/3687424309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f580c004800 0x7f580c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.367+0000 7f58227fc640 1 --2- 172.21.15.148:0/3687424309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f580c0051c0 0x7f580c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.367+0000 7f58227fc640 1 -- 172.21.15.148:0/3687424309 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f582415f8e0 con 0x7f580c003a00 2024-03-29T04:12:48.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.367+0000 7f58227fc640 1 -- 172.21.15.148:0/3687424309 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f580c00c0c0 con 0x7f580c004800 2024-03-29T04:12:48.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.367+0000 7f58227fc640 1 -- 172.21.15.148:0/3687424309 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f580c00c2c0 con 0x7f580c0051c0 2024-03-29T04:12:48.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.367+0000 7f58237fe640 1 --2- 172.21.15.148:0/3687424309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f580c003a00 0x7f580c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.367+0000 7f5822ffd640 1 --2- 172.21.15.148:0/3687424309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f580c004800 0x7f580c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.367+0000 7f5823fff640 1 --2- 172.21.15.148:0/3687424309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f580c0051c0 0x7f580c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.367+0000 7f58237fe640 1 -- 172.21.15.148:0/3687424309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f580c0051c0 msgr2=0x7f580c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:48.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.367+0000 7f58237fe640 1 --2- 172.21.15.148:0/3687424309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f580c0051c0 0x7f580c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.367+0000 7f58237fe640 1 -- 172.21.15.148:0/3687424309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f580c004800 msgr2=0x7f580c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:48.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.367+0000 7f58237fe640 1 --2- 172.21.15.148:0/3687424309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f580c004800 0x7f580c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.367+0000 7f58237fe640 1 -- 172.21.15.148:0/3687424309 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580c003520 con 0x7f580c003a00 2024-03-29T04:12:48.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.367+0000 7f5823fff640 1 -- 172.21.15.148:0/3687424309 reap_dead start 2024-03-29T04:12:48.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.367+0000 7f5822ffd640 1 --2- 172.21.15.148:0/3687424309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f580c004800 0x7f580c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:48.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.368+0000 7f58237fe640 1 --2- 172.21.15.148:0/3687424309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f580c003a00 0x7f580c003e20 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5814009ac0 tx=0x7f580c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30b8953a89f3813 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:48.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.368+0000 7f58227fc640 1 -- 172.21.15.148:0/3687424309 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5814039070 con 0x7f580c003a00 2024-03-29T04:12:48.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.368+0000 7f58227fc640 1 -- 172.21.15.148:0/3687424309 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f581402c6b0 con 0x7f580c003a00 2024-03-29T04:12:48.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.368+0000 7f58227fc640 1 -- 172.21.15.148:0/3687424309 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5814009690 con 0x7f580c003a00 2024-03-29T04:12:48.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.369+0000 7f5829f1c640 1 -- 172.21.15.148:0/3687424309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f580c003a00 msgr2=0x7f580c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.369+0000 7f5829f1c640 1 --2- 172.21.15.148:0/3687424309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f580c003a00 0x7f580c003e20 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5814009ac0 tx=0x7f580c005830 comp rx=0 tx=0).stop 2024-03-29T04:12:48.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.369+0000 7f5829f1c640 1 -- 172.21.15.148:0/3687424309 shutdown_connections 2024-03-29T04:12:48.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.369+0000 7f5829f1c640 1 --2- 172.21.15.148:0/3687424309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f580c003a00 0x7f580c003e20 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.369+0000 7f5829f1c640 1 -- 172.21.15.148:0/3687424309 >> 172.21.15.148:0/3687424309 conn(0x7f58240cd140 msgr2=0x7f58240cd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:48.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.369+0000 7f5829f1c640 1 -- 172.21.15.148:0/3687424309 shutdown_connections 2024-03-29T04:12:48.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.369+0000 7f5829f1c640 1 -- 172.21.15.148:0/3687424309 wait complete. 2024-03-29T04:12:48.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.370+0000 7f5829f1c640 1 Processor -- start 2024-03-29T04:12:48.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.370+0000 7f5829f1c640 1 -- start start 2024-03-29T04:12:48.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.371+0000 7f5829f1c640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f580c003a00 0x7f5824158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.371+0000 7f5829f1c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f582415b000 0x7f582414a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.371+0000 7f5829f1c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f582406f790 0x7f582406fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.371+0000 7f5829f1c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f58240cc080 con 0x7f582406f790 2024-03-29T04:12:48.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.371+0000 7f5829f1c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f58240cbf00 con 0x7f582415b000 2024-03-29T04:12:48.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.371+0000 7f5829f1c640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f582415ef10 con 0x7f580c003a00 2024-03-29T04:12:48.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.371+0000 7f5823fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f582406f790 0x7f582406fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.371+0000 7f5823fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f582406f790 0x7f582406fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56652/0 (socket says 172.21.15.148:56652) 2024-03-29T04:12:48.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.371+0000 7f5823fff640 1 -- 172.21.15.148:0/2028802248 learned_addr learned my addr 172.21.15.148:0/2028802248 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:48.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.371+0000 7f5823fff640 1 -- 172.21.15.148:0/2028802248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f580c003a00 msgr2=0x7f5824158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.371+0000 7f5822ffd640 1 --2- 172.21.15.148:0/2028802248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f582415b000 0x7f582414a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.371+0000 7f58237fe640 1 --2- 172.21.15.148:0/2028802248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f580c003a00 0x7f5824158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.371+0000 7f5823fff640 1 --2- 172.21.15.148:0/2028802248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f580c003a00 0x7f5824158f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.371+0000 7f5823fff640 1 -- 172.21.15.148:0/2028802248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f582415b000 msgr2=0x7f582414a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.371+0000 7f5823fff640 1 --2- 172.21.15.148:0/2028802248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f582415b000 0x7f582414a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.371+0000 7f5823fff640 1 -- 172.21.15.148:0/2028802248 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5824159690 con 0x7f582406f790 2024-03-29T04:12:48.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.371+0000 7f5822ffd640 1 --2- 172.21.15.148:0/2028802248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f582415b000 0x7f582414a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:12:48.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.371+0000 7f5823fff640 1 --2- 172.21.15.148:0/2028802248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f582406f790 0x7f582406fc10 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5824068390 tx=0x7f581c00e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:48.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.371+0000 7f580bfff640 1 -- 172.21.15.148:0/2028802248 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c00eb10 con 0x7f582406f790 2024-03-29T04:12:48.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.371+0000 7f580bfff640 1 -- 172.21.15.148:0/2028802248 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f581c00ecb0 con 0x7f582406f790 2024-03-29T04:12:48.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.371+0000 7f580bfff640 1 -- 172.21.15.148:0/2028802248 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c014670 con 0x7f582406f790 2024-03-29T04:12:48.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.371+0000 7f5829f1c640 1 -- 172.21.15.148:0/2028802248 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5824159970 con 0x7f582406f790 2024-03-29T04:12:48.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.371+0000 7f5829f1c640 1 -- 172.21.15.148:0/2028802248 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5824159f30 con 0x7f582406f790 2024-03-29T04:12:48.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.372+0000 7f5829f1c640 1 -- 172.21.15.148:0/2028802248 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5824068740 con 0x7f582406f790 2024-03-29T04:12:48.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.373+0000 7f580bfff640 1 -- 172.21.15.148:0/2028802248 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f581c014810 con 0x7f582406f790 2024-03-29T04:12:48.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.373+0000 7f580bfff640 1 --2- 172.21.15.148:0/2028802248 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f581006c560 0x7f581006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.373+0000 7f580bfff640 1 -- 172.21.15.148:0/2028802248 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f581c08b2f0 con 0x7f582406f790 2024-03-29T04:12:48.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.373+0000 7f58237fe640 1 --2- 172.21.15.148:0/2028802248 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f581006c560 0x7f581006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.373+0000 7f580bfff640 1 --2- 172.21.15.148:0/2028802248 >> [v2:172.21.15.157:6800/1932472006,v1:172.21.15.157:6801/1932472006] conn(0x7f5810072a00 0x7f5810074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.373+0000 7f580bfff640 1 -- 172.21.15.148:0/2028802248 --> [v2:172.21.15.157:6800/1932472006,v1:172.21.15.157:6801/1932472006] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5810075510 con 0x7f5810072a00 2024-03-29T04:12:48.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.373+0000 7f580bfff640 1 -- 172.21.15.148:0/2028802248 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f581c092050 con 0x7f582406f790 2024-03-29T04:12:48.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.374+0000 7f5822ffd640 1 --2- 172.21.15.148:0/2028802248 >> [v2:172.21.15.157:6800/1932472006,v1:172.21.15.157:6801/1932472006] conn(0x7f5810072a00 0x7f5810074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.374+0000 7f58237fe640 1 --2- 172.21.15.148:0/2028802248 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f581006c560 0x7f581006ea20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5814036000 tx=0x7f5814030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:48.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.374+0000 7f5822ffd640 1 --2- 172.21.15.148:0/2028802248 >> [v2:172.21.15.157:6800/1932472006,v1:172.21.15.157:6801/1932472006] conn(0x7f5810072a00 0x7f5810074e40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:48.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.375+0000 7f580bfff640 1 -- 172.21.15.148:0/2028802248 <== osd.4 v2:172.21.15.157:6800/1932472006 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5810075510 con 0x7f5810072a00 2024-03-29T04:12:48.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.388+0000 7f5829f1c640 1 -- 172.21.15.148:0/2028802248 --> [v2:172.21.15.157:6800/1932472006,v1:172.21.15.157:6801/1932472006] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5824067c90 con 0x7f5810072a00 2024-03-29T04:12:48.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.388+0000 7f580bfff640 1 -- 172.21.15.148:0/2028802248 <== osd.4 v2:172.21.15.157:6800/1932472006 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5824067c90 con 0x7f5810072a00 2024-03-29T04:12:48.368 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-03-29T04:12:48.369 INFO:teuthology.orchestra.run.smithi148.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:12:48.369 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-03-29T04:12:48.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.389+0000 7f5829f1c640 1 -- 172.21.15.148:0/2028802248 >> [v2:172.21.15.157:6800/1932472006,v1:172.21.15.157:6801/1932472006] conn(0x7f5810072a00 msgr2=0x7f5810074e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.389+0000 7f5829f1c640 1 --2- 172.21.15.148:0/2028802248 >> [v2:172.21.15.157:6800/1932472006,v1:172.21.15.157:6801/1932472006] conn(0x7f5810072a00 0x7f5810074e40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.389+0000 7f5829f1c640 1 -- 172.21.15.148:0/2028802248 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f581006c560 msgr2=0x7f581006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.389+0000 7f5829f1c640 1 --2- 172.21.15.148:0/2028802248 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f581006c560 0x7f581006ea20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5814036000 tx=0x7f5814030000 comp rx=0 tx=0).stop 2024-03-29T04:12:48.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.389+0000 7f5829f1c640 1 -- 172.21.15.148:0/2028802248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f582406f790 msgr2=0x7f582406fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.389+0000 7f5829f1c640 1 --2- 172.21.15.148:0/2028802248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f582406f790 0x7f582406fc10 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5824068390 tx=0x7f581c00e160 comp rx=0 tx=0).stop 2024-03-29T04:12:48.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.389+0000 7f5823fff640 1 -- 172.21.15.148:0/2028802248 reap_dead start 2024-03-29T04:12:48.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.389+0000 7f5829f1c640 1 -- 172.21.15.148:0/2028802248 shutdown_connections 2024-03-29T04:12:48.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.389+0000 7f5829f1c640 1 -- 172.21.15.148:0/2028802248 >> 172.21.15.148:0/2028802248 conn(0x7f58240cd140 msgr2=0x7f582415cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:48.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.389+0000 7f5829f1c640 1 -- 172.21.15.148:0/2028802248 shutdown_connections 2024-03-29T04:12:48.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.389+0000 7f5829f1c640 1 -- 172.21.15.148:0/2028802248 wait complete. 2024-03-29T04:12:48.379 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-29T04:12:48.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.502+0000 7f83f5d67640 1 Processor -- start 2024-03-29T04:12:48.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.502+0000 7f83f5d67640 1 -- start start 2024-03-29T04:12:48.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.502+0000 7f83f5d67640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f83f015e3e0 0x7f83f00d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.502+0000 7f83f5d67640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83f015e7c0 con 0x7f83f00d10d0 2024-03-29T04:12:48.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.502+0000 7f83f5d67640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83f00d2c30 con 0x7f83f00ba030 2024-03-29T04:12:48.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.502+0000 7f83f5d67640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f83f00d3840 con 0x7f83f015e3e0 2024-03-29T04:12:48.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.503+0000 7f83effff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f83f00d10d0 0x7f83f0167f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44044/0 (socket says 172.21.15.148:44044) 2024-03-29T04:12:48.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.503+0000 7f83effff640 1 -- 172.21.15.148:0/908568601 learned_addr learned my addr 172.21.15.148:0/908568601 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:48.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.503+0000 7f83ef7fe640 1 --2- 172.21.15.148:0/908568601 >> v2:172.21.15.157:3300/0 conn(0x7f83f015e3e0 0x7f83f00d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.503+0000 7f83ee7fc640 1 -- 172.21.15.148:0/908568601 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3872361202 0 0) 0x7f83f015e7c0 con 0x7f83f00d10d0 2024-03-29T04:12:48.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.503+0000 7f83ee7fc640 1 -- 172.21.15.148:0/908568601 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83d8003520 con 0x7f83f00d10d0 2024-03-29T04:12:48.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.503+0000 7f83ee7fc640 1 -- 172.21.15.148:0/908568601 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1690821832 0 0) 0x7f83d8003520 con 0x7f83f00d10d0 2024-03-29T04:12:48.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.503+0000 7f83ee7fc640 1 -- 172.21.15.148:0/908568601 >> v2:172.21.15.157:3300/0 conn(0x7f83f015e3e0 msgr2=0x7f83f00d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:48.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.503+0000 7f83ee7fc640 1 --2- 172.21.15.148:0/908568601 >> v2:172.21.15.157:3300/0 conn(0x7f83f015e3e0 0x7f83f00d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.503+0000 7f83ee7fc640 1 -- 172.21.15.148:0/908568601 >> v1:172.21.15.157:6789/0 conn(0x7f83f00ba030 legacy=0x7f83f00d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.503+0000 7f83ee7fc640 1 -- 172.21.15.148:0/908568601 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f00d2f10 con 0x7f83f00d10d0 2024-03-29T04:12:48.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.503+0000 7f83ee7fc640 1 -- 172.21.15.148:0/908568601 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f83e80019f0 con 0x7f83f00d10d0 2024-03-29T04:12:48.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.503+0000 7f83ee7fc640 1 -- 172.21.15.148:0/908568601 >> v1:172.21.15.148:6789/0 conn(0x7f83f00d10d0 legacy=0x7f83f0167f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.503+0000 7f83ee7fc640 1 --2- 172.21.15.148:0/908568601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83d80039b0 0x7f83d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.503+0000 7f83ee7fc640 1 --2- 172.21.15.148:0/908568601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83d8004800 0x7f83d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.503+0000 7f83ee7fc640 1 --2- 172.21.15.148:0/908568601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83d80051c0 0x7f83d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.503+0000 7f83ee7fc640 1 -- 172.21.15.148:0/908568601 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f83f00d3840 con 0x7f83d80039b0 2024-03-29T04:12:48.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.503+0000 7f83ee7fc640 1 -- 172.21.15.148:0/908568601 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f83d800c070 con 0x7f83d8004800 2024-03-29T04:12:48.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.503+0000 7f83ee7fc640 1 -- 172.21.15.148:0/908568601 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f83d800c270 con 0x7f83d80051c0 2024-03-29T04:12:48.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.503+0000 7f83ef7fe640 1 --2- 172.21.15.148:0/908568601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83d80039b0 0x7f83d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.503+0000 7f83eeffd640 1 --2- 172.21.15.148:0/908568601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83d8004800 0x7f83d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.503+0000 7f83effff640 1 --2- 172.21.15.148:0/908568601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83d80051c0 0x7f83d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.503+0000 7f83ef7fe640 1 -- 172.21.15.148:0/908568601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83d80051c0 msgr2=0x7f83d800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:48.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.503+0000 7f83ef7fe640 1 --2- 172.21.15.148:0/908568601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83d80051c0 0x7f83d800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.503+0000 7f83ef7fe640 1 -- 172.21.15.148:0/908568601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83d8004800 msgr2=0x7f83d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:48.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.503+0000 7f83ef7fe640 1 --2- 172.21.15.148:0/908568601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83d8004800 0x7f83d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.503+0000 7f83ef7fe640 1 -- 172.21.15.148:0/908568601 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d8003520 con 0x7f83d80039b0 2024-03-29T04:12:48.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.503+0000 7f83effff640 1 -- 172.21.15.148:0/908568601 reap_dead start 2024-03-29T04:12:48.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.503+0000 7f83eeffd640 1 --2- 172.21.15.148:0/908568601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83d8004800 0x7f83d8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:48.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.504+0000 7f83ef7fe640 1 --2- 172.21.15.148:0/908568601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83d80039b0 0x7f83d8003dd0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009ac0 tx=0x7f83d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1c472de29a8367 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:48.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.504+0000 7f83ee7fc640 1 -- 172.21.15.148:0/908568601 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e4039070 con 0x7f83d80039b0 2024-03-29T04:12:48.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.504+0000 7f83ee7fc640 1 -- 172.21.15.148:0/908568601 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83e402c6d0 con 0x7f83d80039b0 2024-03-29T04:12:48.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.504+0000 7f83ee7fc640 1 -- 172.21.15.148:0/908568601 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e4009690 con 0x7f83d80039b0 2024-03-29T04:12:48.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.505+0000 7f83f5d67640 1 -- 172.21.15.148:0/908568601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83d80039b0 msgr2=0x7f83d8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.505+0000 7f83f5d67640 1 --2- 172.21.15.148:0/908568601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83d80039b0 0x7f83d8003dd0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009ac0 tx=0x7f83d8005830 comp rx=0 tx=0).stop 2024-03-29T04:12:48.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.505+0000 7f83f5d67640 1 -- 172.21.15.148:0/908568601 shutdown_connections 2024-03-29T04:12:48.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.505+0000 7f83f5d67640 1 --2- 172.21.15.148:0/908568601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83d80039b0 0x7f83d8003dd0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.505+0000 7f83f5d67640 1 -- 172.21.15.148:0/908568601 >> 172.21.15.148:0/908568601 conn(0x7f83f00b6210 msgr2=0x7f83f00b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:48.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.505+0000 7f83f5d67640 1 -- 172.21.15.148:0/908568601 shutdown_connections 2024-03-29T04:12:48.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.505+0000 7f83f5d67640 1 -- 172.21.15.148:0/908568601 wait complete. 2024-03-29T04:12:48.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.506+0000 7f83f5d67640 1 Processor -- start 2024-03-29T04:12:48.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.506+0000 7f83f5d67640 1 -- start start 2024-03-29T04:12:48.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.507+0000 7f83f5d67640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83d80039b0 0x7f83f015b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.507+0000 7f83f5d67640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83f00cdda0 0x7f83f01f7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.507+0000 7f83f5d67640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83f01f8480 0x7f83f015cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.507+0000 7f83f5d67640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f83f00d3cd0 con 0x7f83f00cdda0 2024-03-29T04:12:48.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.507+0000 7f83f5d67640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f83f00d34d0 con 0x7f83f01f8480 2024-03-29T04:12:48.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.507+0000 7f83f5d67640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f83f00d3e50 con 0x7f83d80039b0 2024-03-29T04:12:48.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.507+0000 7f83eeffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83f00cdda0 0x7f83f01f7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.507+0000 7f83eeffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83f00cdda0 0x7f83f01f7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56676/0 (socket says 172.21.15.148:56676) 2024-03-29T04:12:48.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.507+0000 7f83eeffd640 1 -- 172.21.15.148:0/489797264 learned_addr learned my addr 172.21.15.148:0/489797264 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:48.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.507+0000 7f83ef7fe640 1 --2- 172.21.15.148:0/489797264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83d80039b0 0x7f83f015b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.507+0000 7f83effff640 1 --2- 172.21.15.148:0/489797264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83f01f8480 0x7f83f015cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.507+0000 7f83eeffd640 1 -- 172.21.15.148:0/489797264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83d80039b0 msgr2=0x7f83f015b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.507+0000 7f83eeffd640 1 --2- 172.21.15.148:0/489797264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83d80039b0 0x7f83f015b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.507+0000 7f83eeffd640 1 -- 172.21.15.148:0/489797264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83f01f8480 msgr2=0x7f83f015cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.507+0000 7f83eeffd640 1 --2- 172.21.15.148:0/489797264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83f01f8480 0x7f83f015cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.507+0000 7f83eeffd640 1 -- 172.21.15.148:0/489797264 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f015b640 con 0x7f83f00cdda0 2024-03-29T04:12:48.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.507+0000 7f83ef7fe640 1 --2- 172.21.15.148:0/489797264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83d80039b0 0x7f83f015b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:12:48.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.507+0000 7f83effff640 1 --2- 172.21.15.148:0/489797264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83f01f8480 0x7f83f015cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:12:48.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.507+0000 7f83eeffd640 1 --2- 172.21.15.148:0/489797264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83f00cdda0 0x7f83f01f7f40 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f83e000e480 tx=0x7f83e000e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:48.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.508+0000 7f83d7fff640 1 -- 172.21.15.148:0/489797264 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e00052e0 con 0x7f83f00cdda0 2024-03-29T04:12:48.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.508+0000 7f83d7fff640 1 -- 172.21.15.148:0/489797264 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83e0007020 con 0x7f83f00cdda0 2024-03-29T04:12:48.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.508+0000 7f83f5d67640 1 -- 172.21.15.148:0/489797264 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f015b900 con 0x7f83f00cdda0 2024-03-29T04:12:48.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.508+0000 7f83d7fff640 1 -- 172.21.15.148:0/489797264 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e0007210 con 0x7f83f00cdda0 2024-03-29T04:12:48.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.508+0000 7f83f5d67640 1 -- 172.21.15.148:0/489797264 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f015c340 con 0x7f83f00cdda0 2024-03-29T04:12:48.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.509+0000 7f83f5d67640 1 -- 172.21.15.148:0/489797264 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83f00687e0 con 0x7f83f00cdda0 2024-03-29T04:12:48.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.509+0000 7f83d7fff640 1 -- 172.21.15.148:0/489797264 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f83e001a020 con 0x7f83f00cdda0 2024-03-29T04:12:48.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.510+0000 7f83d7fff640 1 --2- 172.21.15.148:0/489797264 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f83dc06c560 0x7f83dc06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.510+0000 7f83d7fff640 1 -- 172.21.15.148:0/489797264 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f83e008aaa0 con 0x7f83f00cdda0 2024-03-29T04:12:48.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.510+0000 7f83ef7fe640 1 --2- 172.21.15.148:0/489797264 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f83dc06c560 0x7f83dc06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.510+0000 7f83ef7fe640 1 --2- 172.21.15.148:0/489797264 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f83dc06c560 0x7f83dc06ea20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f83e4036000 tx=0x7f83e4030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:48.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.514+0000 7f83d7fff640 1 -- 172.21.15.148:0/489797264 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f83e00592c0 con 0x7f83f00cdda0 2024-03-29T04:12:48.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.688+0000 7f83f5d67640 1 -- 172.21.15.148:0/489797264 --> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f83f00ce200 con 0x7f83dc06c560 2024-03-29T04:12:48.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.688+0000 7f83d7fff640 1 -- 172.21.15.148:0/489797264 <== mgr.4113 v2:172.21.15.148:6832/3271445075 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f83f00ce200 con 0x7f83dc06c560 2024-03-29T04:12:48.668 INFO:teuthology.orchestra.run.smithi148.stderr:instructed osd(s) 4 to deep-scrub 2024-03-29T04:12:48.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.690+0000 7f83f5d67640 1 -- 172.21.15.148:0/489797264 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f83dc06c560 msgr2=0x7f83dc06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.690+0000 7f83f5d67640 1 --2- 172.21.15.148:0/489797264 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f83dc06c560 0x7f83dc06ea20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f83e4036000 tx=0x7f83e4030000 comp rx=0 tx=0).stop 2024-03-29T04:12:48.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.690+0000 7f83f5d67640 1 -- 172.21.15.148:0/489797264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83f00cdda0 msgr2=0x7f83f01f7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.690+0000 7f83f5d67640 1 --2- 172.21.15.148:0/489797264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83f00cdda0 0x7f83f01f7f40 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f83e000e480 tx=0x7f83e000e940 comp rx=0 tx=0).stop 2024-03-29T04:12:48.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.690+0000 7f83f5d67640 1 -- 172.21.15.148:0/489797264 shutdown_connections 2024-03-29T04:12:48.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.690+0000 7f83f5d67640 1 --2- 172.21.15.148:0/489797264 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f83dc06c560 0x7f83dc06ea20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.690+0000 7f83f5d67640 1 --2- 172.21.15.148:0/489797264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83f01f8480 0x7f83f015cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.690+0000 7f83f5d67640 1 --2- 172.21.15.148:0/489797264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83f00cdda0 0x7f83f01f7f40 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.690+0000 7f83f5d67640 1 --2- 172.21.15.148:0/489797264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83d80039b0 0x7f83f015b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.690+0000 7f83f5d67640 1 -- 172.21.15.148:0/489797264 >> 172.21.15.148:0/489797264 conn(0x7f83f00b6210 msgr2=0x7f83f006cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:48.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.690+0000 7f83f5d67640 1 -- 172.21.15.148:0/489797264 shutdown_connections 2024-03-29T04:12:48.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.691+0000 7f83f5d67640 1 -- 172.21.15.148:0/489797264 wait complete. 2024-03-29T04:12:48.682 INFO:tasks.ceph:Scrubbing osd.5 2024-03-29T04:12:48.682 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:12:48.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.806+0000 7f8b59ffd640 1 Processor -- start 2024-03-29T04:12:48.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.806+0000 7f8b59ffd640 1 -- start start 2024-03-29T04:12:48.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.807+0000 7f8b59ffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b5406c5c0 0x7f8b5406ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.807+0000 7f8b59ffd640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b5406e570 con 0x7f8b540cbb30 2024-03-29T04:12:48.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.807+0000 7f8b59ffd640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b540cbf10 con 0x7f8b54073400 2024-03-29T04:12:48.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.807+0000 7f8b59ffd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8b5406d9d0 con 0x7f8b5406c5c0 2024-03-29T04:12:48.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.807+0000 7f8b537fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8b540cbb30 0x7f8b5406de60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44052/0 (socket says 172.21.15.148:44052) 2024-03-29T04:12:48.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.807+0000 7f8b537fe640 1 -- 172.21.15.148:0/2726542746 learned_addr learned my addr 172.21.15.148:0/2726542746 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:48.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.807+0000 7f8b527fc640 1 -- 172.21.15.148:0/2726542746 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2841338393 0 0) 0x7f8b5406e570 con 0x7f8b540cbb30 2024-03-29T04:12:48.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.807+0000 7f8b527fc640 1 -- 172.21.15.148:0/2726542746 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b3c003520 con 0x7f8b540cbb30 2024-03-29T04:12:48.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.807+0000 7f8b53fff640 1 --2- 172.21.15.148:0/2726542746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b5406c5c0 0x7f8b5406ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.807+0000 7f8b527fc640 1 -- 172.21.15.148:0/2726542746 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1301004328 0 0) 0x7f8b3c003520 con 0x7f8b540cbb30 2024-03-29T04:12:48.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.807+0000 7f8b527fc640 1 -- 172.21.15.148:0/2726542746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b5406c5c0 msgr2=0x7f8b5406ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:48.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.807+0000 7f8b527fc640 1 --2- 172.21.15.148:0/2726542746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b5406c5c0 0x7f8b5406ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.807+0000 7f8b527fc640 1 -- 172.21.15.148:0/2726542746 >> v1:172.21.15.157:6789/0 conn(0x7f8b54073400 legacy=0x7f8b5415c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.807+0000 7f8b527fc640 1 -- 172.21.15.148:0/2726542746 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b5415d720 con 0x7f8b540cbb30 2024-03-29T04:12:48.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.807+0000 7f8b527fc640 1 -- 172.21.15.148:0/2726542746 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7f8b44001670 con 0x7f8b540cbb30 2024-03-29T04:12:48.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.808+0000 7f8b527fc640 1 -- 172.21.15.148:0/2726542746 >> v1:172.21.15.148:6789/0 conn(0x7f8b540cbb30 legacy=0x7f8b5406de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.808+0000 7f8b527fc640 1 --2- 172.21.15.148:0/2726542746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b3c0039e0 0x7f8b3c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.808+0000 7f8b527fc640 1 --2- 172.21.15.148:0/2726542746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b3c004800 0x7f8b3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.808+0000 7f8b527fc640 1 --2- 172.21.15.148:0/2726542746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b5406c5c0 0x7f8b3c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.808+0000 7f8b527fc640 1 -- 172.21.15.148:0/2726542746 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5406d9d0 con 0x7f8b3c0039e0 2024-03-29T04:12:48.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.808+0000 7f8b527fc640 1 -- 172.21.15.148:0/2726542746 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c00bb20 con 0x7f8b3c004800 2024-03-29T04:12:48.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.808+0000 7f8b527fc640 1 -- 172.21.15.148:0/2726542746 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8b3c00bd20 con 0x7f8b5406c5c0 2024-03-29T04:12:48.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.808+0000 7f8b537fe640 1 --2- 172.21.15.148:0/2726542746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b3c0039e0 0x7f8b3c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.808+0000 7f8b52ffd640 1 --2- 172.21.15.148:0/2726542746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b3c004800 0x7f8b3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.808+0000 7f8b53fff640 1 --2- 172.21.15.148:0/2726542746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b5406c5c0 0x7f8b3c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.808+0000 7f8b537fe640 1 -- 172.21.15.148:0/2726542746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b5406c5c0 msgr2=0x7f8b3c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:48.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.808+0000 7f8b537fe640 1 --2- 172.21.15.148:0/2726542746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b5406c5c0 0x7f8b3c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.808+0000 7f8b537fe640 1 -- 172.21.15.148:0/2726542746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b3c004800 msgr2=0x7f8b3c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:48.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.808+0000 7f8b537fe640 1 --2- 172.21.15.148:0/2726542746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b3c004800 0x7f8b3c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.808+0000 7f8b537fe640 1 -- 172.21.15.148:0/2726542746 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b3c003520 con 0x7f8b3c0039e0 2024-03-29T04:12:48.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.808+0000 7f8b52ffd640 1 --2- 172.21.15.148:0/2726542746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b3c004800 0x7f8b3c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:48.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.809+0000 7f8b537fe640 1 --2- 172.21.15.148:0/2726542746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b3c0039e0 0x7f8b3c003e00 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8b4400c240 tx=0x7f8b3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ee6bb3e00b9da7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:48.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.809+0000 7f8b527fc640 1 -- 172.21.15.148:0/2726542746 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b44007d10 con 0x7f8b3c0039e0 2024-03-29T04:12:48.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.809+0000 7f8b527fc640 1 -- 172.21.15.148:0/2726542746 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b44030070 con 0x7f8b3c0039e0 2024-03-29T04:12:48.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.809+0000 7f8b527fc640 1 -- 172.21.15.148:0/2726542746 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4402cd40 con 0x7f8b3c0039e0 2024-03-29T04:12:48.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.809+0000 7f8b59ffd640 1 -- 172.21.15.148:0/2726542746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b3c0039e0 msgr2=0x7f8b3c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.809+0000 7f8b59ffd640 1 --2- 172.21.15.148:0/2726542746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b3c0039e0 0x7f8b3c003e00 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8b4400c240 tx=0x7f8b3c005830 comp rx=0 tx=0).stop 2024-03-29T04:12:48.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.809+0000 7f8b53fff640 1 -- 172.21.15.148:0/2726542746 reap_dead start 2024-03-29T04:12:48.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.809+0000 7f8b59ffd640 1 -- 172.21.15.148:0/2726542746 shutdown_connections 2024-03-29T04:12:48.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.809+0000 7f8b59ffd640 1 -- 172.21.15.148:0/2726542746 >> 172.21.15.148:0/2726542746 conn(0x7f8b540b1b70 msgr2=0x7f8b540b1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:48.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.809+0000 7f8b59ffd640 1 -- 172.21.15.148:0/2726542746 shutdown_connections 2024-03-29T04:12:48.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.809+0000 7f8b59ffd640 1 -- 172.21.15.148:0/2726542746 wait complete. 2024-03-29T04:12:48.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.810+0000 7f8b59ffd640 1 Processor -- start 2024-03-29T04:12:48.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.811+0000 7f8b59ffd640 1 -- start start 2024-03-29T04:12:48.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.811+0000 7f8b59ffd640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3c0039e0 0x7f8b540cef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.811+0000 7f8b59ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b541e72d0 0x7f8b540b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.811+0000 7f8b59ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b540b4f60 0x7f8b5415b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.811+0000 7f8b59ffd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5406d660 con 0x7f8b540b4f60 2024-03-29T04:12:48.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.811+0000 7f8b59ffd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5415d440 con 0x7f8b541e72d0 2024-03-29T04:12:48.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.811+0000 7f8b59ffd640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8b5415ce70 con 0x7f8b3c0039e0 2024-03-29T04:12:48.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.811+0000 7f8b53fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b540b4f60 0x7f8b5415b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.811+0000 7f8b53fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b540b4f60 0x7f8b5415b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56688/0 (socket says 172.21.15.148:56688) 2024-03-29T04:12:48.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.811+0000 7f8b53fff640 1 -- 172.21.15.148:0/3775710271 learned_addr learned my addr 172.21.15.148:0/3775710271 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:48.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.811+0000 7f8b537fe640 1 --2- 172.21.15.148:0/3775710271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3c0039e0 0x7f8b540cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.812+0000 7f8b52ffd640 1 --2- 172.21.15.148:0/3775710271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b541e72d0 0x7f8b540b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.812+0000 7f8b53fff640 1 -- 172.21.15.148:0/3775710271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3c0039e0 msgr2=0x7f8b540cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.812+0000 7f8b53fff640 1 --2- 172.21.15.148:0/3775710271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b3c0039e0 0x7f8b540cef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.812+0000 7f8b53fff640 1 -- 172.21.15.148:0/3775710271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b541e72d0 msgr2=0x7f8b540b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.812+0000 7f8b53fff640 1 --2- 172.21.15.148:0/3775710271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b541e72d0 0x7f8b540b4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.812+0000 7f8b53fff640 1 -- 172.21.15.148:0/3775710271 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b541f4a20 con 0x7f8b540b4f60 2024-03-29T04:12:48.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.812+0000 7f8b52ffd640 1 --2- 172.21.15.148:0/3775710271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b541e72d0 0x7f8b540b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:48.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.812+0000 7f8b53fff640 1 --2- 172.21.15.148:0/3775710271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b540b4f60 0x7f8b5415b880 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8b540ca410 tx=0x7f8b4c009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:48.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.812+0000 7f8b3bfff640 1 -- 172.21.15.148:0/3775710271 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c007d20 con 0x7f8b540b4f60 2024-03-29T04:12:48.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.812+0000 7f8b3bfff640 1 -- 172.21.15.148:0/3775710271 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b4c019070 con 0x7f8b540b4f60 2024-03-29T04:12:48.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.812+0000 7f8b3bfff640 1 -- 172.21.15.148:0/3775710271 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c014650 con 0x7f8b540b4f60 2024-03-29T04:12:48.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.812+0000 7f8b59ffd640 1 -- 172.21.15.148:0/3775710271 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b541f4d00 con 0x7f8b540b4f60 2024-03-29T04:12:48.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.812+0000 7f8b59ffd640 1 -- 172.21.15.148:0/3775710271 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b541f52c0 con 0x7f8b540b4f60 2024-03-29T04:12:48.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.813+0000 7f8b59ffd640 1 -- 172.21.15.148:0/3775710271 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8b54068740 con 0x7f8b540b4f60 2024-03-29T04:12:48.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.813+0000 7f8b3bfff640 1 -- 172.21.15.148:0/3775710271 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8b4c0147f0 con 0x7f8b540b4f60 2024-03-29T04:12:48.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.814+0000 7f8b3bfff640 1 --2- 172.21.15.148:0/3775710271 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f8b4006c530 0x7f8b4006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.814+0000 7f8b3bfff640 1 -- 172.21.15.148:0/3775710271 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f8b4c08b960 con 0x7f8b540b4f60 2024-03-29T04:12:48.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.814+0000 7f8b537fe640 1 --2- 172.21.15.148:0/3775710271 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f8b4006c530 0x7f8b4006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.814+0000 7f8b3bfff640 1 --2- 172.21.15.148:0/3775710271 >> [v2:172.21.15.157:6816/1608291780,v1:172.21.15.157:6817/1608291780] conn(0x7f8b400729d0 0x7f8b40074e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.814+0000 7f8b3bfff640 1 -- 172.21.15.148:0/3775710271 --> [v2:172.21.15.157:6816/1608291780,v1:172.21.15.157:6817/1608291780] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8b400754e0 con 0x7f8b400729d0 2024-03-29T04:12:48.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.814+0000 7f8b3bfff640 1 -- 172.21.15.148:0/3775710271 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f8b4c0902c0 con 0x7f8b540b4f60 2024-03-29T04:12:48.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.814+0000 7f8b52ffd640 1 --2- 172.21.15.148:0/3775710271 >> [v2:172.21.15.157:6816/1608291780,v1:172.21.15.157:6817/1608291780] conn(0x7f8b400729d0 0x7f8b40074e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.814+0000 7f8b537fe640 1 --2- 172.21.15.148:0/3775710271 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f8b4006c530 0x7f8b4006e9f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8b44034000 tx=0x7f8b44009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:48.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.815+0000 7f8b52ffd640 1 --2- 172.21.15.148:0/3775710271 >> [v2:172.21.15.157:6816/1608291780,v1:172.21.15.157:6817/1608291780] conn(0x7f8b400729d0 0x7f8b40074e10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:48.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.815+0000 7f8b3bfff640 1 -- 172.21.15.148:0/3775710271 <== osd.5 v2:172.21.15.157:6816/1608291780 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8b400754e0 con 0x7f8b400729d0 2024-03-29T04:12:48.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.828+0000 7f8b59ffd640 1 -- 172.21.15.148:0/3775710271 --> [v2:172.21.15.157:6816/1608291780,v1:172.21.15.157:6817/1608291780] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8b541f5700 con 0x7f8b400729d0 2024-03-29T04:12:48.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.829+0000 7f8b3bfff640 1 -- 172.21.15.148:0/3775710271 <== osd.5 v2:172.21.15.157:6816/1608291780 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8b541f5700 con 0x7f8b400729d0 2024-03-29T04:12:48.809 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-03-29T04:12:48.809 INFO:teuthology.orchestra.run.smithi148.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:12:48.809 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-03-29T04:12:48.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.830+0000 7f8b59ffd640 1 -- 172.21.15.148:0/3775710271 >> [v2:172.21.15.157:6816/1608291780,v1:172.21.15.157:6817/1608291780] conn(0x7f8b400729d0 msgr2=0x7f8b40074e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.830+0000 7f8b59ffd640 1 --2- 172.21.15.148:0/3775710271 >> [v2:172.21.15.157:6816/1608291780,v1:172.21.15.157:6817/1608291780] conn(0x7f8b400729d0 0x7f8b40074e10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.830+0000 7f8b59ffd640 1 -- 172.21.15.148:0/3775710271 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f8b4006c530 msgr2=0x7f8b4006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.830+0000 7f8b59ffd640 1 --2- 172.21.15.148:0/3775710271 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f8b4006c530 0x7f8b4006e9f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8b44034000 tx=0x7f8b44009030 comp rx=0 tx=0).stop 2024-03-29T04:12:48.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.830+0000 7f8b59ffd640 1 -- 172.21.15.148:0/3775710271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b540b4f60 msgr2=0x7f8b5415b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.830+0000 7f8b59ffd640 1 --2- 172.21.15.148:0/3775710271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b540b4f60 0x7f8b5415b880 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8b540ca410 tx=0x7f8b4c009d20 comp rx=0 tx=0).stop 2024-03-29T04:12:48.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.830+0000 7f8b53fff640 1 -- 172.21.15.148:0/3775710271 reap_dead start 2024-03-29T04:12:48.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.830+0000 7f8b59ffd640 1 -- 172.21.15.148:0/3775710271 shutdown_connections 2024-03-29T04:12:48.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.830+0000 7f8b59ffd640 1 -- 172.21.15.148:0/3775710271 >> 172.21.15.148:0/3775710271 conn(0x7f8b540b1b70 msgr2=0x7f8b5415d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:48.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.830+0000 7f8b59ffd640 1 -- 172.21.15.148:0/3775710271 shutdown_connections 2024-03-29T04:12:48.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.830+0000 7f8b59ffd640 1 -- 172.21.15.148:0/3775710271 wait complete. 2024-03-29T04:12:48.820 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-29T04:12:48.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.942+0000 7f10c661b640 1 Processor -- start 2024-03-29T04:12:48.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.942+0000 7f10c661b640 1 -- start start 2024-03-29T04:12:48.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.942+0000 7f10c661b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10c015a740 0x7f10c015e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.942+0000 7f10c661b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f10c00b3ef0 0x7f10c015ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.942+0000 7f10c661b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10c00cf790 con 0x7f10c015e910 2024-03-29T04:12:48.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.942+0000 7f10c661b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f10c00b4330 con 0x7f10c00b3ef0 2024-03-29T04:12:48.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.942+0000 7f10c661b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f10c00728d0 con 0x7f10c015a740 2024-03-29T04:12:48.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.943+0000 7f10c4bb8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f10c00b3ef0 0x7f10c015ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.943+0000 7f10bf7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f10c015e910 0x7f10c006d6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44068/0 (socket says 172.21.15.148:44068) 2024-03-29T04:12:48.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.943+0000 7f10c4bb8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f10c00b3ef0 0x7f10c015ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56702/0 (socket says 172.21.15.148:56702) 2024-03-29T04:12:48.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.943+0000 7f10bf7fe640 1 -- 172.21.15.148:0/862749476 learned_addr learned my addr 172.21.15.148:0/862749476 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:48.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.943+0000 7f10bffff640 1 --2- 172.21.15.148:0/862749476 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10c015a740 0x7f10c015e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.943+0000 7f10c4bb8640 1 -- 172.21.15.148:0/862749476 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10c015a740 msgr2=0x7f10c015e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:48.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.943+0000 7f10c4bb8640 1 --2- 172.21.15.148:0/862749476 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10c015a740 0x7f10c015e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.943+0000 7f10c4bb8640 1 -- 172.21.15.148:0/862749476 >> v1:172.21.15.148:6789/0 conn(0x7f10c015e910 legacy=0x7f10c006d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.943+0000 7f10c4bb8640 1 -- 172.21.15.148:0/862749476 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c0072bb0 con 0x7f10c00b3ef0 2024-03-29T04:12:48.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.943+0000 7f10beffd640 1 -- 172.21.15.148:0/862749476 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3435736592 0 0) 0x7f10b4001110 con 0x7f10c015e910 2024-03-29T04:12:48.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.944+0000 7f10c4bb8640 1 --2- 172.21.15.148:0/862749476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10c00b3ef0 0x7f10c015ed70 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009d90 tx=0x7f10b802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0bd5edede4d007a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:48.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.944+0000 7f10beffd640 1 -- 172.21.15.148:0/862749476 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b8039070 con 0x7f10c00b3ef0 2024-03-29T04:12:48.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.944+0000 7f10beffd640 1 -- 172.21.15.148:0/862749476 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10b802cbc0 con 0x7f10c00b3ef0 2024-03-29T04:12:48.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.944+0000 7f10beffd640 1 -- 172.21.15.148:0/862749476 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b8034440 con 0x7f10c00b3ef0 2024-03-29T04:12:48.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.944+0000 7f10c661b640 1 -- 172.21.15.148:0/862749476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10c00b3ef0 msgr2=0x7f10c015ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.944+0000 7f10c661b640 1 --2- 172.21.15.148:0/862749476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10c00b3ef0 0x7f10c015ed70 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009d90 tx=0x7f10b802be10 comp rx=0 tx=0).stop 2024-03-29T04:12:48.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.944+0000 7f10c661b640 1 -- 172.21.15.148:0/862749476 shutdown_connections 2024-03-29T04:12:48.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.944+0000 7f10c661b640 1 --2- 172.21.15.148:0/862749476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10c00b3ef0 0x7f10c015ed70 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.944+0000 7f10c661b640 1 --2- 172.21.15.148:0/862749476 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10c015a740 0x7f10c015e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.944+0000 7f10c661b640 1 -- 172.21.15.148:0/862749476 >> 172.21.15.148:0/862749476 conn(0x7f10c00cfed0 msgr2=0x7f10c00d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:48.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.944+0000 7f10c661b640 1 -- 172.21.15.148:0/862749476 shutdown_connections 2024-03-29T04:12:48.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.944+0000 7f10c661b640 1 -- 172.21.15.148:0/862749476 wait complete. 2024-03-29T04:12:48.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.945+0000 7f10c661b640 1 Processor -- start 2024-03-29T04:12:48.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.946+0000 7f10c661b640 1 -- start start 2024-03-29T04:12:48.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.946+0000 7f10c661b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10c00b3ef0 0x7f10c015d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.946+0000 7f10c661b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10c015a740 0x7f10c0144ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.946+0000 7f10c661b640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10c015e910 0x7f10c00cd660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.946+0000 7f10c661b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f10c015ab20 con 0x7f10c00b3ef0 2024-03-29T04:12:48.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.946+0000 7f10c661b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f10c0073170 con 0x7f10c015a740 2024-03-29T04:12:48.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.946+0000 7f10c661b640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f10c015aca0 con 0x7f10c015e910 2024-03-29T04:12:48.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.946+0000 7f10bffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10c00b3ef0 0x7f10c015d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.946+0000 7f10bffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10c00b3ef0 0x7f10c015d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56708/0 (socket says 172.21.15.148:56708) 2024-03-29T04:12:48.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.947+0000 7f10bffff640 1 -- 172.21.15.148:0/1615865327 learned_addr learned my addr 172.21.15.148:0/1615865327 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:48.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.947+0000 7f10bf7fe640 1 --2- 172.21.15.148:0/1615865327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10c015a740 0x7f10c0144ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.947+0000 7f10c4bb8640 1 --2- 172.21.15.148:0/1615865327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10c015e910 0x7f10c00cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.947+0000 7f10bffff640 1 -- 172.21.15.148:0/1615865327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10c015e910 msgr2=0x7f10c00cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.947+0000 7f10bffff640 1 --2- 172.21.15.148:0/1615865327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10c015e910 0x7f10c00cd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.947+0000 7f10bffff640 1 -- 172.21.15.148:0/1615865327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10c015a740 msgr2=0x7f10c0144ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:48.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.947+0000 7f10bffff640 1 --2- 172.21.15.148:0/1615865327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10c015a740 0x7f10c0144ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:48.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.947+0000 7f10bffff640 1 -- 172.21.15.148:0/1615865327 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c015b370 con 0x7f10c00b3ef0 2024-03-29T04:12:48.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.947+0000 7f10bf7fe640 1 --2- 172.21.15.148:0/1615865327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10c015a740 0x7f10c0144ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:48.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.947+0000 7f10c4bb8640 1 --2- 172.21.15.148:0/1615865327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10c015e910 0x7f10c00cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:48.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.947+0000 7f10bffff640 1 --2- 172.21.15.148:0/1615865327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10c00b3ef0 0x7f10c015d740 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f10b000e9e0 tx=0x7f10b000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:48.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.947+0000 7f10bcff9640 1 -- 172.21.15.148:0/1615865327 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b0017070 con 0x7f10c00b3ef0 2024-03-29T04:12:48.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.947+0000 7f10bcff9640 1 -- 172.21.15.148:0/1615865327 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10b000c9d0 con 0x7f10c00b3ef0 2024-03-29T04:12:48.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.947+0000 7f10c661b640 1 -- 172.21.15.148:0/1615865327 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c015b650 con 0x7f10c00b3ef0 2024-03-29T04:12:48.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.947+0000 7f10bcff9640 1 -- 172.21.15.148:0/1615865327 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b000cdc0 con 0x7f10c00b3ef0 2024-03-29T04:12:48.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.947+0000 7f10c661b640 1 -- 172.21.15.148:0/1615865327 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c015bc10 con 0x7f10c00b3ef0 2024-03-29T04:12:48.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.948+0000 7f10c661b640 1 -- 172.21.15.148:0/1615865327 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10c00687e0 con 0x7f10c00b3ef0 2024-03-29T04:12:48.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.949+0000 7f10bcff9640 1 -- 172.21.15.148:0/1615865327 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f10b001a070 con 0x7f10c00b3ef0 2024-03-29T04:12:48.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.949+0000 7f10bcff9640 1 --2- 172.21.15.148:0/1615865327 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f10ac06c530 0x7f10ac06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:48.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.949+0000 7f10bcff9640 1 -- 172.21.15.148:0/1615865327 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f10b008b940 con 0x7f10c00b3ef0 2024-03-29T04:12:48.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.949+0000 7f10bf7fe640 1 --2- 172.21.15.148:0/1615865327 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f10ac06c530 0x7f10ac06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:48.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.949+0000 7f10bf7fe640 1 --2- 172.21.15.148:0/1615865327 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f10ac06c530 0x7f10ac06e9f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f10c00d0960 tx=0x7f10b4007ca0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:48.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:48.953+0000 7f10bcff9640 1 -- 172.21.15.148:0/1615865327 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f10b005a0d0 con 0x7f10c00b3ef0 2024-03-29T04:12:49.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.128+0000 7f10c661b640 1 -- 172.21.15.148:0/1615865327 --> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f10c00716e0 con 0x7f10ac06c530 2024-03-29T04:12:49.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.128+0000 7f10bcff9640 1 -- 172.21.15.148:0/1615865327 <== mgr.4113 v2:172.21.15.148:6832/3271445075 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f10c00716e0 con 0x7f10ac06c530 2024-03-29T04:12:49.109 INFO:teuthology.orchestra.run.smithi148.stderr:instructed osd(s) 5 to deep-scrub 2024-03-29T04:12:49.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.131+0000 7f10c661b640 1 -- 172.21.15.148:0/1615865327 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f10ac06c530 msgr2=0x7f10ac06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.131+0000 7f10c661b640 1 --2- 172.21.15.148:0/1615865327 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f10ac06c530 0x7f10ac06e9f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f10c00d0960 tx=0x7f10b4007ca0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.131+0000 7f10c661b640 1 -- 172.21.15.148:0/1615865327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10c00b3ef0 msgr2=0x7f10c015d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.131+0000 7f10c661b640 1 --2- 172.21.15.148:0/1615865327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10c00b3ef0 0x7f10c015d740 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f10b000e9e0 tx=0x7f10b000eea0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.131+0000 7f10c661b640 1 -- 172.21.15.148:0/1615865327 shutdown_connections 2024-03-29T04:12:49.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.131+0000 7f10c661b640 1 --2- 172.21.15.148:0/1615865327 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f10ac06c530 0x7f10ac06e9f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.131+0000 7f10c661b640 1 --2- 172.21.15.148:0/1615865327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10c015e910 0x7f10c00cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.131+0000 7f10c661b640 1 --2- 172.21.15.148:0/1615865327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10c015a740 0x7f10c0144ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.131+0000 7f10c661b640 1 --2- 172.21.15.148:0/1615865327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10c00b3ef0 0x7f10c015d740 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.131+0000 7f10c661b640 1 -- 172.21.15.148:0/1615865327 >> 172.21.15.148:0/1615865327 conn(0x7f10c00cfed0 msgr2=0x7f10c006e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:49.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.131+0000 7f10c661b640 1 -- 172.21.15.148:0/1615865327 shutdown_connections 2024-03-29T04:12:49.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.131+0000 7f10c661b640 1 -- 172.21.15.148:0/1615865327 wait complete. 2024-03-29T04:12:49.121 INFO:tasks.ceph:Scrubbing osd.6 2024-03-29T04:12:49.122 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:12:49.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.247+0000 7fc4a299d640 1 Processor -- start 2024-03-29T04:12:49.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.248+0000 7fc4a299d640 1 -- start start 2024-03-29T04:12:49.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.248+0000 7fc4a299d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c0cef70 0x7fc49c06c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:49.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.248+0000 7fc4a299d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc49c0cebf0 con 0x7fc49c06cb00 2024-03-29T04:12:49.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.248+0000 7fc4a299d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc49c0ce350 con 0x7fc49c144b10 2024-03-29T04:12:49.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.248+0000 7fc4a299d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc49c06cf70 con 0x7fc49c0cef70 2024-03-29T04:12:49.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.248+0000 7fc4a0f3a640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc49c06cb00 0x7fc49c072810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44074/0 (socket says 172.21.15.148:44074) 2024-03-29T04:12:49.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.249+0000 7fc4a0f3a640 1 -- 172.21.15.148:0/3342431336 learned_addr learned my addr 172.21.15.148:0/3342431336 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:49.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.249+0000 7fc49b7fe640 1 --2- 172.21.15.148:0/3342431336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c0cef70 0x7fc49c06c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:49.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.249+0000 7fc49affd640 1 -- 172.21.15.148:0/3342431336 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1268498269 0 0) 0x7fc49c0cebf0 con 0x7fc49c06cb00 2024-03-29T04:12:49.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.249+0000 7fc49affd640 1 -- 172.21.15.148:0/3342431336 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc484003520 con 0x7fc49c06cb00 2024-03-29T04:12:49.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.249+0000 7fc49affd640 1 -- 172.21.15.148:0/3342431336 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 700077677 0 0) 0x7fc49c0ce350 con 0x7fc49c144b10 2024-03-29T04:12:49.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.249+0000 7fc49affd640 1 -- 172.21.15.148:0/3342431336 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc49c0cebf0 con 0x7fc49c144b10 2024-03-29T04:12:49.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.249+0000 7fc49affd640 1 -- 172.21.15.148:0/3342431336 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2372090728 0 0) 0x7fc484003520 con 0x7fc49c06cb00 2024-03-29T04:12:49.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.249+0000 7fc49affd640 1 -- 172.21.15.148:0/3342431336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c0cef70 msgr2=0x7fc49c06c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:49.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.249+0000 7fc49affd640 1 --2- 172.21.15.148:0/3342431336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c0cef70 0x7fc49c06c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.249+0000 7fc49affd640 1 -- 172.21.15.148:0/3342431336 >> v1:172.21.15.157:6789/0 conn(0x7fc49c144b10 legacy=0x7fc49c0cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.249+0000 7fc49affd640 1 -- 172.21.15.148:0/3342431336 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49c0ce630 con 0x7fc49c06cb00 2024-03-29T04:12:49.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.249+0000 7fc49affd640 1 -- 172.21.15.148:0/3342431336 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3492041099 0 0) 0x7fc4940019f0 con 0x7fc49c06cb00 2024-03-29T04:12:49.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.249+0000 7fc49b7fe640 1 --2- 172.21.15.148:0/3342431336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c0cef70 0x7fc49c06c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:12:49.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.249+0000 7fc49affd640 1 -- 172.21.15.148:0/3342431336 >> v1:172.21.15.148:6789/0 conn(0x7fc49c06cb00 legacy=0x7fc49c072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.249+0000 7fc49affd640 1 --2- 172.21.15.148:0/3342431336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c0cef70 0x7fc484003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:49.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.249+0000 7fc49affd640 1 --2- 172.21.15.148:0/3342431336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4840040b0 0x7fc484004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:49.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.249+0000 7fc49affd640 1 --2- 172.21.15.148:0/3342431336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc484004d40 0x7fc4840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:49.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.249+0000 7fc49affd640 1 -- 172.21.15.148:0/3342431336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c06cf70 con 0x7fc4840040b0 2024-03-29T04:12:49.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.249+0000 7fc49affd640 1 -- 172.21.15.148:0/3342431336 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc48400dd10 con 0x7fc484004d40 2024-03-29T04:12:49.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.249+0000 7fc49affd640 1 -- 172.21.15.148:0/3342431336 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc48400df10 con 0x7fc49c0cef70 2024-03-29T04:12:49.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.249+0000 7fc49b7fe640 1 --2- 172.21.15.148:0/3342431336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4840040b0 0x7fc484004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:49.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.249+0000 7fc49bfff640 1 --2- 172.21.15.148:0/3342431336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c0cef70 0x7fc484003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:49.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.249+0000 7fc4a0f3a640 1 --2- 172.21.15.148:0/3342431336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc484004d40 0x7fc4840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:49.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.249+0000 7fc49b7fe640 1 -- 172.21.15.148:0/3342431336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c0cef70 msgr2=0x7fc484003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:49.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.249+0000 7fc49b7fe640 1 --2- 172.21.15.148:0/3342431336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c0cef70 0x7fc484003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.249+0000 7fc49b7fe640 1 -- 172.21.15.148:0/3342431336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc484004d40 msgr2=0x7fc4840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:49.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.249+0000 7fc49b7fe640 1 --2- 172.21.15.148:0/3342431336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc484004d40 0x7fc4840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.250+0000 7fc49b7fe640 1 -- 172.21.15.148:0/3342431336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc484003520 con 0x7fc4840040b0 2024-03-29T04:12:49.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.250+0000 7fc49bfff640 1 --2- 172.21.15.148:0/3342431336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c0cef70 0x7fc484003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:49.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.250+0000 7fc49b7fe640 1 --2- 172.21.15.148:0/3342431336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4840040b0 0x7fc484004800 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc490009c40 tx=0x7fc484005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986f60488e79565c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:49.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.250+0000 7fc49affd640 1 -- 172.21.15.148:0/3342431336 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc490039070 con 0x7fc4840040b0 2024-03-29T04:12:49.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.250+0000 7fc49affd640 1 -- 172.21.15.148:0/3342431336 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc49002c8d0 con 0x7fc4840040b0 2024-03-29T04:12:49.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.250+0000 7fc49affd640 1 -- 172.21.15.148:0/3342431336 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc490009840 con 0x7fc4840040b0 2024-03-29T04:12:49.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.251+0000 7fc4a299d640 1 -- 172.21.15.148:0/3342431336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4840040b0 msgr2=0x7fc484004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.251+0000 7fc4a299d640 1 --2- 172.21.15.148:0/3342431336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4840040b0 0x7fc484004800 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc490009c40 tx=0x7fc484005830 comp rx=0 tx=0).stop 2024-03-29T04:12:49.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.251+0000 7fc4a0f3a640 1 -- 172.21.15.148:0/3342431336 reap_dead start 2024-03-29T04:12:49.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.251+0000 7fc4a299d640 1 -- 172.21.15.148:0/3342431336 shutdown_connections 2024-03-29T04:12:49.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.251+0000 7fc4a299d640 1 -- 172.21.15.148:0/3342431336 >> 172.21.15.148:0/3342431336 conn(0x7fc49c0b1b70 msgr2=0x7fc49c0b1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:49.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.251+0000 7fc4a299d640 1 -- 172.21.15.148:0/3342431336 shutdown_connections 2024-03-29T04:12:49.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.251+0000 7fc4a299d640 1 -- 172.21.15.148:0/3342431336 wait complete. 2024-03-29T04:12:49.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.252+0000 7fc4a299d640 1 Processor -- start 2024-03-29T04:12:49.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.252+0000 7fc4a299d640 1 -- start start 2024-03-29T04:12:49.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.253+0000 7fc4a299d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4840040b0 0x7fc49c13c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:49.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.253+0000 7fc4a299d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc49c13fc90 0x7fc49c1efd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:49.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.253+0000 7fc4a299d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c1f02c0 0x7fc49c15c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:49.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.253+0000 7fc4a299d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c0b2e70 con 0x7fc4840040b0 2024-03-29T04:12:49.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.253+0000 7fc4a299d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c0ce0f0 con 0x7fc49c13fc90 2024-03-29T04:12:49.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.253+0000 7fc4a299d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc49c06c5c0 con 0x7fc49c1f02c0 2024-03-29T04:12:49.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.253+0000 7fc49bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4840040b0 0x7fc49c13c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:49.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.253+0000 7fc49bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4840040b0 0x7fc49c13c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56718/0 (socket says 172.21.15.148:56718) 2024-03-29T04:12:49.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.253+0000 7fc49bfff640 1 -- 172.21.15.148:0/4240380104 learned_addr learned my addr 172.21.15.148:0/4240380104 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:49.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.253+0000 7fc4a0f3a640 1 --2- 172.21.15.148:0/4240380104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c1f02c0 0x7fc49c15c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:49.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.253+0000 7fc49b7fe640 1 --2- 172.21.15.148:0/4240380104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc49c13fc90 0x7fc49c1efd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:49.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.254+0000 7fc49bfff640 1 -- 172.21.15.148:0/4240380104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c1f02c0 msgr2=0x7fc49c15c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.254+0000 7fc49bfff640 1 --2- 172.21.15.148:0/4240380104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c1f02c0 0x7fc49c15c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.254+0000 7fc49bfff640 1 -- 172.21.15.148:0/4240380104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc49c13fc90 msgr2=0x7fc49c1efd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.254+0000 7fc49bfff640 1 --2- 172.21.15.148:0/4240380104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc49c13fc90 0x7fc49c1efd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.254+0000 7fc49bfff640 1 -- 172.21.15.148:0/4240380104 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49c13ced0 con 0x7fc4840040b0 2024-03-29T04:12:49.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.254+0000 7fc49bfff640 1 --2- 172.21.15.148:0/4240380104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4840040b0 0x7fc49c13c990 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00e7b0 tx=0x7fc48c00ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:49.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.254+0000 7fc498ff9640 1 -- 172.21.15.148:0/4240380104 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c0050d0 con 0x7fc4840040b0 2024-03-29T04:12:49.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.254+0000 7fc498ff9640 1 -- 172.21.15.148:0/4240380104 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc48c005270 con 0x7fc4840040b0 2024-03-29T04:12:49.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.254+0000 7fc4a299d640 1 -- 172.21.15.148:0/4240380104 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc49c0b46b0 con 0x7fc4840040b0 2024-03-29T04:12:49.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.254+0000 7fc498ff9640 1 -- 172.21.15.148:0/4240380104 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c005420 con 0x7fc4840040b0 2024-03-29T04:12:49.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.254+0000 7fc4a299d640 1 -- 172.21.15.148:0/4240380104 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc49c0b4c70 con 0x7fc4840040b0 2024-03-29T04:12:49.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.255+0000 7fc4a299d640 1 -- 172.21.15.148:0/4240380104 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc49c0b5330 con 0x7fc4840040b0 2024-03-29T04:12:49.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.255+0000 7fc498ff9640 1 -- 172.21.15.148:0/4240380104 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc48c01a020 con 0x7fc4840040b0 2024-03-29T04:12:49.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.256+0000 7fc498ff9640 1 --2- 172.21.15.148:0/4240380104 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc48806c560 0x7fc48806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:49.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.256+0000 7fc498ff9640 1 -- 172.21.15.148:0/4240380104 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7fc48c08ab80 con 0x7fc4840040b0 2024-03-29T04:12:49.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.256+0000 7fc49b7fe640 1 --2- 172.21.15.148:0/4240380104 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc48806c560 0x7fc48806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:49.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.256+0000 7fc498ff9640 1 --2- 172.21.15.148:0/4240380104 >> [v2:172.21.15.157:6808/3576931649,v1:172.21.15.157:6809/3576931649] conn(0x7fc488072a00 0x7fc488074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:49.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.256+0000 7fc498ff9640 1 -- 172.21.15.148:0/4240380104 --> [v2:172.21.15.157:6808/3576931649,v1:172.21.15.157:6809/3576931649] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc48c008e50 con 0x7fc488072a00 2024-03-29T04:12:49.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.256+0000 7fc498ff9640 1 -- 172.21.15.148:0/4240380104 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc48c08f2a0 con 0x7fc4840040b0 2024-03-29T04:12:49.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.256+0000 7fc4a0f3a640 1 --2- 172.21.15.148:0/4240380104 >> [v2:172.21.15.157:6808/3576931649,v1:172.21.15.157:6809/3576931649] conn(0x7fc488072a00 0x7fc488074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:49.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.256+0000 7fc49b7fe640 1 --2- 172.21.15.148:0/4240380104 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc48806c560 0x7fc48806ea20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc490036000 tx=0x7fc490030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:49.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.256+0000 7fc4a0f3a640 1 --2- 172.21.15.148:0/4240380104 >> [v2:172.21.15.157:6808/3576931649,v1:172.21.15.157:6809/3576931649] conn(0x7fc488072a00 0x7fc488074e40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:49.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.257+0000 7fc498ff9640 1 -- 172.21.15.148:0/4240380104 <== osd.6 v2:172.21.15.157:6808/3576931649 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc48c008e50 con 0x7fc488072a00 2024-03-29T04:12:49.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.270+0000 7fc4a299d640 1 -- 172.21.15.148:0/4240380104 --> [v2:172.21.15.157:6808/3576931649,v1:172.21.15.157:6809/3576931649] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc49c068740 con 0x7fc488072a00 2024-03-29T04:12:49.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.271+0000 7fc498ff9640 1 -- 172.21.15.148:0/4240380104 <== osd.6 v2:172.21.15.157:6808/3576931649 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc49c068740 con 0x7fc488072a00 2024-03-29T04:12:49.250 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-03-29T04:12:49.250 INFO:teuthology.orchestra.run.smithi148.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:12:49.251 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-03-29T04:12:49.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.271+0000 7fc4a299d640 1 -- 172.21.15.148:0/4240380104 >> [v2:172.21.15.157:6808/3576931649,v1:172.21.15.157:6809/3576931649] conn(0x7fc488072a00 msgr2=0x7fc488074e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.271+0000 7fc4a299d640 1 --2- 172.21.15.148:0/4240380104 >> [v2:172.21.15.157:6808/3576931649,v1:172.21.15.157:6809/3576931649] conn(0x7fc488072a00 0x7fc488074e40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.271+0000 7fc4a299d640 1 -- 172.21.15.148:0/4240380104 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc48806c560 msgr2=0x7fc48806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.271+0000 7fc4a299d640 1 --2- 172.21.15.148:0/4240380104 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc48806c560 0x7fc48806ea20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc490036000 tx=0x7fc490030000 comp rx=0 tx=0).stop 2024-03-29T04:12:49.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.271+0000 7fc4a299d640 1 -- 172.21.15.148:0/4240380104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4840040b0 msgr2=0x7fc49c13c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.271+0000 7fc4a299d640 1 --2- 172.21.15.148:0/4240380104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4840040b0 0x7fc49c13c990 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00e7b0 tx=0x7fc48c00ebb0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.271+0000 7fc4a0f3a640 1 -- 172.21.15.148:0/4240380104 reap_dead start 2024-03-29T04:12:49.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.271+0000 7fc4a299d640 1 -- 172.21.15.148:0/4240380104 shutdown_connections 2024-03-29T04:12:49.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.271+0000 7fc4a299d640 1 -- 172.21.15.148:0/4240380104 >> 172.21.15.148:0/4240380104 conn(0x7fc49c0b1b70 msgr2=0x7fc49c0cbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:49.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.271+0000 7fc4a299d640 1 -- 172.21.15.148:0/4240380104 shutdown_connections 2024-03-29T04:12:49.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.271+0000 7fc4a299d640 1 -- 172.21.15.148:0/4240380104 wait complete. 2024-03-29T04:12:49.261 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-29T04:12:49.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.383+0000 7f6f6afa8640 1 Processor -- start 2024-03-29T04:12:49.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.383+0000 7f6f6afa8640 1 -- start start 2024-03-29T04:12:49.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.384+0000 7f6f6afa8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6f6415a740 0x7f6f6415e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:49.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.384+0000 7f6f6afa8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f640b3ef0 0x7f6f6415eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:49.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.384+0000 7f6f6afa8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f640b4360 con 0x7f6f6415e910 2024-03-29T04:12:49.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.384+0000 7f6f6afa8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f6f640cf790 con 0x7f6f6415a740 2024-03-29T04:12:49.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.384+0000 7f6f6afa8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6f640728d0 con 0x7f6f640b3ef0 2024-03-29T04:12:49.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.384+0000 7f6f68d44640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6f6415a740 0x7f6f6415e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:49.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.384+0000 7f6f63fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6f6415e910 0x7f6f6406d6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44076/0 (socket says 172.21.15.148:44076) 2024-03-29T04:12:49.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.384+0000 7f6f68d44640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6f6415a740 0x7f6f6415e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56724/0 (socket says 172.21.15.148:56724) 2024-03-29T04:12:49.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.384+0000 7f6f63fff640 1 -- 172.21.15.148:0/369712960 learned_addr learned my addr 172.21.15.148:0/369712960 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:49.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.384+0000 7f6f69545640 1 --2- 172.21.15.148:0/369712960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f640b3ef0 0x7f6f6415eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:49.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.384+0000 7f6f68d44640 1 -- 172.21.15.148:0/369712960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f640b3ef0 msgr2=0x7f6f6415eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:49.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.384+0000 7f6f68d44640 1 --2- 172.21.15.148:0/369712960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f640b3ef0 0x7f6f6415eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.384+0000 7f6f68d44640 1 -- 172.21.15.148:0/369712960 >> v1:172.21.15.148:6789/0 conn(0x7f6f6415e910 legacy=0x7f6f6406d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.384+0000 7f6f68d44640 1 -- 172.21.15.148:0/369712960 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f64072bb0 con 0x7f6f6415a740 2024-03-29T04:12:49.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.384+0000 7f6f637fe640 1 -- 172.21.15.148:0/369712960 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2851354827 0 0) 0x7f6f640b4360 con 0x7f6f6415e910 2024-03-29T04:12:49.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.385+0000 7f6f68d44640 1 --2- 172.21.15.148:0/369712960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f6415a740 0x7f6f6415e3d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6f580097a0 tx=0x7f6f5802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540433ce6fbb01f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:49.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.385+0000 7f6f637fe640 1 -- 172.21.15.148:0/369712960 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f580315a0 con 0x7f6f6415a740 2024-03-29T04:12:49.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.385+0000 7f6f637fe640 1 -- 172.21.15.148:0/369712960 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f58037070 con 0x7f6f6415a740 2024-03-29T04:12:49.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.385+0000 7f6f637fe640 1 -- 172.21.15.148:0/369712960 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58031960 con 0x7f6f6415a740 2024-03-29T04:12:49.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.385+0000 7f6f6afa8640 1 -- 172.21.15.148:0/369712960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f6415a740 msgr2=0x7f6f6415e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.385+0000 7f6f6afa8640 1 --2- 172.21.15.148:0/369712960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f6415a740 0x7f6f6415e3d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6f580097a0 tx=0x7f6f5802b840 comp rx=0 tx=0).stop 2024-03-29T04:12:49.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.386+0000 7f6f6afa8640 1 -- 172.21.15.148:0/369712960 shutdown_connections 2024-03-29T04:12:49.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.386+0000 7f6f6afa8640 1 --2- 172.21.15.148:0/369712960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f640b3ef0 0x7f6f6415eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.386+0000 7f6f6afa8640 1 --2- 172.21.15.148:0/369712960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f6415a740 0x7f6f6415e3d0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.386+0000 7f6f6afa8640 1 -- 172.21.15.148:0/369712960 >> 172.21.15.148:0/369712960 conn(0x7f6f640cfed0 msgr2=0x7f6f640d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:49.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.386+0000 7f6f6afa8640 1 -- 172.21.15.148:0/369712960 shutdown_connections 2024-03-29T04:12:49.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.386+0000 7f6f6afa8640 1 -- 172.21.15.148:0/369712960 wait complete. 2024-03-29T04:12:49.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.387+0000 7f6f6afa8640 1 Processor -- start 2024-03-29T04:12:49.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.387+0000 7f6f6afa8640 1 -- start start 2024-03-29T04:12:49.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.388+0000 7f6f6afa8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f640b3ef0 0x7f6f6415b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:49.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.388+0000 7f6f6afa8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f6415a740 0x7f6f641e3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:49.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.388+0000 7f6f6afa8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f6415e910 0x7f6f6406e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:49.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.388+0000 7f6f6afa8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6415ab20 con 0x7f6f640b3ef0 2024-03-29T04:12:49.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.388+0000 7f6f6afa8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6f64073170 con 0x7f6f6415a740 2024-03-29T04:12:49.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.388+0000 7f6f6afa8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6f6415aca0 con 0x7f6f6415e910 2024-03-29T04:12:49.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.388+0000 7f6f68d44640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f640b3ef0 0x7f6f6415b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:49.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.388+0000 7f6f68d44640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f640b3ef0 0x7f6f6415b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56730/0 (socket says 172.21.15.148:56730) 2024-03-29T04:12:49.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.388+0000 7f6f68d44640 1 -- 172.21.15.148:0/3135393850 learned_addr learned my addr 172.21.15.148:0/3135393850 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:49.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.388+0000 7f6f69545640 1 --2- 172.21.15.148:0/3135393850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f6415e910 0x7f6f6406e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:49.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.388+0000 7f6f63fff640 1 --2- 172.21.15.148:0/3135393850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f6415a740 0x7f6f641e3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:49.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.388+0000 7f6f68d44640 1 -- 172.21.15.148:0/3135393850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f6415e910 msgr2=0x7f6f6406e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.388+0000 7f6f68d44640 1 --2- 172.21.15.148:0/3135393850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f6415e910 0x7f6f6406e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.388+0000 7f6f68d44640 1 -- 172.21.15.148:0/3135393850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f6415a740 msgr2=0x7f6f641e3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.388+0000 7f6f68d44640 1 --2- 172.21.15.148:0/3135393850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f6415a740 0x7f6f641e3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.388+0000 7f6f68d44640 1 -- 172.21.15.148:0/3135393850 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6413fac0 con 0x7f6f640b3ef0 2024-03-29T04:12:49.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.388+0000 7f6f63fff640 1 --2- 172.21.15.148:0/3135393850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f6415a740 0x7f6f641e3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:12:49.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.388+0000 7f6f68d44640 1 --2- 172.21.15.148:0/3135393850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f640b3ef0 0x7f6f6415b240 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6f58034040 tx=0x7f6f580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:49.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.388+0000 7f6f617fa640 1 -- 172.21.15.148:0/3135393850 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58032840 con 0x7f6f640b3ef0 2024-03-29T04:12:49.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.388+0000 7f6f617fa640 1 -- 172.21.15.148:0/3135393850 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f58032a00 con 0x7f6f640b3ef0 2024-03-29T04:12:49.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.388+0000 7f6f6afa8640 1 -- 172.21.15.148:0/3135393850 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f6413fda0 con 0x7f6f640b3ef0 2024-03-29T04:12:49.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.388+0000 7f6f617fa640 1 -- 172.21.15.148:0/3135393850 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58032df0 con 0x7f6f640b3ef0 2024-03-29T04:12:49.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.388+0000 7f6f6afa8640 1 -- 172.21.15.148:0/3135393850 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f64140360 con 0x7f6f640b3ef0 2024-03-29T04:12:49.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.389+0000 7f6f6afa8640 1 -- 172.21.15.148:0/3135393850 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f640716c0 con 0x7f6f640b3ef0 2024-03-29T04:12:49.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.390+0000 7f6f617fa640 1 -- 172.21.15.148:0/3135393850 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6f5804e430 con 0x7f6f640b3ef0 2024-03-29T04:12:49.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.390+0000 7f6f617fa640 1 --2- 172.21.15.148:0/3135393850 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6f4806c560 0x7f6f4806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:49.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.390+0000 7f6f617fa640 1 -- 172.21.15.148:0/3135393850 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f6f5802f030 con 0x7f6f640b3ef0 2024-03-29T04:12:49.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.390+0000 7f6f63fff640 1 --2- 172.21.15.148:0/3135393850 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6f4806c560 0x7f6f4806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:49.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.391+0000 7f6f63fff640 1 --2- 172.21.15.148:0/3135393850 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6f4806c560 0x7f6f4806ea20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6f641f78d0 tx=0x7f6f50009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:49.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.395+0000 7f6f617fa640 1 -- 172.21.15.148:0/3135393850 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6f640716c0 con 0x7f6f640b3ef0 2024-03-29T04:12:49.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.569+0000 7f6f6afa8640 1 -- 172.21.15.148:0/3135393850 --> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f6f640687e0 con 0x7f6f4806c560 2024-03-29T04:12:49.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.569+0000 7f6f617fa640 1 -- 172.21.15.148:0/3135393850 <== mgr.4113 v2:172.21.15.148:6832/3271445075 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6f640687e0 con 0x7f6f4806c560 2024-03-29T04:12:49.549 INFO:teuthology.orchestra.run.smithi148.stderr:instructed osd(s) 6 to deep-scrub 2024-03-29T04:12:49.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.571+0000 7f6f6afa8640 1 -- 172.21.15.148:0/3135393850 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6f4806c560 msgr2=0x7f6f4806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.571+0000 7f6f6afa8640 1 --2- 172.21.15.148:0/3135393850 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6f4806c560 0x7f6f4806ea20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6f641f78d0 tx=0x7f6f50009040 comp rx=0 tx=0).stop 2024-03-29T04:12:49.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.571+0000 7f6f6afa8640 1 -- 172.21.15.148:0/3135393850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f640b3ef0 msgr2=0x7f6f6415b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.571+0000 7f6f6afa8640 1 --2- 172.21.15.148:0/3135393850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f640b3ef0 0x7f6f6415b240 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6f58034040 tx=0x7f6f580046c0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.571+0000 7f6f6afa8640 1 -- 172.21.15.148:0/3135393850 shutdown_connections 2024-03-29T04:12:49.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.571+0000 7f6f6afa8640 1 --2- 172.21.15.148:0/3135393850 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f6f4806c560 0x7f6f4806ea20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.571+0000 7f6f6afa8640 1 --2- 172.21.15.148:0/3135393850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f6415e910 0x7f6f6406e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.571+0000 7f6f6afa8640 1 --2- 172.21.15.148:0/3135393850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f6415a740 0x7f6f641e3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.571+0000 7f6f6afa8640 1 --2- 172.21.15.148:0/3135393850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f640b3ef0 0x7f6f6415b240 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.571+0000 7f6f6afa8640 1 -- 172.21.15.148:0/3135393850 >> 172.21.15.148:0/3135393850 conn(0x7f6f640cfed0 msgr2=0x7f6f640af970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:49.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.571+0000 7f6f6afa8640 1 -- 172.21.15.148:0/3135393850 shutdown_connections 2024-03-29T04:12:49.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.571+0000 7f6f6afa8640 1 -- 172.21.15.148:0/3135393850 wait complete. 2024-03-29T04:12:49.562 INFO:tasks.ceph:Scrubbing osd.7 2024-03-29T04:12:49.562 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:12:49.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.687+0000 7ff35d552640 1 Processor -- start 2024-03-29T04:12:49.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.687+0000 7ff35d552640 1 -- start start 2024-03-29T04:12:49.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.687+0000 7ff35d552640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3580d3840 0x7ff3580d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:49.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.687+0000 7ff35d552640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3580d27b0 con 0x7ff3580b36e0 2024-03-29T04:12:49.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.687+0000 7ff35d552640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3580d2a50 con 0x7ff358161ce0 2024-03-29T04:12:49.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.687+0000 7ff35d552640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff3580d2d30 con 0x7ff3580d3840 2024-03-29T04:12:49.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.688+0000 7ff3567fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff3580b36e0 0x7ff3581615d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44092/0 (socket says 172.21.15.148:44092) 2024-03-29T04:12:49.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.688+0000 7ff3567fc640 1 -- 172.21.15.148:0/4060900002 learned_addr learned my addr 172.21.15.148:0/4060900002 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:49.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.688+0000 7ff356ffd640 1 --2- 172.21.15.148:0/4060900002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3580d3840 0x7ff3580d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:49.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.688+0000 7ff355ffb640 1 -- 172.21.15.148:0/4060900002 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4230834365 0 0) 0x7ff3580d27b0 con 0x7ff3580b36e0 2024-03-29T04:12:49.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.688+0000 7ff355ffb640 1 -- 172.21.15.148:0/4060900002 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff344003520 con 0x7ff3580b36e0 2024-03-29T04:12:49.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.688+0000 7ff355ffb640 1 -- 172.21.15.148:0/4060900002 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 298989066 0 0) 0x7ff3580d2a50 con 0x7ff358161ce0 2024-03-29T04:12:49.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.688+0000 7ff355ffb640 1 -- 172.21.15.148:0/4060900002 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3580d27b0 con 0x7ff358161ce0 2024-03-29T04:12:49.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.688+0000 7ff355ffb640 1 -- 172.21.15.148:0/4060900002 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2317359841 0 0) 0x7ff344003520 con 0x7ff3580b36e0 2024-03-29T04:12:49.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.688+0000 7ff356ffd640 1 -- 172.21.15.148:0/4060900002 >> v1:172.21.15.157:6789/0 conn(0x7ff358161ce0 legacy=0x7ff35806bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.688+0000 7ff356ffd640 1 -- 172.21.15.148:0/4060900002 >> v1:172.21.15.148:6789/0 conn(0x7ff3580b36e0 legacy=0x7ff3581615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.688+0000 7ff356ffd640 1 -- 172.21.15.148:0/4060900002 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3580d3010 con 0x7ff3580d3840 2024-03-29T04:12:49.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.689+0000 7ff356ffd640 1 --2- 172.21.15.148:0/4060900002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3580d3840 0x7ff3580d2270 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff350009dc0 tx=0x7ff35002bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac8ab208d934cfc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:49.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.689+0000 7ff355ffb640 1 -- 172.21.15.148:0/4060900002 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff350039070 con 0x7ff3580d3840 2024-03-29T04:12:49.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.689+0000 7ff355ffb640 1 -- 172.21.15.148:0/4060900002 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff35002c910 con 0x7ff3580d3840 2024-03-29T04:12:49.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.689+0000 7ff355ffb640 1 -- 172.21.15.148:0/4060900002 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff340001690 con 0x7ff3580d3840 2024-03-29T04:12:49.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.690+0000 7ff35d552640 1 -- 172.21.15.148:0/4060900002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3580d3840 msgr2=0x7ff3580d2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.690+0000 7ff35d552640 1 --2- 172.21.15.148:0/4060900002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3580d3840 0x7ff3580d2270 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff350009dc0 tx=0x7ff35002bde0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.690+0000 7ff35d552640 1 -- 172.21.15.148:0/4060900002 shutdown_connections 2024-03-29T04:12:49.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.690+0000 7ff35d552640 1 --2- 172.21.15.148:0/4060900002 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3580d3840 0x7ff3580d2270 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.690+0000 7ff35d552640 1 -- 172.21.15.148:0/4060900002 >> 172.21.15.148:0/4060900002 conn(0x7ff3580d42d0 msgr2=0x7ff3580d46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:49.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.690+0000 7ff35d552640 1 -- 172.21.15.148:0/4060900002 shutdown_connections 2024-03-29T04:12:49.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.690+0000 7ff35d552640 1 -- 172.21.15.148:0/4060900002 wait complete. 2024-03-29T04:12:49.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.691+0000 7ff35d552640 1 Processor -- start 2024-03-29T04:12:49.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.691+0000 7ff35d552640 1 -- start start 2024-03-29T04:12:49.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.692+0000 7ff35d552640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3580b36e0 0x7ff3580cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:49.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.692+0000 7ff35d552640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3580d3840 0x7ff3580cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:49.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.692+0000 7ff35d552640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff358161ce0 0x7ff358152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:49.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.692+0000 7ff35d552640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff3580d3da0 con 0x7ff3580d3840 2024-03-29T04:12:49.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.692+0000 7ff35d552640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff3580d3c20 con 0x7ff358161ce0 2024-03-29T04:12:49.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.692+0000 7ff35d552640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff3580d3f20 con 0x7ff3580b36e0 2024-03-29T04:12:49.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.692+0000 7ff3567fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3580d3840 0x7ff3580cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:49.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.692+0000 7ff3567fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3580d3840 0x7ff3580cda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56740/0 (socket says 172.21.15.148:56740) 2024-03-29T04:12:49.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.692+0000 7ff3567fc640 1 -- 172.21.15.148:0/694358613 learned_addr learned my addr 172.21.15.148:0/694358613 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:49.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.692+0000 7ff3567fc640 1 -- 172.21.15.148:0/694358613 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3580b36e0 msgr2=0x7ff3580cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.692+0000 7ff356ffd640 1 --2- 172.21.15.148:0/694358613 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3580b36e0 0x7ff3580cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:49.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.692+0000 7ff3577fe640 1 --2- 172.21.15.148:0/694358613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff358161ce0 0x7ff358152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:49.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.692+0000 7ff3567fc640 1 --2- 172.21.15.148:0/694358613 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3580b36e0 0x7ff3580cc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.692+0000 7ff3567fc640 1 -- 172.21.15.148:0/694358613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff358161ce0 msgr2=0x7ff358152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.692+0000 7ff3567fc640 1 --2- 172.21.15.148:0/694358613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff358161ce0 0x7ff358152340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.692+0000 7ff3567fc640 1 -- 172.21.15.148:0/694358613 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff35815c090 con 0x7ff3580d3840 2024-03-29T04:12:49.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.692+0000 7ff3567fc640 1 --2- 172.21.15.148:0/694358613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3580d3840 0x7ff3580cda90 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff34000c730 tx=0x7ff34000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:49.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.692+0000 7ff34f7fe640 1 -- 172.21.15.148:0/694358613 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff340018070 con 0x7ff3580d3840 2024-03-29T04:12:49.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.692+0000 7ff34f7fe640 1 -- 172.21.15.148:0/694358613 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3400072d0 con 0x7ff3580d3840 2024-03-29T04:12:49.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.692+0000 7ff35d552640 1 -- 172.21.15.148:0/694358613 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3580ccef0 con 0x7ff3580d3840 2024-03-29T04:12:49.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.692+0000 7ff34f7fe640 1 -- 172.21.15.148:0/694358613 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3400047e0 con 0x7ff3580d3840 2024-03-29T04:12:49.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.692+0000 7ff35d552640 1 -- 172.21.15.148:0/694358613 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3580cbed0 con 0x7ff3580d3840 2024-03-29T04:12:49.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.693+0000 7ff35d552640 1 -- 172.21.15.148:0/694358613 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff358068740 con 0x7ff3580d3840 2024-03-29T04:12:49.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.694+0000 7ff34f7fe640 1 -- 172.21.15.148:0/694358613 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff34001b070 con 0x7ff3580d3840 2024-03-29T04:12:49.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.694+0000 7ff34f7fe640 1 --2- 172.21.15.148:0/694358613 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff33806c530 0x7ff33806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:49.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.694+0000 7ff34f7fe640 1 -- 172.21.15.148:0/694358613 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7ff34008aaf0 con 0x7ff3580d3840 2024-03-29T04:12:49.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.694+0000 7ff356ffd640 1 --2- 172.21.15.148:0/694358613 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff33806c530 0x7ff33806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:49.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.694+0000 7ff34f7fe640 1 --2- 172.21.15.148:0/694358613 >> [v2:172.21.15.157:6824/2669171853,v1:172.21.15.157:6825/2669171853] conn(0x7ff3380729d0 0x7ff338074e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:49.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.694+0000 7ff34f7fe640 1 -- 172.21.15.148:0/694358613 --> [v2:172.21.15.157:6824/2669171853,v1:172.21.15.157:6825/2669171853] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff3380754e0 con 0x7ff3380729d0 2024-03-29T04:12:49.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.694+0000 7ff34f7fe640 1 -- 172.21.15.148:0/694358613 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ff34008f2c0 con 0x7ff3580d3840 2024-03-29T04:12:49.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.695+0000 7ff356ffd640 1 --2- 172.21.15.148:0/694358613 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff33806c530 0x7ff33806e9f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff350036000 tx=0x7ff350030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:49.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.695+0000 7ff3577fe640 1 --2- 172.21.15.148:0/694358613 >> [v2:172.21.15.157:6824/2669171853,v1:172.21.15.157:6825/2669171853] conn(0x7ff3380729d0 0x7ff338074e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:49.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.695+0000 7ff3577fe640 1 --2- 172.21.15.148:0/694358613 >> [v2:172.21.15.157:6824/2669171853,v1:172.21.15.157:6825/2669171853] conn(0x7ff3380729d0 0x7ff338074e10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:49.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.696+0000 7ff34f7fe640 1 -- 172.21.15.148:0/694358613 <== osd.7 v2:172.21.15.157:6824/2669171853 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff3380754e0 con 0x7ff3380729d0 2024-03-29T04:12:49.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.709+0000 7ff35d552640 1 -- 172.21.15.148:0/694358613 --> [v2:172.21.15.157:6824/2669171853,v1:172.21.15.157:6825/2669171853] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff3580d2050 con 0x7ff3380729d0 2024-03-29T04:12:49.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.709+0000 7ff34f7fe640 1 -- 172.21.15.148:0/694358613 <== osd.7 v2:172.21.15.157:6824/2669171853 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff3580d2050 con 0x7ff3380729d0 2024-03-29T04:12:49.689 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-03-29T04:12:49.690 INFO:teuthology.orchestra.run.smithi148.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:12:49.690 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-03-29T04:12:49.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.710+0000 7ff35d552640 1 -- 172.21.15.148:0/694358613 >> [v2:172.21.15.157:6824/2669171853,v1:172.21.15.157:6825/2669171853] conn(0x7ff3380729d0 msgr2=0x7ff338074e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.710+0000 7ff35d552640 1 --2- 172.21.15.148:0/694358613 >> [v2:172.21.15.157:6824/2669171853,v1:172.21.15.157:6825/2669171853] conn(0x7ff3380729d0 0x7ff338074e10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.710+0000 7ff35d552640 1 -- 172.21.15.148:0/694358613 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff33806c530 msgr2=0x7ff33806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.710+0000 7ff35d552640 1 --2- 172.21.15.148:0/694358613 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7ff33806c530 0x7ff33806e9f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff350036000 tx=0x7ff350030000 comp rx=0 tx=0).stop 2024-03-29T04:12:49.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.710+0000 7ff35d552640 1 -- 172.21.15.148:0/694358613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3580d3840 msgr2=0x7ff3580cda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.710+0000 7ff35d552640 1 --2- 172.21.15.148:0/694358613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3580d3840 0x7ff3580cda90 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff34000c730 tx=0x7ff34000cbf0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.710+0000 7ff3577fe640 1 -- 172.21.15.148:0/694358613 reap_dead start 2024-03-29T04:12:49.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.710+0000 7ff35d552640 1 -- 172.21.15.148:0/694358613 shutdown_connections 2024-03-29T04:12:49.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.710+0000 7ff35d552640 1 -- 172.21.15.148:0/694358613 >> 172.21.15.148:0/694358613 conn(0x7ff3580d42d0 msgr2=0x7ff3580cef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:49.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.710+0000 7ff35d552640 1 -- 172.21.15.148:0/694358613 shutdown_connections 2024-03-29T04:12:49.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.710+0000 7ff35d552640 1 -- 172.21.15.148:0/694358613 wait complete. 2024-03-29T04:12:49.701 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-29T04:12:49.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.822+0000 7f5a6b1a8640 1 Processor -- start 2024-03-29T04:12:49.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.822+0000 7f5a6b1a8640 1 -- start start 2024-03-29T04:12:49.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.823+0000 7f5a6b1a8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5a640b2ee0 0x7f5a6415f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:49.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.823+0000 7f5a6b1a8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5a6406c5c0 0x7f5a6406ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:49.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.823+0000 7f5a6b1a8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a640b3630 con 0x7f5a6415b060 2024-03-29T04:12:49.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.823+0000 7f5a6b1a8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7f5a640b3910 con 0x7f5a640b2ee0 2024-03-29T04:12:49.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.823+0000 7f5a6b1a8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5a640b3bf0 con 0x7f5a6406c5c0 2024-03-29T04:12:49.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.823+0000 7f5a63fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5a640b2ee0 0x7f5a6415f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:49.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.823+0000 7f5a68f44640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5a6415b060 0x7f5a640b46d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44098/0 (socket says 172.21.15.148:44098) 2024-03-29T04:12:49.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.823+0000 7f5a68f44640 1 -- 172.21.15.148:0/3186053974 learned_addr learned my addr 172.21.15.148:0/3186053974 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:12:49.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.824+0000 7f5a69745640 1 --2- 172.21.15.148:0/3186053974 >> v2:172.21.15.157:3300/0 conn(0x7f5a6406c5c0 0x7f5a6406ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:49.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.824+0000 7f5a63fff640 1 -- 172.21.15.148:0/3186053974 >> v2:172.21.15.157:3300/0 conn(0x7f5a6406c5c0 msgr2=0x7f5a6406ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:49.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.824+0000 7f5a63fff640 1 --2- 172.21.15.148:0/3186053974 >> v2:172.21.15.157:3300/0 conn(0x7f5a6406c5c0 0x7f5a6406ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.824+0000 7f5a63fff640 1 -- 172.21.15.148:0/3186053974 >> v1:172.21.15.148:6789/0 conn(0x7f5a6415b060 legacy=0x7f5a640b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.824+0000 7f5a637fe640 1 -- 172.21.15.148:0/3186053974 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 179759497 0 0) 0x7f5a640b3630 con 0x7f5a6415b060 2024-03-29T04:12:49.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.824+0000 7f5a63fff640 1 -- 172.21.15.148:0/3186053974 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a640b3ed0 con 0x7f5a640b2ee0 2024-03-29T04:12:49.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.824+0000 7f5a69745640 1 --2- 172.21.15.148:0/3186053974 >> v2:172.21.15.157:3300/0 conn(0x7f5a6406c5c0 0x7f5a6406ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:12:49.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.824+0000 7f5a63fff640 1 --2- 172.21.15.148:0/3186053974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a640b2ee0 0x7f5a6415f6d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5a64071cf0 tx=0x7f5a5802b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfafd624836bf993 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:49.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.824+0000 7f5a637fe640 1 -- 172.21.15.148:0/3186053974 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a58038070 con 0x7f5a640b2ee0 2024-03-29T04:12:49.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.824+0000 7f5a637fe640 1 -- 172.21.15.148:0/3186053974 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a58031710 con 0x7f5a640b2ee0 2024-03-29T04:12:49.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.824+0000 7f5a637fe640 1 -- 172.21.15.148:0/3186053974 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a58031ae0 con 0x7f5a640b2ee0 2024-03-29T04:12:49.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.825+0000 7f5a6b1a8640 1 -- 172.21.15.148:0/3186053974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a640b2ee0 msgr2=0x7f5a6415f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.825+0000 7f5a6b1a8640 1 --2- 172.21.15.148:0/3186053974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a640b2ee0 0x7f5a6415f6d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5a64071cf0 tx=0x7f5a5802b7b0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.825+0000 7f5a6b1a8640 1 -- 172.21.15.148:0/3186053974 shutdown_connections 2024-03-29T04:12:49.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.825+0000 7f5a6b1a8640 1 --2- 172.21.15.148:0/3186053974 >> v2:172.21.15.157:3300/0 conn(0x7f5a6406c5c0 0x7f5a6406ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.825+0000 7f5a6b1a8640 1 --2- 172.21.15.148:0/3186053974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a640b2ee0 0x7f5a6415f6d0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.825+0000 7f5a6b1a8640 1 -- 172.21.15.148:0/3186053974 >> 172.21.15.148:0/3186053974 conn(0x7f5a640cfed0 msgr2=0x7f5a640d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:49.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.825+0000 7f5a6b1a8640 1 -- 172.21.15.148:0/3186053974 shutdown_connections 2024-03-29T04:12:49.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.825+0000 7f5a6b1a8640 1 -- 172.21.15.148:0/3186053974 wait complete. 2024-03-29T04:12:49.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.826+0000 7f5a6b1a8640 1 Processor -- start 2024-03-29T04:12:49.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.826+0000 7f5a6b1a8640 1 -- start start 2024-03-29T04:12:49.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.827+0000 7f5a6b1a8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a6406c5c0 0x7f5a64159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:49.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.827+0000 7f5a6b1a8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a640b2ee0 0x7f5a641f8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:49.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.827+0000 7f5a6b1a8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a6415b060 0x7f5a640ca940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:49.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.827+0000 7f5a6b1a8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6415b440 con 0x7f5a6406c5c0 2024-03-29T04:12:49.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.827+0000 7f5a6b1a8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5a640b3340 con 0x7f5a6415b060 2024-03-29T04:12:49.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.827+0000 7f5a6b1a8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5a640cd530 con 0x7f5a640b2ee0 2024-03-29T04:12:49.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.827+0000 7f5a68f44640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a6406c5c0 0x7f5a64159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:49.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.827+0000 7f5a68f44640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a6406c5c0 0x7f5a64159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56748/0 (socket says 172.21.15.148:56748) 2024-03-29T04:12:49.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.827+0000 7f5a68f44640 1 -- 172.21.15.148:0/1247336620 learned_addr learned my addr 172.21.15.148:0/1247336620 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:49.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.828+0000 7f5a68f44640 1 -- 172.21.15.148:0/1247336620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a640b2ee0 msgr2=0x7f5a641f8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.828+0000 7f5a68f44640 1 --2- 172.21.15.148:0/1247336620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a640b2ee0 0x7f5a641f8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.828+0000 7f5a69745640 1 --2- 172.21.15.148:0/1247336620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a6415b060 0x7f5a640ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:49.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.828+0000 7f5a68f44640 1 -- 172.21.15.148:0/1247336620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a6415b060 msgr2=0x7f5a640ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.828+0000 7f5a68f44640 1 --2- 172.21.15.148:0/1247336620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a6415b060 0x7f5a640ca940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.828+0000 7f5a68f44640 1 -- 172.21.15.148:0/1247336620 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6414e070 con 0x7f5a6406c5c0 2024-03-29T04:12:49.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.828+0000 7f5a68f44640 1 --2- 172.21.15.148:0/1247336620 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a6406c5c0 0x7f5a64159c20 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400c430 tx=0x7f5a5400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:49.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.828+0000 7f5a617fa640 1 -- 172.21.15.148:0/1247336620 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a540074e0 con 0x7f5a6406c5c0 2024-03-29T04:12:49.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.828+0000 7f5a617fa640 1 -- 172.21.15.148:0/1247336620 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a540047e0 con 0x7f5a6406c5c0 2024-03-29T04:12:49.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.828+0000 7f5a6b1a8640 1 -- 172.21.15.148:0/1247336620 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a64158b80 con 0x7f5a6406c5c0 2024-03-29T04:12:49.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.828+0000 7f5a617fa640 1 -- 172.21.15.148:0/1247336620 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a54004bd0 con 0x7f5a6406c5c0 2024-03-29T04:12:49.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.828+0000 7f5a6b1a8640 1 -- 172.21.15.148:0/1247336620 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a64159140 con 0x7f5a6406c5c0 2024-03-29T04:12:49.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.829+0000 7f5a6b1a8640 1 -- 172.21.15.148:0/1247336620 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a640687e0 con 0x7f5a6406c5c0 2024-03-29T04:12:49.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.829+0000 7f5a617fa640 1 -- 172.21.15.148:0/1247336620 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5a54011070 con 0x7f5a6406c5c0 2024-03-29T04:12:49.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.830+0000 7f5a617fa640 1 --2- 172.21.15.148:0/1247336620 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5a5006c530 0x7f5a5006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:49.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.830+0000 7f5a617fa640 1 -- 172.21.15.148:0/1247336620 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f5a5408b1e0 con 0x7f5a6406c5c0 2024-03-29T04:12:49.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.830+0000 7f5a63fff640 1 --2- 172.21.15.148:0/1247336620 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5a5006c530 0x7f5a5006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:49.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.830+0000 7f5a63fff640 1 --2- 172.21.15.148:0/1247336620 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5a5006c530 0x7f5a5006e9f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5a58035000 tx=0x7f5a58030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:49.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:49.835+0000 7f5a617fa640 1 -- 172.21.15.148:0/1247336620 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5a54059970 con 0x7f5a6406c5c0 2024-03-29T04:12:49.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.010+0000 7f5a6b1a8640 1 -- 172.21.15.148:0/1247336620 --> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f5a640721e0 con 0x7f5a5006c530 2024-03-29T04:12:49.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.010+0000 7f5a617fa640 1 -- 172.21.15.148:0/1247336620 <== mgr.4113 v2:172.21.15.148:6832/3271445075 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5a640721e0 con 0x7f5a5006c530 2024-03-29T04:12:49.990 INFO:teuthology.orchestra.run.smithi148.stderr:instructed osd(s) 7 to deep-scrub 2024-03-29T04:12:49.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.013+0000 7f5a6b1a8640 1 -- 172.21.15.148:0/1247336620 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5a5006c530 msgr2=0x7f5a5006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.013+0000 7f5a6b1a8640 1 --2- 172.21.15.148:0/1247336620 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5a5006c530 0x7f5a5006e9f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5a58035000 tx=0x7f5a58030000 comp rx=0 tx=0).stop 2024-03-29T04:12:49.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.013+0000 7f5a6b1a8640 1 -- 172.21.15.148:0/1247336620 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a6406c5c0 msgr2=0x7f5a64159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:49.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.013+0000 7f5a6b1a8640 1 --2- 172.21.15.148:0/1247336620 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a6406c5c0 0x7f5a64159c20 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400c430 tx=0x7f5a5400c8f0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.013+0000 7f5a6b1a8640 1 -- 172.21.15.148:0/1247336620 shutdown_connections 2024-03-29T04:12:49.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.013+0000 7f5a6b1a8640 1 --2- 172.21.15.148:0/1247336620 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f5a5006c530 0x7f5a5006e9f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.013+0000 7f5a6b1a8640 1 --2- 172.21.15.148:0/1247336620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a6415b060 0x7f5a640ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.013+0000 7f5a6b1a8640 1 --2- 172.21.15.148:0/1247336620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a640b2ee0 0x7f5a641f8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.013+0000 7f5a6b1a8640 1 --2- 172.21.15.148:0/1247336620 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a6406c5c0 0x7f5a64159c20 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:49.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.013+0000 7f5a6b1a8640 1 -- 172.21.15.148:0/1247336620 >> 172.21.15.148:0/1247336620 conn(0x7f5a640cfed0 msgr2=0x7f5a64150d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:49.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.013+0000 7f5a6b1a8640 1 -- 172.21.15.148:0/1247336620 shutdown_connections 2024-03-29T04:12:49.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.013+0000 7f5a6b1a8640 1 -- 172.21.15.148:0/1247336620 wait complete. 2024-03-29T04:12:50.002 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:12:50.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.123+0000 7f8d0a8c8640 1 Processor -- start 2024-03-29T04:12:50.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.123+0000 7f8d0a8c8640 1 -- start start 2024-03-29T04:12:50.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.124+0000 7f8d0a8c8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8d040b3710 0x7f8d040b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:50.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.124+0000 7f8d0a8c8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d0406b5b0 0x7f8d0406ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:50.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.124+0000 7f8d0a8c8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d0406dd30 con 0x7f8d04158c30 2024-03-29T04:12:50.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.124+0000 7f8d0a8c8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8d0406e010 con 0x7f8d040b3710 2024-03-29T04:12:50.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.124+0000 7f8d0a8c8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8d0406e2f0 con 0x7f8d0406b5b0 2024-03-29T04:12:50.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.124+0000 7f8d08e65640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d0406b5b0 0x7f8d0406ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:50.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.125+0000 7f8d08e65640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d0406b5b0 0x7f8d0406ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.148:37796/0 (socket says 172.21.15.148:37796) 2024-03-29T04:12:50.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.125+0000 7f8d08e65640 1 -- 172.21.15.148:0/1875176153 learned_addr learned my addr 172.21.15.148:0/1875176153 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:50.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.125+0000 7f8d03fff640 1 --1- 172.21.15.148:0/1875176153 >> v1:172.21.15.157:6789/0 conn(0x7f8d04158c30 0x7f8d0415f480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.148:38688/0 (socket says 172.21.15.148:38688) 2024-03-29T04:12:50.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.125+0000 7f8d037fe640 1 --2- 172.21.15.148:0/1875176153 >> v2:172.21.15.157:3300/0 conn(0x7f8d040b3710 0x7f8d040b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:50.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.125+0000 7f8d02ffd640 1 -- 172.21.15.148:0/1875176153 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3685562235 0 0) 0x7f8d0406dd30 con 0x7f8d04158c30 2024-03-29T04:12:50.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.125+0000 7f8d08e65640 1 -- 172.21.15.148:0/1875176153 >> v2:172.21.15.157:3300/0 conn(0x7f8d040b3710 msgr2=0x7f8d040b3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:50.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.125+0000 7f8d08e65640 1 --2- 172.21.15.148:0/1875176153 >> v2:172.21.15.157:3300/0 conn(0x7f8d040b3710 0x7f8d040b3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:50.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.125+0000 7f8d08e65640 1 -- 172.21.15.148:0/1875176153 >> v1:172.21.15.157:6789/0 conn(0x7f8d04158c30 legacy=0x7f8d0415f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:50.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.125+0000 7f8d08e65640 1 -- 172.21.15.148:0/1875176153 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0406e5d0 con 0x7f8d0406b5b0 2024-03-29T04:12:50.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.126+0000 7f8d08e65640 1 --2- 172.21.15.148:0/1875176153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d0406b5b0 0x7f8d0406ba20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc009e50 tx=0x7f8cfc02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cea0d0af05494406 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:50.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.126+0000 7f8d02ffd640 1 -- 172.21.15.148:0/1875176153 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc02ca70 con 0x7f8d0406b5b0 2024-03-29T04:12:50.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.126+0000 7f8d02ffd640 1 -- 172.21.15.148:0/1875176153 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cfc02cc10 con 0x7f8d0406b5b0 2024-03-29T04:12:50.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.126+0000 7f8d02ffd640 1 -- 172.21.15.148:0/1875176153 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc0343f0 con 0x7f8d0406b5b0 2024-03-29T04:12:50.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.126+0000 7f8d0a8c8640 1 -- 172.21.15.148:0/1875176153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d0406b5b0 msgr2=0x7f8d0406ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:50.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.126+0000 7f8d0a8c8640 1 --2- 172.21.15.148:0/1875176153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d0406b5b0 0x7f8d0406ba20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc009e50 tx=0x7f8cfc02bed0 comp rx=0 tx=0).stop 2024-03-29T04:12:50.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.126+0000 7f8d0a8c8640 1 -- 172.21.15.148:0/1875176153 shutdown_connections 2024-03-29T04:12:50.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.126+0000 7f8d0a8c8640 1 --2- 172.21.15.148:0/1875176153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d0406b5b0 0x7f8d0406ba20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:50.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.126+0000 7f8d0a8c8640 1 --2- 172.21.15.148:0/1875176153 >> v2:172.21.15.157:3300/0 conn(0x7f8d040b3710 0x7f8d040b3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:50.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.126+0000 7f8d0a8c8640 1 -- 172.21.15.148:0/1875176153 >> 172.21.15.148:0/1875176153 conn(0x7f8d040d0700 msgr2=0x7f8d040d0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:50.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.126+0000 7f8d0a8c8640 1 -- 172.21.15.148:0/1875176153 shutdown_connections 2024-03-29T04:12:50.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.127+0000 7f8d0a8c8640 1 -- 172.21.15.148:0/1875176153 wait complete. 2024-03-29T04:12:50.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.128+0000 7f8d0a8c8640 1 Processor -- start 2024-03-29T04:12:50.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.128+0000 7f8d0a8c8640 1 -- start start 2024-03-29T04:12:50.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.128+0000 7f8d0a8c8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d0406b5b0 0x7f8d04152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:50.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.129+0000 7f8d0a8c8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d040b3710 0x7f8d040aef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:50.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.129+0000 7f8d0a8c8640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d04158c30 0x7f8d04160f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:50.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.129+0000 7f8d0a8c8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8d040b4400 con 0x7f8d040b3710 2024-03-29T04:12:50.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.129+0000 7f8d0a8c8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0406da00 con 0x7f8d0406b5b0 2024-03-29T04:12:50.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.129+0000 7f8d0a8c8640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8d040b4580 con 0x7f8d04158c30 2024-03-29T04:12:50.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.129+0000 7f8d037fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d040b3710 0x7f8d040aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:50.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.129+0000 7f8d037fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d040b3710 0x7f8d040aef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56756/0 (socket says 172.21.15.148:56756) 2024-03-29T04:12:50.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.129+0000 7f8d037fe640 1 -- 172.21.15.148:0/1608029912 learned_addr learned my addr 172.21.15.148:0/1608029912 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:12:50.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.129+0000 7f8d03fff640 1 --2- 172.21.15.148:0/1608029912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d0406b5b0 0x7f8d04152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:50.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.129+0000 7f8d08e65640 1 --2- 172.21.15.148:0/1608029912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d04158c30 0x7f8d04160f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:50.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.129+0000 7f8d037fe640 1 -- 172.21.15.148:0/1608029912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d04158c30 msgr2=0x7f8d04160f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:50.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.129+0000 7f8d037fe640 1 --2- 172.21.15.148:0/1608029912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d04158c30 0x7f8d04160f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:50.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.129+0000 7f8d037fe640 1 -- 172.21.15.148:0/1608029912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d0406b5b0 msgr2=0x7f8d04152af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:50.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.129+0000 7f8d037fe640 1 --2- 172.21.15.148:0/1608029912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d0406b5b0 0x7f8d04152af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:50.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.129+0000 7f8d037fe640 1 -- 172.21.15.148:0/1608029912 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d041533a0 con 0x7f8d040b3710 2024-03-29T04:12:50.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.129+0000 7f8d03fff640 1 --2- 172.21.15.148:0/1608029912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d0406b5b0 0x7f8d04152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:50.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.129+0000 7f8d037fe640 1 --2- 172.21.15.148:0/1608029912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d040b3710 0x7f8d040aef30 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f8cf800e9b0 tx=0x7f8cf800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:50.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.129+0000 7f8d00ff9640 1 -- 172.21.15.148:0/1608029912 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8017070 con 0x7f8d040b3710 2024-03-29T04:12:50.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.129+0000 7f8d00ff9640 1 -- 172.21.15.148:0/1608029912 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cf800c9a0 con 0x7f8d040b3710 2024-03-29T04:12:50.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.129+0000 7f8d0a8c8640 1 -- 172.21.15.148:0/1608029912 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d04153680 con 0x7f8d040b3710 2024-03-29T04:12:50.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.129+0000 7f8d00ff9640 1 -- 172.21.15.148:0/1608029912 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf800cd90 con 0x7f8d040b3710 2024-03-29T04:12:50.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.129+0000 7f8d0a8c8640 1 -- 172.21.15.148:0/1608029912 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d040cd620 con 0x7f8d040b3710 2024-03-29T04:12:50.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.130+0000 7f8d0a8c8640 1 -- 172.21.15.148:0/1608029912 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d040744a0 con 0x7f8d040b3710 2024-03-29T04:12:50.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.131+0000 7f8d00ff9640 1 -- 172.21.15.148:0/1608029912 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8cf801a070 con 0x7f8d040b3710 2024-03-29T04:12:50.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.131+0000 7f8d00ff9640 1 --2- 172.21.15.148:0/1608029912 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f8cf006c530 0x7f8cf006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:50.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.131+0000 7f8d00ff9640 1 -- 172.21.15.148:0/1608029912 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f8cf808b8b0 con 0x7f8d040b3710 2024-03-29T04:12:50.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.131+0000 7f8d03fff640 1 --2- 172.21.15.148:0/1608029912 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f8cf006c530 0x7f8cf006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:50.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.132+0000 7f8d03fff640 1 --2- 172.21.15.148:0/1608029912 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f8cf006c530 0x7f8cf006e9f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4000f60 tx=0x7f8cf4007ca0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:50.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.137+0000 7f8d00ff9640 1 -- 172.21.15.148:0/1608029912 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8cf80599f0 con 0x7f8d040b3710 2024-03-29T04:12:50.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.309+0000 7f8d0a8c8640 1 -- 172.21.15.148:0/1608029912 --> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8d040687e0 con 0x7f8cf006c530 2024-03-29T04:12:50.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.310+0000 7f8d00ff9640 1 -- 172.21.15.148:0/1608029912 <== mgr.4113 v2:172.21.15.148:6832/3271445075 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13495 (secure 0 0 0) 0x7f8d040687e0 con 0x7f8cf006c530 2024-03-29T04:12:50.290 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:12:50.290 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-03-29T04:12:50.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.313+0000 7f8d0a8c8640 1 -- 172.21.15.148:0/1608029912 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f8cf006c530 msgr2=0x7f8cf006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:50.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.313+0000 7f8d0a8c8640 1 --2- 172.21.15.148:0/1608029912 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f8cf006c530 0x7f8cf006e9f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4000f60 tx=0x7f8cf4007ca0 comp rx=0 tx=0).stop 2024-03-29T04:12:50.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.313+0000 7f8d0a8c8640 1 -- 172.21.15.148:0/1608029912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d040b3710 msgr2=0x7f8d040aef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:50.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.313+0000 7f8d0a8c8640 1 --2- 172.21.15.148:0/1608029912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d040b3710 0x7f8d040aef30 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f8cf800e9b0 tx=0x7f8cf800ee70 comp rx=0 tx=0).stop 2024-03-29T04:12:50.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.313+0000 7f8d0a8c8640 1 -- 172.21.15.148:0/1608029912 shutdown_connections 2024-03-29T04:12:50.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.313+0000 7f8d0a8c8640 1 --2- 172.21.15.148:0/1608029912 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f8cf006c530 0x7f8cf006e9f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:50.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.313+0000 7f8d0a8c8640 1 --2- 172.21.15.148:0/1608029912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d04158c30 0x7f8d04160f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:50.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.313+0000 7f8d0a8c8640 1 --2- 172.21.15.148:0/1608029912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d040b3710 0x7f8d040aef30 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:50.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.313+0000 7f8d0a8c8640 1 --2- 172.21.15.148:0/1608029912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d0406b5b0 0x7f8d04152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:50.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.313+0000 7f8d0a8c8640 1 -- 172.21.15.148:0/1608029912 >> 172.21.15.148:0/1608029912 conn(0x7f8d040d0700 msgr2=0x7f8d04156cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:50.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.313+0000 7f8d0a8c8640 1 -- 172.21.15.148:0/1608029912 shutdown_connections 2024-03-29T04:12:50.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:12:50.313+0000 7f8d0a8c8640 1 -- 172.21.15.148:0/1608029912 wait complete. 2024-03-29T04:12:50.302 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-03-29T04:12:49.913941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":238032,"kb_used_data":5224,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736688,"statfs":{"total":773094113280,"available":772850368512,"internally_reserved":0,"allocated":5349376,"data_stored":2999328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-173890432,"num_objects":-116,"num_object_clones":0,"num_object_copies":-232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-116,"num_whiteouts":0,"num_read":-41,"num_read_kb":0,"num_write":-175,"num_write_kb":-169853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.680472"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":216,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-03-29T04:12:43.467149+0000","last_change":"2024-03-29T04:09:49.002570+0000","last_active":"2024-03-29T04:12:43.467149+0000","last_peered":"2024-03-29T04:12:43.467149+0000","last_clean":"2024-03-29T04:12:43.467149+0000","last_became_active":"2024-03-29T04:09:49.002235+0000","last_became_peered":"2024-03-29T04:09:49.002235+0000","last_unstale":"2024-03-29T04:12:43.467149+0000","last_undegraded":"2024-03-29T04:12:43.467149+0000","last_fullsized":"2024-03-29T04:12:43.467149+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:47.900733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:47.900733+0000","last_clean_scrub_stamp":"2024-03-29T04:09:47.900733+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:44:04.608265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1114112,"data_stored":817692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31688,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":520192,"data_stored":227324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31688,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":520192,"data_stored":227324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31692,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340148,"statfs":{"total":96636764160,"available":96604311552,"internally_reserved":0,"allocated":520192,"data_stored":227324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":227324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27532,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":520192,"data_stored":227324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":227324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1114112,"data_stored":817692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:12:50.305 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-29T04:09:47.900733+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=29, tm_hour=4, tm_min=9, tm_sec=47, tm_wday=4, tm_yday=89, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=29, tm_hour=4, tm_min=12, tm_sec=45, tm_wday=4, tm_yday=89, tm_isdst=0) 2024-03-29T04:12:50.305 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-29T04:13:02.056 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:13:02.059 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:13:10.309 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:13:10.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.443+0000 7fc47b700640 1 Processor -- start 2024-03-29T04:13:10.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.444+0000 7fc47b700640 1 -- start start 2024-03-29T04:13:10.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.444+0000 7fc47b700640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc47415a740 0x7fc47406c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:13:10.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.445+0000 7fc47b700640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc47415ef70 0x7fc4740b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:13:10.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.445+0000 7fc47b700640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4740cf790 con 0x7fc47415e3d0 2024-03-29T04:13:10.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.445+0000 7fc47b700640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 -- 0x7fc4740728d0 con 0x7fc47415ef70 2024-03-29T04:13:10.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.445+0000 7fc47b700640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc474072bb0 con 0x7fc47415a740 2024-03-29T04:13:10.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.445+0000 7fc479c9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc47415ef70 0x7fc4740b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:13:10.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.445+0000 7fc479c9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc47415ef70 0x7fc4740b3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39952/0 (socket says 172.21.15.148:39952) 2024-03-29T04:13:10.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.445+0000 7fc479c9d640 1 -- 172.21.15.148:0/1812379746 learned_addr learned my addr 172.21.15.148:0/1812379746 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:13:10.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.445+0000 7fc479c9d640 1 -- 172.21.15.148:0/1812379746 >> v2:172.21.15.157:3300/0 conn(0x7fc47415a740 msgr2=0x7fc47406c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:13:10.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.445+0000 7fc479c9d640 1 --2- 172.21.15.148:0/1812379746 >> v2:172.21.15.157:3300/0 conn(0x7fc47415a740 0x7fc47406c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:13:10.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.445+0000 7fc479c9d640 1 -- 172.21.15.148:0/1812379746 >> v1:172.21.15.157:6789/0 conn(0x7fc47415e3d0 legacy=0x7fc47415e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:13:10.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.445+0000 7fc479c9d640 1 -- 172.21.15.148:0/1812379746 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc474072e90 con 0x7fc47415ef70 2024-03-29T04:13:10.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.446+0000 7fc479c9d640 1 --2- 172.21.15.148:0/1812379746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc47415ef70 0x7fc4740b3ef0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fc4740714a0 tx=0x7fc46c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec74840bf9748bc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:13:10.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.446+0000 7fc46bfff640 1 -- 172.21.15.148:0/1812379746 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c02c9f0 con 0x7fc47415ef70 2024-03-29T04:13:10.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.446+0000 7fc46bfff640 1 -- 172.21.15.148:0/1812379746 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc46c02cb90 con 0x7fc47415ef70 2024-03-29T04:13:10.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.446+0000 7fc46bfff640 1 -- 172.21.15.148:0/1812379746 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c0343f0 con 0x7fc47415ef70 2024-03-29T04:13:10.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.446+0000 7fc47b700640 1 -- 172.21.15.148:0/1812379746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc47415ef70 msgr2=0x7fc4740b3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:13:10.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.446+0000 7fc47b700640 1 --2- 172.21.15.148:0/1812379746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc47415ef70 0x7fc4740b3ef0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fc4740714a0 tx=0x7fc46c02be50 comp rx=0 tx=0).stop 2024-03-29T04:13:10.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.447+0000 7fc47b700640 1 -- 172.21.15.148:0/1812379746 shutdown_connections 2024-03-29T04:13:10.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.447+0000 7fc47b700640 1 --2- 172.21.15.148:0/1812379746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc47415ef70 0x7fc4740b3ef0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:13:10.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.447+0000 7fc47b700640 1 --2- 172.21.15.148:0/1812379746 >> v2:172.21.15.157:3300/0 conn(0x7fc47415a740 0x7fc47406c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:13:10.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.447+0000 7fc47b700640 1 -- 172.21.15.148:0/1812379746 >> 172.21.15.148:0/1812379746 conn(0x7fc4740cfed0 msgr2=0x7fc4740d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:13:10.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.447+0000 7fc47b700640 1 -- 172.21.15.148:0/1812379746 shutdown_connections 2024-03-29T04:13:10.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.447+0000 7fc47b700640 1 -- 172.21.15.148:0/1812379746 wait complete. 2024-03-29T04:13:10.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.448+0000 7fc47b700640 1 Processor -- start 2024-03-29T04:13:10.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.448+0000 7fc47b700640 1 -- start start 2024-03-29T04:13:10.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.448+0000 7fc47b700640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc47415a740 0x7fc4741f2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:13:10.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.449+0000 7fc47b700640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc47415e3d0 0x7fc4741f2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:13:10.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.449+0000 7fc47b700640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc47415ef70 0x7fc4741567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:13:10.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.449+0000 7fc47b700640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc47415aca0 con 0x7fc47415ef70 2024-03-29T04:13:10.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.449+0000 7fc47b700640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc47415ab20 con 0x7fc47415e3d0 2024-03-29T04:13:10.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.449+0000 7fc47b700640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc47406cb00 con 0x7fc47415a740 2024-03-29T04:13:10.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.449+0000 7fc479c9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc47415ef70 0x7fc4741567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:13:10.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.449+0000 7fc479c9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc47415ef70 0x7fc4741567b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39964/0 (socket says 172.21.15.148:39964) 2024-03-29T04:13:10.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.449+0000 7fc479c9d640 1 -- 172.21.15.148:0/1394289229 learned_addr learned my addr 172.21.15.148:0/1394289229 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:13:10.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.449+0000 7fc478c9b640 1 --2- 172.21.15.148:0/1394289229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc47415e3d0 0x7fc4741f2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:13:10.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.449+0000 7fc479c9d640 1 -- 172.21.15.148:0/1394289229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc47415a740 msgr2=0x7fc4741f2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:13:10.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.449+0000 7fc47949c640 1 --2- 172.21.15.148:0/1394289229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc47415a740 0x7fc4741f2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:13:10.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.449+0000 7fc479c9d640 1 --2- 172.21.15.148:0/1394289229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc47415a740 0x7fc4741f2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:13:10.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.449+0000 7fc479c9d640 1 -- 172.21.15.148:0/1394289229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc47415e3d0 msgr2=0x7fc4741f2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:13:10.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.449+0000 7fc479c9d640 1 --2- 172.21.15.148:0/1394289229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc47415e3d0 0x7fc4741f2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:13:10.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.449+0000 7fc479c9d640 1 -- 172.21.15.148:0/1394289229 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47415dc30 con 0x7fc47415ef70 2024-03-29T04:13:10.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.449+0000 7fc478c9b640 1 --2- 172.21.15.148:0/1394289229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc47415e3d0 0x7fc4741f2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:13:10.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.449+0000 7fc47949c640 1 --2- 172.21.15.148:0/1394289229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc47415a740 0x7fc4741f2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:13:10.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.449+0000 7fc479c9d640 1 --2- 172.21.15.148:0/1394289229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc47415ef70 0x7fc4741567b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc46c036040 tx=0x7fc46c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:13:10.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.449+0000 7fc469ffb640 1 -- 172.21.15.148:0/1394289229 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c043020 con 0x7fc47415ef70 2024-03-29T04:13:10.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.449+0000 7fc469ffb640 1 -- 172.21.15.148:0/1394289229 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc46c03e450 con 0x7fc47415ef70 2024-03-29T04:13:10.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.449+0000 7fc47b700640 1 -- 172.21.15.148:0/1394289229 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4741559f0 con 0x7fc47415ef70 2024-03-29T04:13:10.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.449+0000 7fc469ffb640 1 -- 172.21.15.148:0/1394289229 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c03e820 con 0x7fc47415ef70 2024-03-29T04:13:10.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.450+0000 7fc47b700640 1 -- 172.21.15.148:0/1394289229 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc474155fb0 con 0x7fc47415ef70 2024-03-29T04:13:10.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.450+0000 7fc47b700640 1 -- 172.21.15.148:0/1394289229 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4740687e0 con 0x7fc47415ef70 2024-03-29T04:13:10.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.451+0000 7fc469ffb640 1 -- 172.21.15.148:0/1394289229 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc46c046020 con 0x7fc47415ef70 2024-03-29T04:13:10.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.451+0000 7fc469ffb640 1 --2- 172.21.15.148:0/1394289229 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc45806c530 0x7fc45806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:13:10.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.451+0000 7fc469ffb640 1 -- 172.21.15.148:0/1394289229 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7fc46c0adfc0 con 0x7fc47415ef70 2024-03-29T04:13:10.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.451+0000 7fc47949c640 1 --2- 172.21.15.148:0/1394289229 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc45806c530 0x7fc45806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:13:10.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.452+0000 7fc47949c640 1 --2- 172.21.15.148:0/1394289229 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc45806c530 0x7fc45806e9f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc470006fd0 tx=0x7fc470008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:13:10.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.456+0000 7fc469ffb640 1 -- 172.21.15.148:0/1394289229 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc46c04d030 con 0x7fc47415ef70 2024-03-29T04:13:10.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.628+0000 7fc47b700640 1 -- 172.21.15.148:0/1394289229 --> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc4740cfcb0 con 0x7fc45806c530 2024-03-29T04:13:10.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.629+0000 7fc469ffb640 1 -- 172.21.15.148:0/1394289229 <== mgr.4113 v2:172.21.15.148:6832/3271445075 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13359 (secure 0 0 0) 0x7fc4740cfcb0 con 0x7fc45806c530 2024-03-29T04:13:10.608 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:13:10.608 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-03-29T04:13:10.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.631+0000 7fc47b700640 1 -- 172.21.15.148:0/1394289229 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc45806c530 msgr2=0x7fc45806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:13:10.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.631+0000 7fc47b700640 1 --2- 172.21.15.148:0/1394289229 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc45806c530 0x7fc45806e9f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc470006fd0 tx=0x7fc470008040 comp rx=0 tx=0).stop 2024-03-29T04:13:10.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.631+0000 7fc47b700640 1 -- 172.21.15.148:0/1394289229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc47415ef70 msgr2=0x7fc4741567b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:13:10.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.631+0000 7fc47b700640 1 --2- 172.21.15.148:0/1394289229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc47415ef70 0x7fc4741567b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc46c036040 tx=0x7fc46c004a40 comp rx=0 tx=0).stop 2024-03-29T04:13:10.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.631+0000 7fc47b700640 1 -- 172.21.15.148:0/1394289229 shutdown_connections 2024-03-29T04:13:10.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.631+0000 7fc47b700640 1 --2- 172.21.15.148:0/1394289229 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7fc45806c530 0x7fc45806e9f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:13:10.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.631+0000 7fc47b700640 1 --2- 172.21.15.148:0/1394289229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc47415ef70 0x7fc4741567b0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:13:10.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.631+0000 7fc47b700640 1 --2- 172.21.15.148:0/1394289229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc47415e3d0 0x7fc4741f2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:13:10.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.631+0000 7fc47b700640 1 --2- 172.21.15.148:0/1394289229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc47415a740 0x7fc4741f2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:13:10.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.631+0000 7fc47b700640 1 -- 172.21.15.148:0/1394289229 >> 172.21.15.148:0/1394289229 conn(0x7fc4740cfed0 msgr2=0x7fc4740d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:13:10.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.631+0000 7fc47b700640 1 -- 172.21.15.148:0/1394289229 shutdown_connections 2024-03-29T04:13:10.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.631+0000 7fc47b700640 1 -- 172.21.15.148:0/1394289229 wait complete. 2024-03-29T04:13:10.620 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2024-03-29T04:13:09.917967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":238024,"kb_used_data":5224,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736696,"statfs":{"total":773094113280,"available":772850376704,"internally_reserved":0,"allocated":5349376,"data_stored":2999328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002571"},"pg_stats":[{"pgid":"1.0","version":"44'119","reported_seq":227,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-03-29T04:12:50.398145+0000","last_change":"2024-03-29T04:12:50.398183+0000","last_active":"2024-03-29T04:12:50.398145+0000","last_peered":"2024-03-29T04:12:50.398145+0000","last_clean":"2024-03-29T04:12:50.398145+0000","last_became_active":"2024-03-29T04:09:49.002235+0000","last_became_peered":"2024-03-29T04:09:49.002235+0000","last_unstale":"2024-03-29T04:12:50.398145+0000","last_undegraded":"2024-03-29T04:12:50.398145+0000","last_fullsized":"2024-03-29T04:12:50.398145+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'119","last_scrub_stamp":"2024-03-29T04:12:50.398046+0000","last_deep_scrub":"44'119","last_deep_scrub_stamp":"2024-03-29T04:12:50.398046+0000","last_clean_scrub_stamp":"2024-03-29T04:12:50.398046+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:35:34.040454+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1114112,"data_stored":817692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31688,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":520192,"data_stored":227324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31688,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":520192,"data_stored":227324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574891,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31692,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340148,"statfs":{"total":96636764160,"available":96604311552,"internally_reserved":0,"allocated":520192,"data_stored":227324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574891,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":227324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27532,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":520192,"data_stored":227324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574891,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":227324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574891,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1114112,"data_stored":817692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:13:10.622 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-29T04:13:10.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.744+0000 7f9a9904e640 1 Processor -- start 2024-03-29T04:13:10.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.744+0000 7f9a9904e640 1 -- start start 2024-03-29T04:13:10.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.744+0000 7f9a9904e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415e3c0 0x7f9a940c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:13:10.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.744+0000 7f9a9904e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9a94162860 0x7f9a940c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:13:10.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.744+0000 7f9a9904e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a94162ec0 con 0x7f9a940cf290 2024-03-29T04:13:10.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.744+0000 7f9a9904e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9a940cee00 con 0x7f9a94162860 2024-03-29T04:13:10.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.744+0000 7f9a9904e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9a940ce4d0 con 0x7f9a9415e3c0 2024-03-29T04:13:10.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.745+0000 7f9a92d9d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415e3c0 0x7f9a940c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:13:10.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.745+0000 7f9a9359e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9a94162860 0x7f9a940c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:13:10.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.745+0000 7f9a9259c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f9a940cf290 0x7f9a94162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.148:40236/0 (socket says 172.21.15.148:40236) 2024-03-29T04:13:10.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.745+0000 7f9a9259c640 1 -- 172.21.15.148:0/928125792 learned_addr learned my addr 172.21.15.148:0/928125792 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:13:10.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.745+0000 7f9a92d9d640 1 -- 172.21.15.148:0/928125792 >> v2:172.21.15.157:3300/0 conn(0x7f9a94162860 msgr2=0x7f9a940c9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:13:10.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.745+0000 7f9a92d9d640 1 --2- 172.21.15.148:0/928125792 >> v2:172.21.15.157:3300/0 conn(0x7f9a94162860 0x7f9a940c9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:13:10.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.745+0000 7f9a92d9d640 1 -- 172.21.15.148:0/928125792 >> v1:172.21.15.157:6789/0 conn(0x7f9a940cf290 legacy=0x7f9a94162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:13:10.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.745+0000 7f9a91d9b640 1 -- 172.21.15.148:0/928125792 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2690155218 0 0) 0x7f9a94162ec0 con 0x7f9a940cf290 2024-03-29T04:13:10.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.745+0000 7f9a92d9d640 1 -- 172.21.15.148:0/928125792 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a940ce7b0 con 0x7f9a9415e3c0 2024-03-29T04:13:10.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.746+0000 7f9a92d9d640 1 --2- 172.21.15.148:0/928125792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415e3c0 0x7f9a940c99f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9a880097a0 tx=0x7f9a8802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b71a758977e4f20 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:13:10.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.746+0000 7f9a91d9b640 1 -- 172.21.15.148:0/928125792 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a88035020 con 0x7f9a9415e3c0 2024-03-29T04:13:10.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.746+0000 7f9a91d9b640 1 -- 172.21.15.148:0/928125792 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a88031830 con 0x7f9a9415e3c0 2024-03-29T04:13:10.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.746+0000 7f9a91d9b640 1 -- 172.21.15.148:0/928125792 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a88031c00 con 0x7f9a9415e3c0 2024-03-29T04:13:10.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.747+0000 7f9a9904e640 1 -- 172.21.15.148:0/928125792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415e3c0 msgr2=0x7f9a940c99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:13:10.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.747+0000 7f9a9904e640 1 --2- 172.21.15.148:0/928125792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415e3c0 0x7f9a940c99f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9a880097a0 tx=0x7f9a8802b8d0 comp rx=0 tx=0).stop 2024-03-29T04:13:10.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.747+0000 7f9a9904e640 1 -- 172.21.15.148:0/928125792 shutdown_connections 2024-03-29T04:13:10.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.747+0000 7f9a9904e640 1 --2- 172.21.15.148:0/928125792 >> v2:172.21.15.157:3300/0 conn(0x7f9a94162860 0x7f9a940c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:13:10.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.747+0000 7f9a9904e640 1 --2- 172.21.15.148:0/928125792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a9415e3c0 0x7f9a940c99f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:13:10.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.747+0000 7f9a9904e640 1 -- 172.21.15.148:0/928125792 >> 172.21.15.148:0/928125792 conn(0x7f9a940cd160 msgr2=0x7f9a940cd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:13:10.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.747+0000 7f9a9904e640 1 -- 172.21.15.148:0/928125792 shutdown_connections 2024-03-29T04:13:10.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.747+0000 7f9a9904e640 1 -- 172.21.15.148:0/928125792 wait complete. 2024-03-29T04:13:10.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.748+0000 7f9a9904e640 1 Processor -- start 2024-03-29T04:13:10.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.748+0000 7f9a9904e640 1 -- start start 2024-03-29T04:13:10.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.749+0000 7f9a9904e640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a940cf290 0x7f9a94153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:13:10.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.749+0000 7f9a9904e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a9415e3c0 0x7f9a940b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:13:10.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.749+0000 7f9a9904e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a94162860 0x7f9a941607f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:13:10.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.749+0000 7f9a9904e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9415e920 con 0x7f9a9415e3c0 2024-03-29T04:13:10.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.749+0000 7f9a9904e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9415e7a0 con 0x7f9a94162860 2024-03-29T04:13:10.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.749+0000 7f9a9904e640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9a940cc050 con 0x7f9a940cf290 2024-03-29T04:13:10.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.749+0000 7f9a9259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a9415e3c0 0x7f9a940b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:13:10.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.749+0000 7f9a9259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a9415e3c0 0x7f9a940b5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39978/0 (socket says 172.21.15.148:39978) 2024-03-29T04:13:10.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.749+0000 7f9a9259c640 1 -- 172.21.15.148:0/1792857717 learned_addr learned my addr 172.21.15.148:0/1792857717 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:13:10.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.749+0000 7f9a9359e640 1 --2- 172.21.15.148:0/1792857717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a94162860 0x7f9a941607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:13:10.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.749+0000 7f9a92d9d640 1 --2- 172.21.15.148:0/1792857717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a940cf290 0x7f9a94153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:13:10.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.750+0000 7f9a9259c640 1 -- 172.21.15.148:0/1792857717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a940cf290 msgr2=0x7f9a94153230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:13:10.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.750+0000 7f9a9259c640 1 --2- 172.21.15.148:0/1792857717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a940cf290 0x7f9a94153230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:13:10.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.750+0000 7f9a9259c640 1 -- 172.21.15.148:0/1792857717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a94162860 msgr2=0x7f9a941607f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:13:10.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.750+0000 7f9a9259c640 1 --2- 172.21.15.148:0/1792857717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a94162860 0x7f9a941607f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:13:10.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.750+0000 7f9a9259c640 1 -- 172.21.15.148:0/1792857717 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a941612b0 con 0x7f9a9415e3c0 2024-03-29T04:13:10.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.750+0000 7f9a9259c640 1 --2- 172.21.15.148:0/1792857717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a9415e3c0 0x7f9a940b5110 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9a8400c4b0 tx=0x7f9a8400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:13:10.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.750+0000 7f9a7b7fe640 1 -- 172.21.15.148:0/1792857717 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a84018070 con 0x7f9a9415e3c0 2024-03-29T04:13:10.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.750+0000 7f9a7b7fe640 1 -- 172.21.15.148:0/1792857717 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a84007500 con 0x7f9a9415e3c0 2024-03-29T04:13:10.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.750+0000 7f9a7b7fe640 1 -- 172.21.15.148:0/1792857717 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a84004a20 con 0x7f9a9415e3c0 2024-03-29T04:13:10.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.750+0000 7f9a9904e640 1 -- 172.21.15.148:0/1792857717 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a940b4690 con 0x7f9a9415e3c0 2024-03-29T04:13:10.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.750+0000 7f9a9904e640 1 -- 172.21.15.148:0/1792857717 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a9406f9e0 con 0x7f9a9415e3c0 2024-03-29T04:13:10.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.751+0000 7f9a9904e640 1 -- 172.21.15.148:0/1792857717 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a94068850 con 0x7f9a9415e3c0 2024-03-29T04:13:10.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.751+0000 7f9a7b7fe640 1 -- 172.21.15.148:0/1792857717 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9a8401b070 con 0x7f9a9415e3c0 2024-03-29T04:13:10.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.752+0000 7f9a7b7fe640 1 --2- 172.21.15.148:0/1792857717 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9a8006c530 0x7f9a8006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:13:10.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.752+0000 7f9a7b7fe640 1 -- 172.21.15.148:0/1792857717 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f9a8408a820 con 0x7f9a9415e3c0 2024-03-29T04:13:10.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.752+0000 7f9a92d9d640 1 --2- 172.21.15.148:0/1792857717 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9a8006c530 0x7f9a8006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:13:10.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.752+0000 7f9a92d9d640 1 --2- 172.21.15.148:0/1792857717 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9a8006c530 0x7f9a8006e9f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9a88036000 tx=0x7f9a8803c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:13:10.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.756+0000 7f9a7b7fe640 1 -- 172.21.15.148:0/1792857717 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9a940b4690 con 0x7f9a9415e3c0 2024-03-29T04:13:10.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.935+0000 7f9a9904e640 1 -- 172.21.15.148:0/1792857717 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f9a940704c0 con 0x7f9a9415e3c0 2024-03-29T04:13:10.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.938+0000 7f9a7b7fe640 1 -- 172.21.15.148:0/1792857717 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7f9a84058a10 con 0x7f9a9415e3c0 2024-03-29T04:13:10.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.939+0000 7f9a7b7fe640 1 -- 172.21.15.148:0/1792857717 <== mon.0 v2:172.21.15.148:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f9a8405cd10 con 0x7f9a9415e3c0 2024-03-29T04:13:10.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.941+0000 7f9a9904e640 1 -- 172.21.15.148:0/1792857717 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9a8006c530 msgr2=0x7f9a8006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:13:10.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.941+0000 7f9a9904e640 1 --2- 172.21.15.148:0/1792857717 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9a8006c530 0x7f9a8006e9f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9a88036000 tx=0x7f9a8803c040 comp rx=0 tx=0).stop 2024-03-29T04:13:10.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.941+0000 7f9a9904e640 1 -- 172.21.15.148:0/1792857717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a9415e3c0 msgr2=0x7f9a940b5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:13:10.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.941+0000 7f9a9904e640 1 --2- 172.21.15.148:0/1792857717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a9415e3c0 0x7f9a940b5110 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9a8400c4b0 tx=0x7f9a8400c970 comp rx=0 tx=0).stop 2024-03-29T04:13:10.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.941+0000 7f9a9904e640 1 -- 172.21.15.148:0/1792857717 shutdown_connections 2024-03-29T04:13:10.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.941+0000 7f9a9904e640 1 --2- 172.21.15.148:0/1792857717 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f9a8006c530 0x7f9a8006e9f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:13:10.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.941+0000 7f9a9904e640 1 --2- 172.21.15.148:0/1792857717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a94162860 0x7f9a941607f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:13:10.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.941+0000 7f9a9904e640 1 --2- 172.21.15.148:0/1792857717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a9415e3c0 0x7f9a940b5110 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:13:10.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.941+0000 7f9a9904e640 1 --2- 172.21.15.148:0/1792857717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a940cf290 0x7f9a94153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:13:10.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.941+0000 7f9a9904e640 1 -- 172.21.15.148:0/1792857717 >> 172.21.15.148:0/1792857717 conn(0x7f9a940cd160 msgr2=0x7f9a940cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:13:10.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.941+0000 7f9a9904e640 1 -- 172.21.15.148:0/1792857717 shutdown_connections 2024-03-29T04:13:10.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:10.941+0000 7f9a9904e640 1 -- 172.21.15.148:0/1792857717 wait complete. 2024-03-29T04:13:10.931 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-29T04:13:10.931 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:13:11.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.053+0000 7f460b00d640 1 Processor -- start 2024-03-29T04:13:11.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.054+0000 7f460b00d640 1 -- start start 2024-03-29T04:13:11.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.054+0000 7f460b00d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f460406ccd0 0x7f46040ce610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:13:11.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.054+0000 7f460b00d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f460415d210 0x7f46040b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:13:11.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.054+0000 7f460b00d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f460415d6a0 con 0x7f46040cf780 2024-03-29T04:13:11.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.054+0000 7f460b00d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f460406da30 con 0x7f460415d210 2024-03-29T04:13:11.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.054+0000 7f460b00d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f460406dd10 con 0x7f460406ccd0 2024-03-29T04:13:11.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.055+0000 7f46095aa640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f460415d210 0x7f46040b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:13:11.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.055+0000 7f45fbfff640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f460406ccd0 0x7f46040ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:13:11.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.055+0000 7f4608da9640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f46040cf780 0x7f460406c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.148:40252/0 (socket says 172.21.15.148:40252) 2024-03-29T04:13:11.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.055+0000 7f4608da9640 1 -- 172.21.15.148:0/2077586420 learned_addr learned my addr 172.21.15.148:0/2077586420 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-03-29T04:13:11.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.055+0000 7f45fbfff640 1 -- 172.21.15.148:0/2077586420 >> v2:172.21.15.157:3300/0 conn(0x7f460415d210 msgr2=0x7f46040b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:13:11.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.055+0000 7f45fbfff640 1 --2- 172.21.15.148:0/2077586420 >> v2:172.21.15.157:3300/0 conn(0x7f460415d210 0x7f46040b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:13:11.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.055+0000 7f45fbfff640 1 -- 172.21.15.148:0/2077586420 >> v1:172.21.15.157:6789/0 conn(0x7f46040cf780 legacy=0x7f460406c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:13:11.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.055+0000 7f45fbfff640 1 -- 172.21.15.148:0/2077586420 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f460406dff0 con 0x7f460406ccd0 2024-03-29T04:13:11.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.055+0000 7f46095aa640 1 --2- 172.21.15.148:0/2077586420 >> v2:172.21.15.157:3300/0 conn(0x7f460415d210 0x7f46040b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:13:11.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.056+0000 7f45fbfff640 1 --2- 172.21.15.148:0/2077586420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f460406ccd0 0x7f46040ce610 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f45f40097a0 tx=0x7f45f402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31f9adfe3635fbfb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:13:11.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.056+0000 7f45fb7fe640 1 -- 172.21.15.148:0/2077586420 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f40315a0 con 0x7f460406ccd0 2024-03-29T04:13:11.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.056+0000 7f45fb7fe640 1 -- 172.21.15.148:0/2077586420 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f45f4037070 con 0x7f460406ccd0 2024-03-29T04:13:11.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.056+0000 7f45fb7fe640 1 -- 172.21.15.148:0/2077586420 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4031980 con 0x7f460406ccd0 2024-03-29T04:13:11.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.056+0000 7f460b00d640 1 -- 172.21.15.148:0/2077586420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f460406ccd0 msgr2=0x7f46040ce610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:13:11.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.056+0000 7f460b00d640 1 --2- 172.21.15.148:0/2077586420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f460406ccd0 0x7f46040ce610 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f45f40097a0 tx=0x7f45f402b840 comp rx=0 tx=0).stop 2024-03-29T04:13:11.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.057+0000 7f460b00d640 1 -- 172.21.15.148:0/2077586420 shutdown_connections 2024-03-29T04:13:11.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.057+0000 7f460b00d640 1 --2- 172.21.15.148:0/2077586420 >> v2:172.21.15.157:3300/0 conn(0x7f460415d210 0x7f46040b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:13:11.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.057+0000 7f460b00d640 1 --2- 172.21.15.148:0/2077586420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f460406ccd0 0x7f46040ce610 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:13:11.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.057+0000 7f460b00d640 1 -- 172.21.15.148:0/2077586420 >> 172.21.15.148:0/2077586420 conn(0x7f46040cfed0 msgr2=0x7f46040d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:13:11.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.057+0000 7f460b00d640 1 -- 172.21.15.148:0/2077586420 shutdown_connections 2024-03-29T04:13:11.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.057+0000 7f460b00d640 1 -- 172.21.15.148:0/2077586420 wait complete. 2024-03-29T04:13:11.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.058+0000 7f460b00d640 1 Processor -- start 2024-03-29T04:13:11.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.058+0000 7f460b00d640 1 -- start start 2024-03-29T04:13:11.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.059+0000 7f460b00d640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f460406ccd0 0x7f4604156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:13:11.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.059+0000 7f460b00d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46040cf780 0x7f46041585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:13:11.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.059+0000 7f460b00d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460415d210 0x7f4604158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:13:11.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.059+0000 7f460b00d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f460406d5c0 con 0x7f46040cf780 2024-03-29T04:13:11.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.059+0000 7f460b00d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f460406f340 con 0x7f460415d210 2024-03-29T04:13:11.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.059+0000 7f460b00d640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f460406d740 con 0x7f460406ccd0 2024-03-29T04:13:11.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.059+0000 7f45fbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46040cf780 0x7f46041585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:13:11.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.059+0000 7f45fbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46040cf780 0x7f46041585d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39980/0 (socket says 172.21.15.148:39980) 2024-03-29T04:13:11.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.059+0000 7f45fbfff640 1 -- 172.21.15.148:0/1489604438 learned_addr learned my addr 172.21.15.148:0/1489604438 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T04:13:11.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.059+0000 7f46095aa640 1 --2- 172.21.15.148:0/1489604438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460415d210 0x7f4604158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:13:11.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.059+0000 7f4608da9640 1 --2- 172.21.15.148:0/1489604438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f460406ccd0 0x7f4604156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:13:11.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.059+0000 7f45fbfff640 1 -- 172.21.15.148:0/1489604438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f460406ccd0 msgr2=0x7f4604156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:13:11.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.059+0000 7f45fbfff640 1 --2- 172.21.15.148:0/1489604438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f460406ccd0 0x7f4604156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:13:11.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.059+0000 7f45fbfff640 1 -- 172.21.15.148:0/1489604438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460415d210 msgr2=0x7f4604158b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:13:11.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.059+0000 7f45fbfff640 1 --2- 172.21.15.148:0/1489604438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460415d210 0x7f4604158b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:13:11.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.059+0000 7f45fbfff640 1 -- 172.21.15.148:0/1489604438 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4604157530 con 0x7f46040cf780 2024-03-29T04:13:11.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.059+0000 7f45fbfff640 1 --2- 172.21.15.148:0/1489604438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46040cf780 0x7f46041585d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f45f4000c00 tx=0x7f45f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:13:11.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.059+0000 7f45f97fa640 1 -- 172.21.15.148:0/1489604438 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4032840 con 0x7f46040cf780 2024-03-29T04:13:11.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.059+0000 7f45f97fa640 1 -- 172.21.15.148:0/1489604438 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f45f4032a00 con 0x7f46040cf780 2024-03-29T04:13:11.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.059+0000 7f460b00d640 1 -- 172.21.15.148:0/1489604438 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4604157810 con 0x7f46040cf780 2024-03-29T04:13:11.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.059+0000 7f45f97fa640 1 -- 172.21.15.148:0/1489604438 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4032e10 con 0x7f46040cf780 2024-03-29T04:13:11.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.059+0000 7f460b00d640 1 -- 172.21.15.148:0/1489604438 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4604157dd0 con 0x7f46040cf780 2024-03-29T04:13:11.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.060+0000 7f460b00d640 1 -- 172.21.15.148:0/1489604438 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46040687e0 con 0x7f46040cf780 2024-03-29T04:13:11.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.061+0000 7f45f97fa640 1 -- 172.21.15.148:0/1489604438 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f45f404e430 con 0x7f46040cf780 2024-03-29T04:13:11.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.061+0000 7f45f97fa640 1 --2- 172.21.15.148:0/1489604438 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f45f006c560 0x7f45f006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:13:11.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.061+0000 7f4608da9640 1 --2- 172.21.15.148:0/1489604438 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f45f006c560 0x7f45f006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:13:11.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.061+0000 7f45f97fa640 1 -- 172.21.15.148:0/1489604438 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f45f402f030 con 0x7f46040cf780 2024-03-29T04:13:11.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.062+0000 7f4608da9640 1 --2- 172.21.15.148:0/1489604438 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f45f006c560 0x7f45f006ea20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4600000f60 tx=0x7f4600009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:13:11.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.065+0000 7f45f97fa640 1 -- 172.21.15.148:0/1489604438 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f45f4040070 con 0x7f46040cf780 2024-03-29T04:13:11.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.276+0000 7f460b00d640 1 -- 172.21.15.148:0/1489604438 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f460406c4f0 con 0x7f46040cf780 2024-03-29T04:13:11.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.276+0000 7f45f97fa640 1 -- 172.21.15.148:0/1489604438 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+3881 (secure 0 0 0) 0x7f45f4031e90 con 0x7f46040cf780 2024-03-29T04:13:11.256 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T04:13:11.256 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":21,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/2618617956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2618617956},{"type":"v1","addr":"172.21.15.157:6834","nonce":2618617956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5260,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1177855619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1177855619},{"type":"v1","addr":"172.21.15.157:6835","nonce":1177855619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5562,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3610463670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3610463670},{"type":"v1","addr":"172.21.15.148:6835","nonce":3610463670}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":6033,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1764480077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1764480077},{"type":"v1","addr":"172.21.15.148:6837","nonce":1764480077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[]} 2024-03-29T04:13:11.256 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 21 2024-03-29T04:13:11.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.278+0000 7f460b00d640 1 -- 172.21.15.148:0/1489604438 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f45f006c560 msgr2=0x7f45f006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:13:11.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.278+0000 7f460b00d640 1 --2- 172.21.15.148:0/1489604438 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f45f006c560 0x7f45f006ea20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4600000f60 tx=0x7f4600009040 comp rx=0 tx=0).stop 2024-03-29T04:13:11.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.278+0000 7f460b00d640 1 -- 172.21.15.148:0/1489604438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46040cf780 msgr2=0x7f46041585d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:13:11.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.278+0000 7f460b00d640 1 --2- 172.21.15.148:0/1489604438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46040cf780 0x7f46041585d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f45f4000c00 tx=0x7f45f40046c0 comp rx=0 tx=0).stop 2024-03-29T04:13:11.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.278+0000 7f460b00d640 1 -- 172.21.15.148:0/1489604438 shutdown_connections 2024-03-29T04:13:11.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.278+0000 7f460b00d640 1 --2- 172.21.15.148:0/1489604438 >> [v2:172.21.15.148:6832/3271445075,v1:172.21.15.148:6833/3271445075] conn(0x7f45f006c560 0x7f45f006ea20 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:13:11.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.278+0000 7f460b00d640 1 --2- 172.21.15.148:0/1489604438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460415d210 0x7f4604158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:13:11.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.278+0000 7f460b00d640 1 --2- 172.21.15.148:0/1489604438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46040cf780 0x7f46041585d0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:13:11.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.278+0000 7f460b00d640 1 --2- 172.21.15.148:0/1489604438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f460406ccd0 0x7f4604156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:13:11.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.278+0000 7f460b00d640 1 -- 172.21.15.148:0/1489604438 >> 172.21.15.148:0/1489604438 conn(0x7f46040cfed0 msgr2=0x7f460415dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:13:11.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.278+0000 7f460b00d640 1 -- 172.21.15.148:0/1489604438 shutdown_connections 2024-03-29T04:13:11.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T04:13:11.279+0000 7f460b00d640 1 -- 172.21.15.148:0/1489604438 wait complete. 2024-03-29T04:13:11.268 DEBUG:tasks.cephfs.filesystem:already dead... 2024-03-29T04:13:11.269 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-29T04:13:11.269 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-29T04:13:11.269 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:13:11.278 INFO:tasks.ceph.mds.a:Stopped 2024-03-29T04:13:11.278 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-29T04:13:11.278 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:13:11.328 INFO:tasks.ceph.mds.c:Stopped 2024-03-29T04:13:11.328 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-29T04:13:11.329 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:13:11.340 INFO:tasks.ceph.mds.b:Stopped 2024-03-29T04:13:11.340 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-29T04:13:11.340 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:13:11.392 INFO:tasks.ceph.mds.d:Stopped 2024-03-29T04:13:11.392 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-29T04:13:11.392 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-29T04:13:11.392 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:13:11.419 INFO:tasks.ceph.osd.0:Stopped 2024-03-29T04:13:11.419 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-29T04:13:11.420 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:13:11.485 INFO:tasks.ceph.osd.1:Stopped 2024-03-29T04:13:11.485 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-29T04:13:11.485 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:13:11.550 INFO:tasks.ceph.osd.2:Stopped 2024-03-29T04:13:11.551 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-29T04:13:11.551 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:13:11.615 INFO:tasks.ceph.osd.3:Stopped 2024-03-29T04:13:11.615 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-29T04:13:11.615 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:13:11.647 INFO:tasks.ceph.osd.4:Stopped 2024-03-29T04:13:11.647 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-29T04:13:11.648 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:13:11.728 INFO:tasks.ceph.osd.5:Stopped 2024-03-29T04:13:11.728 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-29T04:13:11.728 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:13:11.802 INFO:tasks.ceph.osd.6:Stopped 2024-03-29T04:13:11.802 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-29T04:13:11.802 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:13:11.876 INFO:tasks.ceph.osd.7:Stopped 2024-03-29T04:13:11.876 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-29T04:13:11.876 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-29T04:13:11.876 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:13:11.903 INFO:tasks.ceph.mgr.y:Stopped 2024-03-29T04:13:11.903 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-29T04:13:11.903 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:13:11.945 INFO:tasks.ceph.mgr.x:Stopped 2024-03-29T04:13:11.945 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-29T04:13:11.945 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-29T04:13:11.945 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:13:11.957 INFO:tasks.ceph.mon.a:Stopped 2024-03-29T04:13:11.957 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-29T04:13:11.957 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:13:11.998 INFO:tasks.ceph.mon.b:Stopped 2024-03-29T04:13:11.998 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-29T04:13:11.998 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:13:12.051 INFO:tasks.ceph.mon.c:Stopped 2024-03-29T04:13:12.051 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-29T04:13:12.051 DEBUG:teuthology.orchestra.run.smithi148:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-03-29T04:13:12.087 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi148.front.sepia.ceph.com 2024-03-29T04:13:12.088 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-29T04:13:12.276 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi148.front.sepia.ceph.com 2024-03-29T04:13:12.276 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-29T04:13:12.364 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi148.front.sepia.ceph.com 2024-03-29T04:13:12.364 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-29T04:13:12.421 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi148.front.sepia.ceph.com 2024-03-29T04:13:12.421 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-29T04:13:12.490 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi157.front.sepia.ceph.com 2024-03-29T04:13:12.490 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-29T04:13:12.614 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi157.front.sepia.ceph.com 2024-03-29T04:13:12.614 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-29T04:13:12.685 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi157.front.sepia.ceph.com 2024-03-29T04:13:12.685 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-29T04:13:12.743 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi157.front.sepia.ceph.com 2024-03-29T04:13:12.743 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-29T04:13:12.801 INFO:tasks.ceph:Archiving mon data... 2024-03-29T04:13:12.802 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629083/data/mon.a.tgz 2024-03-29T04:13:12.802 DEBUG:teuthology.orchestra.run.smithi148:> mktemp 2024-03-29T04:13:12.817 INFO:teuthology.orchestra.run.smithi148.stdout:/tmp/tmp.hQKGkjspRp 2024-03-29T04:13:12.818 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.hQKGkjspRp 2024-03-29T04:13:12.941 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0666 /tmp/tmp.hQKGkjspRp 2024-03-29T04:13:13.025 DEBUG:teuthology.orchestra.remote:smithi148:/tmp/tmp.hQKGkjspRp is 320KB 2024-03-29T04:13:13.122 DEBUG:teuthology.orchestra.run.smithi148:> rm -fr /tmp/tmp.hQKGkjspRp 2024-03-29T04:13:13.136 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629083/data/mon.b.tgz 2024-03-29T04:13:13.137 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2024-03-29T04:13:13.153 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.XGNVozDb8t 2024-03-29T04:13:13.153 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.XGNVozDb8t 2024-03-29T04:13:13.309 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.XGNVozDb8t 2024-03-29T04:13:13.394 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.XGNVozDb8t is 426KB 2024-03-29T04:13:13.491 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.XGNVozDb8t 2024-03-29T04:13:13.506 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629083/data/mon.c.tgz 2024-03-29T04:13:13.506 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2024-03-29T04:13:13.559 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.xcY3MWv7Zg 2024-03-29T04:13:13.560 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.xcY3MWv7Zg 2024-03-29T04:13:13.703 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.xcY3MWv7Zg 2024-03-29T04:13:13.789 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.xcY3MWv7Zg is 426KB 2024-03-29T04:13:13.885 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.xcY3MWv7Zg 2024-03-29T04:13:13.899 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-29T04:13:13.900 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-29T04:13:13.902 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-29T04:13:13.984 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-29T04:13:14.021 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-29T04:13:14.021 INFO:tasks.ceph:Archiving crash dumps... 2024-03-29T04:13:14.022 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629083/remote/smithi148/crash 2024-03-29T04:13:14.023 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-29T04:13:14.052 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629083/remote/smithi157/crash 2024-03-29T04:13:14.053 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-29T04:13:14.087 INFO:tasks.ceph:Shutting down logrotate 2024-03-29T04:13:14.087 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:13:14.097 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:13:14.179 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-29T04:13:14.206 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-29T04:13:14.233 INFO:tasks.ceph:Compressing logs... 2024-03-29T04:13:14.233 DEBUG:teuthology.orchestra.run.smithi148:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-29T04:13:14.249 DEBUG:teuthology.orchestra.run.smithi157:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-29T04:13:14.332 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-03-29T04:13:14.333 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44482.log 2024-03-29T04:13:14.333 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.43759.log 2024-03-29T04:13:14.333 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.1.44482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37878.log 2024-03-29T04:13:14.333 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.3.43759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37838.log 2024-03-29T04:13:14.333 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-29T04:13:14.333 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.37878.log: /var/log/ceph/ceph-client.admin.37838.log: 91.3% -- replaced with /var/log/ceph/ceph-client.3.43759.log.gz 2024-03-29T04:13:14.333 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-03-29T04:13:14.334 INFO:teuthology.orchestra.run.smithi157.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37878.log.gz 2024-03-29T04:13:14.334 INFO:teuthology.orchestra.run.smithi157.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37838.log.gz 2024-03-29T04:13:14.334 INFO:teuthology.orchestra.run.smithi157.stderr:gzip/var/log/ceph/ceph.audit.log: -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-03-29T04:13:14.334 INFO:teuthology.orchestra.run.smithi157.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.1.44482.log.gz 2024-03-29T04:13:14.334 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43302.log 2024-03-29T04:13:14.334 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-03-29T04:13:14.334 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.1.43302.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-03-29T04:13:14.335 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37598.log 2024-03-29T04:13:14.335 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-osd.5.log: 91.5% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-03-29T04:13:14.335 INFO:teuthology.orchestra.run.smithi157.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.1.43302.log.gz 2024-03-29T04:13:14.335 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37758.log 2024-03-29T04:13:14.335 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.37598.log: /var/log/ceph/ceph-mgr.x.log 2024-03-29T04:13:14.335 INFO:teuthology.orchestra.run.smithi157.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-29T04:13:14.335 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.37758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37718.log 2024-03-29T04:13:14.335 INFO:teuthology.orchestra.run.smithi157.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37598.log.gz 2024-03-29T04:13:14.335 INFO:teuthology.orchestra.run.smithi157.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37758.log.gz 2024-03-29T04:13:14.336 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37798.log 2024-03-29T04:13:14.336 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.37718.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-29T04:13:14.336 INFO:teuthology.orchestra.run.smithi157.stderr: 87.5%/var/log/ceph/ceph-client.admin.37798.log: -- replaced with /var/log/ceph/ceph-client.admin.37718.log.gz 2024-03-29T04:13:14.336 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37678.log 2024-03-29T04:13:14.336 INFO:teuthology.orchestra.run.smithi157.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37798.log.gz 2024-03-29T04:13:14.336 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37638.log 2024-03-29T04:13:14.336 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-03-29T04:13:14.336 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.37638.log: /var/log/ceph/ceph-client.admin.37678.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37638.log.gz 2024-03-29T04:13:14.337 INFO:teuthology.orchestra.run.smithi157.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37678.log.gz 2024-03-29T04:13:14.337 INFO:teuthology.orchestra.run.smithi157.stderr: 91.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-29T04:13:14.337 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-03-29T04:13:14.337 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-03-29T04:13:14.337 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.c.log: 91.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-03-29T04:13:14.337 INFO:teuthology.orchestra.run.smithi157.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-29T04:13:14.390 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50002.log 2024-03-29T04:13:14.390 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49708.log 2024-03-29T04:13:14.390 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.50002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45741.log 2024-03-29T04:13:14.391 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57128.log 2024-03-29T04:13:14.391 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50002.log.gz 2024-03-29T04:13:14.391 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.45741.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49708.log.gz 2024-03-29T04:13:14.391 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37401.log 2024-03-29T04:13:14.391 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57128.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45741.log.gz 2024-03-29T04:13:14.391 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-29T04:13:14.391 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.37401.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57128.log.gz 2024-03-29T04:13:14.391 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48800.log 2024-03-29T04:13:14.391 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-mon.a.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37401.log.gz 2024-03-29T04:13:14.392 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43810.log 2024-03-29T04:13:14.392 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.48800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45867.log 2024-03-29T04:13:14.392 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50044.log 2024-03-29T04:13:14.392 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48800.log.gz 2024-03-29T04:13:14.392 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.45867.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43810.log.gz 2024-03-29T04:13:14.392 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54259.log 2024-03-29T04:13:14.392 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.50044.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.45867.log.gz 2024-03-29T04:13:14.392 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43874.log 2024-03-29T04:13:14.392 INFO:teuthology.orchestra.run.smithi148.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50044.log.gz 2024-03-29T04:13:14.393 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.54259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51459.log 2024-03-29T04:13:14.393 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43874.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54259.log.gz 2024-03-29T04:13:14.393 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52279.log 2024-03-29T04:13:14.393 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43874.log.gz 2024-03-29T04:13:14.393 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.51459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45201.log 2024-03-29T04:13:14.393 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51459.log.gz 2024-03-29T04:13:14.393 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.52279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57942.log 2024-03-29T04:13:14.393 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.45201.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52279.log.gz 2024-03-29T04:13:14.394 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45573.log 2024-03-29T04:13:14.394 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57942.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45201.log.gz 2024-03-29T04:13:14.394 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53596.log 2024-03-29T04:13:14.394 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57942.log.gz 2024-03-29T04:13:14.394 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.45573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52027.log 2024-03-29T04:13:14.394 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.53596.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45573.log.gz 2024-03-29T04:13:14.394 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55029.log 2024-03-29T04:13:14.394 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.52027.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53596.log.gz 2024-03-29T04:13:14.394 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49834.log 2024-03-29T04:13:14.395 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52027.log.gz 2024-03-29T04:13:14.395 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.55029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50181.log 2024-03-29T04:13:14.395 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52195.log 2024-03-29T04:13:14.395 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49834.log.gz 2024-03-29T04:13:14.395 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.50181.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.55029.log.gz 2024-03-29T04:13:14.395 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55752.log 2024-03-29T04:13:14.395 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50181.log.gz 2024-03-29T04:13:14.395 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.52195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54627.log 2024-03-29T04:13:14.396 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.55752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57716.log 2024-03-29T04:13:14.396 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52195.log.gz 2024-03-29T04:13:14.396 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55752.log.gz 2024-03-29T04:13:14.396 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.54627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50989.log 2024-03-29T04:13:14.396 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51901.log 2024-03-29T04:13:14.396 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.50989.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57716.log.gz 2024-03-29T04:13:14.396 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46702.log 2024-03-29T04:13:14.396 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50989.log.gz 2024-03-29T04:13:14.396 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51333.log 2024-03-29T04:13:14.397 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.51901.log: /var/log/ceph/ceph-client.admin.46702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45159.log 2024-03-29T04:13:14.397 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51901.log.gz 2024-03-29T04:13:14.397 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.51333.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46702.log.gz 2024-03-29T04:13:14.397 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45447.log 2024-03-29T04:13:14.397 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51333.log.gz 2024-03-29T04:13:14.397 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54217.log 2024-03-29T04:13:14.397 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.45159.log: /var/log/ceph/ceph-client.admin.45447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43714.log 2024-03-29T04:13:14.397 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45159.log.gz 2024-03-29T04:13:14.397 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45447.log.gz 2024-03-29T04:13:14.398 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.54217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55836.log 2024-03-29T04:13:14.398 INFO:teuthology.orchestra.run.smithi148.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54217.log.gz 2024-03-29T04:13:14.398 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49876.log 2024-03-29T04:13:14.398 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43714.log: /var/log/ceph/ceph-client.admin.55836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47043.log 2024-03-29T04:13:14.398 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43714.log.gz 2024-03-29T04:13:14.398 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49876.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55836.log.gz 2024-03-29T04:13:14.398 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53793.log 2024-03-29T04:13:14.398 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49876.log.gz 2024-03-29T04:13:14.399 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52531.log 2024-03-29T04:13:14.399 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47043.log.gz 2024-03-29T04:13:14.399 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46380.log 2024-03-29T04:13:14.399 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.53793.log: /var/log/ceph/ceph-client.admin.52531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53070.log 2024-03-29T04:13:14.399 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53793.log.gz 2024-03-29T04:13:14.399 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52531.log.gz 2024-03-29T04:13:14.399 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46659.log 2024-03-29T04:13:14.399 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46703.log 2024-03-29T04:13:14.399 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.53070.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46380.log.gz 2024-03-29T04:13:14.400 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46659.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53070.log.gz 2024-03-29T04:13:14.400 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56157.log 2024-03-29T04:13:14.400 INFO:teuthology.orchestra.run.smithi148.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46659.log.gz 2024-03-29T04:13:14.400 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48994.log 2024-03-29T04:13:14.400 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46703.log: /var/log/ceph/ceph-client.admin.56157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51249.log 2024-03-29T04:13:14.400 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46703.log.gz 2024-03-29T04:13:14.400 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.48994.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56157.log.gz 2024-03-29T04:13:14.400 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52405.log 2024-03-29T04:13:14.400 INFO:teuthology.orchestra.run.smithi148.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48994.log.gz 2024-03-29T04:13:14.401 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.51249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44278.log 2024-03-29T04:13:14.401 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51249.log.gz 2024-03-29T04:13:14.401 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50947.log 2024-03-29T04:13:14.401 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.52405.log: /var/log/ceph/ceph-client.admin.44278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45531.log 2024-03-29T04:13:14.401 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52405.log.gz 2024-03-29T04:13:14.401 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.50947.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44278.log.gz 2024-03-29T04:13:14.401 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49498.log 2024-03-29T04:13:14.401 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49666.log 2024-03-29T04:13:14.402 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.45531.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50947.log.gz 2024-03-29T04:13:14.402 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49498.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45531.log.gz 2024-03-29T04:13:14.402 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53658.log 2024-03-29T04:13:14.402 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49498.log.gz 2024-03-29T04:13:14.402 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49414.log 2024-03-29T04:13:14.402 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49666.log: /var/log/ceph/ceph-client.admin.53658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51133.log 2024-03-29T04:13:14.402 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49666.log.gz 2024-03-29T04:13:14.402 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49414.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53658.log.gz 2024-03-29T04:13:14.403 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45615.log 2024-03-29T04:13:14.403 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49414.log.gz 2024-03-29T04:13:14.403 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45243.log 2024-03-29T04:13:14.403 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.51133.log: /var/log/ceph/ceph-client.admin.45615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49330.log 2024-03-29T04:13:14.403 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51133.log.gz 2024-03-29T04:13:14.403 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.45243.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45615.log.gz 2024-03-29T04:13:14.403 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51417.log 2024-03-29T04:13:14.403 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49330.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45243.log.gz 2024-03-29T04:13:14.403 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46810.log 2024-03-29T04:13:14.404 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49330.log.gz 2024-03-29T04:13:14.404 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56493.log 2024-03-29T04:13:14.404 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.51417.log: /var/log/ceph/ceph-client.admin.46810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45285.log 2024-03-29T04:13:14.404 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51417.log.gz 2024-03-29T04:13:14.404 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46810.log.gz 2024-03-29T04:13:14.404 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.56493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51859.log 2024-03-29T04:13:14.404 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47127.log 2024-03-29T04:13:14.404 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.45285.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56493.log.gz 2024-03-29T04:13:14.405 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.51859.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45285.log.gz 2024-03-29T04:13:14.405 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57758.log 2024-03-29T04:13:14.405 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51859.log.gz 2024-03-29T04:13:14.405 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53342.log 2024-03-29T04:13:14.405 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47127.log: /var/log/ceph/ceph-client.admin.57758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52153.log 2024-03-29T04:13:14.405 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47127.log.gz 2024-03-29T04:13:14.405 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.53342.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57758.log.gz 2024-03-29T04:13:14.405 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50265.log 2024-03-29T04:13:14.405 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53342.log.gz 2024-03-29T04:13:14.406 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47169.log 2024-03-29T04:13:14.406 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.52153.log: /var/log/ceph/ceph-client.admin.50265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52783.log 2024-03-29T04:13:14.406 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52153.log.gz 2024-03-29T04:13:14.406 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47169.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50265.log.gz 2024-03-29T04:13:14.406 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44044.log 2024-03-29T04:13:14.406 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4%/var/log/ceph/ceph-client.admin.52783.log: -- replaced with /var/log/ceph/ceph-client.admin.47169.log.gz 2024-03-29T04:13:14.406 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45489.log 2024-03-29T04:13:14.406 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52783.log.gz 2024-03-29T04:13:14.406 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51175.log 2024-03-29T04:13:14.407 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.44044.log: /var/log/ceph/ceph-client.admin.45489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47313.log 2024-03-29T04:13:14.407 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44044.log.gz 2024-03-29T04:13:14.407 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.51175.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45489.log.gz 2024-03-29T04:13:14.407 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43544.log 2024-03-29T04:13:14.407 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51175.log.gz 2024-03-29T04:13:14.407 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53174.log 2024-03-29T04:13:14.407 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47313.log: /var/log/ceph/ceph-client.admin.43544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53839.log 2024-03-29T04:13:14.407 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47313.log.gz 2024-03-29T04:13:14.408 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.53174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46376.log 2024-03-29T04:13:14.408 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43544.log.gz 2024-03-29T04:13:14.408 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3%/var/log/ceph/ceph-client.admin.53839.log: -- replaced with /var/log/ceph/ceph-client.admin.53174.log.gz 2024-03-29T04:13:14.408 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52741.log 2024-03-29T04:13:14.408 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53839.log.gz 2024-03-29T04:13:14.408 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57800.log 2024-03-29T04:13:14.408 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46376.log.gz 2024-03-29T04:13:14.408 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55650.log 2024-03-29T04:13:14.408 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.52741.log: /var/log/ceph/ceph-client.admin.57800.log: gzip -5 --verbose -- 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52741.log.gz 2024-03-29T04:13:14.409 INFO:teuthology.orchestra.run.smithi148.stderr: /var/log/ceph/ceph-client.admin.53468.log 2024-03-29T04:13:14.409 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0%/var/log/ceph/ceph-client.admin.55650.log: -- replaced with /var/log/ceph/ceph-client.admin.57800.log.gz 2024-03-29T04:13:14.409 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50575.log 2024-03-29T04:13:14.409 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55650.log.gz 2024-03-29T04:13:14.409 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.53468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48952.log 2024-03-29T04:13:14.409 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.50575.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53468.log.gz 2024-03-29T04:13:14.409 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45369.log 2024-03-29T04:13:14.409 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50575.log.gz 2024-03-29T04:13:14.409 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43670.log 2024-03-29T04:13:14.410 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.48952.log: /var/log/ceph/ceph-client.admin.45369.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.31119.log 2024-03-29T04:13:14.410 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48952.log.gz 2024-03-29T04:13:14.410 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43670.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45369.log.gz 2024-03-29T04:13:14.410 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43918.log 2024-03-29T04:13:14.410 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43670.log.gz 2024-03-29T04:13:14.410 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph.tmp-client.admin.31119.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.31119.log.gz 2024-03-29T04:13:14.410 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53552.log 2024-03-29T04:13:14.410 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49792.log 2024-03-29T04:13:14.411 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43918.log: /var/log/ceph/ceph-client.admin.53552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51517.log 2024-03-29T04:13:14.413 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43918.log.gz 2024-03-29T04:13:14.413 INFO:teuthology.orchestra.run.smithi148.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53552.log.gz 2024-03-29T04:13:14.413 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44086.log 2024-03-29T04:13:14.413 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49792.log.gz 2024-03-29T04:13:14.413 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57548.log 2024-03-29T04:13:14.413 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.51517.log: /var/log/ceph/ceph-client.admin.44086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57860.log 2024-03-29T04:13:14.414 INFO:teuthology.orchestra.run.smithi148.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51517.log.gz 2024-03-29T04:13:14.414 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44086.log.gz 2024-03-29T04:13:14.414 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46379.log 2024-03-29T04:13:14.414 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47271.log 2024-03-29T04:13:14.414 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57860.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57548.log.gz 2024-03-29T04:13:14.414 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46379.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-29T04:13:14.414 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57860.log.gz 2024-03-29T04:13:14.414 INFO:teuthology.orchestra.run.smithi148.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46379.log.gz 2024-03-29T04:13:14.415 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51031.log 2024-03-29T04:13:14.415 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47271.log.gz 2024-03-29T04:13:14.415 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57338.log 2024-03-29T04:13:14.415 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.51031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54784.log 2024-03-29T04:13:14.415 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57338.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51031.log.gz 2024-03-29T04:13:14.415 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51602.log 2024-03-29T04:13:14.415 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49120.log 2024-03-29T04:13:14.415 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57338.log.gz 2024-03-29T04:13:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.54784.log: /var/log/ceph/ceph-client.admin.51602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52489.log 2024-03-29T04:13:14.416 INFO:teuthology.orchestra.run.smithi148.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.54784.log.gz 2024-03-29T04:13:14.416 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51602.log.gz 2024-03-29T04:13:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53384.log 2024-03-29T04:13:14.416 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-29T04:13:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.52489.log: /var/log/ceph/ceph-client.admin.53965.log 2024-03-29T04:13:14.416 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49120.log.gz 2024-03-29T04:13:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.53384.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52489.log.gz 2024-03-29T04:13:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46381.log 2024-03-29T04:13:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45783.log 2024-03-29T04:13:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.53965.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53384.log.gz 2024-03-29T04:13:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46381.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53965.log.gz 2024-03-29T04:13:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52843.log 2024-03-29T04:13:14.417 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46381.log.gz 2024-03-29T04:13:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45059.log 2024-03-29T04:13:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.45783.log: /var/log/ceph/ceph-client.admin.52843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47085.log 2024-03-29T04:13:14.417 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45783.log.gz 2024-03-29T04:13:14.418 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.45059.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52843.log.gz 2024-03-29T04:13:14.418 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52321.log 2024-03-29T04:13:14.418 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47085.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45059.log.gz 2024-03-29T04:13:14.418 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46917.log 2024-03-29T04:13:14.418 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.52321.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47085.log.gz 2024-03-29T04:13:14.418 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51696.log 2024-03-29T04:13:14.418 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46917.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52321.log.gz 2024-03-29T04:13:14.418 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46043.log 2024-03-29T04:13:14.418 INFO:teuthology.orchestra.run.smithi148.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46917.log.gz 2024-03-29T04:13:14.419 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.51696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53510.log 2024-03-29T04:13:14.419 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51696.log.gz 2024-03-29T04:13:14.419 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46661.log 2024-03-29T04:13:14.419 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46043.log: /var/log/ceph/ceph-client.admin.53510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45825.log 2024-03-29T04:13:14.419 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46043.log.gz 2024-03-29T04:13:14.419 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46661.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.53510.log.gz 2024-03-29T04:13:14.419 INFO:teuthology.orchestra.run.smithi148.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46039.log 2024-03-29T04:13:14.419 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46661.log.gz 2024-03-29T04:13:14.420 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.45825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53132.log 2024-03-29T04:13:14.420 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45825.log.gz 2024-03-29T04:13:14.420 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47397.log 2024-03-29T04:13:14.420 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53426.log 2024-03-29T04:13:14.420 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.53132.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46039.log.gz 2024-03-29T04:13:14.420 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57632.log 2024-03-29T04:13:14.420 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53132.log.gz 2024-03-29T04:13:14.420 INFO:teuthology.orchestra.run.smithi148.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47397.log.gz 2024-03-29T04:13:14.420 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49246.log/var/log/ceph/ceph-client.admin.53426.log: 2024-03-29T04:13:14.421 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46049.log 2024-03-29T04:13:14.421 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53426.log.gz 2024-03-29T04:13:14.421 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54301.log 2024-03-29T04:13:14.421 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57632.log.gz 2024-03-29T04:13:14.421 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56409.log 2024-03-29T04:13:14.421 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46049.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49246.log.gz 2024-03-29T04:13:14.421 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.54301.log: gzip -5 --verbose -- 87.7% /var/log/ceph/ceph-client.admin.50821.log 2024-03-29T04:13:14.421 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46049.log.gz 2024-03-29T04:13:14.421 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53258.log 2024-03-29T04:13:14.422 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.56409.log: /var/log/ceph/ceph-client.admin.50821.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.54301.log.gz 2024-03-29T04:13:14.422 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37505.log 2024-03-29T04:13:14.422 INFO:teuthology.orchestra.run.smithi148.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56409.log.gz 2024-03-29T04:13:14.422 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50821.log.gz 2024-03-29T04:13:14.422 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.53258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56764.log 2024-03-29T04:13:14.422 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53751.log 2024-03-29T04:13:14.422 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.37505.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53258.log.gz 2024-03-29T04:13:14.422 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.56764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53216.log 2024-03-29T04:13:14.423 INFO:teuthology.orchestra.run.smithi148.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37505.log.gz 2024-03-29T04:13:14.423 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56764.log.gz 2024-03-29T04:13:14.423 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50617.log 2024-03-29T04:13:14.423 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.53751.log: /var/log/ceph/ceph-client.admin.53216.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-29T04:13:14.423 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53751.log.gz 2024-03-29T04:13:14.423 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53216.log.gz 2024-03-29T04:13:14.423 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.50617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50349.log 2024-03-29T04:13:14.423 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50617.log.gz 2024-03-29T04:13:14.424 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46382.log 2024-03-29T04:13:14.424 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.50349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54049.log 2024-03-29T04:13:14.424 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46382.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50349.log.gz 2024-03-29T04:13:14.424 INFO:teuthology.orchestra.run.smithi148.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.57900.log 2024-03-29T04:13:14.424 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.54049.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46382.log.gz 2024-03-29T04:13:14.424 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49582.log 2024-03-29T04:13:14.424 INFO:teuthology.orchestra.run.smithi148.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54049.log.gz 2024-03-29T04:13:14.424 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52363.log 2024-03-29T04:13:14.425 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57900.log: /var/log/ceph/ceph-client.admin.49582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57464.log 2024-03-29T04:13:14.425 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57900.log.gz 2024-03-29T04:13:14.425 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.52363.log: gzip -5 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49582.log.gz 2024-03-29T04:13:14.425 INFO:teuthology.orchestra.run.smithi148.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44699.log 2024-03-29T04:13:14.425 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57464.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52363.log.gz 2024-03-29T04:13:14.425 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55998.log 2024-03-29T04:13:14.425 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57464.log.gz 2024-03-29T04:13:14.425 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56040.log 2024-03-29T04:13:14.426 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.44699.log: /var/log/ceph/ceph-client.admin.55998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50905.log 2024-03-29T04:13:14.426 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.56040.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55998.log.gz 2024-03-29T04:13:14.426 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48302.log 2024-03-29T04:13:14.426 INFO:teuthology.orchestra.run.smithi148.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56040.log.gz 2024-03-29T04:13:14.426 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.50905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57254.log 2024-03-29T04:13:14.426 INFO:teuthology.orchestra.run.smithi148.stderr:gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50905.log.gz 2024-03-29T04:13:14.426 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.48302.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43148.log 2024-03-29T04:13:14.426 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57254.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48302.log.gz 2024-03-29T04:13:14.427 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53300.log 2024-03-29T04:13:14.427 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57254.log.gz 2024-03-29T04:13:14.427 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43148.log: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.44699.log.gz 2024-03-29T04:13:14.427 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54175.log 2024-03-29T04:13:14.427 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55095.log 2024-03-29T04:13:14.427 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43148.log.gz 2024-03-29T04:13:14.427 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.53300.log: /var/log/ceph/ceph-client.admin.54175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50737.log 2024-03-29T04:13:14.427 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53300.log.gz 2024-03-29T04:13:14.428 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54175.log.gz 2024-03-29T04:13:14.428 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.55095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50779.log 2024-03-29T04:13:14.428 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44891.log 2024-03-29T04:13:14.428 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.50737.log: 90.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49960.log 2024-03-29T04:13:14.428 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.50779.log: -- replaced with /var/log/ceph/ceph-client.admin.55095.log.gz 2024-03-29T04:13:14.428 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50737.log.gz 2024-03-29T04:13:14.428 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.44891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49162.log 2024-03-29T04:13:14.428 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50779.log.gz 2024-03-29T04:13:14.428 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49960.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54007.log 2024-03-29T04:13:14.429 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44891.log.gz 2024-03-29T04:13:14.429 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49162.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49960.log.gz 2024-03-29T04:13:14.429 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54826.log 2024-03-29T04:13:14.429 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49162.log.gz 2024-03-29T04:13:14.429 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.54007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45017.log 2024-03-29T04:13:14.429 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46040.log 2024-03-29T04:13:14.429 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.54826.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54007.log.gz 2024-03-29T04:13:14.429 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.45017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56806.log 2024-03-29T04:13:14.430 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54826.log.gz 2024-03-29T04:13:14.430 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56199.log 2024-03-29T04:13:14.430 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46040.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45017.log.gz 2024-03-29T04:13:14.430 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.56806.log: gzip -5 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46040.log.gz 2024-03-29T04:13:14.430 INFO:teuthology.orchestra.run.smithi148.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47355.log 2024-03-29T04:13:14.430 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.56199.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56806.log.gz 2024-03-29T04:13:14.430 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46048.log 2024-03-29T04:13:14.430 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47355.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56199.log.gz 2024-03-29T04:13:14.430 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55794.log 2024-03-29T04:13:14.431 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47355.log.gz 2024-03-29T04:13:14.431 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52928.log 2024-03-29T04:13:14.431 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56637.log 2024-03-29T04:13:14.431 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.55794.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46048.log.gz 2024-03-29T04:13:14.431 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.52928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51739.log 2024-03-29T04:13:14.431 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55794.log.gz 2024-03-29T04:13:14.431 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.56637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53881.log 2024-03-29T04:13:14.431 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52928.log.gz 2024-03-29T04:13:14.432 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56637.log.gz 2024-03-29T04:13:14.432 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.51739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50449.log 2024-03-29T04:13:14.432 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47481.log 2024-03-29T04:13:14.432 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.53881.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51739.log.gz 2024-03-29T04:13:14.432 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.50449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50407.log 2024-03-29T04:13:14.432 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50449.log.gz 2024-03-29T04:13:14.432 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53881.log.gz 2024-03-29T04:13:14.432 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47481.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-03-29T04:13:14.433 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.50407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52111.log 2024-03-29T04:13:14.433 INFO:teuthology.orchestra.run.smithi148.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47481.log.gz 2024-03-29T04:13:14.433 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50407.log.gz 2024-03-29T04:13:14.433 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52657.log 2024-03-29T04:13:14.433 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.52111.log: -- /var/log/ceph/ceph-client.admin.43459.log 2024-03-29T04:13:14.433 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.52657.log: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49372.log 2024-03-29T04:13:14.433 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52111.log.gz 2024-03-29T04:13:14.433 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52657.log.gz 2024-03-29T04:13:14.433 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49036.log 2024-03-29T04:13:14.434 INFO:teuthology.orchestra.run.smithi148.stderr:gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43459.log.gz 2024-03-29T04:13:14.434 INFO:teuthology.orchestra.run.smithi148.stderr: -5/var/log/ceph/ceph-client.admin.49372.log: --verbose -- /var/log/ceph/ceph-client.admin.46581.log 2024-03-29T04:13:14.434 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49036.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49372.log.gz 2024-03-29T04:13:14.434 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45699.log 2024-03-29T04:13:14.434 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49036.log.gz 2024-03-29T04:13:14.434 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51375.log 2024-03-29T04:13:14.434 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46581.log: /var/log/ceph/ceph-client.admin.45699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49288.log 2024-03-29T04:13:14.434 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46581.log.gz 2024-03-29T04:13:14.435 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.51375.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45699.log.gz 2024-03-29T04:13:14.435 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47607.log 2024-03-29T04:13:14.435 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49288.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51375.log.gz 2024-03-29T04:13:14.435 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46371.log 2024-03-29T04:13:14.435 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49288.log.gz 2024-03-29T04:13:14.435 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55692.log 2024-03-29T04:13:14.435 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47607.log: /var/log/ceph/ceph-client.admin.46371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54091.log 2024-03-29T04:13:14.435 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47607.log.gz 2024-03-29T04:13:14.436 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.55692.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46371.log.gz 2024-03-29T04:13:14.436 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46041.log 2024-03-29T04:13:14.436 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55692.log.gz 2024-03-29T04:13:14.436 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.54091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49540.log 2024-03-29T04:13:14.436 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46041.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.54091.log.gz 2024-03-29T04:13:14.436 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44170.log 2024-03-29T04:13:14.436 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46041.log.gz 2024-03-29T04:13:14.436 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47788.log 2024-03-29T04:13:14.436 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.44170.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49540.log.gz 2024-03-29T04:13:14.437 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57002.log 2024-03-29T04:13:14.437 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44170.log.gz 2024-03-29T04:13:14.437 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49624.log 2024-03-29T04:13:14.437 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47788.log: /var/log/ceph/ceph-client.admin.57002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50533.log 2024-03-29T04:13:14.437 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47788.log.gz/var/log/ceph/ceph-client.admin.49624.log: 2024-03-29T04:13:14.437 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57002.log.gz 2024-03-29T04:13:14.437 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56082.log 2024-03-29T04:13:14.437 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49624.log.gz 2024-03-29T04:13:14.437 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.50533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.48678.log 2024-03-29T04:13:14.438 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50533.log.gz 2024-03-29T04:13:14.438 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.56082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57506.log 2024-03-29T04:13:14.438 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56082.log.gz 2024-03-29T04:13:14.438 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.2.48678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44002.log 2024-03-29T04:13:14.438 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45101.log 2024-03-29T04:13:14.438 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.44002.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57506.log.gz 2024-03-29T04:13:14.438 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46658.log 2024-03-29T04:13:14.439 INFO:teuthology.orchestra.run.smithi148.stderr: 91.4% 88.7% -- replaced with /var/log/ceph/ceph-client.2.48678.log.gz 2024-03-29T04:13:14.439 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44002.log.gz 2024-03-29T04:13:14.439 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51291.log 2024-03-29T04:13:14.439 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.45101.log: /var/log/ceph/ceph-client.admin.46658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51817.log 2024-03-29T04:13:14.439 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.51291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52699.log 2024-03-29T04:13:14.439 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46658.log.gz -- replaced with /var/log/ceph/ceph-client.admin.45101.log.gz 2024-03-29T04:13:14.439 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-03-29T04:13:14.439 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.51817.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51291.log.gz 2024-03-29T04:13:14.439 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48842.log 2024-03-29T04:13:14.440 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51817.log.gz 2024-03-29T04:13:14.440 INFO:teuthology.orchestra.run.smithi148.stderr:gzip/var/log/ceph/ceph-client.admin.52699.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44128.log 2024-03-29T04:13:14.440 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.48842.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52699.log.gz 2024-03-29T04:13:14.440 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49078.log 2024-03-29T04:13:14.440 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48842.log.gz 2024-03-29T04:13:14.440 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.44128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55956.log 2024-03-29T04:13:14.440 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49078.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44128.log.gz 2024-03-29T04:13:14.440 INFO:teuthology.orchestra.run.smithi148.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52886.log 2024-03-29T04:13:14.440 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49078.log.gz 2024-03-29T04:13:14.441 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.55956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52573.log 2024-03-29T04:13:14.441 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55956.log.gz 2024-03-29T04:13:14.441 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.52886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46046.log 2024-03-29T04:13:14.441 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52886.log.gz 2024-03-29T04:13:14.441 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.52573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57674.log 2024-03-29T04:13:14.441 INFO:teuthology.orchestra.run.smithi148.stderr:gzip/var/log/ceph/ceph-client.admin.46046.log: 89.4% -5 --verbose -- /var/log/ceph/ceph-client.admin.50307.log 2024-03-29T04:13:14.441 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52573.log.gz 2024-03-29T04:13:14.441 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50695.log 2024-03-29T04:13:14.442 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46046.log.gz 2024-03-29T04:13:14.442 INFO:teuthology.orchestra.run.smithi148.stderr:gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57674.log.gz -5 2024-03-29T04:13:14.442 INFO:teuthology.orchestra.run.smithi148.stderr: --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-03-29T04:13:14.442 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.50307.log: /var/log/ceph/ceph-client.admin.50695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52069.log 2024-03-29T04:13:14.442 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2%/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44975.log 2024-03-29T04:13:14.442 INFO:teuthology.orchestra.run.smithi148.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.50307.log.gz 2024-03-29T04:13:14.442 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50695.log.gz 2024-03-29T04:13:14.442 INFO:teuthology.orchestra.run.smithi148.stderr:gzip/var/log/ceph/ceph-client.admin.52069.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46042.log 2024-03-29T04:13:14.443 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.44975.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52069.log.gz 2024-03-29T04:13:14.443 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47001.log 2024-03-29T04:13:14.443 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44975.log.gz 2024-03-29T04:13:14.443 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51560.log/var/log/ceph/ceph-client.admin.46042.log: 2024-03-29T04:13:14.443 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50139.log 2024-03-29T04:13:14.443 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46042.log.gz 2024-03-29T04:13:14.443 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47001.log.gz 2024-03-29T04:13:14.443 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.51560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44933.log 2024-03-29T04:13:14.443 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2%/var/log/ceph/ceph-client.admin.50139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56722.log 2024-03-29T04:13:14.444 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51560.log.gz/var/log/ceph/ceph-client.admin.44933.log: 2024-03-29T04:13:14.444 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50139.log.gz 2024-03-29T04:13:14.444 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48180.log 2024-03-29T04:13:14.444 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44933.log.gz 2024-03-29T04:13:14.444 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.56722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55179.log 2024-03-29T04:13:14.444 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.48180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56451.log 2024-03-29T04:13:14.444 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56722.log.gz 2024-03-29T04:13:14.444 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.55179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54367.log 2024-03-29T04:13:14.445 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.56451.log: 90.4% -- replaced with /var/log/ceph/ceph-client.0.48180.log.gz 2024-03-29T04:13:14.445 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56451.log.gz 2024-03-29T04:13:14.445 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55914.log 2024-03-29T04:13:14.445 INFO:teuthology.orchestra.run.smithi148.stderr: 90.8%/var/log/ceph/ceph-client.admin.54367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56241.log 2024-03-29T04:13:14.445 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55179.log.gz/var/log/ceph/ceph-client.admin.55914.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54367.log.gz 2024-03-29T04:13:14.445 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-03-29T04:13:14.445 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43586.log 2024-03-29T04:13:14.445 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55914.log.gz 2024-03-29T04:13:14.445 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.56241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47658.log 2024-03-29T04:13:14.446 INFO:teuthology.orchestra.run.smithi148.stderr: 87.7%/var/log/ceph/ceph-client.admin.43586.log: -- replaced with /var/log/ceph/ceph-client.admin.56241.log.gz 2024-03-29T04:13:14.446 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45327.log 2024-03-29T04:13:14.446 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43586.log.gz 2024-03-29T04:13:14.446 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-03-29T04:13:14.446 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47658.log: /var/log/ceph/ceph-client.admin.45327.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45327.log.gz 2024-03-29T04:13:14.446 INFO:teuthology.orchestra.run.smithi148.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.47658.log.gz 2024-03-29T04:13:14.446 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57212.log 2024-03-29T04:13:14.446 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53709.log 2024-03-29T04:13:14.447 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43417.log 2024-03-29T04:13:14.447 INFO:teuthology.orchestra.run.smithi148.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57212.log.gz/var/log/ceph/ceph-client.admin.53709.log: 2024-03-29T04:13:14.447 INFO:teuthology.orchestra.run.smithi148.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.53709.log.gz 2024-03-29T04:13:14.447 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52447.log 2024-03-29T04:13:14.447 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43758.log 2024-03-29T04:13:14.447 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43417.log.gz 2024-03-29T04:13:14.447 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.52447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44212.log 2024-03-29T04:13:14.447 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43758.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52447.log.gz 2024-03-29T04:13:14.447 INFO:teuthology.orchestra.run.smithi148.stderr:gzip 87.7% -5 --verbose -- /var/log/ceph/ceph-client.admin.50223.log 2024-03-29T04:13:14.448 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43758.log.gz 2024-03-29T04:13:14.448 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49456.log 2024-03-29T04:13:14.448 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.44212.log: /var/log/ceph/ceph-client.admin.50223.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50223.log.gz 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44212.log.gz 2024-03-29T04:13:14.448 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-03-29T04:13:14.448 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56535.log 2024-03-29T04:13:14.448 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45657.log 2024-03-29T04:13:14.448 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49456.log: /var/log/ceph/ceph-client.admin.56535.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49456.log.gz 2024-03-29T04:13:14.448 INFO:teuthology.orchestra.run.smithi148.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56535.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49918.log 2024-03-29T04:13:14.449 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-03-29T04:13:14.449 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.45657.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45657.log.gz 2024-03-29T04:13:14.449 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50491.log 2024-03-29T04:13:14.449 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43240.log 2024-03-29T04:13:14.449 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49918.log.gz 2024-03-29T04:13:14.449 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.50491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56680.log 2024-03-29T04:13:14.449 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43240.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50491.log.gz 2024-03-29T04:13:14.449 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52237.log 2024-03-29T04:13:14.449 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43240.log.gz 2024-03-29T04:13:14.450 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.56680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43628.log 2024-03-29T04:13:14.450 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.52237.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56680.log.gz 2024-03-29T04:13:14.450 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52237.log.gz 2024-03-29T04:13:14.450 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43960.log 2024-03-29T04:13:14.450 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56960.log 2024-03-29T04:13:14.450 INFO:teuthology.orchestra.run.smithi148.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43628.log.gz 2024-03-29T04:13:14.450 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43960.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43960.log.gz 2024-03-29T04:13:14.450 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-29T04:13:14.451 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57296.log 2024-03-29T04:13:14.451 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.56960.log: /var/log/ceph/ceph.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56960.log.gz 2024-03-29T04:13:14.451 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-03-29T04:13:14.451 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57296.log: 85.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-29T04:13:14.451 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57296.log.gz 2024-03-29T04:13:14.451 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49750.log 2024-03-29T04:13:14.451 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56325.log 2024-03-29T04:13:14.451 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.49750.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49750.log.gz 2024-03-29T04:13:14.451 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47565.log 2024-03-29T04:13:14.452 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.56325.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56325.log.gz 2024-03-29T04:13:14.453 INFO:teuthology.orchestra.run.smithi157.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-03-29T04:13:14.453 INFO:teuthology.orchestra.run.smithi157.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-03-29T04:13:14.466 INFO:teuthology.orchestra.run.smithi148.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-03-29T04:13:14.466 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37151.log 2024-03-29T04:13:14.466 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47565.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-03-29T04:13:14.466 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.37151.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47565.log.gz 2024-03-29T04:13:14.467 INFO:teuthology.orchestra.run.smithi148.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37151.log.gz 2024-03-29T04:13:14.467 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51652.log 2024-03-29T04:13:14.467 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57086.log 2024-03-29T04:13:14.467 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.51652.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51652.log.gz 2024-03-29T04:13:14.467 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46660.log 2024-03-29T04:13:14.467 INFO:teuthology.orchestra.run.smithi148.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-03-29T04:13:14.467 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57086.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57086.log.gz 2024-03-29T04:13:14.467 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51073.log 2024-03-29T04:13:14.468 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51943.log 2024-03-29T04:13:14.468 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46660.log: /var/log/ceph/ceph-client.admin.51073.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46660.log.gz 2024-03-29T04:13:14.468 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51073.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43336.log 2024-03-29T04:13:14.468 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-03-29T04:13:14.468 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.51943.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51943.log.gz 2024-03-29T04:13:14.468 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49204.log 2024-03-29T04:13:14.468 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47439.log 2024-03-29T04:13:14.468 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43336.log: /var/log/ceph/ceph-client.admin.49204.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43336.log.gz 2024-03-29T04:13:14.468 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46959.log 2024-03-29T04:13:14.469 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49204.log.gz 2024-03-29T04:13:14.469 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47439.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47439.log.gz 2024-03-29T04:13:14.469 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53923.log 2024-03-29T04:13:14.469 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56367.log 2024-03-29T04:13:14.469 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46959.log: /var/log/ceph/ceph-client.admin.53923.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46959.log.gz 2024-03-29T04:13:14.469 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53923.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43502.log 2024-03-29T04:13:14.469 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-03-29T04:13:14.469 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.56367.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56367.log.gz 2024-03-29T04:13:14.469 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57044.log 2024-03-29T04:13:14.470 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56577.log 2024-03-29T04:13:14.470 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43502.log: /var/log/ceph/ceph-client.admin.57044.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43502.log.gz 2024-03-29T04:13:14.470 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57044.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50863.log 2024-03-29T04:13:14.470 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-03-29T04:13:14.470 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.56577.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56577.log.gz 2024-03-29T04:13:14.470 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54885.log 2024-03-29T04:13:14.470 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57170.log 2024-03-29T04:13:14.470 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.50863.log: /var/log/ceph/ceph-client.admin.54885.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50863.log.gz 2024-03-29T04:13:14.471 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54885.log.gz 2024-03-29T04:13:14.471 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57422.log 2024-03-29T04:13:14.471 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57590.log 2024-03-29T04:13:14.471 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57170.log: /var/log/ceph/ceph-client.admin.57422.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57170.log.gz 2024-03-29T04:13:14.471 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57422.log.gz 2024-03-29T04:13:14.471 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51985.log 2024-03-29T04:13:14.471 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57590.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57590.log.gz 2024-03-29T04:13:14.471 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46378.log 2024-03-29T04:13:14.472 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52615.log 2024-03-29T04:13:14.472 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.51985.log: /var/log/ceph/ceph-client.admin.46378.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51985.log.gz 2024-03-29T04:13:14.472 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46378.log.gz 2024-03-29T04:13:14.472 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46385.log 2024-03-29T04:13:14.472 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.52615.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52615.log.gz 2024-03-29T04:13:14.472 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56283.log 2024-03-29T04:13:14.472 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54133.log 2024-03-29T04:13:14.472 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46385.log: /var/log/ceph/ceph-client.admin.56283.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46385.log.gz 2024-03-29T04:13:14.473 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56283.log.gz 2024-03-29T04:13:14.473 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47211.log 2024-03-29T04:13:14.473 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-03-29T04:13:14.473 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.54133.log: /var/log/ceph/ceph-client.admin.47211.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54133.log.gz 2024-03-29T04:13:14.473 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47211.log.gz 2024-03-29T04:13:14.473 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47523.log 2024-03-29T04:13:14.473 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52970.log 2024-03-29T04:13:14.473 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47523.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47523.log.gz 2024-03-29T04:13:14.474 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57380.log 2024-03-29T04:13:14.474 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.52970.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52970.log.gz 2024-03-29T04:13:14.474 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57380.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57380.log.gz 2024-03-29T04:13:14.518 INFO:teuthology.orchestra.run.smithi148.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-29T04:13:14.780 INFO:teuthology.orchestra.run.smithi148.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-03-29T04:13:15.021 INFO:teuthology.orchestra.run.smithi157.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-03-29T04:13:15.036 INFO:teuthology.orchestra.run.smithi157.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-03-29T04:13:15.037 INFO:teuthology.orchestra.run.smithi148.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-29T04:13:15.091 INFO:teuthology.orchestra.run.smithi157.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-03-29T04:13:15.117 INFO:teuthology.orchestra.run.smithi157.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-03-29T04:13:15.119 INFO:teuthology.orchestra.run.smithi157.stderr: 2024-03-29T04:13:15.119 INFO:teuthology.orchestra.run.smithi157.stderr:real 0m0.826s 2024-03-29T04:13:15.120 INFO:teuthology.orchestra.run.smithi157.stderr:user 0m3.149s 2024-03-29T04:13:15.120 INFO:teuthology.orchestra.run.smithi157.stderr:sys 0m0.146s 2024-03-29T04:13:15.225 INFO:teuthology.orchestra.run.smithi148.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-29T04:13:15.244 INFO:teuthology.orchestra.run.smithi148.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-29T04:13:15.273 INFO:teuthology.orchestra.run.smithi148.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-29T04:13:15.731 INFO:teuthology.orchestra.run.smithi148.stderr: 97.1% -- replaced with /var/log/ceph/ceph-client.0.54627.log.gz 2024-03-29T04:13:15.733 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-03-29T04:13:15.733 INFO:teuthology.orchestra.run.smithi148.stderr:real 0m1.472s 2024-03-29T04:13:15.733 INFO:teuthology.orchestra.run.smithi148.stderr:user 0m5.490s 2024-03-29T04:13:15.734 INFO:teuthology.orchestra.run.smithi148.stderr:sys 0m0.351s 2024-03-29T04:13:15.734 INFO:tasks.ceph:Archiving logs... 2024-03-29T04:13:15.735 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629083/remote/smithi148/log 2024-03-29T04:13:15.736 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-29T04:13:16.636 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629083/remote/smithi157/log 2024-03-29T04:13:16.637 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-29T04:13:17.127 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-29T04:13:17.137 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-29T04:13:17.137 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-29T04:13:17.140 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-29T04:13:17.239 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-29T04:13:17.249 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-29T04:13:17.250 DEBUG:teuthology.orchestra.run.smithi148:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T04:13:17.253 DEBUG:teuthology.orchestra.run.smithi157:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T04:13:17.265 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-03-29T04:13:17.269 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2024-03-29T04:13:17.270 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-29T04:13:17.270 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-03-29T04:13:17.270 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 47 +65us[ +60us] +/- 80ms 2024-03-29T04:13:17.270 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 47 -1307us[-1312us] +/- 71ms 2024-03-29T04:13:17.270 INFO:teuthology.orchestra.run.smithi148.stdout:^* hv03.front.sepia.ceph.com 3 6 377 47 +1190us[+1185us] +/- 68ms 2024-03-29T04:13:17.271 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:13:17.274 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-29T04:13:17.274 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-03-29T04:13:17.274 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 45 +72us[ +72us] +/- 80ms 2024-03-29T04:13:17.274 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 47 -1289us[-1289us] +/- 71ms 2024-03-29T04:13:17.274 INFO:teuthology.orchestra.run.smithi157.stdout:^* hv03.front.sepia.ceph.com 3 6 377 47 +1209us[+1207us] +/- 68ms 2024-03-29T04:13:17.275 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:13:17.275 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-29T04:13:17.285 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-29T04:13:17.285 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-29T04:13:17.305 DEBUG:teuthology.orchestra.run.smithi148:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-29T04:13:17.390 DEBUG:teuthology.orchestra.run.smithi157:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-29T04:13:17.465 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-29T04:13:17.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:13:17.497 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-29T04:13:17.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:13:17.528 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-29T04:13:17.538 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-29T04:13:17.556 INFO:teuthology.task.internal:Duration was 630.711010 seconds 2024-03-29T04:13:17.556 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-29T04:13:17.580 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-29T04:13:17.580 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-29T04:13:17.583 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-29T04:13:17.624 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-29T04:13:17.629 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-29T04:13:18.027 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-29T04:13:18.027 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi148.front.sepia.ceph.com 2024-03-29T04:13:18.027 DEBUG:teuthology.orchestra.run.smithi148:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-29T04:13:18.087 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi157.front.sepia.ceph.com 2024-03-29T04:13:18.087 DEBUG:teuthology.orchestra.run.smithi157:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-29T04:13:18.115 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-29T04:13:18.115 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-29T04:13:18.130 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-29T04:13:18.184 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-29T04:13:18.185 DEBUG:teuthology.orchestra.run.smithi148:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-29T04:13:18.200 DEBUG:teuthology.orchestra.run.smithi157:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-29T04:13:18.324 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-29T04:13:18.335 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-29T04:13:18.335 DEBUG:teuthology.orchestra.run.smithi148:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-29T04:13:18.363 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-29T04:13:18.391 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-29T04:13:18.401 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-29T04:13:18.406 DEBUG:teuthology.orchestra.run.smithi157:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-29T04:13:18.428 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = core 2024-03-29T04:13:18.456 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = core 2024-03-29T04:13:18.472 DEBUG:teuthology.orchestra.run.smithi148:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-29T04:13:18.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:13:18.499 DEBUG:teuthology.orchestra.run.smithi157:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-29T04:13:18.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:13:18.526 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-29T04:13:18.537 INFO:teuthology.task.internal:Transferring archived files... 2024-03-29T04:13:18.537 DEBUG:teuthology.misc:Transferring archived files from smithi148:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629083/remote/smithi148 2024-03-29T04:13:18.538 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-29T04:13:18.575 DEBUG:teuthology.misc:Transferring archived files from smithi157:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629083/remote/smithi157 2024-03-29T04:13:18.575 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-29T04:13:18.611 INFO:teuthology.task.internal:Removing archive directory... 2024-03-29T04:13:18.612 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-29T04:13:18.614 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-29T04:13:18.660 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-29T04:13:18.672 INFO:teuthology.task.internal:Not uploading archives. 2024-03-29T04:13:18.672 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-29T04:13:18.690 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-29T04:13:18.690 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-29T04:13:18.692 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-29T04:13:18.706 INFO:teuthology.orchestra.run.smithi148.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 29 04:13 /home/ubuntu/cephtest 2024-03-29T04:13:18.715 INFO:teuthology.orchestra.run.smithi157.stdout: 265778 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 29 04:13 /home/ubuntu/cephtest 2024-03-29T04:13:18.716 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-29T04:13:18.727 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-29T04:13:18.774 INFO:teuthology.nuke:Checking targets against current locks 2024-03-29T04:13:18.801 DEBUG:teuthology.nuke:shortname: smithi148 2024-03-29T04:13:18.801 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-29T04:13:18.828 DEBUG:teuthology.nuke:shortname: smithi157 2024-03-29T04:13:18.828 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-29T04:13:18.832 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629083', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 03:53:41.239490', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE2Ekop2D5/qeW+aNVldgxNC+pD18+a42gfuvyjNTcjUOI88NVUYIADLe947cAob/WzHHAqLMW1lHVreP4Li6xg='} 2024-03-29T04:13:18.851 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629083', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 03:53:41.241399', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBznV4lGC+LsGWKVf03Y7PBNSd75NFHyTgX517Oz732WskLXmpV6VD/C0X9zCxweqK1PwYSM0nz3uz+E2foVWM8='} 2024-03-29T04:13:18.884 INFO:teuthology.orchestra.console.smithi148:Power off 2024-03-29T04:13:18.885 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-29T04:13:18.908 DEBUG:teuthology.orchestra.console.smithi148:power off output: Chassis Power Control: Down/Off 2024-03-29T04:13:18.909 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T04:13:18.926 INFO:teuthology.orchestra.console.smithi157:Power off 2024-03-29T04:13:18.927 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-29T04:13:18.941 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2024-03-29T04:13:18.953 DEBUG:teuthology.orchestra.console.smithi157:power off output: Chassis Power Control: Down/Off 2024-03-29T04:13:18.954 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T04:13:18.979 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2024-03-29T04:13:22.942 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T04:13:22.981 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T04:13:23.052 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2024-03-29T04:13:23.091 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2024-03-29T04:13:27.054 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T04:13:27.092 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T04:13:27.165 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2024-03-29T04:13:27.202 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off 2024-03-29T04:13:27.302 INFO:teuthology.orchestra.console.smithi157:Power off completed 2024-03-29T04:13:31.170 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T04:13:31.280 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is off 2024-03-29T04:13:31.381 INFO:teuthology.orchestra.console.smithi148:Power off completed 2024-03-29T04:13:31.486 INFO:teuthology.run:Summary data: description: fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/uninlining} duration: 630.7110097408295 failure_reason: 'Test failure: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining)' flavor: default owner: scheduled_mchangir@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=c1f9a4edd6f3472b8a05cc4bf58637a0 status: fail success: false 2024-03-29T04:13:31.486 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-29T04:13:31.568 INFO:teuthology.run:FAIL